Audio and video data uploading method and system, and storage medium

Abstract

Provided in the present invention are an audio and video data uploading method, an audio and video data uploading system, and a computer-readable storage medium. The audio and video data uploading method comprises the following steps: acquiring audio and video data, the audio and video data being divided into a plurality of GOPs; on the basis of the plurality of GOPs, separately sending a plurality of uploading requests to an HTTP interface of a resource pool to upload the plurality of GOPs, the uploading requests comprising offset values; and in response to the completion of uploading of the audio and video data, concatenating the plurality of GOPs on the basis of the offset values to form a concatenated file. The audio and video data uploading method, the audio and video data uploading system and the computer-readable storage medium provided by the present invention can optimize uploading modes of a cloud video recording function without changing original hardware resource configurations, thereby improving the success rate of audio and video data uploading.

Description

Method, system and storage medium for uploading audio and video data Technical Field

[0001] The present invention relates to the field of security video, and in particular to a method for uploading audio and video data, a system for uploading audio and video data, and a computer-readable storage medium. Background Art

[0002] With the public's growing awareness of safety, an increasing number of Internet Protocol Camera (IPC) terminals are being installed in factories, offices, stores, and other places requiring surveillance. Local recording is one of the basic functions of an IP camera. It allows the camera to continuously record video and store the audio and video data on a hard drive or SD card, allowing users to retrieve local footage for use as evidence. Cloud recording, on the other hand, uploads locally recorded audio and video data to the cloud, allowing users to access the IP camera's audio and video data from the cloud via mobile applications. Local recording has drawbacks such as being easily lost, easily damaged, and costly. Therefore, enabling both local and cloud recording greatly improves the security of audio and video data.

[0003] In the prior art, cloud recording uses a chunked transfer encoding (Chunked Transfer Encoding) method based on the Hypertext Transfer Protocol (HTTP) to upload audio and video data.

[0004] Please refer to FIG1 , which shows a schematic diagram of uploading audio and video data through block transmission encoding in the prior art.

[0005] As shown in Figure 1, the audio and video data uploading system for a network surveillance camera includes an audio and video data acquisition module 110, an audio and video data transmission module 120, and a cloud-based resource pool 130. Chunked transfer encoding works by continuously acquiring real-time audio and video data from the audio and video data acquisition module 110 and uploading it to the resource pool 130 via the audio and video data transmission module 120. At the end of a period, the audio and video data acquisition module 110 sends an end packet to the audio and video data transmission module 120, which then sends a chunked terminator to the resource pool 130 to complete the upload.

[0006] However, when the user's network fluctuates, the current cloud recording may not be able to upload the audio and video data completely, resulting in the loss of the video. Therefore, uploading audio and video data through block transfer encoding has very high network requirements. In the case of network fluctuations, the upload success rate of audio and video data is low, and a large amount of server resources are wasted.

[0007] In order to overcome the above-mentioned defects of the existing technology, this field urgently needs an audio and video data uploading technology that optimizes the uploading method of the cloud recording function without changing the original hardware resource configuration and improves the upload success rate of audio and video data. Summary of the Invention

[0008] The following is a brief summary of one or more aspects to provide a basic understanding of these aspects. This summary is not an exhaustive overview of all conceivable aspects and is neither intended to identify key or critical elements of all aspects nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that will be provided later.

[0009] In order to overcome the above-mentioned defects of the prior art, the present invention provides a method for uploading audio and video data, a system for uploading audio and video data, and a computer-readable storage medium. Without changing the original hardware resource configuration, the present invention optimizes the uploading method of the cloud recording function and improves the success rate of uploading audio and video data.

[0010] Specifically, the above-mentioned method for uploading audio and video data provided according to the first aspect of the present invention includes the steps of: obtaining audio and video data, wherein the audio and video data is divided into multiple GOPs; based on the multiple GOPs, sending multiple upload requests to the HTTP interface of the resource pool to upload the multiple GOPs, and the upload request includes an offset value; in response to the completion of uploading the audio and video data, splicing the multiple GOPs according to the offset value to form a spliced ​​file.

[0011] Preferably, in one embodiment of the present invention, the completion of uploading the audio and video data includes: determining that the uploading of the audio and video data is completed based on not receiving the upload request within a preset time range; and further, determining that the uploading of the audio and video data is completed based on the received end character.

[0012] Preferably, in one embodiment of the present invention, the audio and video data includes real-time audio and video data and non-real-time audio and video data, and the step of obtaining audio and video data includes: obtaining the real-time audio and video data via an audio and video ring buffer; in addition, obtaining the non-real-time audio and video data via a local video recording cache.

[0013] Preferably, in one embodiment of the present invention, the real-time audio and video data generates the multiple GOPs one by one, and the step of sending multiple upload requests to the HTTP interface of the resource pool to upload the multiple GOPs includes: based on the multiple GOPs generated one by one, sending the upload requests to the HTTP interface of the resource pool in sequence to upload the multiple GOPs.

[0014] Preferably, in one embodiment of the present invention, the audio and video data is non-real-time audio and video data, and the step of sending multiple upload requests to the HTTP interface of the resource pool respectively to upload the multiple GOPs includes: based on the multiple GOPs, sending the upload requests to the HTTP interface of the resource pool respectively and in parallel to upload the multiple GOPs.

[0015] Preferably, in one embodiment of the present invention, the offset value is determined by the position of the starting point of the GOP in the entire data segment of the audio and video data.

[0016] Furthermore, the audio and video data uploading system provided in accordance with the second aspect of the present invention includes a memory and a processor. The memory stores computer instructions. The processor is connected to the memory and configured to execute the computer instructions stored in the memory to implement the audio and video data uploading method provided in any of the above embodiments.

[0017] Furthermore, the computer-readable storage medium provided in accordance with the third aspect of the present invention stores computer instructions, which, when executed by a processor, implement the method for uploading audio and video data provided in any one of the above embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above features and advantages of the present invention will be better understood after reading the detailed description of the embodiments of the present disclosure in conjunction with the following drawings. In the drawings, the components are not necessarily drawn to scale, and components with similar related properties or characteristics may have the same or similar reference numerals.

[0019] FIG1 is a schematic diagram showing a block-wise transmission encoding method for uploading audio and video data in the prior art;

[0020] FIG2 shows a flow chart of a method for uploading audio and video data according to some embodiments of the present invention;

[0021] FIG3 shows a schematic diagram of a system for uploading audio and video data according to some embodiments of the present invention;

[0022] FIG4 is a schematic diagram showing a plurality of GOPs and offset values ​​provided according to some embodiments of the present invention;

[0023] 5A-5B are schematic diagrams showing methods for uploading audio and video data according to some embodiments of the present invention; and

[0024] FIG6 shows a schematic diagram of a spliced ​​file provided according to some embodiments of the present invention.

[0025] Reference numerals: 110, 310: audio and video data acquisition module; 120, 320: audio and video data sending module; 130, 330: resource pool; 200: audio and video data uploading method; S210-S230: steps; 300: audio and video data uploading system; and 321: sending buffer. DETAILED DESCRIPTION

[0026] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. Note that the various aspects described below with reference to the accompanying drawings and specific embodiments are merely exemplary and should not be construed as limiting the scope of protection of the present invention.

[0027] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0028] Furthermore, the terms "upper," "lower," "left," "right," "top," "bottom," "horizontal," and "vertical" used in the following description should be understood to refer to the orientations depicted in that section and the accompanying drawings. These relative terms are used solely for convenience of description and do not necessarily imply that the devices described herein must be manufactured or operated in a specific orientation. Therefore, they should not be construed as limiting the present invention.

[0029] It will be understood that although the terms "first," "second," "third," etc. may be used herein to describe various components, regions, layers, and / or portions, these components, regions, layers, and / or portions should not be limited by these terms, and these terms are merely used to distinguish different components, regions, layers, and / or portions. Thus, a first component, region, layer, and / or portion discussed below may be referred to as a second component, region, layer, and / or portion without departing from some embodiments of the present invention.

[0030] As mentioned above, uploading audio and video data through block transfer encoding has very high requirements on the network. In the case of network fluctuations, the success rate of uploading audio and video data is low, and a large amount of server resources will be wasted.

[0031] Chunked transfer encoding sends a single upload request to the HTTP interface when uploading audio and video data. The chunked audio and video data is then uploaded continuously and in a single batch in chunked packets. After receiving the "chunked" terminator, the resource pool automatically reassembles the chunked audio and video data. The resource pool can return a status code of 200 to indicate a successful request.

[0032] In order to overcome the above-mentioned defects of the prior art, the present invention provides a method for uploading audio and video data, a system for uploading audio and video data, and a computer-readable storage medium. Without changing the original hardware resource configuration, the present invention optimizes the uploading method of the cloud recording function and improves the success rate of uploading audio and video data.

[0033] In some non-limiting embodiments, the audio and video data uploading method provided in the first aspect of the present invention can be implemented via the audio and video data uploading system provided in the second aspect of the present invention. Specifically, the system can be configured with a memory and a processor. The memory includes, but is not limited to, the computer-readable storage medium provided in the third aspect of the present invention, on which computer instructions are stored. The processor is connected to the memory and is configured to execute the computer instructions stored in the memory to implement the audio and video data uploading method provided in the first aspect of the present invention.

[0034] The following will first describe the working principle of the above-mentioned audio and video data uploading system in conjunction with some embodiments of the audio and video data uploading method. Those skilled in the art will understand that the embodiments of these audio and video data uploading methods are only some non-restrictive implementation methods provided by the present invention, which are intended to clearly demonstrate the main concept of the present invention and provide some specific solutions that are convenient for the public to implement, rather than to limit all functions or all working methods of the system. Similarly, the audio and video data uploading system is also only a non-restrictive implementation method provided by the present invention, and does not constitute a limitation on the execution subject and execution order of each step in these audio and video data uploading methods.

[0035] Please refer to FIG2 , which shows a flowchart of a method for uploading audio and video data according to some embodiments of the present invention.

[0036] As shown in FIG. 2 , the method 200 for uploading audio and video data includes step S210 : acquiring audio and video data, where the audio and video data is divided into a plurality of GOPs.

[0037] Audio and video data can be divided into multiple GOPs (Group of Pictures). Each GOP can be segmented based on a fixed time period or by using keyframes as nodes. For example, if there are 10 keyframes, the audio and video data can be divided into 9 GOPs.

[0038] Please refer to FIG3 , which shows a schematic diagram of a system for uploading audio and video data according to some embodiments of the present invention.

[0039] As shown in Figure 3, the audio and video data upload system 300 may include an audio and video data acquisition module 310, an audio and video data sending module 320 and a cloud resource pool 330. The audio and video data upload system 300 can obtain audio and video data from the audio and video data acquisition module 310, and the audio and video data can be divided into multiple GOPs.

[0040] Here, the audio and video data acquisition module 310 can obtain audio and video data from the audio and video ring buffer or the local video cache. The cloud recording of the network surveillance camera can be divided into real-time cloud recording and non-real-time cloud recording. The real-time cloud recording function depends on the audio and video ring buffer in the software development kit (SDK, Software Development Kit). The network surveillance camera can write real-time audio and video data into the audio and video ring buffer to trigger the real-time cloud recording function. The non-real-time cloud recording function of the network surveillance camera depends on the local recording function, and the local recording function depends on the local video cache formed by the hard disk or SD card in the network surveillance camera. The network surveillance camera can write non-real-time audio and video data into the local video cache to trigger the non-real-time cloud recording function. In this way, the audio and video data upload system 300 can obtain real-time audio and video data via the audio and video ring buffer and / or obtain non-real-time audio and video data via the local video cache.

[0041] The method 200 for uploading audio and video data includes step S220: based on multiple GOPs, sending upload requests to the HTTP interface of the resource pool respectively to upload the multiple GOPs, where the upload requests include offset values.

[0042] As shown in Figure 3 , the audio and video data uploading system 300 can send an upload request to the HTTP interface of the resource pool 330 via the audio and video data sending module 320 to upload a GOP. Each upload request corresponds to one GOP. After the GOP is successfully uploaded, the resource pool 330 returns a status code 200 to indicate the success of the GOP upload request. The resource pool 330 can also return other status codes to indicate whether the upload was successful or whether there are other problems. The specific method for resource pool 330 to return status codes does not involve the technical improvements of the present invention and is not described in detail here.

[0043] Each upload request includes an offset value corresponding to a GOP, which is determined by the position of the starting point of the GOP in the entire data segment of the audio and video data.

[0044] Please refer to FIG. 4 , which shows a schematic diagram of multiple GOPs and offset values ​​provided according to some embodiments of the present invention.

[0045] As shown in Figure 4 , the upload request includes an offset value Ox, which can be determined based on the position of the GOP's starting point within the entire data segment of the audio and video data. Based on the offset value included in the upload request, the uploaded GOP is stored in the corresponding address in the resource pool. The audio and video data upload system 300 can determine the order of the GOPs based on the offset values ​​determined when the audio and video data is divided into multiple GOPs.

[0046] Specifically, in the embodiment shown in FIG4 , the audio and video data is segmented according to fixed time periods, with one GOP being 4 seconds. The audio and video data uploading system 300 successfully uploads GOP 1 (0-4s), GOP 3 (8-12s), GOP 4 (12-16s), GOP 5 (16-20s), GOP 7 (24-28s), GOP 10 (36-40s), GOP 11 (40-44s), GOP 12 (44-48s), and GOP 13 (48-52s) to the resource pool 330.

[0047] Furthermore, the starting point of the GOP with sequence number 1 is set to position 0 in the entire data segment of the audio and video data, and the offset value of the GOP with sequence number 1 is determined to be 0. Similarly, the starting point of the GOP with sequence number 2 can be positioned at position O1 in the entire data segment of the audio and video data, and the address segments 0 to O1 are the data segment size occupied by the GOP with sequence number 1. The starting point of the GOP with sequence number 3 can be positioned at position O2 in the entire data segment of the audio and video data, and the address segments 0 to O2 are the total data segment size occupied by the GOP with sequence number 1 and the GOP with sequence number 2. In the embodiment shown in FIG4 , the GOP with sequence number 2 fails to be uploaded, and therefore the GOP with sequence number 2 is not stored in the corresponding address segments O1 to O2 of the resource pool 330.

[0048] Please refer to FIG. 5A to FIG. 5B , which are schematic diagrams illustrating methods for uploading audio and video data according to some embodiments of the present invention.

[0049] As shown in Figure 5A, the audio and video data acquisition module 310 can collect real-time audio and video data from the audio and video ring buffer area, and generate multiple GOPs of real-time audio and video data one by one until the recording stops after a preset period and notifies the server of the audio and video data upload system 300 that the upload is completed.

[0050] Specifically, the audio and video data uploading system 300 can sequentially send upload requests to the HTTP interface of the resource pool 330 based on the multiple GOPs generated one by one to upload the multiple GOPs. The audio and video data acquisition module 310 continuously obtains a segment of GOP from the audio and video ring buffer and writes it to the sending buffer 321. The audio and video data sending module 320 continuously obtains a segment of GOP from the sending buffer 321 and uploads it to the resource pool 330 in the cloud. When the recording time reaches the threshold T, an end symbol is sent to notify the server of the audio and video data uploading system 300 that the upload has ended. Here, the sending buffer 321 is implemented as a first-in-first-out queue.

[0051] In this way, the uploading of real-time audio and video data adopts a single-channel slicing method, which can improve the success rate of uploading the entire audio and video data.

[0052] As shown in FIG5B , the audio and video data acquisition module 310 can collect non-real-time audio and video data from the local video buffer, and the non-real-time audio and video data is divided into multiple GOPs. During the operation of the network surveillance camera, there may be a period of time when there is no network, and the real-time cloud recording function cannot be realized. In order to ensure that the recording during this period is not lost, the network surveillance camera can record the time period when there is no network and the cloud recording is not uploaded, and upload the non-real-time audio and video data after the network is restored. The supplementary upload relies on the local video buffer function. By obtaining the local video (such as MP4 file) within the corresponding time period, the local non-real-time audio and video data is converted into a format and uploaded to the cloud resource pool 330, and after completion, the server of the audio and video data upload system 300 is notified that the upload is complete.

[0053] Specifically, the audio and video data upload system 300 can upload multiple GOPs in parallel by sending upload requests to the resource pool's HTTP interface. The audio and video data acquisition module 310 continuously retrieves a GOP from the local video buffer for the corresponding time period, converts the format, and writes it to the transmission buffer 321. The audio and video data transmission module 320 creates multiple threads to upload multiple GOPs in parallel until all the audio and video data from the local video buffer for the corresponding time period is retrieved. The module then sends a termination symbol to notify the server of the audio and video data upload system 300 that the upload has ended.

[0054] In this way, the uploading of non-real-time audio and video data adopts the method of multi-channel slice parallel uploading, which can improve the network throughput when uploading audio and video data.

[0055] In addition to determining that the audio and video data upload is complete based on the received terminator, the server of the audio and video data upload system 300 may also determine that the audio and video data upload is complete when detecting that no upload request is received within a preset period of time.

[0056] 2 , the method 200 for uploading audio and video data further includes step S230 : in response to the completion of uploading the audio and video data, splicing multiple GOPs according to the offset value to form a spliced ​​file.

[0057] After the audio and video data are uploaded, the system 300 for uploading the audio and video data sorts the multiple GOPs according to their offset values ​​and merges them into a spliced ​​file.

[0058] Please refer to FIG6 , which shows a schematic diagram of a spliced ​​file provided according to some embodiments of the present invention.

[0059] As shown in Figure 6, the server of the audio and video data uploading system 300 splices the GOP (0~4s) with sequence number 1, GOP (8~12s) with sequence number 3, GOP (12~16s) with sequence number 4, GOP (16~20s) with sequence number 5, GOP (24~28s) with sequence number 7, GOP (36~40s) with sequence number 11, GOP (40~44s) with sequence number 12, GOP (44~48s) and GOP (48~52s) with sequence number 13 in the embodiment shown in Figure 4 into a spliced ​​file, and the total video length of the spliced ​​file is 36s.

[0060] In summary, the method and system for uploading audio and video data provided by the present invention improve the ability of uploading audio and video data to resist network fluctuations by sending multiple upload requests via the HTTP interface.

[0061] Although the above methods are illustrated and described as a series of acts for simplicity of explanation, it is to be understood and appreciated that these methods are not limited by the order of the acts, as some acts may occur in a different order and / or concurrently with other acts from those illustrated and described herein or not illustrated and described herein but understandable to those skilled in the art according to one or more embodiments.

[0062] The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples and designs described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for uploading audio and video data, characterized in that: Includes steps: Acquire audio and video data, where the audio and video data is divided into a plurality of GOPs; Based on the multiple GOPs, sending multiple upload requests to the HTTP interface of the resource pool respectively to upload the multiple GOPs, wherein the upload requests include offset values; In response to the completion of uploading the audio and video data, the multiple GOPs are spliced ​​according to the offset value to form a spliced ​​file.

2. The uploading method according to claim 1, characterized in that: The completion of uploading the audio and video data includes: Determining that the uploading of the audio and video data is complete based on the fact that the upload request is not received within a preset time range; and / or Based on the received end character, it is determined that the uploading of the audio and video data is completed.

3. The uploading method according to claim 1, characterized in that: The audio and video data includes real-time audio and video data and non-real-time audio and video data, and the step of obtaining the audio and video data includes: Acquiring the real-time audio and video data via an audio and video ring buffer; and / or The non-real-time audio and video data is obtained via a local video recording buffer.

4. The uploading method according to claim 3, characterized in that: The real-time audio and video data generates the multiple GOPs one by one, and the step of sending multiple upload requests to the HTTP interface of the resource pool to upload the multiple GOPs comprises: Based on the multiple GOPs generated one by one, the upload requests are sent to the HTTP interface of the resource pool in sequence to upload the multiple GOPs.

5. The uploading method according to claim 3, characterized in that: The audio and video data is non-real-time audio and video data, and the step of sending multiple upload requests to the HTTP interface of the resource pool to upload the multiple GOPs includes: Based on the multiple GOPs, the upload requests are respectively and concurrently sent to the HTTP interface of the resource pool to upload the multiple GOPs.

6. The uploading method according to claim 1, characterized in that: The offset value is determined by the position of the starting point of the GOP in the entire data segment of the audio and video data.

7. A system for uploading audio and video data, characterized in that: include: a memory having computer instructions stored thereon; as well as A processor is connected to the memory and is configured to execute computer instructions stored in the memory to implement the method for uploading audio and video data according to any one of claims 1 to 6.

8. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by the processor, the method for uploading audio and video data according to any one of claims 1 to 6 is implemented.

Images