Private frame grouping method, device and computer equipment of code stream

Abstract

The application relates to a private frame grouping method, device and computer equipment of a code stream, wherein the private frame grouping method of the code stream comprises the following steps: determining the frame category of a to-be-processed private frame; determining the target grouping strategy corresponding to the to-be-processed private frame according to the frame category of the to-be-processed private frame; wherein the target grouping strategy is used for defining the insertion rule of the to-be-processed private frame in the code stream through at least one insertion parameter; and grouping the to-be-processed private frame into the code stream based on the target grouping strategy. Through the application, the problem that there is no private frame insertion mechanism which can be flexibly set and thus the code rate cannot be controlled is solved, the private frame insertion mechanism which can be flexibly set is provided, and the code rate can be adjusted so as to control the code rate.

Description

Technical Field

[0001] This application relates to the field of security monitoring, and in particular to methods, apparatus and computer equipment for private frame assembly of bitstreams. Background Technology

[0002] In current video surveillance and intelligent analysis systems, to achieve value-added functions such as device management and intelligent services, the bitstream, while adhering to video coding standards such as H.264 / H.265, audio coding standards such as AAC / G.711, and encapsulation specifications such as PS / TS, often inserts proprietary frames (also known as intelligent frames) to carry vendor-defined extended information. These proprietary frames, as a supplement to standard audio and video data, are used to transmit information such as device information, service configuration parameters, and intelligent analysis results.

[0003] However, with the popularization of front-end intelligent applications and the deepening of application scenarios, various intelligent analysis services often run in parallel, and each video stream needs to carry private frames from different services. This leads to a sharp increase in the generation frequency and data volume of private frames, significantly increasing their proportion in the overall stream, which in turn leads to an increase in bitrate and consumes a large amount of network bandwidth and storage space.

[0004] There is currently no effective solution to the problem that the relevant technologies lack a flexible private frame insertion mechanism, which makes it impossible to control the bitrate. Summary of the Invention

[0005] This embodiment provides a method, apparatus, and computer device for private frame assembly of a bitstream to solve the problem in related technologies that lack a flexibly configurable private frame insertion mechanism, thus making it impossible to control the bitrate.

[0006] Firstly, this embodiment provides a method for framing private frames of a bitstream, including:

[0007] Determine the frame category of the private frame to be processed;

[0008] Based on the frame category of the private frame to be processed, a target framing strategy corresponding to the private frame to be processed is determined; wherein, the target framing strategy is used to define the insertion rules of the private frame to be processed in the bitstream through at least one insertion parameter;

[0009] Based on the target framing strategy, the private frames to be processed are framed into the bitstream.

[0010] In some embodiments, determining the frame category of the private frame to be processed includes:

[0011] The private frames to be processed are classified based on functional and spatiotemporal dimensions to obtain the frame category of the private frames to be processed.

[0012] The functional dimension is associated with the rendering characteristics of the private frame to be processed; the spatiotemporal dimension is associated with the spatiotemporal dependency relationship of the private frame to be processed.

[0013] In some embodiments, the insertion parameters include one or more combinations of the insertion position, insertion frequency, and insertion time of the private frame to be processed.

[0014] In some of these embodiments, the frame type of the private frame to be processed is a metadata frame;

[0015] The step of framing the private frames to be processed into the bitstream based on the target framing strategy includes:

[0016] When the frame data of the metadata frame is fixed information, the private frame to be processed is framed into the bitstream according to the preset insertion mode;

[0017] If the frame data of the metadata frame is change information, the private frame to be processed is inserted after the video frame corresponding to the time when the change information is generated.

[0018] In some embodiments, the method further includes:

[0019] When the preset insertion mode is the first insertion mode, in response to the establishment of the video stream session, the private frame to be processed is inserted after the video frame corresponding to the time when the video stream session is established;

[0020] When the preset insertion mode is the second insertion mode, the private frame to be processed is inserted into the bitstream at a period of one or more image groups.

[0021] In some of these embodiments, the frame category of the private frame to be processed is a style frame;

[0022] The step of framing the private frames to be processed into the bitstream based on the target framing strategy includes:

[0023] According to the preset control mode, the private frames to be processed are framed into the bitstream; the preset control mode corresponds to the style description object of the style frame.

[0024] In some embodiments, the method further includes:

[0025] When the style description object is a predefined type and / or style category of the corresponding content frame, the control mode is a general control mode;

[0026] In the general control mode, in response to the activation of an associated service, the private frame to be processed is inserted after the video frame corresponding to the activation time of the associated service; or,

[0027] In the general control mode, the private frame to be processed is inserted into the bitstream at a period of one or more image groups.

[0028] In some embodiments, the method further includes:

[0029] When the style description object is the content index of the corresponding content frame, the control mode is a fine-grained control mode;

[0030] In the fine control mode, in response to a change in the content index, the private frame to be processed is inserted after the video frame corresponding to the moment the content index changes.

[0031] In some of these embodiments, the frame type of the private frame to be processed is a continuous frame;

[0032] The step of framing the private frames to be processed into the bitstream based on the target framing strategy includes:

[0033] In response to a configuration operation of the intelligent service associated with the persistent frame, the private frame to be processed is inserted after the video frame corresponding to the execution time of the configuration operation; or,

[0034] In response to the configuration operation of the intelligent service associated with the continuous frame, the private frame to be processed is inserted into the bitstream periodically, with one or more image groups as the cycle.

[0035] In some embodiments, the frame type of the private frame to be processed is a transient frame; the step of framing the private frame to be processed into the bitstream based on the target framing strategy includes:

[0036] In response to the generation of data content for the instantaneous frame, the instantaneous frame is inserted after the video frame corresponding to the time of data content generation.

[0037] In some embodiments, the frame category of the private frame to be processed is an additional frame; the target framing strategy is determined based on the associated object of the additional frame; wherein the associated object includes continuous frames and / or instantaneous frames.

[0038] In some embodiments, the step of framing the private frames to be processed into the bitstream based on the target framing strategy includes:

[0039] If the associated object contains only a continuous frame, and the additional frame is inserted into the bitstream for the first time, then in response to the insertion of the continuous frame into the bitstream, the additional frame is inserted after the continuous frame;

[0040] If the associated object contains only continuous frames and the additional frame is not the first time it is inserted into the bitstream, then in response to a change in the data content of the additional frame, the additional frame is inserted after the video frame corresponding to the time when the data content changes.

[0041] If the associated object contains only a momentary frame, then in response to the insertion of the momentary frame into the bitstream, the additional frame is inserted after the momentary frame;

[0042] If the associated object contains both continuous frames and instantaneous frames, then in response to the generation of the data content of the additional frame, the additional frame is inserted after the video frame corresponding to the time of data content generation.

[0043] In some embodiments, before framing the private frames to be processed into the bitstream based on the target framing strategy, the method further includes:

[0044] According to the associated service and / or the frame category of the private frame to be processed, the frame type of the private frame to be processed is marked so that the player used to parse the bitstream can identify the frame type and perform corresponding decoding operations based on the identification result.

[0045] Secondly, this embodiment provides a private frame assembly device for a bitstream, comprising:

[0046] The classification module is used to determine the frame category of the private frame to be processed;

[0047] The matching module is used to determine the target framing strategy corresponding to the private frame to be processed based on the frame category of the private frame to be processed; wherein, the target framing strategy is used to define the insertion rules of the private frame to be processed in the bitstream through at least one insertion parameter;

[0048] The framing module is used to frame the private frames to be processed into the bitstream based on the target framing strategy.

[0049] Thirdly, this embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the private frame framing method for the bitstream described in the first aspect above.

[0050] Fourthly, this embodiment provides a storage medium storing a computer program that, when executed by a processor, implements the private frame framing method for the bitstream described in the first aspect above.

[0051] Compared with related technologies, the private frame assembly method, apparatus, and computer device for bitstream provided in this embodiment determine the frame category of the private frame to be processed; determine the target framing strategy corresponding to the private frame to be processed based on the frame category; wherein, the target framing strategy is used to define the insertion rules of the private frame to be processed in the bitstream through at least one insertion parameter; and frame the private frame to be processed into the bitstream based on the target framing strategy, thereby solving the problem of lacking a flexibly configurable private frame insertion mechanism, which makes it impossible to control the bitrate, and realizing the provision of a flexibly configurable private frame insertion mechanism, so that the bitrate is adjustable and can be controlled.

[0052] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0053] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0054] Figure 1 This is a hardware structure block diagram of a terminal device for a private frame grouping method for a bitstream provided in an embodiment of this application;

[0055] Figure 2 This is a flowchart of a private frame assembly method for a bitstream provided in an embodiment of this application;

[0056] Figure 3 This is a schematic diagram illustrating an example of a people counting service provided in an embodiment of this application;

[0057] Figure 4 This is a framing example diagram of the first insertion mode provided in an embodiment of this application;

[0058] Figure 5 This is a framing example diagram of a balanced insertion mode provided in an embodiment of this application;

[0059] Figure 6 This is a framing example diagram of the extreme compression insertion mode provided in an embodiment of this application;

[0060] Figure 7 This is a framing example diagram of the extreme compression insertion mode provided in another embodiment of this application;

[0061] Figure 8This is a structural block diagram of a private frame assembly device for a bitstream provided in an embodiment of this application.

[0062] In the diagram: 102, processor; 104, memory; 106, transmission device; 108, input / output device; 10, classification module; 20, matching module; 30, framing module. Detailed Implementation

[0063] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0064] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0065] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the private frame assembly method of the bitstream in this embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0066] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the private frame assembly method of the bitstream in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0067] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0068] This embodiment provides a method for private frame assembly of a bitstream. Figure 2 This is a flowchart of the private frame assembly method for the bitstream in this embodiment, as follows: Figure 2 As shown, the process includes the following steps:

[0069] Step S210: Determine the frame category of the private frame to be processed;

[0070] Step S220: Determine the target framing strategy corresponding to the private frame to be processed based on the frame category of the private frame to be processed; wherein, the target framing strategy is used to define the insertion rules of the private frame to be processed in the bitstream through at least one insertion parameter.

[0071] Step S230: Based on the target framing strategy, the private frames to be processed are framed into the bitstream.

[0072] Specifically, the private frames to be processed are classified to determine their frame category. Frame categories include metadata frames, style frames, persistent frames, transient frames, and additional frames, as defined below:

[0073] 1) Metadata frame: Unrelated to decoding and rendering, and usually unrelated to business, it contains data such as manufacturer information, device information, invisible watermarks, sensor information, and sensor data.

[0074] 2) Style Frame: Used to describe the rendering effect of the content frame. Its data types include, but are not limited to, wireframe color, thickness, dashed and solid lines and line endpoint styles, text font, size and color, and animation effects such as blinking, transition, color change and bolding.

[0075] 3) Continuous frames: Data frames that are continuously displayed after decoding. They may contain visible watermarks that are unrelated to intelligent services, or configuration rules, intelligent algorithm analysis areas, etc. that are related to intelligent services.

[0076] 4) Instant frame: A data frame that is only displayed once after decoding. It usually includes data such as the location of the target detected by the algorithm and related target information. It should be noted that the player can use smoothing strategies such as flicker removal to make its visual effect last for multiple frames (the display position may change).

[0077] 5) Additional frames: Used to supplement the description of continuous frames or instantaneous frames, such as statistical analysis data bound to a certain configuration rule in a continuous frame, the association relationship of multiple detection targets bound to an instantaneous frame, and the event triggering relationship of a certain configuration rule in a continuous frame and multiple detection targets in an instantaneous frame.

[0078] Among them, style frames, continuous frames, instantaneous frames, and additional frames are collectively referred to as visual frames, while continuous frames, instantaneous frames, and additional frames are collectively referred to as content frames.

[0079] like Figure 3 As shown, taking the headcount statistics business as an example, the style frame structure includes application identifiers (such as...). Figure 3The statistical template includes the following components: "People Count - 1" (frame type identifier, frame ID, region style definition (such as border color, width and type), text style definition (such as font color, font size and font), etc.; the continuous frame structure includes the application identifier, frame type identifier, frame ID, rule list (such as rule ID and name, area where the rule applies, detection target type and format template for statistical result output), etc.; the supplementary frame structure includes the application identifier, frame type identifier, frame ID, bound continuous frame ID and / or instantaneous frame ID, associated rule ID, specific statistical data (such as the number of people counted in the current region is 5), etc.

[0080] Furthermore, based on the frame category to which the private frame to be processed belongs, a target framing strategy corresponding to the private frame to be processed is determined. This target framing strategy is used to define the insertion rules of the private frame to be processed in the bitstream through at least one insertion parameter.

[0081] It should be noted that different application scenarios (such as different manufacturers or systems) may have their own independent frame structure definitions, which typically include structural fields such as frame header, frame trailer, frame length, and check type. This embodiment mainly describes the key information in the framing process and the parameters related to the insertion behavior of private frames in the bitstream. Specific insertion parameters include insertion position, insertion frequency, and insertion time. In addition, the insertion of the private frame to be processed refers to the insertion of the frame relative to the video frame (including I-frames and P-frames) position in the original audio and video bitstream according to the corresponding target framing strategy.

[0082] Then, according to the insertion rules specified by the target framing strategy, the private frames to be processed are framed into the bitstream. For example, if the frame type of the private frame to be processed is a metadata frame, the framing operation is performed according to the insertion mode corresponding to the data type of the metadata frame; if the frame type of the private frame to be processed is a style frame, the private frame to be processed is inserted into the corresponding position in the bitstream according to the preset control mode corresponding to its style description object. It is understood that the insertion rules defined by different target framing strategies can be flexibly configured based on the characteristics of the private frame to be processed, such as frame type, data type, business scenario, and network status, to meet different application requirements such as bitrate control. The specific implementation method is not limited to the above example.

[0083] With the popularization of front-end intelligent applications and the deepening of application scenarios, various intelligent analysis services often run in parallel, and each video stream needs to carry private frames from different services. This leads to a sharp increase in the generation frequency and data volume of private frames, significantly increasing their proportion in the overall stream, which in turn leads to an increase in bitrate and consumes a large amount of network bandwidth and storage space.

[0084] Compared to existing technologies, this application determines the frame category of the private frame to be processed; based on the frame category, it determines the target framing strategy corresponding to the private frame to be processed; wherein, the target framing strategy is used to define the insertion rules of the private frame to be processed in the bitstream through at least one insertion parameter; and based on the target framing strategy, the private frame to be processed is framed into the bitstream. Based on this, by finely classifying private frames and adopting differentiated framing strategies according to different categories, redundant information between frames is effectively removed, and inter-frame coupling is reduced. This allows for dynamic adjustment of the private frame insertion strategy according to actual service needs, network conditions, and other factors, solving the problem of lacking a flexibly configurable private frame insertion mechanism and being unable to control the bitrate. It provides a flexibly configurable private frame insertion mechanism, making the bitrate adjustable for control.

[0085] In some embodiments, determining the frame category of the private frame to be processed includes the following steps:

[0086] Based on the functional and spatiotemporal dimensions, the private frames to be processed are classified to obtain the frame category of the private frames to be processed.

[0087] Among them, the functional dimension is related to the rendering characteristics of the private frame to be processed; the spatiotemporal dimension is related to the spatiotemporal dependency relationship of the private frame to be processed.

[0088] In this embodiment, the private frames to be processed are classified in two dimensions. Specifically, in terms of functionality, the classification is related to the rendering characteristics of the private frames to be processed. Based on this, the private frames are divided into visual frames that need to be rendered and metadata frames that do not need to be rendered. Furthermore, according to the difference in rendering purpose, visual frames can be further divided into content frames that carry specific display content and style frames that define rendering styles.

[0089] In the spatiotemporal dimension, the classification is related to the spatiotemporal dependencies of the private frames to be processed. Among them, according to the independence of data frames, content frames can be divided into independent frames and additional frames, and according to the length of rendering time, independent frames can be divided into continuous frames and instantaneous frames.

[0090] In this embodiment, the private frames to be processed are classified based on the functional dimension and the spatiotemporal dimension to obtain the frame category of the private frames to be processed, so as to achieve accurate classification of private frames.

[0091] In some embodiments, the insertion parameters include one or more combinations of the insertion position, insertion frequency, and insertion time of the private frame to be processed.

[0092] In this embodiment, the target framing strategy defines the insertion rules of the private frame to be processed in the bitstream through at least one insertion parameter. The insertion parameters include the insertion position, insertion frequency, and insertion time of the private frame to be processed. Specifically, the insertion operation of the private frame to be processed refers to inserting it relative to the position of the video frame (including I-frames and P-frames) in the original audio and video bitstream according to the target framing strategy.

[0093] Understandably, metadata frames contain data unrelated to decoding and rendering, such as manufacturer information, device information, invisible watermarks, sensor information, and sensor-collected data. Their insertion position, frequency, and timing in the bitstream are less restricted, allowing for flexible framing strategies based on goals such as reducing the number of insertions. Visual frames, on the other hand, are related to rendering and require strict consideration of their insertion position, frequency, and timing to ensure rendering accuracy and temporal consistency.

[0094] This embodiment allows for the definition of insertion rules for private frames to be processed based on insertion parameters such as insertion position, insertion frequency, and insertion time. This enables the flexible adoption of different insertion strategies according to the characteristics of different private frames and application requirements.

[0095] In some of these embodiments, the frame type of the private frame to be processed is a metadata frame;

[0096] Based on the target framing strategy, the private frames to be processed are framed into the bitstream, including the following steps:

[0097] When the frame data of the metadata frame is fixed information, the private frames to be processed are framed into the bitstream according to the preset insertion mode;

[0098] If the frame data of the metadata frame contains change information, the private frame to be processed will be inserted after the video frame corresponding to the time when the change information was generated.

[0099] Specifically, when the frame type of the private frame to be processed is a metadata frame, the frame data type of the metadata frame is determined. The frame data type includes fixed information and variable information.

[0100] If the frame data of the metadata frame is fixed information (such as manufacturer information or device information), the metadata frame is assembled into the bitstream according to the preset insertion mode. The preset insertion mode can be configured to insert periodically or at key frames to achieve efficient data synchronization and redundancy control.

[0101] If the frame data of the metadata frame is change information (such as sensor-collected data), the metadata frame is inserted after the video frame corresponding to the time when the change information is generated, so as to ensure that dynamic data such as sensor readings are aligned with the relevant video content in time, and to meet the needs of subsequent data association-based analysis.

[0102] In this embodiment, when the frame type of the private frame to be processed is a metadata frame, the corresponding frame assembly operation is performed according to the frame data type of the metadata frame, thereby realizing the flexible insertion of different types of metadata frames.

[0103] In some embodiments, the private frame framing method for the above-mentioned bitstream further includes the following steps:

[0104] When the preset insertion mode is the first insertion mode, in response to the establishment of the video stream session, the private frame to be processed is inserted after the video frame corresponding to the time when the video stream session is established.

[0105] When the preset insertion mode is the second insertion mode, the private frames to be processed are inserted into the bitstream in a cycle of one or more image groups.

[0106] Specifically, when the frame data of the metadata frame is fixed information, the preset insertion modes include a first insertion mode for implementing extreme compression insertion and a second insertion mode for implementing balanced insertion.

[0107] If the first insertion mode is used, the private frame to be processed is inserted after the video frame corresponding to the video stream session establishment time only when the video stream session is established. The video stream session establishment time refers to the moment when the device receives a pull request or actively initiates a push stream to a receiving end according to the configuration. The pull request can come from various terminals or systems, including but not limited to the device's own web interface, connected network video recorders (NVRs), intelligent video surveillance systems (IVSS), and various service platforms.

[0108] For example, such as Figure 4 As shown, without any services being enabled, in response to a pull stream request, a metadata frame is inserted after the video frame corresponding to the moment the pull stream link is established. Figure 4 Between I-frames and P-frames.

[0109] If the second insertion mode is adopted, a Group of Pictures (GOP) is used as the basic unit, and the private frame to be processed is periodically inserted into a specified position of each GOP in the bitstream, or configured to be inserted once every several GOP periods.

[0110] In this embodiment, when the preset insertion mode is the first insertion mode, in response to the establishment of a video stream session, the private frame to be processed is inserted after the video frame corresponding to the establishment time of the video stream session. When the preset insertion mode is the second insertion mode, the private frame to be processed is inserted into the bitstream with one or more image groups as a period, thereby realizing flexible insertion of metadata frames based on different application requirements.

[0111] In some of these embodiments, the frame type of the private frame to be processed is a style frame;

[0112] Based on the target framing strategy, the private frames to be processed are framed into the bitstream, including the following steps:

[0113] According to the preset control mode, the private frames to be processed are framed into the bitstream; the preset control mode corresponds to the style description object of the style frame.

[0114] Specifically, when the frame category of the private frame to be processed is a style frame, the private frame to be processed is assembled into frames according to the preset control mode corresponding to the style description object of the style frame. Among them, the style frame is used to describe the rendering effect of the content frame. For the style frame, a unified style description specification is defined, and its style description object includes the predefined type of the corresponding content frame (such as lines, polygons and text), content index (such as ID number) or style category (such as the class selector).

[0115] It should be noted that the above-mentioned mechanism of separating content from style makes style description more flexible and detailed, and enables dynamic adjustment of styles. This avoids situations such as inflexible expansion due to the default styles of built-in business functions in the player, and differences in default styles between different players, thus ensuring cross-system consistency.

[0116] In this embodiment, different control modes are used to frame the style frame according to the style description object, so as to realize the flexible insertion of style frames.

[0117] In some embodiments, the private frame framing method for the above-mentioned bitstream further includes the following steps:

[0118] When the style description object is a predefined type and / or style category of the corresponding content frame, the control mode is the general control mode;

[0119] In general control mode, in response to the activation of associated services, the private frame to be processed is inserted after the video frame corresponding to the activation time of the associated service; or,

[0120] In general control mode, private frames to be processed are inserted into the bitstream in cycles of one or more image groups.

[0121] In this embodiment, the style description object of the style frame is the predefined type and / or style category of the corresponding content frame. Such style frames can usually be fully determined during the development stage of related services, and their control mode adopts a general control mode. Under the general control mode, extreme compression insertion or balanced insertion strategies can be selected.

[0122] In one approach, extreme compression insertion is used only when the associated service is running, inserting the private frame to be processed after the video frame corresponding to the start time of the associated service, thereby achieving extreme compression of the bitstream data. In another approach, balanced insertion is used, inserting a pattern frame for the first time when the associated service starts, and then periodically inserting the private frame to be processed into the bitstream using a Group of Pictures (GOP) as the basic unit. Alternatively, it can be configured to insert every few GOPs, thus effectively reducing the bitrate while also considering the complexity of player decoding and other video processing.

[0123] It should be noted that for repeatedly inserted frames, they are typically inserted at the end of a Group of Pictures (GOP). If inserted at the beginning or middle of a GOP, when the private frame data changes within the GOP period and immediate insertion is required, the same type of private frame will be inserted multiple times within the same GOP, resulting in information redundancy. By uniformly inserting frames at the end of the GOP, frame data redundancy can be effectively avoided.

[0124] Furthermore, by adding repeated insertion to the extreme compression insertion mode, since the insertion intervals of metadata frames, style frames, and persistence frames can be different, only one of them may appear in the same GOP, or all of them may appear simultaneously (e.g., Figure 5 (As shown).

[0125] In this embodiment, when the style description object is a predefined type and / or style category of the corresponding content frame, the control mode is a general control mode. In the general control mode, extreme compression insertion or balanced insertion strategy is selected to achieve flexible framing of style frames according to the category of style description object.

[0126] In some embodiments, the private frame framing method for the above-mentioned bitstream further includes the following steps:

[0127] When the style description object is the content index of the corresponding content frame, the control mode is fine-grained control mode;

[0128] In fine control mode, in response to a change in the content index, the private frame to be processed is inserted after the video frame corresponding to the moment the content index changes.

[0129] In this embodiment, the style description object of the style frame is the content index of the corresponding content frame. This type of style frame will change dynamically during business operation to achieve more complex animation effects, and its control mode adopts fine control mode.

[0130] In fine-grained control mode, the style frame is first inserted after the corresponding video frame when the associated service starts running to complete initial synchronization. Subsequently, during service operation, whenever the content index changes, the current style frame is inserted after the video frame corresponding to the moment the content index changes.

[0131] In this embodiment, when the style description object is the content index of the corresponding content frame, a fine-grained control mode is used for frame grouping, thereby effectively supporting dynamic style updates and complex animation effects in the video stream, while ensuring the consistency of rendering effects across systems.

[0132] In some of these embodiments, the frame type of the private frame to be processed is a continuous frame;

[0133] Based on the target framing strategy, the private frames to be processed are framed into the bitstream, including the following steps:

[0134] In response to configuration operations of intelligent services associated with continuous frames, the private frame to be processed is inserted after the video frame corresponding to the execution time of the configuration operation; or,

[0135] In response to configuration operations of intelligent services associated with continuous frames, private frames to be processed are inserted into the bitstream periodically, with one or more image groups as the cycle.

[0136] Specifically, when the frame type of the private frame to be processed is a persistent frame, it is determined whether the persistent frame is related to intelligent services. If the persistent frame is not related to intelligent services, for example, if the frame data type of the persistent frame is a visible watermark, then the framing strategy for the persistent frame is the same as the processing method for the metadata frame mentioned above, and will not be repeated here. If the persistent frame is related to intelligent services, for example, if the frame data type of the persistent frame is intelligent service configuration rules, intelligent algorithm analysis area, etc., then extreme compression insertion or balanced insertion strategy can be selected.

[0137] If extreme compression insertion is used, the continuous frame is inserted after the video frame corresponding to the configuration operation only when the service configuration changes. If balanced insertion is used, the continuous frame is inserted once when the service configuration changes, and then the continuous frame is periodically inserted into the bitstream with a GOP as the basic unit, or it is configured to be inserted once every several GOPs.

[0138] It should be noted that for persistent frames, when the service configuration changes, the newly inserted persistent frame marks the end of the previous persistent frame. If the configuration is cleared, an empty persistent frame must be inserted as an end marker to indicate the end of the current persistent frame.

[0139] In this embodiment, when the frame type of the private frame to be processed is a continuous frame, in response to the configuration operation of the intelligent service associated with the continuous frame, the private frame to be processed is inserted after the video frame corresponding to the execution time of the configuration operation, or in response to the configuration operation of the intelligent service associated with the continuous frame, the private frame to be processed is inserted into the bitstream with one or more image groups as a period, thereby realizing flexible frame assembly of continuous frames to meet the needs of different business scenarios.

[0140] In some embodiments, the frame type of the private frame to be processed is a transient frame; based on the target framing strategy, the private frame to be processed is framed into the bitstream, including the following steps:

[0141] In response to the generation of instantaneous frame data content, the instantaneous frame is inserted after the video frame corresponding to the time of data content generation.

[0142] Specifically, if the private frame to be processed is identified as a transient frame, its framing operation is triggered by a specific event. Transient frames typically carry highly time-sensitive data content such as the target location detected by the algorithm and target information. The framing process is initiated when such data content is detected.

[0143] During execution, when the data content of the instant frame is generated, the instant frame is inserted after the video frame corresponding to the time of data content generation to ensure that the dynamic information carried by the instant frame can maintain strict time alignment with the relevant visual scene in the bitstream.

[0144] In this embodiment, when the frame type of the private frame to be processed is a transient frame, in response to the generation of the transient frame's data content, the transient frame is inserted after the video frame corresponding to the time when the data content is generated, thereby realizing flexible framing of transient frames.

[0145] In some embodiments, the frame category of the private frame to be processed is an additional frame; the target framing strategy is determined based on the associated object of the additional frame; wherein the associated object includes continuous frames and / or instantaneous frames.

[0146] It should be noted that supplementary frames, as a description of existing visual frames, typically derive their data from algorithmic analysis, including statistical data such as quantity, traffic, speed, and density. This data needs to be bound to specific objects within the continuous or instantaneous frame content; the framing strategy for supplementary frames depends on their binding relationship with associated objects. Therefore, if a private frame to be processed is identified as a supplementary frame, the associated objects of the supplementary frame must be determined in advance.

[0147] Among them, the associated objects of the additional frames include continuous frames (such as a smart business configuration rule or analysis area), instantaneous frames (such as a detection target), and combinations of continuous frames and instantaneous frames (such as the event triggering relationship of multiple detection targets under a certain rule).

[0148] In this embodiment, when the frame category of the private frame to be processed is an additional frame, the target framing strategy is determined based on the associated object of the additional frame. Based on the characteristics of the associated object, a differentiated framing strategy is formulated, which not only ensures the timely synchronization of key data, but also avoids data redundancy.

[0149] In some of these embodiments, based on a target framing strategy, the private frames to be processed are framed into the bitstream, including the following steps:

[0150] If the associated object contains only a continuous frame, and the additional frame is inserted into the bitstream for the first time, then in response to the continuous frame insertion bitstream, the additional frame is inserted after the continuous frame;

[0151] If the associated object contains only continuous frames and the additional frame is not the first time it is inserted into the bitstream, then in response to a change in the data content of the additional frame, the additional frame will be inserted after the video frame corresponding to the moment when the data content changes.

[0152] If the associated object contains only instantaneous frames, then in response to the instantaneous frame insertion stream, an additional frame is inserted after the instantaneous frame;

[0153] If the associated object contains both continuous frames and instantaneous frames, then in response to the generation of the additional frame's data content, the additional frame is inserted after the video frame corresponding to the time when the data content is generated.

[0154] Specifically, when the associated object of the additional frame only contains continuous frames, it is pre-determined whether the additional frame is being inserted into the bitstream for the first time. If it is being inserted into the bitstream for the first time, since the business analysis data of the additional frame has not yet been generated, an empty additional frame is inserted into the bitstream as an initial value; if it is not being inserted into the bitstream for the first time, that is, the data of the additional frame has changed since the last insertion, the current additional frame is inserted after the video frame corresponding to the moment when the data content changed, to ensure that the analysis results are kept synchronized with the video content.

[0155] When the associated object of the additional frame contains only instantaneous frames, the additional frame is synchronously inserted into the corresponding position after the instantaneous frame as supplementary explanatory data when the associated instantaneous frame is detected in the bitstream.

[0156] When the associated object of the additional frame contains both continuous frames and instantaneous frames, such as when the detection target triggers an event generated by a certain business rule (such as a target tripping event), the additional frame is inserted after the video frame corresponding to the time when the data content of the additional frame is generated, so as to accurately record the associated information in complex business scenarios.

[0157] In this embodiment, a differentiated framing strategy is adopted based on different associated objects of the additional frame, thereby achieving flexible framing of the additional frame.

[0158] In some embodiments, before framing the private frames to be processed into the bitstream based on the target framing strategy, the private frame framing method of the bitstream further includes the following steps:

[0159] Based on the associated service and / or frame category of the private frame to be processed, the frame type of the private frame to be processed is marked so that the player used to parse the bitstream can identify the frame type and perform corresponding decoding operations based on the identification results.

[0160] Specifically, based on the specific business associated with the private frame to be processed and its frame category, each private frame is precisely labeled with a frame type, including metadata frames, style frames, continuous frames, instantaneous frames, and additional frames.

[0161] For visual frames such as style frames, persistent frames, transient frames, and additional frames, an application ID is assigned to each frame, thus grouping them according to application category. On open platform devices, each independent app is considered an application; on non-open platform devices, each intelligent service category is considered an application. This grouping mechanism ensures that style frames and additional frames within each group do not interfere with each other, and each group can employ different insertion strategies. Based on this, the player can control whether to decode and render visual frames for a specific application from an application service perspective.

[0162] Furthermore, since both continuous and instantaneous frames are closely related to specific business functions, they are tagged with types according to business needs (such as rules configured for intelligent services, analysis areas of intelligent algorithms, etc.), i.e., grouped by business function. This type tagging mechanism allows the player to control whether to decode and render a visual frame with fine granularity based on the function type. In addition, IDs are tagged according to business needs (such as multiple rules configured for the same intelligent service), and the IDs facilitate the association of additional frames.

[0163] In this embodiment, the frame type of the private frame to be processed is marked according to the associated service and / or frame category, so that the player used to parse the bitstream can identify the frame type and perform corresponding decoding operations based on the identification results. This allows visual frames of different services to adopt differentiated insertion strategies and rendering styles, while also supporting the player to flexibly enable or disable the rendering and display of specific service categories or even specific functional units according to actual needs.

[0164] The present embodiment will be described and explained below through specific examples.

[0165] Figure 6 This is a frame assembly example diagram of the extreme compression insertion mode in this embodiment, as shown below. Figure 6 As shown, the private frame assembly method for this bitstream includes the following steps:

[0166] In response to a pull request, a metadata frame is inserted after the video frame corresponding to the moment the pull link is established. Figure 6 The position between the I-frame and the P-frame is shown. When the application service is started, the style frame is inserted into the bitstream according to the preset control mode corresponding to the style description object of the style frame.

[0167] During the business configuration phase, corresponding continuous frames are dynamically inserted based on the configuration content. Specifically, when configuring rules, the corresponding continuous frames are inserted after the video frames in the bitstream corresponding to the configuration operation; when configuring the analysis area of ​​the algorithm, the corresponding continuous frames are inserted after the corresponding video frames in the bitstream.

[0168] Furthermore, instantaneous frames carrying the location information of the detected target are inserted into the video frame at the corresponding moment in the bitstream, and subsequent instantaneous frames of the detected target are continuously inserted during service operation. When the configuration rules change, updated continuous frames are inserted into the bitstream to reflect the latest configuration.

[0169] Until the application service stops, for the previously inserted persistent frames, insert the corresponding empty persistent frames as a marker that the rendering has ended, thus completing the closed-loop management of the service lifecycle.

[0170] Figure 7 This is a frame assembly example diagram of the extreme compression insertion mode in this embodiment, as shown below. Figure 7 As shown, the private frame assembly method for this bitstream includes the following steps:

[0171] In response to a pull request, a metadata frame is inserted after the video frame corresponding to the moment the pull link is established. Figure 7 The position between the I-frame and the P-frame is shown. When the application service is started, the style frame is inserted into the bitstream according to the preset control mode corresponding to the style description object of the style frame.

[0172] During the business configuration phase, corresponding continuous frames are dynamically inserted based on the configuration content. Specifically, when configuring rules, the corresponding continuous frames are inserted after the video frames in the bitstream corresponding to the configured operation; when configuring the analysis area of ​​the algorithm, the corresponding continuous frames are inserted after the corresponding video frames in the bitstream. Subsequently, additional frames containing the analysis data bound to the configuration rules in the continuous frames are inserted into the bitstream.

[0173] Furthermore, instantaneous frames carrying the location information of the detected target are inserted into the video frame at the corresponding moment in the bitstream. During service operation, instantaneous frames of subsequent detected targets are continuously inserted, and additional frames bound to the detected targets in the instantaneous frames are inserted into the bitstream. If an alarm event is triggered during service operation, the configuration rules in the continuous frames and the additional frames of the detected targets in the instantaneous frames will be simultaneously inserted into the bitstream.

[0174] This embodiment also provides a private frame assembly device for a bitstream, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below can refer to combinations of software and / or hardware that implement a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0175] Figure 8 This is a structural block diagram of the private frame assembly device for the bitstream in this embodiment, as shown below. Figure 8 As shown, the device includes:

[0176] Classification module 10 is used to determine the frame category of the private frame to be processed;

[0177] The matching module 20 is used to determine the target framing strategy corresponding to the private frame to be processed based on the frame category of the private frame to be processed; wherein, the target framing strategy is used to define the insertion rules of the private frame to be processed in the bitstream through at least one insertion parameter;

[0178] The framing module 30 is used to frame the private frames to be processed into the bitstream based on the target framing strategy.

[0179] The apparatus provided in this embodiment determines the frame category of the private frame to be processed; based on the frame category of the private frame to be processed, it determines the target framing strategy corresponding to the private frame to be processed; wherein, the target framing strategy is used to define the insertion rules of the private frame to be processed in the bitstream through at least one insertion parameter; based on the target framing strategy, the private frame to be processed is framed into the bitstream, which solves the problem of lacking a flexibly configurable private frame insertion mechanism, thus making it impossible to control the bitrate, and realizes the provision of a flexibly configurable private frame insertion mechanism, so that the bitrate is adjustable in order to control the bitrate.

[0180] In some embodiments, the classification module 10 is further configured to classify the private frame to be processed based on the functional dimension and the spatiotemporal dimension to obtain the frame category of the private frame to be processed; wherein, the functional dimension is associated with the rendering characteristics of the private frame to be processed; and the spatiotemporal dimension is associated with the spatiotemporal dependency relationship of the private frame to be processed.

[0181] In some embodiments, the frame type of the private frame to be processed is a metadata frame; the framing module 30 is further configured to frame the private frame to be processed into the bitstream according to a preset insertion mode when the frame data of the metadata frame is fixed information; and to insert the private frame to be processed after the video frame corresponding to the time when the frame data of the metadata frame is changing information.

[0182] In some embodiments, the framing module 30 is further configured to, when the preset insertion mode is the first insertion mode, insert the private frame to be processed after the video frame corresponding to the establishment time of the video stream session in response to the establishment of the video stream session; and when the preset insertion mode is the second insertion mode, insert the private frame to be processed into the bitstream at a period of one or more image groups.

[0183] In some embodiments, the frame category of the private frame to be processed is a style frame; the framing module 30 is also used to frame the private frame to be processed into the bitstream according to a preset control mode; the preset control mode corresponds to the style description object of the style frame.

[0184] In some embodiments, the framing module 30 is further configured to use a general control mode when the style description object is a predefined type and / or style classification of the corresponding content frame; in the general control mode, in response to the start of associated service, insert the private frame to be processed after the video frame corresponding to the start time of associated service; or, in the general control mode, insert the private frame to be processed into the bitstream at a period of one or more image groups.

[0185] In some embodiments, the framing module 30 is further configured to use a fine-grained control mode when the style description object is the content index of the corresponding content frame; in the fine-grained control mode, in response to a change in the content index, the private frame to be processed is inserted after the video frame corresponding to the time of the content index change.

[0186] In some embodiments, the frame type of the private frame to be processed is a continuous frame; the frame grouping module 30 is further configured to insert the private frame to be processed after the video frame corresponding to the execution time of the configuration operation in response to the configuration operation of the intelligent service associated with the continuous frame; or, in response to the configuration operation of the intelligent service associated with the continuous frame, insert the private frame to be processed into the bitstream at a period of one or more image groups.

[0187] In some embodiments, the frame type of the private frame to be processed is a momentary frame; the framing module 30 is also used to insert the momentary frame after the video frame corresponding to the time of data content generation in response to the generation of the momentary frame's data content.

[0188] In some embodiments, the framing module 30 is further configured to: if the associated object contains only continuous frames and the additional frame is inserted into the bitstream for the first time, then, in response to the continuous frame insertion bitstream, insert the additional frame after the continuous frame; if the associated object contains only continuous frames and the additional frame is not inserted into the bitstream for the first time, then, in response to the data content of the additional frame changing, insert the additional frame after the video frame corresponding to the time when the data content changes; if the associated object contains only instantaneous frames, then, in response to the instantaneous frame insertion bitstream, insert the additional frame after the instantaneous frame; if the associated object contains both continuous frames and instantaneous frames, then, in response to the generation of the additional frame's data content, insert the additional frame after the video frame corresponding to the time when the data content is generated.

[0189] In some of these embodiments, in Figure 8 In addition, the device also includes a tagging module, which is used to tag the frame type of the private frame to be processed according to the associated service and / or frame category, so that the player used to parse the bitstream can identify the frame type and perform corresponding decoding operations based on the identification result.

[0190] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0191] This embodiment also provides a computer device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0192] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0193] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0194] S1, determine the frame category of the private frame to be processed;

[0195] S2, determine the target framing strategy corresponding to the private frame to be processed based on the frame category of the private frame to be processed; wherein, the target framing strategy is used to define the insertion rules of the private frame to be processed in the bitstream through at least one insertion parameter;

[0196] S3, based on the target framing strategy, frames the private frames to be processed into the bitstream.

[0197] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0198] Furthermore, in conjunction with the private frame assembly method for the bitstream provided in the above embodiments, this embodiment can also provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the private frame assembly methods for the bitstream in the above embodiments.

[0199] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0200] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0201] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0202] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A method for framing private frames of a bitstream, characterized in that, include: Determining the frame category of the private frame to be processed includes: classifying the private frame to be processed based on a functional dimension and a spatiotemporal dimension to obtain the frame category of the private frame to be processed; wherein, the functional dimension is associated with the rendering characteristics of the private frame to be processed; and the spatiotemporal dimension is associated with the spatiotemporal dependency relationship of the private frame to be processed. Based on the frame category of the private frame to be processed, a target framing strategy corresponding to the private frame to be processed is determined; wherein, the target framing strategy is used to define the insertion rules of the private frame to be processed in the bitstream through at least one insertion parameter; Based on the target framing strategy, the private frames to be processed are framed into the bitstream.

2. The method for framing private frames of a bitstream according to claim 1, characterized in that, The insertion parameters include one or more combinations of the insertion position, insertion frequency, and insertion time of the private frame to be processed.

3. The method for framing private frames of a bitstream according to claim 1, characterized in that, The frame category of the private frame to be processed is a metadata frame; The step of framing the private frames to be processed into the bitstream based on the target framing strategy includes: When the frame data of the metadata frame is fixed information, the private frame to be processed is framed into the bitstream according to the preset insertion mode; If the frame data of the metadata frame is change information, the private frame to be processed is inserted after the video frame corresponding to the time when the change information is generated.

4. The method for framing private frames of a bitstream according to claim 3, characterized in that, The method further includes: When the preset insertion mode is the first insertion mode, in response to the establishment of the video stream session, the private frame to be processed is inserted after the video frame corresponding to the time when the video stream session is established; When the preset insertion mode is the second insertion mode, the private frame to be processed is inserted into the bitstream at a period of one or more image groups.

5. The method for framing private frames of a bitstream according to claim 1, characterized in that, The frame category of the private frame to be processed is a style frame; The step of framing the private frames to be processed into the bitstream based on the target framing strategy includes: According to the preset control mode, the private frames to be processed are framed into the bitstream; the preset control mode corresponds to the style description object of the style frame.

6. The method for framing private frames of a bitstream according to claim 5, characterized in that, The method further includes: When the style description object is a predefined type and / or style category of the corresponding content frame, the control mode is a general control mode; In the general control mode, in response to the activation of an associated service, the private frame to be processed is inserted after the video frame corresponding to the activation time of the associated service; or, In the general control mode, the private frame to be processed is inserted into the bitstream at a period of one or more image groups.

7. The method for framing private frames of a bitstream according to claim 5, characterized in that, The method further includes: When the style description object is the content index of the corresponding content frame, the control mode is a fine-grained control mode; In the fine control mode, in response to a change in the content index, the private frame to be processed is inserted after the video frame corresponding to the moment the content index changes.

8. The method for framing private frames of a bitstream according to claim 1, characterized in that, The frame category of the private frame to be processed is a continuous frame; The step of framing the private frames to be processed into the bitstream based on the target framing strategy includes: In response to a configuration operation of the intelligent service associated with the persistent frame, the private frame to be processed is inserted after the video frame corresponding to the execution time of the configuration operation; or, In response to the configuration operation of the intelligent service associated with the continuous frame, the private frame to be processed is inserted into the bitstream periodically, with one or more image groups as the cycle.

9. The method for framing private frames of a bitstream according to claim 1, characterized in that, The frame category of the private frame to be processed is a transient frame; the step of framing the private frame to be processed into the bitstream based on the target framing strategy includes: In response to the generation of data content for the instantaneous frame, the instantaneous frame is inserted after the video frame corresponding to the time of data content generation.

10. The method for framing private frames of a bitstream according to claim 1, characterized in that, The frame category of the private frame to be processed is an additional frame; the target framing strategy is determined based on the associated object of the additional frame; wherein, the associated object includes continuous frames and / or instantaneous frames.

11. The method for framing private frames of a bitstream according to claim 10, characterized in that, The step of framing the private frames to be processed into the bitstream based on the target framing strategy includes: If the associated object contains only a continuous frame, and the additional frame is inserted into the bitstream for the first time, then in response to the insertion of the continuous frame into the bitstream, the additional frame is inserted after the continuous frame; If the associated object contains only continuous frames and the additional frame is not the first time it is inserted into the bitstream, then in response to a change in the data content of the additional frame, the additional frame is inserted after the video frame corresponding to the time when the data content changes. If the associated object contains only a momentary frame, then in response to the insertion of the momentary frame into the bitstream, the additional frame is inserted after the momentary frame; If the associated object contains both continuous frames and instantaneous frames, then in response to the generation of the data content of the additional frame, the additional frame is inserted after the video frame corresponding to the time of data content generation.

12. The method for framing private frames of a bitstream according to any one of claims 1 to 11, characterized in that, Before framing the private frame to be processed into the bitstream based on the target framing strategy, the method further includes: According to the associated service and / or the frame category of the private frame to be processed, the frame type of the private frame to be processed is marked so that the player used to parse the bitstream can identify the frame type and perform corresponding decoding operations based on the identification result.

13. A private frame assembly device for a bitstream, characterized in that, include: The classification module is used to determine the frame category of the private frame to be processed; The classification module is further configured to classify the private frame to be processed based on the functional dimension and the spatiotemporal dimension to obtain the frame category of the private frame to be processed; wherein, the functional dimension is associated with the rendering characteristics of the private frame to be processed; and the spatiotemporal dimension is associated with the spatiotemporal dependency relationship of the private frame to be processed. The matching module is used to determine the target framing strategy corresponding to the private frame to be processed based on the frame category of the private frame to be processed; wherein, the target framing strategy is used to define the insertion rules of the private frame to be processed in the bitstream through at least one insertion parameter; The framing module is used to frame the private frames to be processed into the bitstream based on the target framing strategy.

14. A computer device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the steps of the private frame framing method for the bitstream of any one of claims 1 to 12.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the private frame framing method for the bitstream according to any one of claims 1 to 12.

Images