Video processing method, device and computing equipment in edge intelligence environment

Abstract

The present invention discloses a video processing method, device and computing equipment in an edge intelligent environment, the method comprising: obtaining resource supply information for the current video encoding strategy of the terminal, and quality index data for the posture estimation processing of the current video data sent by the terminal; according to the resource supply information and the quality index data for the posture estimation processing, the target video encoding strategy is calculated by a preset video encoding strategy model; wherein the preset video encoding strategy model is obtained by fitting the quantitative relationship between the video encoding strategy, the quality index of the posture estimation processing and the video processing resource consumption; the target video encoding strategy is sent to the terminal, so that the terminal can encode the subsequent collected video data according to the target video encoding strategy. Through the above method, the adaptive decision of the video encoding strategy for the posture estimation task driven by resources is realized, making the video encoding strategy more reasonable and scientific.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a video processing method, device and computing equipment in an edge intelligence environment. Background Art

[0002] With the development of mobile Internet and multimedia technology, mobile multimedia data has become one of the most important data sources in the mobile Internet. Edge intelligence has gradually become one of the most effective paradigms for deploying low-latency mobile video intelligent analysis applications. End users can offload all or part of their intelligent video analysis tasks to edge nodes. In the edge intelligence environment, edge intelligent video analysis involves multiple links such as video acquisition, encoding and decoding, video transmission, inference calculation, and result feedback.

[0003] In the existing technology, video transmission takes objective QoS (Quality of Service) or (QoE) as the optimization goal. However, in the edge intelligent video analysis environment, video transmission technology takes machine perception as the ultimate goal. The fundamental change in the purpose of video transmission optimization makes it difficult for traditional video transmission technology to adapt to edge intelligent video analysis applications. Summary of the invention

[0004] In view of the above problems, the present invention is proposed to provide a video processing method, apparatus and computing device in an edge intelligence environment that overcomes the above problems or at least partially solves the above problems.

[0005] According to one aspect of the present invention, a video processing method in an edge intelligence environment is provided, the method comprising:

[0006] Obtain resource supply information for the current video encoding strategy of the terminal, and quality indicator data for the posture estimation processing of the current video data sent by the terminal;

[0007] According to the resource supply information and the quality indicator data of the posture estimation processing, the target video encoding strategy is calculated by using the preset video encoding strategy model;

[0008] The preset video coding strategy model is obtained by fitting the quantitative relationship between the video coding strategy, the quality index of the posture estimation processing, and the video processing resource consumption;

[0009] The target video encoding strategy is sent to the terminal so that the terminal can encode the subsequently collected video data according to the target video encoding strategy.

[0010] According to another aspect of the present invention, a video processing method in an edge intelligence environment is provided, the method comprising:

[0011] Sending the current video data to the edge node; so that the edge node can calculate the resource supply information corresponding to the current video encoding strategy of the terminal, and perform posture estimation processing on the current video data, and detect the quality index data of the posture estimation processing, and calculate the target video encoding strategy through the preset video encoding strategy model according to the resource supply information and the quality index data of the posture estimation processing;

[0012] The preset video coding strategy model is obtained by fitting the quantitative relationship between the feasible video coding strategy, the quality index of the posture estimation processing and the video processing resource consumption;

[0013] The target video encoding strategy fed back by the edge node is received, and the subsequently collected video data is encoded according to the target video encoding strategy.

[0014] According to another aspect of the present invention, a video processing device in an edge intelligence environment is provided, comprising:

[0015] A data acquisition module, adapted to acquire resource supply information for a current video encoding strategy of a terminal, and quality indicator data for a posture estimation process of current video data sent by the terminal;

[0016] A strategy processing module, adapted to calculate a target video coding strategy through a preset video coding strategy model according to resource supply information and quality indicator data of posture estimation processing;

[0017] The preset video coding strategy model is obtained by fitting the quantitative relationship between the video coding strategy, the quality index of the posture estimation processing, and the video processing resource consumption;

[0018] The strategy delivery module is adapted to deliver the target video encoding strategy to the terminal, so that the terminal can encode the subsequently collected video data according to the target video encoding strategy.

[0019] According to another aspect of the present invention, a video processing device in an edge intelligence environment is provided, comprising:

[0020] A sending module, adapted to send the current video data to the edge node; so that the edge node can calculate the resource supply information corresponding to the current video encoding strategy of the terminal, perform posture estimation processing on the current video data, detect the quality index data of the posture estimation processing, and calculate the target video encoding strategy through a preset video encoding strategy model according to the resource supply information and the quality index data of the posture estimation processing; wherein the preset video encoding strategy model is obtained by fitting the quantitative relationship between the feasible video encoding strategy, the quality index of the posture estimation processing, and the video processing resource consumption;

[0021] The preset video coding strategy model is obtained by fitting the quantitative relationship between the feasible video coding strategy, the quality index of the posture estimation processing and the video processing resource consumption;

[0022] A receiving module, adapted to receive a target video encoding strategy fed back by an edge node;

[0023] The video encoding module is suitable for encoding the subsequently collected video data according to the target video encoding strategy.

[0024] According to another aspect of the present invention, there is provided a computing device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0025] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the video processing method in the above-mentioned edge intelligent environment.

[0026] According to another aspect of the present invention, a computer storage medium is provided, wherein the storage medium stores at least one executable instruction, and the executable instruction enables a processor to perform operations corresponding to the video processing method in the above-mentioned edge intelligent environment.

[0027] According to the video processing method, device and computing equipment in the edge intelligent environment of the present invention, the method includes: obtaining resource supply information for the current video encoding strategy of the terminal, and quality index data for the posture estimation processing of the current video data sent by the terminal; according to the resource supply information and the quality index data of the posture estimation processing, the target video encoding strategy is calculated by the preset video encoding strategy model; wherein the preset video encoding strategy model is obtained by fitting the quantitative relationship between the video encoding strategy, the quality index of the posture estimation processing and the video processing resource consumption; the target video encoding strategy is sent to the terminal, so that the terminal can encode the subsequent collected video data according to the target video encoding strategy. Through the above method, considering the influence of resource supply on the video encoding strategy and the quality of posture estimation, with the purpose of optimizing the quality index of the posture estimation processing perceived by the machine, a resource-driven adaptive decision on video transmission of the posture estimation task is realized, so that the video encoding strategy is more reasonable and scientific, which helps to ensure the quality of the posture estimation task, reduce the delay required for video transmission, and improve the utilization of resources.

[0028] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0030] Figure 1 A flow chart of a video processing method in an edge intelligence environment provided by an embodiment of the present invention is shown;

[0031] Figure 2 A flowchart of a video processing method in an edge intelligence environment provided by another embodiment of the present invention is shown;

[0032] Figure 3 A flowchart of a video processing method in an edge intelligence environment provided by another embodiment of the present invention is shown;

[0033] Figure 4 It shows a schematic diagram of the structure of a video processing device in an edge intelligence environment provided by an embodiment of the present invention;

[0034] Figure 5It shows a schematic diagram of the structure of a video processing device in an edge intelligence environment provided by an embodiment of the present invention;

[0035] Figure 6 A schematic diagram of the structure of a computing device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0036] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0037] Figure 1 The flowchart of the video processing method in the edge intelligent environment provided by an embodiment of the present invention is shown. The method of this embodiment is applied to the video transmission scenario of posture estimation video analysis, and the execution subject is the edge node in the edge intelligent environment, such as Figure 1 As shown, the method comprises the following steps:

[0038] Step S110, obtaining resource supply information of the current video encoding strategy for the terminal and quality indicator data of the posture estimation processing for the current video data sent by the terminal.

[0039] Specifically, after the terminal collects the video data, it encodes the collected video data according to the current video encoding strategy to obtain the current video data, and sends the current video data to the edge node. Specifically, the video data can be transmitted in the form of RTMP video stream or HTTP video stream.

[0040] The purpose of posture estimation processing is to output human key point information, which can be used in AR avatar applications. The quality index data of posture estimation processing include the accuracy of posture estimation analysis and transmission delay. The accuracy of posture estimation analysis is determined by analyzing the difference between the output human key point information and the actual human key point information. Transmission delay refers to the delay required for video transmission. The above two quality indicators are key indicators for evaluating the quality of posture estimation tasks and are machine perception indicators. In the method of this embodiment, after the edge node receives the current video data, it performs posture estimation processing and analyzes to obtain the quality index data of posture estimation processing. The deep model of posture estimation is deployed in the edge server for resource estimation processing of the video.

[0041] The current video encoding strategy refers to the encoding strategy adopted by the current terminal for encoding processing, which specifically includes the video resolution dimension, quantization parameter dimension, and encoding area. Resource supply information includes computing resource supply information and bandwidth resource supply information, which is obtained by scheduling and calculating the resource information owned by the current terminal.

[0042] Step S120 , obtaining a target video encoding strategy by calculating a preset video encoding strategy model according to the resource supply information and the quality indicator data of the posture estimation processing.

[0043] The preset video coding strategy model is obtained by fitting the quantitative relationship between the video coding strategy, the quality index of the posture estimation processing and the video processing resource consumption.

[0044] The quantitative relationship between the video coding strategy, the quality live broadcast index of the posture estimation analysis, and the video processing resource consumption is determined in advance through data analysis technology, and the internal correlation and transmission relationship between the resource supply, the video coding strategy, and the quality of the posture estimation processing are clarified to construct a video coding strategy model. Then, in the subsequent process, the video coding strategy model can be used to calculate the optimal video coding strategy under a given resource supply, and the collected video data can be encoded and processed according to the target video coding strategy, which can achieve high quality of posture estimation analysis, low transmission delay, and high resource utilization.

[0045] Step S130: sending the target video encoding strategy to the terminal, so that the terminal can encode the subsequently collected video data according to the target video encoding strategy.

[0046] After the edge node calculates the target video coding strategy, it returns the target video coding strategy to the terminal. Similarly, the target video coding strategy also includes video parameters in three dimensions: video resolution, quantization parameter, and coding area. The terminal receives the target video coding strategy fed back by the edge node, and the subsequent collected video data is encoded according to the video parameters contained in the target video coding strategy.

[0047] According to the video processing method in the edge intelligent environment provided by this embodiment, the method fits the quantitative relationship between the feasible coding video strategy, the quality index of the posture estimation processing and the video processing consumption resources, and then constructs a preset video coding strategy model, uses the preset coding strategy model to process and obtain the target video coding strategy in the current scene, and then uses the target video coding strategy to encode the collected video data. Through the above method, considering the impact of resource supply on the video coding strategy and the quality of posture estimation, with the purpose of optimizing the quality index of the machine-perceived posture estimation processing, a resource-driven adaptive decision on video transmission of the posture estimation task is realized, making the video coding strategy more reasonable and scientific, which helps to ensure the quality of the posture estimation task, reduce the delay required for video transmission, and improve resource utilization.

[0048] Figure 2 A flow chart of a video processing method in an edge intelligence environment provided by another embodiment of the present invention is shown. Figure 2 As shown, the method comprises the following steps:

[0049] Step S210 , fitting a first quantitative relationship between a video encoding strategy and a quality indicator of a posture estimation process.

[0050] The video encoding strategy is obtained by combining video parameters of various dimensions, and the video parameters include: video resolution multiple, quantization parameter and encoding area. The video resolution multiple indicates the multiple by which the resolution of the original video is adjusted, such as increasing to 1.25 times, 1.5 times, 2 times of the original video resolution and reducing to 0.8 times, 0.75 times, 0.66 times, 0.5 times of the original video resolution. The encoding area indicates the area in the video to be encoded. For example, considering the particularity of the posture estimation task, the encoding area is set to include: complete video frame, human body part detected in the video frame, and no high-quality encoding area. For example, if the video encoding strategy is {1.25, 0.8, complete video frame}, it means that the encoding process is: increasing the resolution to 1.23 times the original resolution, setting the quantization parameter to 75%, and encoding the complete video frame.

[0051] For each video coding strategy, the video data is encoded using the video coding strategy, the encoded video data is sent to the edge node for posture estimation, and the quality index of the posture estimation processing is detected, that is, the accuracy and transmission delay of the posture estimation analysis under the coding strategy are obtained. Multiple experiments are carried out to obtain a large amount of sample data, and the sample data is analyzed and fitted to obtain the first quantitative relationship between the video coding strategy and the quality index of the posture estimation processing.

[0052] Step S220: fitting a second quantitative relationship between feasible video encoding strategies and video processing resource consumption.

[0053] Similarly, for each video encoding strategy, the video data is encoded using the video encoding strategy, the encoded video data is sent to the edge node, and the computing resources consumed during video encoding are detected, including CPU usage, memory usage, and bandwidth resources during video transmission, including network traffic, to obtain the resource consumption under the video encoding strategy. Multiple experiments are conducted to obtain a large amount of sample data, and the sample data is analyzed and fitted to obtain the second quantitative relationship between the video encoding strategy and the video processing resource consumption.

[0054] Step S230, based on the first quantization relationship and the second quantization relationship, a third quantization relationship among the video processing resource consumption, the video encoding strategy and the quality index of the posture estimation processing is fitted, and based on the third quantization relationship, a deep reinforcement learning framework is constructed to obtain a video encoding strategy model.

[0055] According to the quantitative relationship between the video coding strategy obtained in the above step S210 and the quality index of the posture estimation processing, and the relationship between the video coding strategy obtained in the above step S220 and the video processing resource consumption, the intrinsic correlation and transmission relationship between resource consumption, video coding strategy and the quality index of posture estimation processing are clarified. Afterwards, according to the third quantitative relationship between the fitted resource consumption, feasible video coding strategy and the quality index of posture estimation processing, a deep reinforcement learning framework is constructed to obtain a video coding strategy model. In the subsequent process, the video coding strategy model can be used to calculate the optimal target video coding strategy under a given resource supply, and the collected video data is encoded and processed according to the target video coding strategy, which can make the posture estimation analysis high in quality, low in transmission delay and high in resource utilization.

[0056] In an alternative approach, use a multivariate function The relationship between the fitted video encoding strategy x and the quality index of the pose estimation process is as follows:

[0057]

[0058] Among them, MSE represents mean square error, and D is the set of video coding strategies.

[0059] Use function g to fit the computational resources required for video encoding strategy x and bandwidth resources The quantitative relationship is as follows:

[0060]

[0061] Finally, the quantitative relationship between the quality indicators of video processing resource consumption, video encoding strategy, and posture estimation processing is fitted. On this basis, a resource-driven low-latency video analysis and reasoning quality model in the edge intelligence environment is constructed. The model is as follows:

[0062]

[0063] Among them, X is an intermediate variable, representing the set of video coding schemes, the independent variable of the q function is the quality index of the posture estimation process, and the dependent variable is QoI, Q(r c ,r n ) represents the reward function, r c ,r n Represented as resource supply.

[0064] This is equivalent to fitting the quantitative relationship between resource consumption, video encoding strategy and quality indicators of posture estimation processing based on a large amount of experimental data, and constructing a video encoding strategy model based on the quantitative relationship. In the subsequent process, the appropriate target video encoding strategy is determined through the model based on the current resource supply.

[0065] Optionally, for the deep reinforcement framework, set the above QoI as the reward function and set P DRL Representing the decision function of the deep reinforcement learning framework of the edge node, the decision method is expressed as:

[0066]

[0067] Among them, x* represents the video coding strategy obtained by decision, and the decision function P DRL The larger the function value is, the more advantageous the corresponding video coding scheme is.

[0068] Step S240, determining resource supply information according to the current video encoding strategy of the terminal and the second quantization relationship; wherein the resource supply information includes: computing resource supply information and network resource supply information.

[0069] According to the quantitative relationship between the video encoding strategy and resource consumption determined in the above steps, and the current video encoding strategy, the computing resources required for encoding the video and the network resources required for transmitting the video under the current video encoding strategy are calculated to obtain resource supply information. This step is the resource scheduling step.

[0070] Step S250, receiving the current video data sent by the terminal, performing posture estimation processing on the current video data, and analyzing to obtain quality indicator data of the posture estimation.

[0071] The terminal processes the collected video data according to the three video parameters contained in the current video encoding strategy, and sends the encoded video data to the edge node. After receiving the encoded video data, the edge node performs posture estimation processing and analyzes the accuracy of the posture estimation processing and the transmission delay.

[0072] Step S260, according to the resource supply information and the quality indicator data of the posture estimation processing, a target video encoding strategy is calculated by a preset video encoding strategy model.

[0073] The calculated resource supply information and the quality indicator data of posture estimation processing are input into the deep reinforcement learning framework for processing. When the value of the decision function of the deep reinforcement learning framework is maximized, the corresponding video encoding strategy is extracted, that is, the target video encoding strategy is obtained.

[0074] Step S270: Send the target video encoding strategy to the terminal, so that the terminal can encode the subsequently collected video data according to the target video encoding strategy.

[0075] After the target video encoding strategy is calculated, the target video encoding strategy is sent to the terminal, and the terminal encodes the subsequent video data according to the target video encoding strategy.

[0076] In an optional manner, the method of this embodiment is executed at intervals of a predetermined time, that is, the video encoding strategy for encoding video data by the terminal is updated every certain period of time.

[0077] In another optional manner, the method of this embodiment is executed in real time, that is, the video encoding strategy of the terminal for encoding video data is detected and updated in real time.

[0078] According to the video processing method in the edge intelligent environment provided by this embodiment, by fitting the quantitative relationship between feasible coding video strategies, quality indicators of posture estimation processing, and video processing consumption resources, a preset video coding strategy model is constructed, and the preset coding strategy model is used to process the target video coding strategy in the current scene, and the target video coding strategy is subsequently used to encode the collected video data. In the above manner, considering the impact of resources on video parameters and posture estimation quality, with the purpose of optimizing the quality indicators of machine-perceived posture estimation processing, a resource-driven adaptive decision on video transmission of posture estimation tasks is realized, which helps to ensure the quality of posture estimation tasks, reduce the delay required for video transmission, and improve resource utilization.

[0079] Figure 3 A flowchart of a video processing method in an edge intelligence environment provided by another embodiment of the present invention is shown. The method of this embodiment is applied to the terminal side, such as Figure 3As shown, the method includes:

[0080] Step S310, sending the current video data to the edge node; so that the edge node can calculate the resource supply information corresponding to the current video encoding strategy of the terminal, and perform posture estimation processing on the current video data, and detect the quality indicator data of the posture estimation processing, and calculate the target video encoding strategy through the preset video encoding strategy model based on the resource supply information and the quality indicator data of the posture estimation processing.

[0081] The preset video coding strategy model is obtained by fitting the quantitative relationship between the feasible video coding strategy, the quality index of the posture estimation processing and the video processing resource consumption;

[0082] Step S320: receiving the target video encoding strategy fed back by the edge node, and encoding the subsequently collected video data according to the target video encoding strategy.

[0083] The method applied on the terminal side in this embodiment corresponds to the method applied on the edge node side in the above embodiment. The specific implementation method can refer to the description in the above embodiment, which will not be described here.

[0084] According to the video processing method in the edge intelligent environment provided by this embodiment, the method constructs a preset video coding strategy model by fitting the quantitative relationship between the feasible coding video strategy, the quality index of the posture estimation processing and the video processing consumption resources, and then uses the preset coding strategy model to process the target video coding strategy in the current scene, and then uses the target video coding strategy to encode the collected video data. Through the above method, considering the impact of resource supply on video parameters and posture estimation quality, with the purpose of optimizing the quality index of machine-perceived posture estimation processing, a resource-driven adaptive decision on video transmission of posture estimation tasks is realized, which helps to ensure the quality of posture estimation tasks, reduce the delay required for video transmission, and improve resource utilization.

[0085] Figure 4 FIG. 1 shows a schematic diagram of the structure of a video processing device in an edge intelligence environment provided by an embodiment of the present invention. Figure 4 As shown, the device comprises:

[0086] A data acquisition module 41, adapted to acquire resource supply information for a current video encoding strategy of a terminal, and quality indicator data for a posture estimation process of current video data sent by the terminal;

[0087] A strategy processing module 42, adapted to calculate a target video coding strategy through a preset video coding strategy model according to resource supply information and quality indicator data of posture estimation processing;

[0088] The preset video coding strategy model is obtained by fitting the quantitative relationship between the video coding strategy, the quality index of the posture estimation processing, and the video processing resource consumption;

[0089] The strategy issuing module 43 is adapted to issue the target video encoding strategy to the terminal, so that the terminal can encode the subsequently collected video data according to the target video encoding strategy.

[0090] In an optional approach, the accuracy of the analysis and the transmission delay are estimated.

[0091] In an optional manner, the target video encoding strategy includes the following video parameters: video resolution multiple, quantization parameter, and encoding area.

[0092] In an optional manner, the device further includes:

[0093] The model building module is suitable for fitting the first quantitative relationship between the quality indicators of the video coding strategy and the posture estimation processing, and fitting the second quantitative relationship between the video coding strategy and the video processing resource consumption; according to the first quantitative relationship and the second quantitative relationship, the third quantitative relationship between the video processing resource consumption, the video coding strategy and the quality indicators of the posture estimation processing is fitted; according to the third quantitative relationship, a deep reinforcement learning framework is constructed to obtain a video coding strategy model.

[0094] In an optional manner, the device further includes:

[0095] The scheduling module is adapted to receive the resource status information sent by the terminal, perform resource scheduling processing through the second quantization relationship, and determine the resource supply information; wherein the resource supply information includes: computing resource supply information and network resource supply information.

[0096] In an optional manner, the device further includes:

[0097] The video analysis module is adapted to receive the current video data sent by the terminal, perform posture estimation processing on the current video data, and analyze to obtain quality indicator data of the posture estimation processing.

[0098] Figure 5 FIG. 1 shows a schematic diagram of the structure of a video processing device in an edge intelligence environment provided by an embodiment of the present invention. Figure 5 As shown, the device comprises:

[0099] The sending module 51 is adapted to send the current video data to the edge node; so that the edge node can calculate the resource supply information corresponding to the current video coding strategy of the terminal, perform posture estimation processing on the current video data, detect the quality index data of the posture estimation processing, and calculate the target video coding strategy through the preset video coding strategy model according to the resource supply information and the quality index data of the posture estimation processing; wherein the preset video coding strategy model is obtained by fitting the quantitative relationship between the feasible video coding strategy, the quality index of the posture estimation processing, and the video processing resource consumption;

[0100] The preset video coding strategy model is obtained by fitting the quantitative relationship between the feasible video coding strategy, the quality index of the posture estimation processing and the video processing resource consumption;

[0101] A receiving module 52, adapted to receive a target video encoding strategy fed back by an edge node;

[0102] The video encoding module 53 is adapted to perform encoding processing on the subsequently collected video data according to the target video encoding strategy.

[0103] In an optional manner, the quality indicators of the posture estimation process include: accuracy of posture estimation analysis and transmission delay.

[0104] In an optional manner, the target video encoding strategy includes the following video parameters: video resolution multiple, quantization parameter, and encoding area.

[0105] An embodiment of the present invention provides a non-volatile computer storage medium, which stores at least one executable instruction. The computer executable instruction can execute the video processing method in an edge intelligence environment in any of the above method embodiments.

[0106] Figure 6 The schematic diagram of the structure of the computing device embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the computing device.

[0107] like Figure 6 As shown, the computing device may include: a processor (processor) 602 , a communications interface (Communications Interface) 604 , a memory (memory) 606 , and a communication bus 608 .

[0108] The processor 602, the communication interface 604, and the memory 606 communicate with each other via a communication bus 608. The communication interface 604 is used to communicate with other devices such as a client or other server network elements. The processor 602 is used to execute a program 610, which can specifically execute the relevant steps in the embodiment of the video processing method in the edge intelligence environment for computing devices.

[0109] Specifically, the program 610 may include program codes, which include computer operation instructions.

[0110] The processor 602 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the computing device may be processors of the same type, such as one or more CPUs; or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0111] The memory 606 is used to store the program 610. The memory 606 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0112] The algorithm or display provided herein is not inherently related to any particular computer, virtual system or other equipment. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the embodiment of the present invention is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description made to specific languages ​​above is for disclosing the best mode of the present invention.

[0113] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0114] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, the various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the following intention: that the claimed invention requires more features than the features explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.

[0115] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0116] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0117] The various component embodiments of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some or all components according to embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., computer program and computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0118] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

Claims

1. A video processing method in an edge intelligence environment, characterized in that: The method comprises: Obtain resource supply information for the current video encoding strategy of the terminal, and quality indicator data for the posture estimation processing of the current video data sent by the terminal; According to the resource supply information and the quality indicator data of the posture estimation processing, a target video encoding strategy is calculated by a preset video encoding strategy model; The preset video coding strategy model is obtained by fitting the quantitative relationship between the video coding strategy, the quality index of the posture estimation processing, and the video processing resource consumption; The target video encoding strategy is issued to the terminal, so that the terminal can encode the subsequently collected video data according to the target video encoding strategy.

2. The method according to claim 1, characterized in that The quality indicators of the posture estimation process include: accuracy of posture estimation analysis and transmission delay.

3. The method according to claim 1, characterized in that The target video encoding strategy includes the following video parameters: video resolution multiple, quantization parameter, and encoding area.

4. The method according to claim 1, characterized in that: Before the method is performed, it further comprises: Fitting a first quantitative relationship between a video coding strategy and a quality indicator of a posture estimation process, and fitting a second quantitative relationship between a video coding strategy and a video processing resource consumption; According to the first quantization relationship and the second quantization relationship, a third quantization relationship among the video processing resource consumption, the video encoding strategy and the quality index of the posture estimation processing is obtained by fitting; According to the third quantization relationship, a deep reinforcement learning framework is constructed to obtain the video encoding strategy model.

5. The method according to claim 4, characterized in that Before the method is performed, it further comprises: The resource status information sent by the receiving terminal is processed by performing resource scheduling through the second quantization relationship to determine the resource supply information; wherein the resource supply information includes: computing resource supply information and network resource supply information.

6. The method according to claim 5, characterized in that Before the method is performed, it further comprises: The current video data sent by the receiving terminal is processed by performing posture estimation on the current video data, and quality indicator data of the posture estimation processing is obtained by analysis.

7. A video processing method in an edge intelligence environment, characterized in that: The method comprises: Sending current video data to an edge node; so that the edge node can calculate resource supply information corresponding to the current video encoding strategy of the terminal, and performing posture estimation processing on the current video data, and detecting quality indicator data of the posture estimation processing, and calculating the target video encoding strategy through a preset video encoding strategy model according to the resource supply information and the quality indicator data of the posture estimation processing; The preset video coding strategy model is obtained by fitting the quantitative relationship between the feasible video coding strategy, the quality index of the posture estimation processing and the video processing resource consumption; The target video encoding strategy fed back by the edge node is received, and the subsequently collected video data is encoded according to the target video encoding strategy.

8. A video processing device in an edge intelligence environment, characterized in that: include: A data acquisition module, adapted to acquire resource supply information for a current video encoding strategy of a terminal, and quality indicator data for a posture estimation process of current video data sent by the terminal; A strategy processing module, adapted to calculate a target video coding strategy through a preset video coding strategy model according to the resource supply information and the quality indicator data of the posture estimation processing; The preset video coding strategy model is obtained by fitting the quantitative relationship between the video coding strategy, the quality index of the posture estimation processing, and the video processing resource consumption; The strategy delivery module is adapted to deliver the target video encoding strategy to the terminal, so that the terminal can perform encoding processing on the subsequently collected video data according to the target video encoding strategy.

9. A computing device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the video processing method in the edge intelligence environment as described in any one of claims 1-6.

10. A computer storage medium, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to perform operations corresponding to the video processing method in an edge intelligence environment as described in any one of claims 1-6.

Images