A video analysis configuration adaptive adjustment method based on serverless architecture

Abstract

The present invention discloses a video analysis configuration adaptive adjustment method based on a serverless architecture, the method comprising: obtaining a video stream and configuring an adaptive adjustment model; decoding the video stream to obtain continuous video frames, extracting key frames in the video frames based on the video frame rate configuration; based on the target detection model configuration and the computing resource configuration, the target detection model detects the key frames to obtain the target detection results corresponding to the key frames, and uses the target detection results as the target to be tracked to obtain the analysis results of non-key frames according to the target detection results and the continuous target tracking algorithm; based on the target detection results and the analysis results of the non-key frames, the computing cost and accuracy model are updated to update the configuration adaptive adjustment model. The method of the present invention can effectively adjust the configuration of video parameters and computing resources, and the algorithm ultimately greatly reduces the computing cost under the constraint of the target accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of configuration adjustment of an Internet application layer, and in particular to a video analysis configuration adaptive adjustment method based on a serverless architecture. Background Art

[0002] In recent years, the number of cameras deployed has surged. The global surveillance camera market is expected to reach $45.93 billion by 2027. To fully unleash the potential of these cameras, video analytics applications have been developed that use computer vision technology to understand the videos collected by cameras and detect objects of interest in them to help various public and private organizations (such as law enforcement agencies and retail enterprises) improve efficiency, reduce expenses, and improve safety. Due to the continuous expansion of camera deployment and the growing demand for video analytics, there is an urgent need for high-performance video analytics systems. The continuous evolution of deep neural networks has greatly improved the accuracy of video analysis, but the demand for computing resources is also increasing. Existing solutions either rely on cloud platforms with rich computing resources to build virtual machine clusters or invest in powerful hardware to build private clusters to cope with the resource challenges faced by large-scale high-precision video analysis. Executing expensive deep neural network models results in very high computing costs. In fact, although cameras provide continuous video streams, it is not necessary to use dedicated clusters to continuously analyze them at the highest configuration, because the ideal configuration and resource requirements for video analysis will change over time. Traditional resource configuration techniques (such as configuration based on peak usage or configuration obtained from one-time offline analysis) lead to resource inefficiency.

[0003] Given that video analysis often encounters bursty input workloads and fine-grained dynamic changes in video content, serverless computing is a promising solution that can prevent unnecessary resource configuration. Serverless computing has evolved as a general computing abstraction to reduce time-consuming management tasks and provide fine-grained automatically scalable computing infrastructure. Function as a Service (FaaS) solutions that reflect the concept of serverless computing have achieved commercial success in recent years (e.g., AWS Lambda and Google Cloud Functions). In the FaaS platform, the overall application code is divided into many functions. Each function implements a microservice that can be configured and called separately. This lightweight virtualization approach allows function instances to be instantly scaled up or down within milliseconds based on their input workload, achieving fast and flexible responses. This feature enables the serverless framework to handle fine-grained input workload changes well without the need for resource management and monitoring, thereby achieving scalability and flexibility. At the same time, the pay-as-you-go pricing model of FaaS also ensures that money is not wasted on unused resources, thereby achieving better benefits.

[0004] In video analysis based on serverless architecture, frames are extracted from videos at different sampling rates and compressed into different resolutions, and then analyzed by different deep neural network models on the serverless computing platform. Users also need to determine the computing resources allocated to the corresponding functions. The specific combination of frame rate, deep neural network model, and computing resources is called a configuration. Obviously, different configurations will lead to different accuracy and computing costs. Related measurement studies have shown that finding the best configuration is crucial to achieving high-accuracy and cost-effective video analysis systems. Dynamically selecting the best configuration based on changes in video content can greatly reduce the computing cost. The decisions on video parameter and computing resource configuration affect each other and are crucial to the overall performance of video analysis. Although industry and academia have made extensive efforts to optimize the configuration of video parameters, the coordinated configuration and adjustment of video parameters and computing resources based on serverless architecture remains an unexplored area. Summary of the invention

[0005] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0006] The growth in the number of videos and the enhancement of deep neural network capabilities have led to a growing demand for video analysis, which requires a large amount of computing resources. Traditional resource allocation strategies, such as configuring clusters according to peak utilization, will lead to inefficient resources. Serverless computing is a very effective way to avoid wasteful resource allocation, because video analysis often encounters sudden workloads and dynamic changes in fine-grained video content. For video analysis systems based on serverless architecture, application configuration (frame rate, detection model and computing resources) will affect several key indicators, such as computing cost and analysis accuracy. The present invention proposes an efficient algorithm for the joint configuration and adjustment of video parameters and computing resources in a video analysis system based on a serverless architecture, namely, a video analysis configuration adaptive adjustment method based on a serverless architecture, which can effectively adjust the configuration of video parameters and computing resources, and the algorithm ultimately greatly reduces the computing cost under the constraint of the target accuracy.

[0007] The second aspect of the present invention is to propose a video analysis configuration adaptive adjustment system based on a serverless architecture.

[0008] A third aspect of the present invention provides a computer device.

[0009] A fourth aspect of the present invention provides a storage medium.

[0010] To achieve the above object, the present invention proposes a video analysis configuration adaptive adjustment method based on a serverless architecture, comprising:

[0011] S1, obtaining a video stream and configuring an adaptive adjustment model; wherein the configuration of the adaptive adjustment model includes video frame rate configuration, target detection model configuration and computing resource configuration;

[0012] S2, decoding the video stream to obtain continuous video frames, and extracting key frames from the video frames based on the video frame rate configuration;

[0013] S3, based on the target detection model configuration and the computing resource configuration, the target detection model detects the key frame to obtain the target detection result corresponding to the key frame, and uses the target detection result as the target to be tracked to obtain the analysis result of the non-key frame according to the target detection result and the continuous target tracking algorithm;

[0014] S4, updating the calculation cost and accuracy model based on the target detection result and the analysis result of the non-key frame to update the configuration adaptive adjustment model.

[0015] In addition, the video analysis configuration adaptive adjustment method based on serverless architecture according to the above embodiment of the present invention may also have the following additional technical features:

[0016] Furthermore, in one embodiment of the present invention, the target detection model includes at least one neural network model.

[0017] Further, in one embodiment of the present invention, the S4 includes:

[0018] S41, at the beginning of each time slot, updating the computational cost and accuracy model according to the video stream content of the current video;

[0019] S42, obtaining a first model selection vector based on the first deep neural network model;

[0020] S43, based on the updated calculation cost and accuracy model and the first model selection vector, solving and obtaining the optimal video parameter configuration and memory configuration for the current time slot;

[0021] S44, calculating a new target value of computing cost according to the optimal video parameter configuration and memory configuration; in the current iteration, obtaining a second deep neural network model according to the target value difference of the new target value of computing cost and the target value of the objective function value corresponding to the old solution and updating it to the first deep neural network model with a first preset probability;

[0022] S45, repeat steps S41-S44 until the maximum number of iterations is reached.

[0023] Furthermore, in one embodiment of the present invention, the result of the target detection includes a target label and a bounding box in a key frame; the method further includes:

[0024] Select different neural network models based on different task requirements, perform inference calculations on the key frames based on the different neural network models and using a deep neural network inference algorithm to obtain the target detection result; and

[0025] The key frames and the target detection results corresponding to the key frames are stored.

[0026] To achieve the above object, the present invention further provides a video analysis configuration adaptive adjustment system based on a serverless architecture, comprising:

[0027] The camera side is used to obtain the video stream;

[0028] The edge server is used to decode the video stream to obtain continuous video frames, and extract key frames from the video frames based on the video frame rate configuration of the adaptive adjustment module;

[0029] A serverless platform, configured to adaptively adjust the target detection model configuration and computing resource configuration of the model based on the configuration, so that the target detection model detects the key frame to obtain the target detection result corresponding to the key frame, and uses the target detection result as the target to be tracked to obtain the analysis result of the non-key frame according to the target detection result and the continuous target tracking algorithm;

[0030] The edge server is used to update the calculation cost and accuracy model based on the target detection result and the analysis result of the non-key frame to update the configuration adaptive adjustment module.

[0031] The third aspect of the present invention proposes a computer device, comprising a processor and a memory; wherein the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the video analysis configuration adaptive adjustment method based on the serverless architecture.

[0032] The fourth aspect of the present invention provides a non-temporary computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the video analysis configuration adaptive adjustment method based on a serverless architecture as described.

[0033] The video analysis configuration adaptive adjustment method, system, computer device and storage medium based on a serverless architecture of the embodiments of the present invention propose an efficient algorithm for the joint configuration adjustment problem of video parameters and computing resources in a video analysis system based on a serverless architecture, which can effectively adjust the configuration of video parameters and computing resources. The algorithm ultimately greatly reduces the computing cost under the constraint of the target accuracy.

[0034] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0036] Figure 1 A flowchart of a method for configuring an adaptive adjustment of a video analysis based on a serverless architecture according to an embodiment of the present invention;

[0037] Figure 2 A schematic diagram of the structure of a video analysis configuration adaptive adjustment system based on a serverless architecture according to an embodiment of the present invention;

[0038] Figure 3 A computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0040] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0041] The following describes a method, system, computer device, and storage medium for adaptively adjusting video analysis configuration based on a serverless architecture according to an embodiment of the present invention with reference to the accompanying drawings.

[0042] Figure 1 It is a flowchart of a method for adaptively adjusting video analysis configuration based on a serverless architecture according to an embodiment of the present invention.

[0043] like Figure 1 As shown, the method includes but is not limited to the following steps:

[0044] S1, obtaining a video stream and configuring an adaptive adjustment model; wherein configuring the adaptive adjustment model includes video frame rate configuration, target detection model configuration, and computing resource configuration;

[0045] S2, decoding the video stream to obtain continuous video frames, and extracting key frames in the video frames based on the video frame rate configuration;

[0046] S3, based on the target detection model configuration and the computing resource configuration, the target detection model detects the key frame to obtain the target detection result corresponding to the key frame, and uses the target detection result as the target to be tracked to obtain the analysis result of the non-key frame according to the target detection result and the continuous target tracking algorithm;

[0047] S4, based on the target detection results and the analysis results of the non-key frames, the calculation cost and accuracy model are updated to update the configuration adaptive adjustment model.

[0048] The present invention first proposes a video analysis architecture based on a serverless platform. The camera is connected to the edge server and continuously transmits the captured video stream to it. The core of the video analysis task is to execute the deep neural network model, which requires considerable computing resources and memory resources. Due to the limited computing power of the edge server, it is difficult to fully support such a large-scale deep neural network model inference calculation on the edge server. However, transmitting all videos to the cloud platform for analysis will increase unnecessary computing and transmission costs, because there are a large number of redundant frames in the video that do not need to be repeatedly transmitted and calculated. Therefore, the present invention proposes an end-edge-cloud collaborative video analysis architecture. The edge server is responsible for performing video caching and lightweight target tracking, while the serverless platform only runs a deep neural network-based target detection model on some key frames to reduce computing costs. In the present invention, some frames sent to the serverless platform for target detection are called key frames. The fine-grained and automatic resource management services provided by serverless computing help this architecture improve resource utilization.

[0049] Furthermore, after the edge server obtains the video stream from the camera, the video decoding module decodes the video stream into continuous video frames. The video caching module caches the key frames and their analysis results according to the target detection result message corresponding to the key frames from the serverless platform. The lightweight target tracking module uses the detection results of the key frames as the target to be tracked and executes the continuous target tracking algorithm to obtain the analysis results of the non-key frames. Using target tracking can reduce the number of video frames sent to the serverless platform to perform the target detection task, thereby reducing the amount of calculation and memory usage, while ensuring that the accuracy of video analysis does not drop excessively. A key module on the edge server is to configure the adaptive adjustment algorithm. The relationship between the solution space and the optimization target is obtained from the cost and accuracy model, which are used as the input of the algorithm. The output of the algorithm is the configuration of the video frame rate, the detection model and the computing resources. The frame rate configuration is directly used locally on the edge server to guide the key frame extraction module. The configuration of the detection model and computing resources is sent to the serverless platform through the control flow to guide the target detection module.

[0050] Specifically, the serverless platform is mainly responsible for performing deep neural network reasoning computing tasks. In addition to receiving key frames from the edge server, it also receives the computing resource configuration and model configuration generated by the configuration adaptive adjustment module. Then, target detection reasoning is performed on the key frames according to these configurations. In order to adapt to different needs, deep neural networks provide a variety of models from small to large. Their structures are similar, but the depth and width of the network are different, and the corresponding parameters and computational workloads are also different. Larger models can obtain detection results with higher accuracy, but at the same time, the corresponding computational cost and memory usage will also be greater. After these models are deployed on the serverless platform, the algorithm can select the corresponding model according to the needs of the scene. After obtaining the results of target detection, including the target label and bounding box in the key frame, the serverless platform feeds these results back to the edge server for subsequent target tracking.

[0051] N parallel neural network models d1, d2, …, d N , corresponding to images of different input sizes. Neural network models with lower input image resolutions have faster processing speeds and require fewer resources. Different neural network models have different input image resolutions, which are defined in their model architectures. That is, when the model selection is determined, its input image resolution is fixed. Time is divided into discrete time slots, and the duration of each time slot matches the time scale on which the configuration can be updated. Introducing a binary variable To represent the model d i Whether it is selected in time slot t. Then use x t To express That is, the set of model selection variables. Similarly, using f t To represent the frame rate selection of the video stream in time slot t, m t Represents a memory allocation value.

[0052] In current serverless platforms, memory size is the only computing resource configuration that can be controlled by the user. The CPU performance allocation of a function is proportional to the amount of memory allocation. Increasing memory allocation also means an increase in CPU performance, which can improve the performance of processing-intensive serverless functions. Therefore, the present invention only considers memory size as a computing resource configuration. Each serverless function can set a maximum memory parameter in the console. The platform also allows limiting the maximum function execution time of a function to prevent the function from getting out of control or hanging and increasing costs. Since serverless functions only run when service requests are generated, they only incur charges when they are used. The general pricing model adopted by serverless providers is based on two aspects of each function call: (1) Maximum memory: This is not the actual memory used by the function, but the maximum memory parameter in the function configuration; (2) Function execution time: The actual clock time required for the function call to run. For each function call, these two values ​​are multiplied to produce a number in GB-sec, which is the computing cost.

[0053] From experience, we know that calculating the cost and memory value m t The relationship between can be fitted as a cubic function: Accuracy and frame rate f t The relationship between can be fitted as an exponential function: Video analysis requires low computational cost and high accuracy. Therefore, when designing and configuring an adaptive adjustment algorithm, the goal is to minimize the computational cost under the constraint of analysis accuracy. The problem can be stated as:

[0054]

[0055]

[0056]

[0057]

[0058] Constraints 1 and 2 ensure that in each time slot, one and only one deep neural network model can be selected. Constraint 3 indicates that the accuracy should be higher than the target value A. This problem is a mixed integer nonlinear programming problem, and it is impossible to find the optimal solution in polynomial time. Therefore, the present invention uses Markov approximation to obtain the solution. Assume that the model selection x t is fixed, there are two more issues to be resolved. The first issue is to optimize memory allocation to reduce computational cost: It can be proved that the optimal memory allocation is: The second problem is to adjust the frame rate to meet the target accuracy: The critical frame rate that just meets the accuracy requirement can be obtained:

[0059] Based on the above analysis, it can be concluded that once the optimal model selection x is found t , the remaining two problems are easy to solve. However, since the model selection variables are binary, the whole problem is a mixed integer nonlinear programming problem and it is impossible to find the optimal solution in polynomial time. The present invention uses Markov approximation to obtain a solution for model selection. Time is divided into discrete time slots. At the beginning of each time slot, the computational cost and accuracy model is updated by the cost and accuracy analyzer based on the current video content. Given the cost and accuracy functions, the optimal video parameter configuration and memory configuration for the current time slot can be found by solving the above problems. First, a new deep neural network model is randomly selected. Then get the new model selection vector Under this vector, the optimal solution can be obtained by solving the above problem and Then calculate the new computational cost target value And C is the old solution {m t ,f t ,x t} The corresponding objective function value. In the current iteration, the algorithm The model to make the decision is updated with probability η as the model The above iterative process will continue until T max Iterations.

[0060] Based on this, the video analysis configuration adaptive adjustment method based on serverless architecture of the present invention may include the following steps:

[0061] Step (1) At the beginning of each time slot, the cost and accuracy analyzer updates the cost and accuracy model according to the current video content.

[0062] Step (2) Randomly select a new deep neural network model Then get the new model selection vector

[0063] Step (3) Given the cost and accuracy model, solve the optimal video parameter configuration for the current time slot and memory configuration

[0064] Step (4) Calculate the new computational cost target value And C is the old solution {m t ,f t ,x t} The corresponding objective function value. In the current iteration, the algorithm The model used to make the decision is based on probability Update to Model Remains unchanged with probability 1-η.

[0065] Step (5) Repeat steps (1)-(4) until T is reached. max Iterations.

[0066] In summary, the present invention proposes an edge-assisted serverless architecture for video analysis. Aiming at the joint configuration and adjustment problem of video parameters and computing resources, the present invention proposes an efficient algorithm based on Markov approximation, which can select the appropriate configuration for the video stream while taking into account the trade-off between accuracy and cost.

[0067] According to the video analysis configuration adaptive adjustment method based on serverless architecture according to the embodiment of the present invention, the configuration of video parameters and computing resources can be effectively adjusted, and the algorithm ultimately greatly reduces the computing cost under the constraint of target accuracy.

[0068] In order to implement the above embodiment, Figure 2 As shown, this embodiment also provides a video analysis configuration adaptive adjustment system 10 based on a serverless architecture. The system 10 includes a camera end 100, an edge server 200, and a serverless platform 300.

[0069] The camera end 100 is used to obtain a video stream;

[0070] The edge server 200 is used to decode the video stream to obtain continuous video frames, and extract key frames from the video frames based on the video frame rate configuration of the adaptive adjustment module;

[0071] The serverless platform 300 is used for configuring the target detection model and computing resource configuration based on the configuration adaptive adjustment model, so that the target detection model detects the key frame to obtain the target detection result corresponding to the key frame, and uses the target detection result as the target to be tracked to obtain the analysis result of the non-key frame according to the target detection result and the continuous target tracking algorithm;

[0072] The edge server 200 is used to update the calculation cost and accuracy model based on the target detection result and the analysis result of the non-key frame to update the configuration adaptive adjustment module.

[0073] Furthermore, the serverless platform 300 includes a computing platform and a storage platform; wherein the computing platform includes a target detection model;

[0074] A storage platform includes at least one neural network model.

[0075] Furthermore, the edge server 200 further includes a video decoding module, a video caching module, a key frame extraction module and a lightweight target tracking module.

[0076] Further, the video caching module is used to cache the key frames and their target detection results according to the target detection results corresponding to the key frames from the serverless platform 300;

[0077] The serverless platform 300 is also used to feed back the target detection result to the edge server 200 for target tracking of the target to be tracked.

[0078] According to the video analysis configuration adaptive adjustment system based on the serverless architecture according to the embodiment of the present invention, the configuration of video parameters and computing resources can be effectively adjusted, and the algorithm ultimately greatly reduces the computing cost under the constraint of the target accuracy.

[0079] In order to implement the method of the above embodiment, the present invention also provides a computer device, such as Figure 3 As shown, the computer device 600 includes a memory 601 and a processor 602; wherein the processor 602 runs a program corresponding to the executable program code by reading the executable program code stored in the memory 601, so as to implement each step of the method described above.

[0080] In order to implement the above embodiments, the present application also proposes a non-temporary computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described in the above embodiments is implemented.

[0081] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0082] The computer readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device. The computer readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device executes the method for determining the sediment content in a flowing water body of the above embodiment.

[0083] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0084] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0085] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0086] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0087] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0088] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0089] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0090] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0091] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A video analysis configuration adaptive adjustment method based on a serverless architecture, characterized in that: The following steps are involved: S1, obtaining a video stream and configuring an adaptive adjustment model; wherein the configuration of the adaptive adjustment model includes video frame rate configuration, target detection model configuration and computing resource configuration; S2, decoding the video stream to obtain continuous video frames, and extracting key frames from the video frames based on the video frame rate configuration; S3, based on the target detection model configuration and the computing resource configuration, the target detection model detects the key frame to obtain the target detection result corresponding to the key frame, and uses the target detection result as the target to be tracked to obtain the analysis result of the non-key frame according to the target detection result and the continuous target tracking algorithm; S4, updating the calculation cost and accuracy model based on the target detection result and the analysis result of the non-key frame to update the configuration adaptive adjustment model.

2. The method according to claim 1, characterized in that: The target detection model includes at least one neural network model.

3. The method according to claim 2, characterized in that The S4 comprises: S41, at the beginning of each time slot, updating the computational cost and accuracy model according to the video stream content of the current video; S42, obtaining a first model selection vector based on the first deep neural network model; S43, based on the updated calculation cost and accuracy model and the first model selection vector, solving and obtaining the optimal video parameter configuration and memory configuration for the current time slot; S44, calculating a new target value of computing cost according to the optimal video parameter configuration and memory configuration; in the current iteration, obtaining a second deep neural network model according to the target value difference of the new target value of computing cost and the target value of the objective function value corresponding to the old solution and updating it to the first deep neural network model with a first preset probability; S45, repeat steps S41-S44 until the maximum number of iterations is reached.

4. The method according to claim 2, characterized in that: The target detection result includes a target label and a bounding box in a key frame; the method further includes: Select different neural network models based on different task requirements, perform inference calculations on the key frames based on the different neural network models and using a deep neural network inference algorithm to obtain the target detection result; and The key frames and the target detection results corresponding to the key frames are stored.

5. A video analysis configuration adaptive adjustment system based on serverless architecture, characterized in that: include: The camera side is used to obtain the video stream; The edge server is used to decode the video stream to obtain continuous video frames, and extract key frames from the video frames based on the video frame rate configuration of the adaptive adjustment model; A serverless platform, configured to adaptively adjust the target detection model configuration and computing resource configuration of the model based on the configuration, so that the target detection model detects the key frame to obtain the target detection result corresponding to the key frame, and uses the target detection result as the target to be tracked to obtain the analysis result of the non-key frame according to the target detection result and the continuous target tracking algorithm; The edge server is used to update the calculation cost and accuracy model based on the target detection result and the analysis result of the non-key frame to update the configuration adaptive adjustment model.

6. The system according to claim 5, characterized in that The serverless platform includes a computing platform and a storage platform; wherein the computing platform includes the target detection model; The storage platform includes at least one neural network model.

7. The system according to claim 5, characterized in that The edge server also includes a video decoding module, a video caching module, a key frame extraction module and a lightweight target tracking module.

8. The system according to claim 7, characterized in that The video caching module is used to cache the key frames and their target detection results according to the target detection results corresponding to the key frames from the serverless platform; The serverless platform is further used to feed back the target detection result to the edge server for target tracking of the target to be tracked.

9. A computer device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images