Video stream joint reasoning method and device based on cloud edge collaboration, medium and equipment

Abstract

The invention provides a video stream joint reasoning method and device based on cloud edge collaboration, a medium and equipment. The method comprises the following steps: performing target detection on a video frame by adopting a first target detection model at an edge end; inputting the video frame into an actor network to obtain a confidence probability; if the confidence probability is greater than the preset probability, sending the video frame and the first target detection result to the cloud to enable the cloud to perform target detection on the video frame by adopting a second target detection model; judging whether the cloud accuracy is higher than the edge accuracy; if yes, outputting the second target detection result to the user; otherwise, outputting the first target detection result to the user; sending feedback information to the edge end; after feedback information is received, an effective unloading ratio is calculated, and state transition information is formed and stored in an experience playback pool; and selecting state transition information from the experience playback pool, and optimizing the actor network. According to the invention, the overall accuracy of target detection can be improved while the real-time performance of target detection is ensured.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a video stream joint reasoning method, device, medium, and equipment based on cloud-edge collaboration. Background Art

[0002] With the large-scale deployment of smart camera devices in 5G IoT scenarios, the requirements for edge real-time video stream reasoning and analysis in target detection scenarios are getting higher and higher in terms of reasoning accuracy and end-to-end latency. Using independent edge nodes or cloud nodes for real-time video stream reasoning and analysis can no longer meet the current complex real-time target detection application requirements. Most of the current solutions that provide edge real-time video stream reasoning use native Kubernetes objects to deploy reasoning tasks, which lacks operation and management flexibility. Most solutions that improve reasoning accuracy by vertically offloading video frames to the cloud are too subjective and difficult to adapt to real-time changing video stream content. Therefore, it is necessary to design a new computing paradigm for edge real-time video stream reasoning and analysis.

[0003] There are many papers on how to combine edge computing with real-time video stream analysis, and edge real-time video stream analysis has been applied to many fields. In the target detection scenario, there are also many papers on how to perform joint reasoning analysis on real-time video streams. The joint reasoning paradigms in most of the work mainly include two types of vertical and horizontal offloading for real-time video frame reasoning tasks.

[0004] In the related research work on vertical offloading of target detection reasoning tasks, when the inference model files used in the cloud and edge are different, resulting in a large difference in the accuracy of target detection reasoning, it is not considered whether the video frames on the edge can significantly improve the reasoning accuracy after being offloaded to the cloud, and it is impossible to effectively and accurately determine whether a certain video frame needs to be offloaded. There are problems such as subjectivity, randomness and roughness when offloading video frames, which will lead to a certain degree of waste of edge network bandwidth and computing power resources, and cannot effectively improve the overall accuracy of real-time video stream target detection reasoning. These research schemes lack certain practicality and application value when offloading at the resource-constrained edge, and it is difficult to ensure the performance robustness and operation efficiency when offloading real-time video stream content that changes dynamically in different regions. In addition, if there is no difference in the target detection reasoning accuracy of the inference model files used by the cloud and edge, the above-mentioned related research work still cannot accurately and effectively determine that all video frame target detection reasoning tasks on the edge do not need to be offloaded to the cloud for further reasoning. If a large number of video frame reasoning tasks are ineffectively offloaded, it will seriously waste resources.

[0005] In response to these situations, under the joint computing architecture based on cloud-edge collaboration, it is urgently necessary to design excellent collaborative reasoning vertical offloading algorithms for real-time video stream target detection joint reasoning tasks to realize the real-time video stream joint reasoning function, so as to improve the overall accuracy of real-time video stream target detection reasoning, enhance the continuous stability, real-time performance, availability and reliability of target detection joint reasoning services during operation, and make up for the shortcomings and performance limitations of using a single edge or cloud node to manage and control real-time video stream target detection reasoning applications. Summary of the invention

[0006] In response to at least one of the above technical problems, an embodiment of the present invention provides a video stream joint reasoning method, device, medium, and equipment based on cloud-edge collaboration.

[0007] According to the first aspect, the video stream joint reasoning method based on cloud-edge collaboration provided by an embodiment of the present invention includes:

[0008] Using a first target detection model at the edge end to perform target detection on a video frame at a current moment in a video stream collected by a video acquisition device, to obtain a first target detection result;

[0009] Inputting the video frame at the current moment into the actor network of the edge deep deterministic policy gradient algorithm to obtain a confidence probability that the video frame at the current moment needs to be offloaded to the cloud;

[0010] Compare the confidence probability corresponding to the video frame at the current moment with the preset probability. If the confidence probability corresponding to the video frame at the current moment is greater than the preset probability, send the video frame at the current moment and the first target detection result to the cloud, so that the cloud uses the second target detection model to perform target detection on the video frame at the current moment to obtain the second target detection result; by comparing the second target detection result with the first target detection result, determine whether the cloud target detection accuracy is higher than the edge target detection accuracy; if so, output the second target detection result to the user as the final detection result; otherwise, output the first target detection result to the user as the final detection result; and send corresponding feedback information to the edge according to the accuracy judgment result;

[0011] After receiving the feedback information, the current effective offloading ratio is calculated, and the confidence probability corresponding to the video frame at the current moment, the current effective offloading ratio, the image features of the video frame at the current moment, and the image features of the video frame at the next moment are stored in the experience replay pool as a set of state transition information;

[0012] A preset number of state transition information is periodically selected from the experience replay pool, and the actor network is optimized according to the selected state transition information.

[0013] In one embodiment, the confidence probability output by the actor network is an initial confidence probability; correspondingly, before comparing the confidence probability corresponding to the video frame at the current moment with the preset probability, the method further includes: adding random noise to the initial confidence probability output by the actor network to obtain a final confidence probability; wherein the random noise is Gaussian noise or Ornstein-Uhlenbeck noise; correspondingly, comparing the confidence probability corresponding to the video frame at the current moment with the preset probability includes: comparing the final confidence probability corresponding to the video frame at the current moment with the preset probability.

[0014] In one embodiment, the method further includes: inputting the video frame at the current moment and the corresponding initial confidence probability into the critic network of the edge deep deterministic policy gradient algorithm, obtaining an evaluation value of the confidence probability output by the actor network, and optimizing the actor network according to the evaluation value.

[0015] In one embodiment, the actor network includes a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a first flattening layer, a first fully connected layer and a first output layer connected in sequence; wherein the first convolutional layer and the second convolutional layer are used to perform convolution processing, the first pooling layer and the second pooling layer are used to perform pooling processing, the first flattening layer is used to convert the image features output by the second pooling layer into a one-dimensional vector, and input the one-dimensional vector to the first fully connected layer, and the first output layer outputs an initial confidence probability based on the output features of the first connection layer; and / or, the commentator network The network includes: a third convolutional layer, a third pooling layer, a fourth convolutional layer, a fourth pooling layer, a second flattening layer, a second fully connected layer and a second output layer; wherein the third convolutional layer and the fourth convolutional layer are used to perform convolution processing, the third pooling layer and the fourth pooling layer are used to perform pooling processing, the second flattening layer is used to convert the image features output by the second pooling layer and the initial confidence probability output by the actor network into a one-dimensional vector, and input the one-dimensional vector to the second connection layer, and the second output layer is used to output the comment value corresponding to the initial confidence probability according to the output features of the second connection layer.

[0016] In one embodiment, the method further includes: if the confidence probability corresponding to the video frame at the current moment is less than or equal to the preset probability, outputting the first target detection result as the final detection result to the user.

[0017] In one embodiment, the process of determining whether the cloud target detection accuracy is higher than the edge target detection accuracy by comparing the second target detection result with the first target detection result in the cloud includes: if the number of targets in the second target detection result is different from the number of targets in the first target detection result, and / or the target type in the second target detection result is different from the target type in the first target detection result, then the cloud target detection accuracy is higher than the edge target detection accuracy.

[0018] In one embodiment, the process of the cloud sending corresponding feedback information to the edge end according to the accuracy judgment result includes: if the cloud target detection accuracy is higher than the edge end target detection accuracy, sending effective unloading feedback information to the edge end; otherwise, sending invalid unloading feedback information to the edge end.

[0019] According to the second aspect, a video stream joint reasoning device based on cloud-edge collaboration provided by an embodiment of the present invention includes:

[0020] A first detection module, configured to perform target detection on a video frame at a current moment in a video stream collected by a video acquisition device using a first target detection model at an edge end, and obtain a first target detection result;

[0021] A confidence determination module, used to input the video frame at the current moment into the actor network of the edge deep deterministic policy gradient algorithm to obtain a confidence probability that the video frame at the current moment needs to be offloaded to the cloud;

[0022] A probability comparison module is used to compare the confidence probability corresponding to the video frame at the current moment with a preset probability. If the confidence probability corresponding to the video frame at the current moment is greater than the preset probability, the video frame at the current moment and the first target detection result are sent to the cloud, so that the cloud uses a second target detection model to perform target detection on the video frame at the current moment to obtain the second target detection result; by comparing the second target detection result with the first target detection result, it is determined whether the cloud target detection accuracy is higher than the edge target detection accuracy; if so, the second target detection result is output to the user as the final detection result; otherwise, the first target detection result is output to the user as the final detection result; and corresponding feedback information is sent to the edge according to the accuracy judgment result;

[0023] An experience collection module, configured to calculate the current effective offloading ratio after receiving the feedback information, and store the confidence probability corresponding to the video frame at the current moment, the current effective offloading ratio, the image features of the video frame at the current moment, and the image features of the video frame at the next moment as a set of state transition information in an experience replay pool;

[0024] The network optimization module is used to periodically select a preset number of state transition information from the experience replay pool and optimize the actor network according to the selected state transition information.

[0025] According to a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, enables the computer to execute the method provided in the first aspect.

[0026] According to a fourth aspect, a computing device provided by an embodiment of the present invention includes a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method provided by the first aspect is implemented.

[0027] The embodiments of the present invention provide a video stream joint reasoning method, device, medium, and equipment based on cloud-edge collaboration. The edge end uses a first target detection model at the edge end to perform target detection on the video frame at the current moment in the video stream collected by the video acquisition device, and obtains a first target detection result; the video frame at the current moment is input into the actor network of the edge deep deterministic policy gradient algorithm to obtain a confidence probability that the video frame at the current moment needs to be unloaded to the cloud; if the confidence probability corresponding to the video frame at the current moment is greater than a preset probability, the video frame at the current moment and the first target detection result are sent to the cloud, so that the cloud uses a second target detection model to perform target detection on the video frame at the current moment to obtain the second target detection result; and the cloud target Whether the target detection accuracy is higher than the target detection accuracy of the edge end; if so, the second target detection result is output to the user as the final detection result; otherwise, the first target detection result is output to the user as the final detection result; and the corresponding feedback information is sent to the edge end according to the accuracy judgment result; after receiving the feedback information, the edge end calculates the current effective offloading ratio, and stores the confidence probability corresponding to the video frame at the current moment, the current effective offloading ratio, the image features of the video frame at the current moment, and the image features of the video frame at the next moment as a set of state transition information in the experience replay pool; regularly selects a preset number of state transition information from the experience replay pool, and optimizes the actor network according to the selected state transition information. It can be seen that the edge end will select a certain number of state transition information from the experience replay pool at regular intervals, and then optimize the actor network according to the selected state transition information, thereby improving the accuracy of the actor network output confidence probability. Moreover, the present invention determines whether the video frame needs to be offloaded to the cloud by comparing the confidence probability that the video frame needs to be offloaded to the cloud with the preset probability. Such an adaptive offloading strategy can ensure the robustness of offloading performance under dynamically changing real-time video streaming content. Target detection through the edge can improve real-time performance, and target detection accuracy can be improved by offloading to the cloud. Since some video frames will be offloaded to the cloud, the waste of computing resources on the edge can be reduced, but not all video frames are offloaded to the cloud, so that too many edge network bandwidth resources are not required. It can be seen that the embodiment of the present invention, based on a joint reasoning mechanism in which the edge provides a low-precision target detection model and the cloud provides a high-precision target detection reasoning model, can achieve complementary reasoning through cloud-edge collaboration. While effectively ensuring the real-time performance of real-time video stream target detection reasoning, it can significantly improve the overall accuracy and reliability of real-time video stream target detection reasoning, and will not significantly increase edge network bandwidth consumption and computing resource waste while improving the overall reasoning accuracy as much as possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flowchart of a video stream joint reasoning method based on cloud-edge collaboration in one embodiment of the present invention;

[0029] Figure 2 This is a structural block diagram of a video stream joint inference device based on cloud-edge collaboration in one embodiment of the present invention. DETAILED DESCRIPTION

[0030] In a first aspect, an embodiment of the present invention provides a video stream joint reasoning method based on cloud-edge collaboration, which is executed by an edge end and includes the following steps S110 to S150:

[0031] S110, using a first target detection model at the edge end to perform target detection on a video frame at a current moment in a video stream collected by a video acquisition device, to obtain a first target detection result;

[0032] For example, a first target detection model is used to perform target detection on a video frame at time t to obtain a first target detection result, which includes target number, target type, target position rectangular frame information, etc.

[0033] S120, inputting the video frame at the current moment into the actor network of the edge deep deterministic policy gradient algorithm to obtain a confidence probability that the video frame at the current moment needs to be offloaded to the cloud;

[0034] Among them, the Edge Deep Deterministic Policy Gradient algorithm is the EdgeDDPG algorithm, that is, the DDPG algorithm executed by the edge. The full spelling of DDPG in English is Deep Deterministic Policy Gradient, and the Chinese is Deep Deterministic Policy Gradient Algorithm. The DDPG algorithm is a reinforcement learning algorithm based on the Actor-Critic mechanism. Actor refers to the actor network and Critic refers to the critic network.

[0035] Among them, the DDPG algorithm can solve both discrete and continuous action problems, and it has unique advantages in terms of convergence of algorithm training, etc. The algorithm has been used in many studies in recent years. Therefore, the embodiment of the present invention takes the DDPG algorithm as a basic guide, and further designs an edge video frame adaptive longitudinal unloading algorithm, namely the EdgeDDPG algorithm, according to the actual application environment of the real-time video stream target detection reasoning task.

[0036] Among them, preprocessing is required before the video frame is input into the model. The preprocessing operation mainly includes converting the edge video frame into RGB format, setting the width and height of the video frame to the specified size, etc. For example, the height and width are unified to 320.

[0037] In one embodiment, the actor network may include a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a first flattening layer, a first fully connected layer and a first output layer connected in sequence; wherein the first convolutional layer and the second convolutional layer are used to perform convolution processing, the first pooling layer and the second pooling layer are used to perform pooling processing, the first flattening layer is used to convert the image features output by the second pooling layer into a one-dimensional vector, and input the one-dimensional vector to the first fully connected layer, and the first output layer outputs an initial confidence probability based on the output features of the first connection layer.

[0038] For example, the initial input data of the Actor network is the preprocessed edge video frame data of size (320,320,3). First, after the first convolution layer (the convolution kernel size of the first convolution layer is 6*3*3, the moving step is 2, and the padding number is 0) and the first pooling layer (the size of the pooling layer is 2*2, and the moving step is 2), an 80*80*6 feature image is output. After the second convolution layer (the convolution kernel size of the second convolution layer is 12*3*3, the moving step is 2, and the padding number is 0) and the second pooling layer (the size of the pooling layer is 3*3, and the moving step is 4), a 5*5*12 feature image is output. Then, a first flattening layer, i.e., a Flatten layer, is connected to convert the 5*5*12 feature image into a one-dimensional vector of size (1,300). Then, a first fully connected layer, i.e., an FC layer, containing 128 neurons is connected to the output layer, i.e., the confidence probability that the edge video frame needs to be unloaded is output.

[0039] In one embodiment, the critic network includes: a third convolutional layer, a third pooling layer, a fourth convolutional layer, a fourth pooling layer, a second flattening layer, a second fully connected layer and a second output layer; wherein the third convolutional layer and the fourth convolutional layer are used to perform convolution processing, the third pooling layer and the fourth pooling layer are used to perform pooling processing, the second flattening layer is used to convert the image features output by the second pooling layer and the initial confidence probability output by the actor network into a one-dimensional vector, and input the one-dimensional vector to the second connected layer, and the second output layer is used to output the comment value corresponding to the initial confidence probability according to the output features of the second connected layer.

[0040] It can be seen that the structures of the critic network and the actor network are similar, except that the second flattening layer not only accesses the output features of the fourth pooling layer, but also accesses the initial confidence probability output by the actor network.

[0041] For example, the initial input data of Critic is also preprocessed edge video frame data of size (320,320,3). The only difference from the Actor network is that the 5*5*12 feature image is converted into a one-dimensional vector of size (1,300) after the second flattening layer, and the number of one-dimensional vectors in the second flattening layer is expanded by 1 to receive the initial confidence probability output by the Actor network. Then, a fully connected layer with 128 neurons is connected after the second flattening layer, and finally a second output layer with a dimension of 1 that can output the comment value, i.e., the Q value, is connected.

[0042] In one embodiment, the method may further include: inputting the video frame at the current moment and the corresponding initial confidence probability into the critic network of the edge deep deterministic policy gradient algorithm, obtaining an evaluation value of the confidence probability output by the actor network, and optimizing the actor network according to the evaluation value.

[0043] It can be seen that after the critic network obtains the comment value, it can know the quality of the confidence probability output by the actor network, and then update the parameters of the actor network according to the comment value to optimize the actor network, thereby improving the accuracy of the confidence probability output by the actor network.

[0044] Wherein, the confidence probability output by the actor network is the initial confidence probability; correspondingly, before executing S130, the method may further include: adding random noise to the initial confidence probability output by the actor network to obtain a final confidence probability; wherein the random noise is Gaussian noise or Ornstein-Uhlenbeck noise; correspondingly, comparing the confidence probability corresponding to the video frame at the current moment with the preset probability in S130 may include: comparing the final confidence probability corresponding to the video frame at the current moment with the preset probability.

[0045] It can be seen that when the EdgeDDPG algorithm is running, in order to increase random exploration, it will adopt the e-greedy strategy, which adds random noise with a certain probability to the initial confidence probability output by the Actor network as the confidence probability that the final edge video frame needs to be unloaded. In this embodiment of the present invention, Gaussian noise or Ornstein-Uhlenbeck noise can be used as random noise.

[0046] Among them, Gaussian noise refers to a type of noise whose probability density function obeys a normal distribution.

[0047] Among them, Ornstein-Uhlenbeck noise is a random process with time-series correlation, which can enable the intelligent agent in reinforcement learning to explore well in an environment with momentum properties.

[0048] S130, comparing the confidence probability corresponding to the video frame at the current moment with the preset probability, if the confidence probability corresponding to the video frame at the current moment is greater than the preset probability, sending the video frame at the current moment and the first target detection result to the cloud, so that the cloud uses the second target detection model to perform target detection on the video frame at the current moment to obtain the second target detection result; by comparing the second target detection result with the first target detection result, determining whether the cloud target detection accuracy is higher than the edge target detection accuracy; if so, outputting the second target detection result to the user as the final detection result; otherwise, outputting the first target detection result to the user as the final detection result; and sending corresponding feedback information to the edge according to the accuracy judgment result;

[0049] The preset probability can be set as needed, for example, set to 0.5.

[0050] It can be seen that if the confidence probability corresponding to the video frame at the current moment is greater than the preset probability, the video frame at the current moment and the first target detection result are sent to the cloud, and the cloud uses the second target detection model to perform target detection on the video frame at the current moment to obtain the second target detection result; then it is determined whether the cloud target detection accuracy is higher than the edge target detection accuracy. If the cloud target detection accuracy is higher than the edge target detection accuracy, this unloading is valid, so the second target detection result is output to the user. If the cloud target detection accuracy is not higher than the edge target detection accuracy, it is considered that this unloading is invalid, so the first target detection result is output to the user.

[0051] In one embodiment, the process of determining whether the cloud target detection accuracy is higher than the edge target detection accuracy by comparing the second target detection result with the first target detection result in the cloud may include: if the number of targets in the second target detection result is different from the number of targets in the first target detection result, and / or the target type in the second target detection result is different from the target type in the first target detection result, then the cloud target detection accuracy is higher than the edge target detection accuracy.

[0052] It can be seen that if the number of targets and / or the type of targets in the second target detection result are different from those in the first target detection result, the cloud target detection accuracy is considered to be higher than the edge target detection accuracy. If the number of targets and the type of targets in the second target detection result are respectively the same as the number of targets and the type of targets in the first target detection result, the cloud target detection accuracy is considered to be not higher than the edge target detection accuracy.

[0053] In one embodiment, the process of the cloud sending corresponding feedback information to the edge end based on the accuracy judgment result may include: if the cloud target detection accuracy is higher than the edge end target detection accuracy, sending effective unloading feedback information to the edge end; otherwise, sending invalid unloading feedback information to the edge end.

[0054] It can be seen that if this is a valid uninstallation, feedback information of the valid uninstallation will be fed back to the edge end. If this is an invalid uninstallation, feedback information of the invalid uninstallation will be fed back to the edge end.

[0055] In one embodiment, the method may further include: if the confidence probability corresponding to the video frame at the current moment is less than or equal to the preset probability, outputting the first target detection result as the final detection result to the user.

[0056] That is to say, if the confidence probability of the video frame at the current moment is less than the preset probability, there is no need to uninstall, and the first target detection result is directly output to the user.

[0057] S140, after receiving the feedback information, calculating the current effective offloading ratio, and storing the confidence probability corresponding to the video frame at the current moment, the current effective offloading ratio, the image features of the video frame at the current moment, and the image features of the video frame at the next moment as a set of state transition information into the experience replay pool;

[0058] Among them, the effective offload ratio refers to the ratio of the number of times the accuracy of edge video frames is improved after being offloaded to the cloud to the total number of times the edge video frames are offloaded to the cloud.

[0059] It can be seen that after receiving the feedback information from the cloud, the edge can know whether the offloading is effective or invalid, and thus update the effective offloading ratio. Then a set of state transition information is formed, and the set of state transition information is stored in the experience replay pool.

[0060] Each set of state transition information contains four pieces of information: the confidence probability corresponding to the video frame at the current moment, the current effective unloading ratio, the image features of the video frame at the current moment, and the image features of the video frame at the next moment. The image features can adopt the one-dimensional feature vector output by the first flattening layer of the actor network, which can reduce the memory usage of directly saving the edge video frame data in RGB format.

[0061] S150, regularly selecting a preset number of state transition information from the experience replay pool, and optimizing the actor network according to the selected state transition information.

[0062] That is, the edge will select a certain amount of state transition information from the experience replay pool at regular intervals, and then optimize the actor network based on the selected state transition information, thereby improving the accuracy of the confidence probability output by the actor network. Of course, the critic network can also be optimized based on the selected state transition information, thereby improving the accuracy of the comment value output by the critic network.

[0063] It can be seen that the embodiments of the present invention can improve the overall reasoning accuracy as much as possible without significantly increasing edge network bandwidth consumption and computing resource waste.

[0064] The present invention determines whether the video frame needs to be unloaded to the cloud by comparing the confidence probability that the video frame needs to be unloaded to the cloud with the preset probability. Such an adaptive unloading strategy can ensure the robustness of the unloading performance under the dynamically changing real-time video stream content. Target detection through the edge can improve real-time performance, and target detection accuracy can be improved by unloading to the cloud. Since some video frames will be unloaded to the cloud, the waste of computing power resources on the edge can be reduced, but not all video frames are unloaded to the cloud, so that too many edge network bandwidth resources will not be required.

[0065] That is to say, under the joint reasoning mechanism based on the edge providing a low-precision target detection model and the cloud providing a high-precision target detection reasoning model, the embodiments of the present invention can achieve complementary reasoning through cloud-edge collaboration, while effectively ensuring the real-time performance of real-time video stream target detection reasoning, it can significantly improve the overall accuracy and reliability of real-time video stream target detection reasoning.

[0066] Based on the method provided by the embodiment of the present invention, the target detection result received by the user is the first target detection result obtained by the first target detection model on the edge, or the second target detection result obtained by the second target detection model on the cloud. Through cloud-edge collaboration, the resources of edge computing and cloud computing are fully utilized, and the flexibility, real-time performance, security, availability, and accuracy limitation of target detection tasks based on real-time video streams are improved when reasoning on the edge.

[0067] The present invention utilizes cloud-native technology to realize real-time video stream target detection joint reasoning based on cloud-edge collaboration, so as to support the automated closed-loop management of real-time video stream target detection joint reasoning tasks, realize flexible control and governance of real-time target detection joint reasoning services, and effectively ensure the low latency, reliability, accuracy, stability and security requirements required for real-time target detection reasoning tasks during operation.

[0068] The embodiment of the present invention designs a cloud-edge collaborative real-time target detection joint reasoning solution based on Kubernetes and KubeEdge environment, solving the current problem of accuracy limitation when using a single edge terminal or cloud server to reason about real-time video streams in a cloud-native environment. Integrating the embodiment of the present invention with various target detection scenarios can create more value for data.

[0069] The present invention takes into account that in the case of real-time video stream content that changes dynamically over time in different regions, it is difficult to artificially measure the performance gap between the target detection model on the edge and the target detection model on the cloud to determine the edge video frames that really need to be unloaded, which is uncertain and unscientific. Unreasonable unloading timing will lead to invalid unloading when the accuracy of the target detection model reasoning on the cloud is not significantly improved compared to the accuracy of the target detection model reasoning on the edge, and the edge video frames that should be unloaded are judged as not needing to be unloaded. Some current edge video frame vertical unloading solutions cannot effectively solve these problems. Therefore, the present invention designs an adaptive unloading strategy for edge video frames to effectively solve the above problems, so as to automatically discover edge video frames that can improve the accuracy of target detection reasoning to a certain extent after being unloaded to the cloud, and improve the overall accuracy and reliability of real-time target detection reasoning tasks as accurately and effectively as possible.

[0070] It can be seen that the embodiment of the present invention is actually an edge video frame adaptive unloading scheme based on DDPG. Compared with the subjectivity and randomness problems existing in some current literature research work schemes when performing unloading judgment processing on edge video frames, the embodiment of the present invention focuses on automatically judging whether each frame of the video stream can truly and effectively improve the target detection accuracy after unloading, and continuously updates the actor network to achieve self-feedback adjustment, and tries to avoid problems such as roughness, subjectivity and randomness when unloading edge video frames. At the resource-constrained edge end, the embodiment of the present invention can effectively improve the overall accuracy and reliability of real-time video stream target detection reasoning, reduce the computing power of edge nodes and the waste of network bandwidth resources, and can adapt to the application requirements of more complex real-time target detection tasks in the future. It can effectively solve the deficiencies and performance limitations existing in some current research work, and has better design theoretical advantages, practicality and application value.

[0071] Since real-time video streams naturally have a certain degree of temporal and spatial locality, the number and position of targets to be detected in multiple continuous edge video frames may not change much. After unloading a certain edge video frame to the cloud, the present invention can effectively determine whether subsequent continuous edge video frames similar to the detection target in the video frame need to be unloaded by comparing the difference in target detection reasoning accuracy between the cloud model and the edge model, and by means of the self-learning and self-feedback mechanism of reinforcement learning, so as to achieve the purpose of automatically determining and automatically adjusting the edge video frame unloading strategy, avoiding human subjective debugging and control of the timing and strategy of video frame unloading.

[0072] The embodiment of the present invention automatically adjusts the adaptive offloading strategy of edge video frames by introducing a reinforcement learning mechanism, and determines the edge video frames that really need to be offloaded as much as possible, thereby improving the effective offloading ratio, that is, the ratio of the number of times the accuracy of edge video frames is improved after being offloaded to the cloud to the total number of times the edge video frames are offloaded to the cloud. While improving the overall inference accuracy as much as possible, it will not significantly increase the edge network bandwidth consumption and computing power resource waste.

[0073] This paper uses the MOT16 dataset to test and analyze the joint reasoning performance of the edge video frame adaptive offloading algorithm, namely EdgeDDPG, under the task of high-volume pedestrian target detection, and uses several edge video frame offloading algorithms integrated in the Sedna open source framework as benchmark algorithms to perform performance comparison analysis with EdgeDDPG. The implementation methods of several video frame vertical offloading benchmark algorithms in the Sedna framework are as follows:

[0074] (1) ThresholdFilter algorithm. This algorithm takes the average confidence of all target detection bounding boxes in the edge video frame as the target detection result confidence of the edge video frame, and compares it with the confidence threshold to decide whether to perform vertical unloading operation on the video frame. The default confidence threshold is 0.5.

[0075] (2) CrossEntropyFilter algorithm. This algorithm considers calculating the entropy of the confidence of all target detection bounding boxes in the edge video frame as the confidence of the target detection result of the edge video frame, and compares it with the confidence threshold to determine the vertical unloading operation of the edge video frame. The default confidence threshold is 0.5.

[0076] (3) IBTFilter algorithm. This algorithm calculates the ratio of the number of bounding boxes that meet the confidence threshold to the number of all target detection bounding boxes in the edge video frame as the confidence of the target detection result of the edge video frame. Then, it determines whether the confidence of the target detection result meets the confidence threshold to decide whether to unload the edge video frame.

[0077] (4) RandomFilter algorithm. This algorithm randomly unloads edge video frames with a preset probability value. The default random probability value is 30%.

[0078] The main steps for joint reasoning performance testing of the four algorithms in Sedna are as follows:

[0079] (1) Use the target detection models Yolov5s and Yolov5m to infer the second video of the MOT16 dataset respectively. After obtaining the pedestrian target detection results, count the video frame numbers where the Yolov5m model has a higher inference accuracy than the Yolov5s model. These video frames are the video frames that really need to be unloaded. There are 600 video frames in the second video of the MOT16 dataset, and there are 453 video frames where the Yolov5m model has a higher inference accuracy than the Yolov5s model, accounting for 75.50% of the total number of frames.

[0080] (2) Create a real-time video stream instance VideoStream-1 and fill in the streaming address of the second video of the MOT16 dataset.

[0081] (3) Create the target detection model file instances AIModel-5s and AIModel-5m, and fill in the Minio object storage addresses of Yolov5s and Yololv5m respectively.

[0082] (4) Create a joint reasoning task Task-1, specify the edge server to be used as EdgeNode-1 and the cloud server to be used as CloudNode-1; specify the real-time video stream instance and target detection model file instance used by the edge server EdgeNode-1 as VideoStream-1 and AIModel-5m respectively; specify the target detection model file instance used by CloudNode-1 as AIModel-5m.

[0083] (5) After EdgeNode-1 completes the target detection reasoning analysis of the second video segment of the MOT16 dataset using the edge video frame vertical offloading algorithm, the video frame numbers actually offloaded to the cloud server CloudNode-1 are counted and compared with the video frame numbers that actually need to be offloaded obtained from step (1).

[0084] (6) Repeat the above steps using the third / fifth / ninth video of the MOT16 dataset to obtain the video frame number comparison results.

[0085] After all the algorithms in Sedna were tested, the average unloading rate, average effective unloading rate, average accuracy improvement rate, average invalid unloading rate and average number of video frames processed for each algorithm on the four videos in the MOT16 dataset were calculated based on the comparison results of the video frame numbers after the test. In addition, the accuracy gap between these algorithms when using the Yolov5m model on the cloud and the Yolov5s model on the edge to perform target detection reasoning on the four videos in the MOT16 dataset is quite large.

[0086] Among them, the average offloading rate is the ratio of all video frames unloaded by the algorithm to the total number of all video frames; the average effective offloading rate is the number of video frames that really need to be unloaded among all video frames unloaded by the algorithm; the average accuracy improvement rate is the ratio of the number of video frames that can improve the target detection reasoning accuracy after unloading to the total number of all video frames obtained from the real-time video stream; the average invalid offloading rate is the ratio of the number of video frames that do not improve the target detection reasoning accuracy after unloading to the number of all unloaded edge video frames; the average number of video frame processing is the number of edge video frames that undergo model reasoning and offloading judgment processing per second when the algorithm is running, which can reflect the efficiency and operating overhead of the algorithm when performing offloading judgment processing on edge video frames.

[0087] The detailed offloading analysis results of several edge video frame vertical offloading algorithms in Sedna are introduced as follows:

[0088] (1) The average offloading rate of the ThresholdFilter algorithm is 25%, the average effective offloading rate is 24.69%, the average accuracy improvement rate is 19.67%, the average invalid offloading rate is 21.33%, and the average number of video frames processed is 3.27 frames.

[0089] (2) The average offloading rate of the CrossEntropyFilter algorithm is 100%, the average effective offloading rate is 100%, the average accuracy improvement rate is 75.5%, the average invalid offloading rate is 24.50%, and the average number of video frames processed is 3.09 frames.

[0090] (3) The average offloading rate of the IBTFilter algorithm is 39.67%, the average effective offloading rate is 39.99%, the average accuracy improvement rate is 30.17%, the average invalid offloading rate is 23.95%, and the average number of video frames processed is 3.15 frames.

[0091] (4) The average offloading rate of the RandomFilter algorithm is 28.17%, the average effective offloading rate is 28.04%, the average accuracy improvement rate is 21.17%, the average invalid offloading rate is 25.30%, and the average number of video frames processed is 3.34 frames.

[0092] Next, the edge video frame adaptive offloading algorithm EdgeDDPG designed by the present invention is tested, and the test steps are consistent with the test of several algorithms in Sedna. After the EdgeDDPG test is completed, the specific offloading analysis results are shown in Table 1.

[0093] Table 1 Offload test analysis results of EdgeDDPG algorithm

[0094]

[0095] A comprehensive analysis of the offloading test results of these four videos in the MOT16 dataset shows that the average offloading rate using the EdgeDDPG algorithm is 70.48%, the average effective offloading rate is 71.31%, the average accuracy improvement rate is 58.26%, the average invalid offloading rate is 15%, and the average number of video frame processing is 2.97 frames.

[0096] In addition, by changing the Actor and Critic network architecture design in EdgeDDPG for multiple times and using Gaussian noise and Ornstein-Uhlenbeck noise in the network respectively, the unloading analysis results of all edge video frames after retesting do not have a very obvious advantage over the above results, and the various result indicators after unloading are also better than several algorithms in Sedna, and the comprehensive unloading performance can be maintained at a good level.

[0097] After comparative test analysis, it can be seen that the EdgeDDPG algorithm has a higher average offloading rate, average effective offloading rate and average accuracy improvement rate than the ThresholdFilter algorithm, IBTFilter algorithm and RandomFilter algorithm. Compared with these three algorithms, EdgeDDPG has the lowest average invalid offloading rate. While greatly improving the inference accuracy, it does not significantly increase the invalid offloading rate, which can effectively avoid the waste of edge network bandwidth and computing resources caused by the invalid offloading of a large number of video frames. In terms of the average number of video frames processed, the EdgeDDPG algorithm can complete the offloading judgment processing and model inference for 2.97 frames of video frames per second, which is not much less than the 3.27 frames, 3.15 frames and 3.34 frames of the other algorithms. Therefore, the EdgeDDPG algorithm does not significantly increase the processing time overhead when making offloading judgments, which is within the ideal and tolerable range. Although EdgeDDPG will slightly increase the processing time overhead during operation, it has a more obvious average accuracy improvement rate. Compared with other algorithms, it can greatly improve the accuracy of target detection reasoning while still having ideal operating performance and efficiency. The overall performance of the EdgeDDPG algorithm is obviously better in terms of discovering edge video frames that really need to be offloaded and offloading efficiency.

[0098] Although the CrossEntryFilter algorithm has the highest average effective offloading rate and average accuracy improvement rate, which can reach 100%, the algorithm unloads all video frames for the four video clips used in the test. In practical applications, if there is no difference in reasoning accuracy between the edge inference model and the cloud inference model, this algorithm is obviously not desirable and will cause serious waste of resources. The embodiment of the present invention believes that this situation is likely to be caused by inappropriate selection of confidence threshold parameters in the algorithm. Therefore, under the dynamically changing real-time video stream content, subjectively determining the relevant parameters in the built-in video frame offloading algorithm in Sedna to ensure offloading performance will be challenging to a certain extent, and lacks reliability and stability.

[0099] The EdgeDDPG algorithm in the embodiment of the present invention has more significant advantages in offloading performance and operating efficiency in actual scene applications than several algorithms in Senda. It avoids the problems of subjectivity, randomness and roughness when judging whether edge video frames need to be offloaded, effectively improves the overall accuracy of real-time video stream target detection reasoning, and reduces resource waste to a certain extent. When facing real-time changing video stream content in different regions, the robustness of offloading performance can be well guaranteed.

[0100] In addition to the edge video frame offloading algorithm in the Sedna framework, several video frame offloading algorithms in the current mainstream literature are also quite representative, such as VideoEdge, DeepDecision, and EdgeOnly, etc. Next, we will perform performance tests and comparisons on them.

[0101] Among them, the VideoEdge algorithm needs to measure the resource configuration relationship between the resource usage requirements and the inference accuracy of the inference model file used in the video frame inference task under a variety of different video frame resolutions offline. When the node CPU resources and network bandwidth resources are certain, the average accuracy of all target detection inference tasks on the edge node is maximized by automatically selecting the optimal resource configuration relationship for different inference tasks. Different from the EdgeDDPG designed on the basis of the architecture of providing a high-precision target detection model on the cloud and a low-precision target detection model on the edge in the embodiment of the present invention, VideoEdge uses the same inference model on the edge and the cloud for each inference task. When the edge node resources meet the requirements, VideoEdge will definitely choose the resource configuration relationship that gives the highest average inference accuracy to all target detection inference tasks. It determines whether the target detection inference of different tasks can be performed on the edge or in the cloud based on the CPU and network bandwidth resource usage of the edge, and can select the best resource configuration parameters and execution location for each target detection inference task to maximize the average inference accuracy. This is different from the architecture mode of the embodiment of the present invention which determines whether to offload edge video frames to the cloud model by comparing the difference in inference accuracy between the two models to maximize the improvement of inference accuracy.

[0102] Among them, the DeepDecision algorithm is roughly the same as the design architecture of the present invention, which will provide a low-precision reasoning model on the local device side and a high-precision reasoning model on the remote server side. However, it does not effectively and accurately determine whether a certain edge video frame needs to be unloaded based on the reasoning accuracy gap between the two models. It only selects unloading variables that make the target detection reasoning accuracy higher as much as possible when resources such as network bandwidth and battery energy consumption change dynamically, and dynamically adjusts the video bit rate and video frame resolution to provide users with the best model parameter selection to maximize the improvement of reasoning accuracy. However, when the reasoning accuracy of the model on the remote server side is not higher than that of the model on the local device side, it will still lead to unnecessary waste of resources. This solution does not effectively and accurately measure the reasoning accuracy gap between the two models, and cannot determine whether the target detection accuracy during reasoning can be improved after the edge video frame is unloaded.

[0103] Whether it is VideoEdge or DeepDecision, if there is no pre-restriction on resources, it will choose the one with the highest accuracy in target detection reasoning from the selectable edge video frame offloading scheme. Therefore, when testing and comparing with EdgeDDPG in the embodiment of the present invention, both of them can offload the edge video frame reasoning tasks to the cloud, and use the high-precision reasoning model in the cloud to maximize the accuracy of real-time video stream reasoning. At the same time, it focuses on comparing with EdgeDDPG in terms of indicators such as average network bandwidth consumption at the edge, time overhead for completing all video frame reasoning tasks, average offloading rate, average effective offloading rate, average accuracy improvement rate, and average invalid offloading rate.

[0104] When testing VideoEdge and DeepDecision, after offloading all pedestrian target detection reasoning tasks for the second, third, fifth, and ninth videos of the MOT16 dataset to the high-precision reasoning model on the cloud, the average network bandwidth consumption of the edge during this period was about 513.6KB / s, and the time overhead for completing all video frame reasoning tasks (i.e., the time it takes for all video frames to be captured and the final target detection reasoning results to be uploaded to the cloud) was about 4419.31 seconds. Its average offloading rate was 100%, the average effective offloading rate was 100%, the average accuracy improvement rate was 75.50%, and the average invalid offloading rate was 24.50%. After testing the EdgeDDPG algorithm, the average network bandwidth consumption of the edge during this period was about 289.5KB / s, and the time overhead for completing all video frame reasoning tasks was about 3625.98 seconds. Its average offloading rate was 70.48%, the average effective offloading rate was 71.31%, the average accuracy improvement rate was 58.26%, and the average invalid offloading rate was 15%.

[0105] In addition, the embodiment of the present invention also tests the strategy of performing reasoning only on the edge, namely the EdgeOnly algorithm, to better illustrate the advantages of EdgeDDPG in improving the overall reasoning accuracy of real-time target detection when there is a large gap in reasoning performance between the cloud and edge models. After the test is completed, the average network bandwidth consumption of the edge during this period is 0KB / s, and the reasoning time overhead for completing all video frame reasoning tasks is about 2481.50 seconds. Its average offload rate is 0%, the average effective offload rate is 0%, the average accuracy improvement rate is 0%, and the average invalid offload rate is 0%.

[0106] After comparative analysis, it can be seen that compared with the strategy of VideoEdge and DeepDecision to offload all edge video frame inference tasks to the high-precision inference model in the cloud, EdgeDDPG has less average edge network bandwidth consumption, shorter time overhead to complete all video frame inference tasks, and more significant improvement in the accuracy of real-time video stream target detection inference. If the performance gap in inference accuracy between the cloud and edge models is not as obvious as in the test environment, and the network bandwidth and computing resources at the edge are also limited, offloading all video frame inference tasks to the cloud will definitely improve the accuracy of target detection inference, but will seriously affect the real-time performance of target detection tasks, and may not meet the latency requirements of user applications, significantly reduce the reliability of target detection inference services, and cause unnecessary waste of resources. Therefore, when the real-time target detection inference results do not need to be completed accurately, when the EdgeDDPG algorithm is running on resource-constrained edge nodes, it has certain theoretical design advantages, practicality and application value, and has excellent comprehensive offloading performance and operation efficiency, compared with the strategy of offloading all edge video frame inference tasks to the cloud without preliminary screening. In actual scenarios, EdgeDDPG or other offloading strategies can be selected based on edge resources and actual application requirements.

[0107] Compared with the EdgeOnly algorithm, which performs reasoning only at the edge, EdgeDDPG has a higher average reasoning accuracy improvement rate, and the average invalid offloading rate is also at a lower level, without significantly wasting edge network bandwidth and computing resources. There is a large gap in the reasoning accuracy performance between the cloud and edge models. At the same time, if the added delay of offloading edge video frames to the cloud is within a tolerable range, EdgeDDPG can greatly improve the accuracy of target detection reasoning, make up for the shortcomings and limitations of reasoning only at the edge, and effectively ensure the operational reliability and accuracy of real-time target detection reasoning applications.

[0108] Based on the above experimental tests and analysis, when there is a large gap in the inference accuracy performance between the cloud and edge models, the EdgeDDPG algorithm in the present invention has a more significant comprehensive advantage in improving the overall inference accuracy of real-time video stream target detection tasks at the resource-constrained edge compared to benchmark solutions such as Sedna, VideoEdge, and DeepDecision. EdgeDDPG also has good performance in terms of time overhead and execution efficiency used during runtime. When facing dynamically changing real-time video stream content in different regions, it can effectively guarantee the effectiveness and performance robustness of edge video frames after unloading, and has stronger practicality and application value.

[0109] In order to better illustrate the performance advantages and design theoretical value of the EdgeDDPG algorithm when vertically unloading edge video frames when there is a large gap in the accuracy of target detection reasoning between the cloud and edge models, the embodiment of the present invention takes the second video in the MOT16 data set as an example to introduce the dynamic changes of the unloading rate, effective unloading rate and invalid unloading rate of the EdgeDDPG, Sedna, VideoEdge and DeepDecision solutions when unloading all video frames in this video. According to the dynamic changes of relevant test indicators, EdgeDDPG has significant advantages in indicators such as unloading rate, effective unloading rate and invalid unloading rate compared with Sedna, VideoEdge and DeepDecision, and the unloading performance and operation stability of the algorithm are better. According to the target detection reasoning accuracy gap between the cloud and edge models, for this video, the unloading rate of EdgeDDPG at runtime is closest to the unloading rate that should be achieved. The number of video frames that should be unloaded in this video to truly improve the accuracy of target detection reasoning is 453 frames, and the unloading rate that should be achieved is 75.50%. EdgeDDPG not only has a higher offloading rate, but also a higher effective offloading rate and the lowest invalid offloading rate. It can effectively and accurately determine whether the edge video frames it offloads can improve the accuracy of target detection reasoning after being offloaded to the cloud, while ensuring the real-time performance of real-time target detection reasoning tasks, thereby avoiding unnecessary waste of resources. In addition, after the same offloading process was performed on the third, fifth, and ninth videos of the MOT16 dataset on EdgeDDPG and Sedna, VideoEdge, and DeepDecision, the comprehensive offloading performance of EdgeDDPG was found to be better through comprehensive comparison of indicators such as the offloading rate, effective offloading rate, and invalid offloading rate. Therefore, it can be proved that EdgeDDPG can effectively guarantee the robustness and reliability in improving the accuracy of target detection reasoning when facing different real-time video stream content. The offloading performance of solutions such as Sedna, VideoEdge and DeepDecision on these videos is not stable. All indicators are inferior to EdgeDDPG. There are significant problems such as roughness and randomness when offloading video frames. It is impossible to effectively and accurately judge whether a certain edge video frame really needs to be offloaded to the cloud. It is difficult to improve the overall accuracy of real-time video stream reasoning and ensure the robustness and reliability of offloading performance.

[0110] The above-mentioned EdgeDDPG and Sedna, VideoEdge, DeepDecision and other benchmark solutions were tested and analyzed for performance comparison. During the test, the inference model files used by the cloud and edge nodes were Yolov5m and Yolov5s respectively. There was a large difference in accuracy when reasoning about the four videos in the MOT16 dataset. EdgeDDPG has a higher average effective offload rate and average accuracy improvement rate, and a lower average invalid offload rate, so its comprehensive offload performance is more significant. If there is no accuracy difference between the inference model files used by the cloud and edge nodes respectively when reasoning about the videos in the MOT16 dataset, all the video frames on the edge should not be offloaded to the cloud at this time, because the offload will definitely be invalid, which will not only fail to improve the accuracy, but also seriously waste the edge network bandwidth resources and computing power resources.

[0111] The embodiment of the present invention also tests and analyzes whether EdgeDDPG, Sedna, VideoEdge, and DeepDecision can accurately determine whether edge video frames still need to be unloaded by setting up an experimental environment in which there is no difference in target detection reasoning accuracy between the cloud and edge models. During the test, the reasoning model files used by the cloud and edge nodes are Yolov5s, and the data sets used are the second, third, fifth, and ninth videos in the MOT16 data set. If there is no difference in the reasoning accuracy performance between the cloud and edge models, if the algorithm determines that the video frames in all videos do not need to be unloaded, that is, the average unloading rate is 0%, the unloading performance at this time is the best, which can effectively avoid the waste of network bandwidth and computing power resources of edge nodes.

[0112] When testing the four algorithms in Sedna, the average offloading rate of the ThresholdFilter algorithm was 25%, the average offloading rate of the CrossEntropyFilter algorithm was 100%, the average offloading rate of the IBTFilter algorithm was 39.67%, and the average offloading rate of the RandomFilter algorithm was 28.17%. These four algorithms all offloaded a large number of video frames, and failed to effectively and accurately determine that all the video frames of the four videos in the MOT16 dataset did not need to be offloaded to the cloud.

[0113] When testing VideoEdge and DeepDecision, since they offload all edge video frames to the inference model files of the cloud nodes, the average offloading rate of the algorithm is 100%, and it is still impossible to effectively and accurately determine that all edge video frames do not need to be offloaded.

[0114] When the EdgeDDPG algorithm in the embodiment of the present invention is tested, the average offloading rate of the algorithm is 11.65%. The number of video frames offloaded to the cloud during operation is relatively small, and it can be effectively and accurately determined that most edge video frames do not need to be offloaded.

[0115] After completing the above tests, the average uninstall rate indicator of the EdgeDDPG algorithm on the four videos was compared with the four algorithms in Sedna, VideoEdge and DeepDecision.

[0116] A comprehensive analysis of the comparison results of the above algorithms in terms of the average offload rate indicator shows that the EdgeDDPG algorithm has the lowest average offload rate indicator. When there is no gap in inference accuracy between the cloud and edge models, it can effectively avoid invalid offloading of edge video frames and effectively reduce the waste of network bandwidth and computing power resources of edge nodes.

[0117] To better illustrate the performance advantages and design theoretical value of the EdgeDDPG algorithm when there is no gap in target detection reasoning accuracy between the cloud and edge models, and when avoiding vertical unloading of edge video frames. The present invention still takes the second video in the MOT16 dataset as an example, and introduces the dynamic changes in the unloading rates of EdgeDDPG, Sedna, VideoEdge and DeepDecision when performing unloading judgment processing on all video frames in this video. Comparing the dynamic changes in the unloading rate indicators, EdgeDDPG has the lowest unloading rate compared to Sedna, VideoEdge and DeepDecision. The dynamic unloading rate of EdgeDDPG has been stably changing at a low level, which can effectively avoid invalid unloading of edge video frames and greatly reduce the waste of edge network bandwidth and computing resources.

[0118] When there is no difference in the inference accuracy between the cloud and edge models and when there is a large difference in the inference accuracy between the cloud and edge models, the comprehensive offloading performance of the EdgeDDPG algorithm is also better. The EdgeDDPG algorithm can not only effectively improve the overall accuracy and reliability of real-time video stream target detection reasoning, but also effectively determine whether the offloading of edge video frames is invalid, which can greatly reduce the waste of edge network bandwidth and computing resources. In actual application scenarios, the performance robustness and stability shown by the self-feedback learning mechanism during offloading are stronger, and the application value and practical value are higher.

[0119] Since the real-time target detection joint reasoning task may need to use large-scale edge nodes during deployment and operation, and allocate different real-time video streams to these nodes, when unloading the real-time video stream content that changes dynamically in different regions, the reasoning performance of the target detection model file for different real-time video streams will be different. If it is still determined in a subjective, random, and rough way whether a certain edge video frame needs to be unloaded, it will be difficult to guarantee the performance and efficiency of unloading large-scale real-time video stream reasoning tasks, and it is impossible to comprehensively and effectively improve the overall accuracy of large-scale real-time video stream target detection reasoning tasks. The EdgeDDPG algorithm in the embodiment of the present invention can dynamically adapt to different real-time video stream contents, can realize the ability to automatically and self-feedback adjust the unloading strategy for different real-time video stream contents, can greatly improve the overall accuracy and reliability of large-scale real-time target detection reasoning tasks, and provide users with reliable real-time target detection reasoning task vertical unloading support.

[0120] It is understandable that the real-time video stream target detection joint reasoning task requires the use of large-scale edge nodes during deployment and operation, and different real-time video streams need to be allocated to these nodes. When unloading the real-time video stream content that changes dynamically in different regions, the EdgeDDPG target detection collaborative reasoning vertical unloading algorithm designed in the embodiment of the present invention is compared with the mainstream algorithm, and it is proved that it can dynamically adapt to different real-time video stream content, and can realize the ability of automatic and self-feedback adjustment of unloading strategies for different real-time video stream content, which can greatly improve the overall accuracy and reliability of large-scale real-time video stream target detection reasoning tasks, and provide users with reliable real-time video stream target detection reasoning task vertical unloading support.

[0121] The EdgeDDPG joint reasoning algorithm can automatically measure the performance gap in inference accuracy between cloud-edge models and make offloading decisions. It can significantly improve the overall accuracy of real-time target detection reasoning tasks while ensuring the real-time performance of edge real-time target detection reasoning tasks, effectively reducing the waste of edge network bandwidth and computing resources. Compared with several widely used mainstream edge video frame vertical offloading algorithms, it has better offloading performance and can effectively discover the real need for a higher effective offloading rate and accuracy improvement rate. At the same time, it will not significantly increase the invalid offloading rate and running time overhead. It has better comprehensive offloading performance on public data sets, which can avoid the subjectivity and uncertainty when manually measuring the performance gap between inference models.

[0122] More specifically, compared with Sedna, a cloud-edge collaboration framework that has attracted widespread attention in recent years, it has initially integrated joint reasoning functions based on cloud-edge collaboration and has been widely used, but it still has certain limitations and shortcomings. The embodiments of the present invention have more advantages than Sedna in terms of joint reasoning functions. When facing different video stream contents, it is difficult to effectively determine the confidence threshold parameters in the algorithm for several edge video frame unloading algorithms provided in Sedna, resulting in uncertainty, subjectivity and randomness in the unloading performance, which is very limited in improving the overall accuracy of real-time video stream target detection reasoning. The method provided in the embodiment of the present invention can automatically adjust the unloading strategy according to the real-time changing video stream content, so as to determine the video frames that can truly improve the accuracy of target detection after unloading to the greatest extent, and can greatly improve the effective unloading rate.

[0123] The deep integration of cloud native technology and artificial intelligence technology is an inevitable trend. Designing a joint reasoning solution for edge real-time video stream target detection can adapt to future artificial intelligence application scenarios with higher requirements for computing power and architecture. The present invention analyzes the pain points of real-time video stream target detection reasoning in a cloud native environment, and uses cloud native technology to propose a joint reasoning algorithm solution to optimize the management cost and efficiency of real-time video stream reasoning, simplify the landing process of target detection joint reasoning service applications, promote the automation level of application execution, and achieve faster updates and launches of joint reasoning services, accelerate the implementation of artificial intelligence innovation in all walks of life, and promote artificial intelligence technology to play a greater social and economic value.

[0124] In a second aspect, an embodiment of the present invention provides a video stream joint reasoning device based on cloud-edge collaboration, see Figure 2 , the device 100 comprises:

[0125] The first detection module 110 is used to perform target detection on the video frame at the current moment in the video stream collected by the video acquisition device using a first target detection model at the edge to obtain a first target detection result;

[0126] A confidence determination module 120, configured to input the video frame at the current moment into an actor network of an edge deep deterministic policy gradient algorithm to obtain a confidence probability that the video frame at the current moment needs to be offloaded to the cloud;

[0127] The probability comparison module 130 is used to compare the confidence probability corresponding to the video frame at the current moment with the preset probability. If the confidence probability corresponding to the video frame at the current moment is greater than the preset probability, the video frame at the current moment and the first target detection result are sent to the cloud, so that the cloud uses the second target detection model to perform target detection on the video frame at the current moment to obtain the second target detection result; by comparing the second target detection result with the first target detection result, it is determined whether the cloud target detection accuracy is higher than the edge target detection accuracy; if so, the second target detection result is output to the user as the final detection result; otherwise, the first target detection result is output to the user as the final detection result; and corresponding feedback information is sent to the edge according to the accuracy judgment result;

[0128] The experience collection module 140 is used to calculate the current effective offloading ratio after receiving the feedback information, and store the confidence probability corresponding to the video frame at the current moment, the current effective offloading ratio, the image features of the video frame at the current moment, and the image features of the video frame at the next moment as a set of state transition information in the experience replay pool;

[0129] The network optimization module 150 is used to periodically select a preset number of state transition information from the experience replay pool and optimize the actor network according to the selected state transition information.

[0130] In one embodiment, the confidence probability output by the actor network is an initial confidence probability; correspondingly, before comparing the confidence probability corresponding to the video frame at the current moment with the preset probability, the confidence determination module is further used to: add random noise to the initial confidence probability output by the actor network to obtain a final confidence probability; wherein the random noise is Gaussian noise or Ornstein-Uhlenbeck noise; correspondingly, the comparing the confidence probability corresponding to the video frame at the current moment with the preset probability in the confidence determination module includes: comparing the final confidence probability corresponding to the video frame at the current moment with the preset probability.

[0131] In one embodiment, the apparatus further comprises:

[0132] The network optimization module is used to input the video frame at the current moment and the corresponding initial confidence probability into the critic network of the edge deep deterministic policy gradient algorithm, obtain the evaluation value of the confidence probability output by the actor network, and optimize the actor network according to the evaluation value.

[0133] In one embodiment, the actor network includes a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a first flattening layer, a first fully connected layer and a first output layer connected in sequence; wherein the first convolutional layer and the second convolutional layer are used to perform convolution processing, the first pooling layer and the second pooling layer are used to perform pooling processing, the first flattening layer is used to convert the image features output by the second pooling layer into a one-dimensional vector, and input the one-dimensional vector to the first fully connected layer, and the first output layer outputs an initial confidence probability based on the output features of the first connection layer; and / or, the commentator network The network includes: a third convolutional layer, a third pooling layer, a fourth convolutional layer, a fourth pooling layer, a second flattening layer, a second fully connected layer and a second output layer; wherein the third convolutional layer and the fourth convolutional layer are used to perform convolution processing, the third pooling layer and the fourth pooling layer are used to perform pooling processing, the second flattening layer is used to convert the image features output by the second pooling layer and the initial confidence probability output by the actor network into a one-dimensional vector, and input the one-dimensional vector to the second connection layer, and the second output layer is used to output the comment value corresponding to the initial confidence probability according to the output features of the second connection layer.

[0134] In one embodiment, the apparatus further comprises:

[0135] The first output module is used to output the first target detection result as the final detection result to the user if the confidence probability corresponding to the video frame at the current moment is less than or equal to the preset probability.

[0136] In one embodiment, the process of determining whether the cloud target detection accuracy is higher than the edge target detection accuracy by comparing the second target detection result with the first target detection result in the cloud includes: if the number of targets in the second target detection result is different from the number of targets in the first target detection result, and / or the target type in the second target detection result is different from the target type in the first target detection result, then the cloud target detection accuracy is higher than the edge target detection accuracy.

[0137] In one embodiment, the process of the cloud sending corresponding feedback information to the edge end according to the accuracy judgment result includes: if the cloud target detection accuracy is higher than the edge end target detection accuracy, sending effective unloading feedback information to the edge end; otherwise, sending invalid unloading feedback information to the edge end.

[0138] It is understandable that the explanation, specific implementation, beneficial effects, examples, etc. of the relevant contents in the device provided in the embodiment of the present invention can be found in the corresponding parts of the method provided in the first aspect, and will not be repeated here.

[0139] In a third aspect, an embodiment of the present invention provides a computer-readable medium, on which computer instructions are stored. When the computer instructions are executed by a processor, the processor executes the method provided in the first aspect.

[0140] Specifically, a system or device equipped with a storage medium can be provided, on which software program code that implements the functions of any of the above-mentioned embodiments is stored, and a computer (or CPU or MPU) of the system or device can be enabled to read and execute the program code stored in the storage medium.

[0141] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0142] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0143] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0144] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion module connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or expansion module is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0145] It is understandable that the explanation, specific implementation methods, beneficial effects, examples, etc. of the relevant contents in the computer-readable medium provided in the embodiment of the present invention can be found in the corresponding parts of the method provided in the first aspect, and will not be repeated here.

[0146] In a fourth aspect, an embodiment of the present specification provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the method in any embodiment of the specification.

[0147] It is understandable that the explanation, specific implementation, beneficial effects, examples, etc. of the relevant contents in the computing device provided in the embodiment of the present invention can be found in the corresponding parts of the method provided in the first aspect, and will not be repeated here.

[0148] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the present invention can be implemented by hardware, software, widgets, or any combination thereof. When implemented by software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0149] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solution of the present invention should be included in the scope of protection of the present invention.

Claims

1. A video stream joint reasoning method based on cloud-edge collaboration, characterized in that: include: Using a first target detection model at the edge end to perform target detection on a video frame at a current moment in a video stream collected by a video acquisition device, to obtain a first target detection result; Inputting the video frame at the current moment into the actor network of the edge deep deterministic policy gradient algorithm to obtain a confidence probability that the video frame at the current moment needs to be offloaded to the cloud; Compare the confidence probability corresponding to the video frame at the current moment with the preset probability. If the confidence probability corresponding to the video frame at the current moment is greater than the preset probability, send the video frame at the current moment and the first target detection result to the cloud, so that the cloud uses the second target detection model to perform target detection on the video frame at the current moment to obtain the second target detection result; and determine whether the cloud target detection accuracy is higher than the edge target detection accuracy by comparing the second target detection result with the first target detection result; If yes, outputting the second target detection result to the user as the final detection result; Otherwise, outputting the first target detection result as the final detection result to the user; And sending corresponding feedback information to the edge end according to the accuracy judgment result; After receiving the feedback information, the current effective offloading ratio is calculated, and the confidence probability corresponding to the video frame at the current moment, the current effective offloading ratio, the image features of the video frame at the current moment, and the image features of the video frame at the next moment are stored in the experience replay pool as a set of state transition information; A preset number of state transition information is periodically selected from the experience replay pool, and the actor network is optimized according to the selected state transition information.

2. The method according to claim 1, characterized in that The confidence probability output by the actor network is the initial confidence probability; Correspondingly, before comparing the confidence probability corresponding to the video frame at the current moment with the preset probability, the method further includes: adding random noise to the initial confidence probability output by the actor network to obtain a final confidence probability; wherein the random noise is Gaussian noise or Ornstein-Uhlenbeck noise; Correspondingly, comparing the confidence probability corresponding to the video frame at the current moment with the preset probability includes: comparing the final confidence probability corresponding to the video frame at the current moment with the preset probability.

3. The method according to claim 2, characterized in that Also includes: The video frame at the current moment and the corresponding initial confidence probability are input into the critic network of the edge deep deterministic policy gradient algorithm to obtain an evaluation value of the confidence probability output by the actor network, and the actor network is optimized according to the evaluation value.

4. The method according to claim 2, characterized in that: The actor network includes a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a first flattening layer, a first fully connected layer and a first output layer connected in sequence; wherein the first convolutional layer and the second convolutional layer are used to perform convolution processing, the first pooling layer and the second pooling layer are used to perform pooling processing, the first flattening layer is used to convert the image features output by the second pooling layer into a one-dimensional vector, and input the one-dimensional vector to the first fully connected layer, and the first output layer outputs an initial confidence probability based on the output features of the first connection layer; And / or, the critic network includes: a third convolutional layer, a third pooling layer, a fourth convolutional layer, a fourth pooling layer, a second flattening layer, a second fully connected layer and a second output layer; wherein the third convolutional layer and the fourth convolutional layer are used to perform convolution processing, the third pooling layer and the fourth pooling layer are used to perform pooling processing, the second flattening layer is used to convert the image features output by the second pooling layer and the initial confidence probability output by the actor network into a one-dimensional vector, and input the one-dimensional vector to the second connected layer, and the second output layer is used to output the comment value corresponding to the initial confidence probability according to the output features of the second connected layer.

5. The method according to claim 1, characterized in that Also includes: If the confidence probability corresponding to the video frame at the current moment is less than or equal to the preset probability, the first target detection result is output to the user as the final detection result.

6. The method according to claim 1, characterized in that The process of determining whether the cloud target detection accuracy is higher than the edge target detection accuracy by comparing the second target detection result with the first target detection result in the cloud includes: if the number of targets in the second target detection result is different from the number of targets in the first target detection result, and / or the target type in the second target detection result is different from the target type in the first target detection result, then the cloud target detection accuracy is higher than the edge target detection accuracy.

7. The method according to claim 1, characterized in that The process of the cloud sending corresponding feedback information to the edge end according to the accuracy judgment result includes: if the cloud target detection accuracy is higher than the edge target detection accuracy, sending effective unloading feedback information to the edge end; otherwise, sending invalid unloading feedback information to the edge end.

8. A video stream joint reasoning device based on cloud-edge collaboration, characterized in that: include: A first detection module, configured to perform target detection on a video frame at a current moment in a video stream collected by a video acquisition device using a first target detection model at an edge end, and obtain a first target detection result; A confidence determination module, used to input the video frame at the current moment into the actor network of the edge deep deterministic policy gradient algorithm to obtain a confidence probability that the video frame at the current moment needs to be offloaded to the cloud; A probability comparison module is used to compare the confidence probability corresponding to the video frame at the current moment with a preset probability. If the confidence probability corresponding to the video frame at the current moment is greater than the preset probability, the video frame at the current moment and the first target detection result are sent to the cloud, so that the cloud uses a second target detection model to perform target detection on the video frame at the current moment to obtain the second target detection result; by comparing the second target detection result with the first target detection result, it is determined whether the cloud target detection accuracy is higher than the edge target detection accuracy; If yes, outputting the second target detection result to the user as the final detection result; Otherwise, outputting the first target detection result as the final detection result to the user; And sending corresponding feedback information to the edge end according to the accuracy judgment result; An experience collection module, configured to calculate the current effective offloading ratio after receiving the feedback information, and store the confidence probability corresponding to the video frame at the current moment, the current effective offloading ratio, the image features of the video frame at the current moment, and the image features of the video frame at the next moment as a set of state transition information in an experience replay pool; The network optimization module is used to periodically select a preset number of state transition information from the experience replay pool and optimize the actor network according to the selected state transition information.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed in a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

10. A computing device, characterized in that The method comprises a memory and a processor, wherein the memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 7 is implemented.

Images