Real-time object detection method for video streaming services in wireless edge networks

Abstract

The present invention discloses a real-time target detection method for video streaming services in wireless edge networks. In view of the dynamics of wireless edge network link quality and the uncertainty of video streaming content, a lightweight standard "frame-anchor frame distance" for measuring the fine-grained dynamics of wireless network links, an integer nonlinear programming model, and a two-stage optimization method are proposed. The method includes: in the first stage, according to the monotonically increasing property of the cumulative tracking error function, the law of optimal average interval selection is revealed, and the selection of unloading frames is guided in turn; in the second stage, based on the unloading frame selection result of the first stage, the resolution of the unloading frame is adjusted, so as to match the link quality with the unloading frame quality, maximize the use of real-time wireless link resources, further improve the accuracy of target detection, and finally improve the detection accuracy under the premise of ensuring the real-time output of the application. The present invention is suitable for the field of edge computing.

Description

Technical Field

[0001] The invention relates to a target detection method, and in particular to a real-time target detection method for a video stream service in a wireless edge network. Background Art

[0002] Vision-based real-time object detection has become a basic service for many smart city applications. It is widely used in fields such as autonomous driving, traffic control, and digital twins. For example, in London's camera network, real-time object detection is often used for intelligent traffic control and safety detection. In autonomous driving systems, vehicles need to detect and track physical objects and road signs to avoid obstacles and navigate autonomously. Object detection usually involves machine learning methods that require a lot of computing resources. Terminal camera devices cannot provide enough computing power and resources, and do not have the computing power to run such applications locally. Therefore, edge-coordinated object detection has become a promising solution.

[0003] In edge-cooperative object detection, the front-end device offloads the captured video frames to a nearby edge server. The server performs the inference process and returns the results to the front-end device. Real-time object detection requires the lowest possible latency, but because video streaming requires huge transmission overhead and computing workload, the end-to-end latency is too large to meet the low latency requirements of the application. For example, analyzing a 720p real-time video at 30FPS (frames per second) requires that the detection results be returned within 33.3ms for each frame. However, the total latency of a frame may be between 100-150ms, and this huge latency cannot meet the requirements.

[0004] In order to reduce the overall latency, the existing approach introduces a detection-tracking architecture that offloads selected frames to the server. Based on the detection results of these selected frames, the front-end device uses a lightweight tracker to locally predict the position of objects in the remaining frames. In this way, the strict latency requirements in real-time object detection can be met.

[0005] However, existing practices often assume that the link is stable, while in reality the link often changes frequently and is highly dynamic. This can lead to unexpected delays or failure to fully utilize link resources, which may significantly reduce detection accuracy and increase total latency.

[0006] In view of this, a method that can reduce the total delay of target detection and improve the accuracy of target detection is urgently needed in the art. Summary of the invention

[0007] In order to solve or alleviate some or all of the above technical problems, the present invention is implemented through the following technical solutions:

[0008] A real-time target detection method for video streaming services in wireless edge networks, in which a front-end device performs local tracking and an edge server performs edge recognition, so that the front-end device outputs the target detection result in real time, selects several key frames to be unloaded to the edge server, and the edge server executes the target detection method to detect the target and returns the recognition result to the front-end device; after the front-end device obtains the result, the tracking method is run locally to track the target object in other frames and predict the target position in the remaining frames; and the front-end device selects the key frame to be unloaded according to the frame-anchor frame distance; wherein the frame-anchor frame distance is defined as: frame The frame-anchor frame distance Defined as: ,in Yes Frame The frame number, Representation frame Anchor frame The sequence number of the anchor frame The result is in the frame Output time The previous frame returned; that is: ;in, Yes Frame Output time; Yes Frame The set of available anchor frames; For Frame The estimated return time of the detection result is calculated by the time the frame is captured, the frame upload time, the edge server processing time, and the download time of the calculation result.

[0009] In one embodiment, the frame with the least accuracy degradation in each window is selected for offloading.

[0010] In one embodiment, selecting the frame with the least degradation in accuracy within each window is based on an objective function:

[0011]

[0012] in, Display Window The set of all frames in , Represents the total resolution defined by the resolution used by all frames. If the resolution of a frame is set to 0, it means that the frame is not unloaded; and,

[0013]

[0014] Representation frame In the window The accuracy degradation, Representation frame Anchor frame Resolution configuration; Indicates a given resolution The corresponding accuracy is Indicates in the window The frame-anchor frame distance The corresponding precision reduction value.

[0015] In one embodiment, the selection of the key frame to be uninstalled is divided into a first stage and a second stage; in the first stage, a fixed resolution configuration is used. To optimize frame selection; in the second stage, the resolution configuration is adjusted based on the results of the first stage to achieve the optimal detection accuracy.

[0016] In one embodiment, the first stage comprises the following steps:

[0017] The front-end device first uses a link quality estimation method to estimate the available bandwidth of the uplink and downlink respectively;

[0018] Then use the historical data to update the function , to incorporate changes in network links and video stream content into accuracy;

[0019] Taking into account the dynamic changes in server resources and the changes in frame calculation amount, the maximum number of unloaded frames is periodically updated in each window. .

[0020] In one embodiment, the first stage further includes the following steps: Select Unload Frame.

[0021] In one embodiment, in the second stage, a basic strategy of unloading at the lowest resolution is obtained, and then the resolution of the frames selected in the strategy is gradually increased.

[0022] In one embodiment, the second stage specifically includes the following steps: first, using the method of the first stage to obtain an unloading decision with the lowest resolution as a benchmark; then, locate the range between the two unloaded frames with the largest cumulative accuracy drop ; After iterating the above two steps to the maximum number of times, the final result is obtained.

[0023] In one embodiment, Representation frame In the window The accuracy of the data is degraded and will be updated periodically in each window.

[0024] In a certain embodiment, the frame-anchor frame distance reflects the impact of dynamic changes in the link.

[0025] In a certain embodiment, the present invention discloses a real-time target detection method for video streaming services in wireless edge networks, which, through local tracking-edge recognition, enables terminal devices with limited computing resources to output complex task target detection results in real time.

[0026] In a certain embodiment, the fine-grained dynamics of wireless network links and the uncertainty of video stream content are taken into consideration to design a real-time video target detection mechanism for link adaptation in dynamic edge networks to perform more efficient offload frame selection, thereby improving detection accuracy while ensuring real-time output of applications.

[0027] In a certain embodiment, the impact of link quality under the local tracking-edge identification mechanism is explored, and a lightweight new metric "frame-anchor frame distance" is proposed to convert the performance impact of real-time link quality on target detection, which is difficult to measure and quantify, into an easily measurable standard, so that the terminal device can take the real-time wireless link quality into consideration when performing frame offloading.

[0028] In a certain embodiment, combined with the aforementioned new metric "frame-anchor frame distance", the frame offloading problem for link adaptive real-time target detection in dynamic edge networks is modeled and formalized as an integer nonlinear programming problem to maximize the average recognition accuracy while meeting strict delay constraints.

[0029] In a certain embodiment, a link adaptive two-stage online frame offloading method is proposed to decide the selection of offloading frames and the regulation of offloading resolution, so that the frame offloading strategy can agilely respond to rapid changes in wireless network links.

[0030] In a certain embodiment, in the first stage of the aforementioned method, based on the monotonically increasing property of the cumulative tracking error function, the rule for selecting the optimal average interval is revealed, which in turn guides the selection of the unloading frame.

[0031] In a certain embodiment, in the second stage of the aforementioned method, based on the unloading frame selection result of the first stage, the resolution of the unloading frame is adjusted, thereby achieving a match between the link quality and the unloading frame quality, maximizing the use of real-time wireless link resources, and further improving the target detection accuracy.

[0032] Some or all of the embodiments of the present invention have the following beneficial technical effects:

[0033] 1) Taking into account the fine-grained link dynamics and uncertainty of video stream content in edge networks, it can adapt to the dynamic changes of links in a timely manner, reduce the overall delay of target detection, and improve the accuracy of target detection.

[0034] 2) By introducing a “detection-tracking” framework for frames in video streams, the present invention can also complete real-time target detection tasks on resource-constrained devices.

[0035] 3) In view of the dynamic nature of wireless network links and the uncertainty of video stream content, the present invention proposes a real-time video target detection mechanism for link adaptation in dynamic edge networks, which selects appropriate frames for offloading, thereby improving the accuracy of target detection while ensuring real-time output.

[0036] 4) The present invention proposes a "frame-anchor frame distance" to measure the impact of link quality under the local tracking-edge recognition mechanism, converting the performance impact of real-time link quality on target detection, which is difficult to measure and quantify, into an easily measurable standard, and models the frame offloading problem of link-adaptive real-time target detection in dynamic edge networks as an integer nonlinear programming problem, maximizing the average recognition accuracy while meeting strict delay constraints.

[0037] 5) The present invention proposes a link-adaptive two-stage online frame offloading method to solve the above-mentioned integer nonlinear programming problem, selects the optimal frame for offloading, maximizes the utilization of real-time wireless link resources, and greatly improves the accuracy of target detection.

[0038] More beneficial effects will be further introduced in the optimized embodiments.

[0039] The technical solutions / features disclosed above are intended to summarize the technical solutions and technical features described in the specific implementation method section, so the scope of the records may not be exactly the same. However, the new technical solutions disclosed in this section are also part of the many technical solutions disclosed in the invention document. The technical features disclosed in this section, the technical features disclosed in the subsequent specific implementation method section, and some contents in the drawings that are not clearly described in the specification, disclose more technical solutions in a reasonable combination with each other.

[0040] The technical solution formed by combining all the technical features disclosed at any position of the present invention is used to support the generalization of the technical solution, the modification of the patent document, and the disclosure of the technical solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a schematic diagram of the local tracking-edge identification framework proposed by the present invention;

[0042] Figure 2 It is a schematic diagram of the "optimal equal distribution" principle that the frame selection method proposed in the present invention relies on. DETAILED DESCRIPTION

[0043] Since it is impossible to describe all alternatives, the following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the key points of the technical solutions in the embodiments of the present invention. For other technical solutions and details not disclosed in detail below, they generally belong to technical goals or technical features that can be achieved by conventional means in the art. Due to space limitations, the present invention will not introduce them in detail.

[0044] Unless it means division, " / " in any position in the present invention means logical "or". The serial numbers such as "first" and "second" in any position in the present invention are only used as distinguishing marks for description, and do not imply an absolute order in time or space, nor do they imply that the terms prefixed with such serial numbers and the same terms prefixed with other attributives must have different references.

[0045] The present invention will describe various key points for combining into various specific embodiments, and these key points will be combined into various methods and products. In the present invention, even if the key points are only described when introducing the method / product solution, it means that the corresponding product / method solution also explicitly includes the technical features.

[0046] When describing the existence or inclusion of a certain step, module, or feature at any position in the present invention, it does not imply that such existence is exclusive and unique. Those skilled in the art can obtain other embodiments based on the technical solution disclosed in the present invention and supplemented by other technical means. Based on the key points described in the specific embodiments of the present invention, those skilled in the art can completely replace, delete, add, combine, and change the order of certain technical features to obtain a technical solution that still follows the concept of the present invention. These solutions that do not deviate from the technical concept of the present invention are also within the scope of protection of the present invention.

[0047] First, a real-time target detection architecture for video streaming services in wireless edge networks proposed by the present invention is introduced. Since target detection usually involves machine learning methods that require a large amount of computing resources, terminal camera devices cannot provide sufficient computing power and resources, and do not have the computing ability to run such applications locally, so edge computing is needed to assist in implementation. Edge computing is a computing paradigm that sinks computing resources to the edge of the network. It deploys edge servers that integrate computing, control, storage resources and network functions at locations close to the user end as a supplement to centralized cloud computing services, solving the problems of long transmission delay and large bandwidth occupancy. For data that originally needed to be transmitted to the cloud for analysis and processing, in the edge network, users can unload their own loads to the edge servers and have them processed by the edge servers. This not only ensures the privacy of user data, but also greatly reduces transmission delays, improves the network's service quality level, and provides a better user experience.

[0048] However, due to the large overhead of video streaming and the amount of computation generated, the end-to-end latency is still high, which cannot meet the requirements for real-time object detection. Therefore, this example selects some key frames to be offloaded to the edge server and uses edge computing to complete the object detection task of the key frames instead of identifying all frames.

[0049] After the edge detects the target object, it returns the result to the front-end device, which then tracks the target object in the next frame. However, due to the uncertainty of the video content, continuous real-time tracking locally will reduce the accuracy of recognition, so edge recognition and local tracking need to be continuously performed to adapt to changes in video stream content.

[0050] That is, this embodiment adopts the local tracking-edge recognition (meaning local tracking and edge recognition, the same below) method, selects some key frames to be unloaded to the edge server, and after the edge server executes the target detection method to detect the target, it returns the recognition result to the front-end device. After the front-end device obtains the result, it runs a lightweight tracking method locally to track the target object in other frames and predict the target position in the remaining frames.

[0051] Figure 1 The basic workflow of the edge detection and terminal tracking framework for real-time video stream object detection applications is demonstrated, where the camera terminal is Capture keyframes at all times And offload it to the edge server. The frame is uploaded to the edge server at the time of of processing time, then spent The processing result is returned to the camera terminal in a certain time. The terminal then performs subsequent tracking based on the detection result.

[0052] In an embodiment of the present invention, a real-time video target detection method that comprehensively considers the fine-grained dynamics of wireless network links and the uncertainty of video stream content is specifically as follows:

[0053] Experiments show that existing schemes that assume stable links cannot achieve expected results in real-world dynamic edge networks. The reason is that frame offloading only occupies a small part of the time window, and the average link quality of the entire window is not enough to evaluate frame-level offloading. When the instantaneous link quality is similar to the expected long-term link quality, this scheme can select the optimal frame. However, when the actual link quality is lower than expected, the offloaded frame may experience a larger delay than expected, which will lead to outdated detection results and further reduction in subsequent tracking accuracy. When the actual link quality is higher than expected, the offloaded frame may miss a better resolution, that is, a lower resolution than the optimal case is selected for offloading, thereby reducing the accuracy of target detection.

[0054] The present invention incorporates link changes into the detection accuracy model to make more efficient offloading decisions. In order to determine the impact of the dynamic nature of the wireless link on edge-based target detection, the present invention designs a new lightweight metric "frame-anchor frame distance" to quantify the performance impact of real-time link quality on target detection, allowing the front-end device to perform a lightweight and link-adaptive frame selection method. In addition, the present invention will regularly update the accuracy degradation of the frame in each window to adapt to the dynamic changes of the video content.

[0055] For the lightweight metric "frame-anchor frame distance" method, the implementation is as follows:

[0056] Existing approaches rely solely on content-level information to select the most appropriate frame for offloading, such as using the number of different pixels between adjacent frames to infer whether a frame needs to be offloaded. However, under different link qualities, the pixel-level differences are significant, which makes it difficult for tracking methods to obtain accurate results.

[0057] In order to determine the impact of the dynamic nature of wireless links on edge-based object detection, a new metric called “frame-anchor frame distance” is designed to quantify the impact of dynamic links on detection accuracy, which is defined as the number of frames between each frame and its previous anchor frame. The "frame-anchor frame distance" The calculation method is as follows:

[0058]

[0059] in, Yes Frame The frame number, Representation frame Anchor frame The sequence number, anchor frame The result is in the frame Output time The previous frame returned, that is:

[0060]

[0061] in, Yes Frame The output time, Yes Frame The set of available anchor frames. For Frame The estimated return time of the detection result can be calculated from the time when the frame is captured, the upload time of the frame, the processing time of the edge server, and the download time of the calculation result. The upload time and the download time are both related to the link quality. Therefore, the "frame-anchor frame distance" can well reflect the impact of the dynamic changes of the link, and it has the advantage of being easy to measure, which enables the front-end device to perform lightweight and link-adaptive selection of key frames to be offloaded.

[0062] Based on different data sets, this paper conducts experiments on the relationship between “frame-anchor frame distance” and tracking accuracy. The experiments show that the tracking accuracy decreases with the increase of “frame-anchor frame distance”, and the relationship between “frame-anchor frame distance” and the decrease of detection accuracy can be quantified as a monotonic convex function. , which represents the “frame-anchor frame distance” The corresponding precision reduction value. The area under the function curve represents the cumulative accuracy drop across all frames.

[0063] Figure 2 Schematic diagram of the "best evenly distributed" principle relied upon by the frame selection method. If all frames are not unloaded, such as Figure 2 As shown on the left, since the video stream content is always updated, it is obvious that the tracking accuracy will continue to decline, and the cumulative value of the accuracy decline is getting higher and higher. If some frames are selected to be unloaded, their subsequent frames will track the new anchor frame and reset their "frame-anchor frame distance", such as Figure 2 As shown on the right, the frame offloading problem is transformed into the problem of finding several segmentation lines to minimize the area under the curve.

[0064] Combined with the aforementioned new metric "frame-anchor frame distance", the frame offloading problem of link-adaptive real-time target detection in dynamic edge networks is modeled and formalized as an integer nonlinear programming problem to maximize the average recognition accuracy while meeting strict delay constraints.

[0065] Experiments show that both the frame-to-anchor distance and frame resolution are important factors affecting detection accuracy. Using the existing resolution-accuracy model, a function Get a given resolution Therefore, the present invention uses the “frame-anchor frame distance” and frame resolution to model the detection accuracy. Representation frame In the window The precision degradation is expressed mathematically as follows:

[0066]

[0067] in, Representation frame Anchor frame Resolution configuration, Indicates in the window The frame-anchor frame distance The corresponding precision drop value. In order to adapt to the dynamic changes of video stream content, this function will be updated regularly in each window.

[0068] In edge-based object detection, we need to select appropriate frames that can achieve the highest detection accuracy for unloading, that is, select some frames with the smallest accuracy degradation in each window for unloading. The objective function is:

[0069]

[0070] The optimization goal of the model is to minimize the overall accuracy drop of all frames. Display Window The set of all frames in , Indicates the total resolution defined by the resolution used by all frames. If the resolution of a frame is set to 0, it means that the frame is not offloaded. In addition, constraints such as server computing power and computing resources need to be met.

[0071] The model has nonlinear objective functions and integer variables. It is proved that the frame selection problem is an NP-hard integer nonlinear programming problem.

[0072] The specific solution to the aforementioned frame offloading problem for link adaptive real-time target detection in dynamic edge networks is a two-stage optimization method. The specific implementation method is as follows:

[0073] In the first stage, the method uses a fixed resolution configuration To optimize frame selection. First, consider the frame selection problem with a fixed frame resolution. In this case, the resolution-accuracy function Becomes a constant, and the accuracy decreases Only with "frame-to-anchor frame distance" However, it is still an integer nonlinear programming problem with a non-differentiable objective function. Based on the intrinsic meaning of frame unloading and "frame-anchor frame distance", the present invention designs a heuristic principle "even partition" to indicate frame selection.

[0074] In the second stage, the method adjusts the resolution configuration based on the results of the first stage to achieve the optimal detection accuracy.

[0075] The two-stage optimization method specifically includes the following two stages and corresponding steps:

[0076] In the first phase, the method consists of two parts: parameter update and offload optimization.

[0077] Step 1.1, parameter updating step;

[0078] The front-end device first uses the existing link quality estimation method to estimate the available bandwidth of the uplink and downlink respectively. Then, the method uses historical data to update the function , to take into account the changes in network links and video stream content. Taking into account the dynamic changes in server resources and the changes in frame calculation amount, the maximum number of unloaded frames is periodically updated in each window. .

[0079] Step 1.2, uninstall optimization step;

[0080] The second part of the method is to divide the current window according to the principle of "even division" Select the unload frame. It includes the following sub-steps:

[0081] Step 1.2.1, the method first calculates the available anchor frame set of the last frame of the current window Considering that the size of the set increases with the increase of frame index, The set of available anchor frames for the entire window. Maximum number of unloaded frames for the window , you can get the number of unloaded frames of the current window , which is exactly the number of partition lines in the transform region partition problem.

[0082] Step 1.2.2, after obtaining the number of division lines, the method uses the "even division" principle to divide use Split the lines and get their positions . is the last anchor frame before the current window, which can be obtained through the history offloading decision.

[0083] Step 1.2.3, since it is not possible to ensure Always close to the current window, we need to determine the first line Specifically, There are two situations for the position of. The location of the division categorizes these two cases.

[0084] In the first case, is a stale anchor frame far away from the current window, i.e. At this time, a frame needs to be unloaded as soon as possible to avoid continuing to track distant old frames, which will lead to a decrease in detection accuracy. Therefore, the method selects the first frame to be unloaded, and selects the corresponding frame for the remaining frames using the "even division" principle.

[0085] In the second case, is a relatively new frame for the current window, that is, At this time, the "even division" principle can be directly used to select the appropriate frame from all the frames in the current window.

[0086] Step 1.2.4, after obtaining the frame division position Then, traverse all frames in the current window and select the ones close to The unloading frame at the location that meets the model constraints.

[0087] Step 1.2.5: For the selected unloaded frames, set their resolution to For other unselected frames, set their resolution to 0, that is, do not select the frame.

[0088] Step 1.2.6, Updated to The last unloaded frame in .

[0089] In the second stage, the method can make the front-end device better utilize dynamic link resources through flexible control of the unloaded frame resolution, thereby obtaining higher detection accuracy. Based on the method of the first stage, the present invention proposes a second stage method - a frame selection method with resolution configuration, which is adaptive to the dynamic changes of the link. The main idea is to obtain a basic strategy of unloading with the lowest resolution, and then gradually improve the resolution of the frames selected in the strategy.

[0090] Step 2.1, first use the first stage method to obtain the offloading decision with the lowest resolution As a benchmark.

[0091] Step 2.2, the method then locates the range between the two unloading frames with the largest cumulative accuracy drop . By adjusting the resolution, you can further optimize The method attempts to offload \left [ {{f}_{i}-β, {f}_{j}-β} \right ] Replace it with a higher resolution frame The method will retain the decision that results in the smallest drop in accuracy.

[0092] Step 2.3, the above steps are This is performed iteratively in rounds to further improve the overall accuracy.

[0093] Although the present invention has been described with reference to specific features and embodiments of the present invention, various modifications, combinations, and substitutions may be made thereto without departing from the present invention. The scope of protection of the present invention is not intended to be limited to the specific embodiments of the processes, machines, manufactures, compositions of matter, devices, methods, and steps described in the specification, and these methods and modules may also be implemented in one or more products and methods that are associated, interdependent, mutually coordinated, and preceding / following stages.

[0094] Therefore, the specification and drawings should be simply regarded as an introduction to some embodiments of the technical solution defined by the attached claims. The attached claims should be interpreted according to the principle of maximum reasonable interpretation, and are intended to cover as much as possible all modifications, changes, combinations or equivalents within the scope of the present invention, while avoiding unreasonable interpretations.

[0095] In order to achieve better technical effects or meet the needs of certain applications, those skilled in the art may make further improvements to the technical solution based on the present invention. However, even if this part of the improvement / design is creative and / or progressive, as long as it relies on the technical concept of the present invention and covers the technical features defined in the claims, the technical solution should also fall within the protection scope of the present invention.

[0096] Some of the technical features mentioned in the attached claims may have alternative technical features, or the order of certain technical processes and the order of material organization may be reorganized. After knowing the present invention, ordinary technicians in this field can easily think of these replacement means, or change the order of technical processes and the order of material organization, and then use basically the same means to solve basically the same technical problems and achieve basically the same technical effects. Therefore, even if the above-mentioned means and / or order are clearly defined in the claims, these modifications, changes, and substitutions should fall within the scope of protection of the claims based on the principle of equivalents.

[0097] In conjunction with the various method steps or modules described in the embodiments disclosed herein, they can be implemented in hardware, software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the steps and components of each embodiment have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application or design constraints of the technical solution. A person of ordinary skill in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered outside the scope of protection claimed by the present invention.

Claims

1. A real-time target detection method for video streaming services in a wireless edge network, wherein the front-end device performs local tracking and the edge server performs edge recognition, so that the front-end device outputs the target detection result in real time, characterized in that: Select several key frames to be unloaded to the edge server. After the edge server detects the target using the target detection method, it returns the recognition result to the front-end device. After the front-end device obtains the result, it runs the tracking method locally, tracks the target object in other frames, and predicts the target position in the remaining frames; Furthermore, the front-end device selects the key frame to be unloaded according to the frame-anchor frame distance; wherein the frame-anchor frame distance is defined as: The frame-anchor frame distance Defined as: ,in Yes Frame The frame number, Representation frame Anchor frame Serial number; Anchor Frame The result is in the frame Output time The previous frame returned; that is: ； in, Yes Frame Output time; Yes Frame The set of available anchor frames; For Frame The estimated return time of the detection result is calculated by the time the frame is captured, the frame upload time, the edge server processing time, and the download time of the calculation result; Select the frame with the least accuracy degradation in each window for offloading; Selecting the frame with the least accuracy degradation within each window is based on the objective function: ； in, Display Window The set of all frames in , Represents the total resolution defined by the resolution used by all frames. If the resolution of a frame is set to 0, it means that the frame is not unloaded; and, ； Representation frame In the window The accuracy degradation, Representation frame Anchor frame Resolution configuration; Indicates the accuracy corresponding to a given resolution r, Indicates in the window The frame-anchor frame distance The corresponding precision reduction value.

2. The real-time target detection method for video streaming services in wireless edge networks according to claim 1, characterized in that: The selection of keyframes to be unloaded is divided into the first stage and the second stage; in the first stage, a fixed resolution configuration is used To optimize frame selection; in the second stage, the resolution configuration is adjusted based on the results of the first stage to achieve the optimal detection accuracy.

3. The real-time target detection method for video streaming services in wireless edge networks according to claim 2, characterized in that: The first stage includes the following steps: The front-end device first uses a link quality estimation method to estimate the available bandwidth of the uplink and downlink respectively; Then use the historical data to update the function , to incorporate changes in network links and video stream content into accuracy; Taking into account the dynamic changes in server resources and the changes in frame calculation amount, the maximum number of unloaded frames is periodically updated in each window. .

4. The real-time target detection method for video streaming services in wireless edge networks according to claim 3, characterized in that: The first stage also includes the following steps: According to the principle of even division, in the current window Select Unload Frame.

5. The real-time target detection method for video streaming services in wireless edge networks according to claim 2, characterized in that: In the second stage, a basic strategy of offloading at the lowest resolution is obtained, and then the resolution of the frames selected in the strategy is gradually increased.

6. The real-time target detection method for video streaming services in wireless edge networks according to claim 2, characterized in that: The second stage specifically includes the following steps: First, the first-stage method is used to obtain the offloading decision with the lowest resolution. As a benchmark; Then, locate the range between the two unloaded frames with the largest cumulative accuracy drop ; After iterating the above two steps to the maximum number of times, the final result is obtained.

7. The real-time target detection method for video streaming services in wireless edge networks according to claim 1, characterized in that: Representation frame In the window The accuracy of the data is degraded and will be updated periodically in each window.

8. The real-time target detection method for video streaming services in wireless edge networks according to any one of claims 1 to 7, characterized in that: The frame-anchor frame distance reflects the impact of dynamic changes in the link.

Images