Real-time video analytics over high-frequency wireless networks

By integrating Wi-Fi control plane and millimeter wave data plane in multi-hop relay networks and optimizing routing and bit rate allocation using stream planning algorithms, the problems of link blocking and delay in existing technology in high-resolution video analysis are solved. , realizes efficient video analysis accuracy and flexibility.

CN115462046BActive Publication Date: 2024-11-22MICROSOFT TECHNOLOGY LICENSING LLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202180031943.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2020-04-29
Filing Date
2021-03-02
Publication Date
2024-11-22
Estimated Expiration
2041-03-02

AI Technical Summary

Technical Problem

Existing Wi-Fi networks have link blocking, latency and throughput problems in high resolution and high frame rate video analysis, and the high throughput of millimeter wave networks relies on clear line of sight and short distances, making it difficult to meet real-time data analysis. need.

Method used

Using a multi-hop relay network, integrating low-latency Wi-Fi control planes with high-frequency data planes, running on hundreds of nodes using scalable stream planning algorithms, optimize network routing and bit rate allocation to maximize data Analyze the accuracy and periodically analyze the bit rate and accuracy relationship of nodes, adjust the video profile to optimize network performance.

Benefits of technology

It realizes reducing link blocking and delay in high-resolution and high frame rate video analysis, improves the overall accuracy and flexibility of the network, adapts to dynamic channel characteristics, avoids the damage of real-time data flow, and ensures efficient video analysis .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115462046B_ABST
    Figure CN115462046B_ABST
Patent Text Reader

Abstract

The multi-hop relay network includes a set of data nodes arranged in a relay topology, wherein the set of data nodes communicate with each other via a data plane including a set of high-frequency data links such as millimeter wave links. The edge node communicates with one or more data nodes via the data plane, and communicates with a set of data nodes on a control plane including a set of low-frequency wireless links such as a Wi-Fi network. The edge node determines path utilization for the set of high-frequency wireless links. When one of the high-frequency wireless links is overutilized, the edge node transmits a command to the data node via the control plane to change the relay topology. When there is no overutilization, the edge node adjusts data analysis parameters for the node to improve the overall accuracy of the network.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0001] Massive data analysis networks, such as camera networks, are driving innovation in smart retail stores, road traffic monitoring, and security applications. Enabling real-time data analysis over these networks is a major challenge that cannot be solved by Wi-Fi and cellular networks alone.

[0002] It is to these and other general considerations that the various aspects disclosed herein relate.In addition, although relatively specific problems may be discussed, it should be understood that the examples should not be limited to solving specific problems identified in the background of this disclosure or elsewhere. Summary of the invention

[0003] The systems and methods described herein relate to a real-time data analysis network system using a multi-hop, high-frequency (e.g., millimeter wave) wireless relay network. To mitigate link congestion, the network integrates a separate low-latency (e.g., Wi-Fi) control plane with the relay data plane, allowing flexible rerouting without suffering the penalty of beam searching for new routes. With the goal of maximizing data analysis accuracy (rather than simply maximizing data throughput), the system uses a novel, scalable flow planning algorithm that runs on hundreds of nodes to simultaneously compute network routes, load balance traffic, and allocate bit rates to each node.

[0004] More specifically, the multi-hop relay network includes a group of data nodes arranged in a relay topology, wherein the group of data nodes communicate with each other via a data plane including a group of high-frequency data links (such as millimeter wave links). The edge node communicates with one or more data nodes in the group of data nodes via the data plane, and communicates with a group of data nodes on a control plane including a group of low-frequency wireless links (such as a Wi-Fi network). The edge node determines the path utilization for the group of high-frequency wireless links. When one of the high-frequency wireless links is overutilized, the edge node transmits a command to change the relay topology to the data nodes in the group of data nodes via the control plane. The edge node also determines whether the relay topology is operating at a target accuracy. When the relay topology is not operating at the target accuracy, the edge node adjusts the data analysis parameters for the node to improve the overall accuracy of the network.

[0005] The present invention summary is provided to introduce some concepts in a simplified form, which will be further described in the following detailed description. The present invention summary is not intended to identify the key features or essential features of the claimed technical solution, nor is it intended to be used to limit the scope of the claimed technical solution. Additional aspects, features, and / or advantages of the examples will be set forth in part in the following description, and in part will be apparent from the description, or can be understood through the practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Non-limiting and non-exhaustive examples are described with reference to the following figures.

[0007] Figure 1 An example of a multi-hop relay network according to aspects of the present disclosure is illustrated.

[0008] Figure 2A An example camera node is illustrated in accordance with aspects of the present disclosure.

[0009] Figure 2B Example edge nodes in accordance with aspects of the present disclosure are illustrated.

[0010] Figure 3A-3C Three alternative topologies of an example multi-hop relay network in accordance with aspects of the present disclosure are illustrated.

[0011] Figure 4

[0013] An example method of determining a resource accuracy profile for a network node in accordance with aspects of the present disclosure is illustrated.

[0012] Figure 5 Illustrated are example methods of determining topology / flow to optimize data analysis accuracy in accordance with aspects of the present disclosure.

[0013] Fig. 6A and Figure 6B Two alternative topologies of an example multi-hop relay network in accordance with aspects of the present disclosure are illustrated.

[0014] Figure 7

[0013] Methods in which a detection system according to aspects of the present disclosure may not be operating with a target accuracy are illustrated.

[0015] Fig. 8A and Figure 8B is a simplified block diagram of a mobile computing device that can be used to practice aspects of the present disclosure. DETAILED DESCRIPTION

[0016] Various aspects of the present disclosure are described more fully below with reference to the accompanying drawings, which are from a portion of the present disclosure and illustrate specific example aspects. However, different aspects of the present disclosure can be implemented in many different forms and should not be construed as being limited to the aspects set forth herein; on the contrary, these aspects are provided to make the present disclosure thorough and complete, and to fully convey the scope of the aspects to those skilled in the art. Various aspects can be practiced as methods, systems, or devices. Accordingly, various aspects can be implemented in hardware, completely in software, or in the form of a combination of software and hardware. Therefore, the following detailed description should not be understood as limiting.

[0017] Video analytics deployments stream video to on-premise edge servers that are equipped with GPUs for executing the visual models involved in their processing. Higher resolution cameras (e.g., 4K or 8K) provide higher video analytics accuracy. For example, the higher the resolution, the more pixels can be used, which generally leads to higher accuracy in identifying objects. The demand for high video analytics accuracy drives higher resolutions (e.g., 4K) and video frame rates (e.g., 60 frames per second). However, these high-resolution cameras are not well suited for traditional Wi-Fi networks. Higher resolutions and frame rates greatly increase the bitrate requirements for video streams. Wi-Fi links do not have enough capacity to stream multiple high-bitrate videos (to edge servers for analysis). In fact, transmitting large video streams over Wi-Fi links can lead to congestion of overutilized Wi-Fi bands and can cause severe interference and latency to other Wi-Fi users. Connecting cameras to wired networks is cumbersome at scale and also reduces deployment flexibility.

[0018] Millimeter wave ("mmWave") networks (e.g., 5G networks) with multi-gigabit / second capacity (e.g., 802.11ad WiGig at 60 GHz) are a promising option for streaming high bitrate video for video analytics. However, mmWave networks have significant limitations: (i) their high throughput depends on clear line of sight, thus making them susceptible to moving objects in buildings, and (ii) their range is short (10-20 meters) and their throughput drops dramatically beyond ca. 10 meters, which is significantly shorter than the typical distance between a camera and an edge server in a typical deployment.

[0019] The present disclosure relates to a real-time data analysis network characterized by data nodes (such as cameras, wireless transceivers / receivers / , cellular phones, etc.) and edge nodes, both of which are equipped with long-range radios such as WiGig radios. A multi-hop relay network of nodes is constructed. The node relays its data stream to one of the edge nodes for analysis and processing. Nodes close to the edge nodes communicate directly with the edge nodes. Nodes in the distance pass their streams to other relay nodes, which in turn forward their streams to the edge nodes. As a result, the network links of the relay nodes close to the edge nodes will experience higher utilization and congestion. These nodes not only send their own data streams to the edge nodes, but also forward data streams from other nodes. In addition, if a node or link in such a network is disconnected due to physical obstruction or interruption, the narrow beam width of the link radio means that the node must search for possible backup reflection paths or backup routes to reconnect the network. This is problematic because the connected node may be busy with ongoing data transmission and may not know that it needs to participate in the search. During such a search, data is not relayed, which inevitably affects throughput, delay, and may cause jitter penalties. In addition, the steps required to broadcast routing information to each node's neighbors in a millimeter wave relay network are challenging because each node's radio must send a beam to each of its neighbors to exchange routing information, which is time-expensive. To address these issues, the present disclosure uses a separate control plane on a lower frequency network such as Wi-Fi, which solves these problems more efficiently than a millimeter wave network for data flow.

[0020] In the relay network of the present disclosure, where nodes provide data flow load to the network, multiple flows need to share a single millimeter wave link. A scalable flow planning algorithm allocates the capacity of the shared link between the various flows and runs on hundreds of nodes with thousands of multi-hop links. For the same reasons as above, a separate control plane distributes instructions to all nodes in the network so that they can implement the routing topology and bit rate allocation generated by the flow planning algorithm.

[0021] Another challenge is the dynamic nature of mmWave wireless channels, which requires a throughput estimation mechanism to avoid disrupting real-time data flows. The performance of a data application may depend on the quality or quantity of data received by the application. For example, the relationship between the bitrate of a video and the accuracy of the video analytics output is calculated dynamically. However, the present invention does not involve continuous and expensive probing traffic for throughput estimation. Given the directionality of WiGig links and the expensive beam search to initiate new links, any continuous probing would severely disrupt the flow of real-time data flows. Instead, a reactive approach to throughput estimation detects when an existing link has reduced throughput or is blocked, and only then probes a new link.

[0022] The system disclosed in this paper periodically analyzes nodes to obtain the relationship between the bit rate of the data stream and its impact on the accuracy of the output. In the process, it weighs the timeliness of the video profile against the throughput cost of sending video segments with the highest frame rate and resolution.

[0023] In the present disclosure, flow planning on a multi-hop relay network is optimized to maximize data analysis accuracy. Therefore, the selected routing and link allocation may not be optimal for network throughput, but will result in better data, such as video, analysis accuracy. In addition, a centralized heuristic uses all link throughputs between nodes and performs global optimization for target accuracy. In all aspects, the target accuracy is the maximum possible accuracy. This contrasts with a decentralized design in which nodes perform neighbor discovery and routing selection by themselves. The heuristic method optimizes both network-layer video stream routing and application-layer video bitrate allocation. Specifically, it first constructs an application-independent routing tree based on the intuition that load balancing between relay nodes produces higher overall video analysis accuracy. Then, it exploits the unique throughput-accuracy trade-off in video analysis (which varies with camera and time), and constructs a mixed integer linear programming (MILP) model to determine the application-dependent bitrate configuration to maximize the accuracy of overall video analysis.

[0024] For the reasons described above, the node reports link throughput to the controller in the edge node as the controller pushes down the routes for the camera to use and the video bitrate to provide. To transmit these control messages, the present disclosure uses a reliable wireless connection, such as, for example, 802.11a / b / g / n / ac / ax Wi-Fi, to enable more reliable propagation of control information. The control protocols have limited capacity overhead, so they do not significantly disrupt existing Wi-Fi traffic. Although specific Wi-Fi and WiGig standards are discussed herein, those skilled in the art will appreciate that the systems and methods disclosed herein can be practiced using other wireless data standards.

[0025] Figure 1An example multi-hop relay network 100 is illustrated in accordance with aspects of the present invention. The network 100 divides messages into two planes - a data plane 130 for streaming data packets (such as video packets) over a high-frequency data link or a high-speed wireless link (such as a millimeter wave network or a 5G network), and a control plane 132 for sending control packets over a low-frequency network or a low-speed wireless link (such as an 802.11a / b / g / n / ac / ax Wi-Fi network). In various aspects, the network 100 has four data nodes C1 104, C2 106, C3 108, and C4 110 that are wirelessly connected to an edge node 102 in a relay topology. In various aspects, the data nodes C1-C4 are cameras capable of sending and receiving video streams over a millimeter wave link via their WiGig radios. In other aspects, the data nodes may be mobile computing devices, such as those in conjunction with Fig. 8A and Figure 8B Those discussed. In various aspects, the edge node 102 is an edge server, but can also be a 5G base station. In a relay topology, each data node can act as both a data producer and a relay for forwarding data streams of other nodes. In a relay topology, not all data nodes are directly connected to the edge node 102. The data node can be connected to the edge node through multiple hops (e.g., multiple relay data nodes and links). Solid arrows 112, 114, 116, and 118 illustrate the millimeter wave links between data nodes C1-C4 and edge nodes 102. For example, node C1 104 sends data packet 120 (e.g., related to video streams, audio data streams, etc.) to node C2 106 on link 112. Node C2 106 sends data packet 122 to node C3 108 on link 114. Node C3 108 sends data packet 124 to node C4 110 on link 116. Node C4 110 sends data packet 126 on link 118. Because network 200 is a network with a relay topology, multiple data streams can be sent on high-speed data links such as mmWave links 120-126. For example, link 114 can send a video stream captured from node C1 104 and node C2 106 to node C3 108. Node C4 sends its own captured video stream and the video streams captured by node C1 104, node C2 106, and node C3 108 to edge node 102.

[0026] In addition to the data plane 130 that uses the millimeter wave link for data flows, the network 100 also includes a control plane 132 that sends control packets 140, 142, 144, and 146 to the edge nodes over low-speed or low-frequency wireless links (such as bidirectional Wi-Fi (dashed lines)) 150, 152, 154, and 156, respectively. In various aspects, the data nodes are directly connected to the edge nodes via low-frequency Wi-Fi links. The overhead of control packets is generally small and therefore does not overwhelm the Wi-Fi network, which is typically used for other functions. By sending control messages on a separate control plane 132, the control messages 140-146 do not disrupt the data flows 120-126 or the data plane 130. Many types of control messages (e.g., packets) can be sent using the control plane 132 that is immediately public. For example, the following types of data packets are sent on the control plane 308: (1) peer discovery packets for discovering nodes during the bootstrap phase; (2) bit rate control packets for reconfiguring the bit rate on the node through the control plane 132; (3) routing control packets for rescheduling the forwarding path for data flows; and (4) network status packets for reporting link throughput and real-time data segments to edge nodes.

[0027] FIG2 illustrates an example camera node 202 according to aspects of the present invention. The camera node 202 is equipped with at least one high frequency band radio 204, such as a WiGig radio, which is capable of transmitting and receiving information over at least a millimeter wave wireless network. Figure 2A A WiGig radio 204 is shown with a transmitter 206 and a receiver 208, in other aspects, the node will have a WiGig radio for transmission and a WiGig radio for reception. The WiGig radio 204 can be backward compatible to transmit high-frequency millimeter wave signals and low-frequency signals for Wi-Fi connections, such as 802.11ac signals. In other aspects, the node 202 also has a Wi-Fi module 211 for sending and receiving Wi-Fi signals in a lower frequency than the millimeter wave network. The node 202 also has a lens 210, an imager 212, a recorder 214, a memory 216, and a processor 218. Optionally, the camera node 202 may also include a video analysis application or engine for analyzing the scene being imaged and streamed and a motion detector 222 for detecting motion in the scene.

[0028] Figure 2B230 . The edge node 230 may be a server or a 5G base station or other type of computing device. The computing device components described below may be applicable to the above computing devices. In a basic configuration, the edge node 230 may include at least one processing unit 232, at least one graphics processing unit (GPU) 234, and a system memory 236. Depending on the configuration and type of the computing device, the system memory 236 may include, but is not limited to, volatile memory (e.g., random access memory), non-volatile memory (e.g., read-only memory), flash memory, or any combination of these memories. The system memory 236 may include an operating system 238 and one or more program tools 240, which are suitable for performing various aspects disclosed herein. For example, the operating system 238 may be suitable for controlling the operation of the edge node 230. In addition, various aspects of the present disclosure may be implemented in conjunction with a graphics library, other operating systems, or any other application and is not limited to any particular application or system. The basic configuration is Figure 2B 244 are illustrated by those components within the dashed line 244.

[0029] The edge node 230 may have additional features or functions. For example, the edge node 230 may also include additional data storage devices (removable and / or non-removable), such as disks, optical disks, or tapes. Such additional storage Figure 2B , illustrated by removable storage device 260 and non-removable storage device 258.

[0030] The edge node 230 is equipped with at least one high frequency band radio 246, such as a WiGig radio, which is capable of transmitting and receiving information over at least a millimeter wave wireless network, such as a 5G network. Figure 2B One WiGig radio 246 is shown with a transmitter 248 and a receiver 250, in other aspects the node will have one WiGig radio for transmitting and one WiGig radio for receiving.

[0031] As described above, many program tools and data files can be stored in system memory 236. When executed on processing unit 232, program tools 240 can perform processes including, but not limited to, various aspects as described herein. For example, edge node 230 includes video analysis application 247. GPU 234 performs DNN reasoning for video processing through video analysis application 247. Other program tools that can be used according to various aspects of the present disclosure can include email and contact applications, word processing applications, spreadsheet applications, database applications, slide presentation applications, drawing or computer-aided applications, etc.

[0032] In addition, aspects of the present disclosure may be implemented using circuits including discrete electronic components, packaged or integrated electronic chips containing logic gates, circuits utilizing a microprocessor, or on a single chip containing electronic components or a microprocessor. For example, aspects of the present disclosure may be practiced via a system on a chip (SOC), wherein Figure 2B Each or more components shown in can be integrated into a single integrated circuit. Such SOC devices may include one or more processing units, graphics units, communication units, system virtualization units, and various application functions, all of which are integrated (or "burned") into the chip substrate as a single integrated circuit. When operated via the SOC, the functions described herein regarding the ability to switch protocols for the client can be operated via application-specific logic integrated with other components of the edge node 230 on a single integrated circuit (chip). Various aspects of the present disclosure may also be practiced using other technologies capable of performing logical operations, such as, for example, AND, OR, and NOT, including but not limited to mechanical, optical, fluid, and quantum technologies. In addition, various aspects of the present disclosure may be practiced in a general-purpose computer or any other circuit or system.

[0033] The edge node 230 may also have one or more (multiple) input devices 256, such as a keyboard, a mouse, a pen, a sound or voice input device, a touch or sliding input device, etc. (Multiple) output devices 254 may also include displays, speakers, printers, etc. The above devices are examples and other devices can be used. The edge node 230 may include one or more communication connections 252, which allow communication with other computing devices 262 (such as other edge nodes in the network). Examples of suitable communication connections 252 include, but are not limited to, Wi-Fi transmitters and receivers, radio frequency (RF) transmitters, receivers and / or transceiver circuits; Universal Serial Bus (USB), and line and / or serial ports.

[0034] The term computer-readable medium as used herein may include computer storage media. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, or program tools. System memory 236, removable storage device 260, and non-removable storage device 258 are all examples of computer storage media (e.g., memory storage). Computer storage media may include RAM, ROM, electrically erasable read-only memory (EEPROM), flash memory or other storage technology, CD-ROM, digital versatile disk (DVD) or other optical storage, magnetic tape, magnetic tape, disk storage or other magnetic storage device, or any other product that can be used to store information and can be accessed by edge node 230. Any such computer storage medium may be part of edge node 230. Computer storage media does not include carrier waves or other propagated or modulated data signals.

[0035] Communication media may be embodied by computer readable instructions, data structures, program tools, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery media. The term "modulated data signal" may describe a signal that has one or more characteristics set or changed in a manner that encodes information in the signal. By way of example and not limitation, communication media may include wired media, such as a wired network or direct wired connection, and wireless media, such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0036] Figure 2B The edge node 230 in may be a master server, in which case it also includes a controller 264, such as Figure 1 The controller 106 shown in FIG. 264 handles the topology and global optimization or the entire network. In addition, the controller distributes the video processing workload among the other edge nodes 230 in the network.

[0037] For example, the controller 264 uses the optimizer 270 to decide which nodes use which links and which flows go to which edge nodes, etc. In various aspects, the controller 264 is not optimized to maximize total throughput, but rather to maximize overall data analysis accuracy. Video analysis is an example of a data analysis application. When performing video analysis, the accuracy of the analysis is related to frame rate and resolution. That is, streaming at a certain frame rate and resolution will produce a certain accuracy, which is also proportional to throughput. The higher the accuracy, the more throughput is required, and the lower the accuracy, the less throughput is required. The controller must also consider what the camera is capturing. The more activity occurs in the scene being streamed, the higher the resolution required to improve the accuracy of the video analysis. In other words, when there is a lot of activity in the scene being streamed, the accuracy will change significantly based on the resolution. On the other hand, if the scene of the camera is relatively empty, the accuracy will not change much between lower and higher resolutions. Therefore, the controller not only considers the throughput of each camera in the system, but also considers the relationship between the scene and natural throughput and accuracy. It uses this information to try to optimize the highest overall accuracy for all cameras in the network.

[0038] When the optimizer 270 determines that a change to the network topology is needed, the network scheduler 268 reschedules the network flows. As described in more detail below, changes may need to be made in any number of situations, including but not limited to interfering links, equipment problems, over-utilized links, and scene changes that require greater or less video analytics accuracy. Figure 4 As described in more detail in , the network scheduler also schedules periodic checks of the video accuracy profile from each node and stores the resource accuracy profile 266.

[0039] Figure 3A-3C Various example topologies (eg, which nodes are currently connected to which nodes) are shown in accordance with aspects of the present disclosure.

[0040] Figure 3A-3CA building 302 is illustrated with eleven nodes C1 330, C2 332, C3 334, C4 316, C5 318, C6 320, C7 322, C8 324, C9 326, CIO 328, and C11 330 and one master edge node 304. Cameras C10 328 and C11 330 are located in a room 170 within the building 302 so that camera C11 330 does not have a line of sight to the edge node 304. Although eleven nodes and one edge node are shown, any number of nodes and edge nodes may be deployed as part of the instant disclosure. Typically, the edge nodes are interconnected via a wired network (not shown). Solid unidirectional arrows indicate active millimeter wave links between nodes. Dashed bidirectional arrows indicate possible but unutilized millimeter wave links between nodes, which can move in either direction. A node can use a link to send data packets (such as a video stream) to another node - this is called an upstream link. A node can also receive data packets from multiple nodes - these are called downstream links. In this way, Figure 3A-3C The topology shown in forms a tree having a root node including edge node 304 .

[0041] In some aspects, a particular topology is centrally controlled by a controller 306, rather than autonomously controlled by each node. Although the controller 306 is shown as part of the edge node 304, it can reside anywhere in the system, including in a 5G base station. The node is responsible for reporting to the controller 306 which nearby millimeter wave links are available and which are not available. In the initial setup (e.g., the boot phase), the system performs a one-time scan via the data plane to obtain the throughput of all links. The throughput is sent to the controller 306 via the control plane and stored at the end node. When an active link is disconnected, the controller 306 automatically selects the next best link based on the information from the initial scan. After the system starts using the suboptimal link, if the measured throughput is different from what was shown in the previous measurement, the updated and actual throughput is sent to the controller 306 via the control plane and stored to replace the outdated throughput.

[0042] For example, in Figure 3A, camera C4 316 may report to controller 306 that it has 5 possible mmWave links, that it is using link 350 as an upstream link to connect to edge node 304, that it is connected to node C5 318 via downstream link 355, and to node C9 326 via downstream link 362, and that links 352 and 364 are available. As another example, node C5 318 may report to controller 306 that it is using upstream link 355 to connect to node C4 316, that it is connected to node C6 320 via downstream link 338, and to node C7 322 via downstream link 336, and that links 342 and 344 are available. Node C1-Node C11 on control plane 132 ( Figure 1 In various aspects, the network scheduler in the controller is responsible for managing the collection of this information. In addition to reporting which neighboring links are available and which are not available, the node also reports the throughput of each link it is utilizing on the control plane.

[0043] In various aspects, a network includes multiple edge nodes, one of which is designated as a master edge node. Each edge node is connected to its own relay topology of data nodes (e.g., a routing tree), and the edge nodes are connected together via communication links, such as wired or wireless network connections. The master edge node receives data streams (data packets) from data nodes that are not in its own relay topology via communication links from edge servers connected to the sending data nodes. In various aspects, the master edge node sends and receives control packets to and from data nodes outside of its own relay topology in the same manner. In other aspects, all data nodes are directly connected to the master edge node over a low-frequency wireless network, such as Figure 1 In the control plane 132, in this example, the master edge node sends and receives control data packets directly from all data nodes via the control plane.

[0044] Figure 4 An example method of managing resource accuracy profiles for a node according to aspects of the present disclosure is illustrated. In various aspects, method 400 may be performed by Figure 2B The network scheduler 268 of the controller 264 is shown to execute. The network scheduler 268 schedules each node in the network to periodically send samples of its highest resolution data stream to the controller of the master edge node on its current mmWave link in a round-robin manner.

[0045] The general sequence of operations for method 400 is as follows: Figure 4 Typically, method 400 begins with start operation 402 and ends with operation 422. Method 400 may include more or fewer steps, or may be arranged in a manner similar to that of FIG. Figure 4The method 400 may be performed as a computer-executable instruction set executed by a computer system and encoded or stored on a computer-readable medium. In addition, the method 400 may be performed by gates or circuits associated with a processor, ASIC, FPGA, SOC, or other hardware device. Hereinafter, the method 400 will be referred to in conjunction with Figure 1 Figure 2A , Figure 2B , Figure 3A-3C , Figure 5 , Fig. 6A , Figure 6B , Figure 7 , Fig. 8A ,and Figure 8B The described systems, components, devices, modules, software, data structures, data characteristic representations, signaling diagrams, methods, etc. are explained.

[0046] Method 400 begins at operation 402, where a controller receives a request from a first node (such as Figure 3A Node C1310 shown in FIG. 1 receives a high-resolution data stream. The first node sends the highest quality data stream it is able to send at the time. In various aspects, the first node sends the data stream samples in response to receiving a control packet from a scheduler in a controller (not shown) on a control plane.

[0047] At operation 404, the controller reduces the sample to a lower quality stream and determines the relationship between the accuracy of each quality level, the quality analysis (e.g., resolution, frame rate in the example of the video stream) and the throughput. The quality analysis or parameter (e.g., knob) will depend on the type of data stream sent and is collectively referred to as the knob index. The higher the knob index, the higher the quality of the data stream. In various aspects, the knob index for the video stream can consider resolution and / or frame rate in other analyses. For example, node C1 can send a video stream with 8K resolution. The controller will downgrade the video stream to various resolutions (e.g., 4K, 1080p, and 720p) and determine the accuracy of each resolution and throughput, including the resolution and throughput of the received samples. Additionally or alternatively, the controller will change the frame rate of the video stream and determine the accuracy and each frame rate and throughput, including these aspects of the received samples. These determinations are referred to as resource accuracy profiles in this article. In various aspects, the video analysis engine 247 uses DNN (deep neural network) reasoning and non-DNN operations to determine the resource accuracy profile. The resource accuracy profile can vary greatly depending on what is happening in the scene being imaged in the samples sent. For example, when there is a lot of motion happening in the scene, the accuracy can vary greatly. On the other hand, if there is minimal motion happening in the scene, the accuracy may not change much with increasing resolution and / or throughput.

[0048] Example asset accuracy profile for an active scene

[0049] Knob Index 1 2 3 4 Resolution 720p 1080p 4K 8K Throughput 60 200 540 900 Accuracy 35% 60% 93% 99%

[0050] Example asset accuracy profile for quiet scenes

[0051] Knob Index 1 2 3 4 Resolution 720p 1080p 4K 8K Throughput 10 20 35 150 Accuracy 75% 80% 93% 99%

[0052] In various aspects, the video analytics engine runs a cascade of operations to determine the resource-accurate profile. The background subtraction module is first called on the downsampled frames as an early filter to detect changes in each frame. If this module detects motion in the "region of interest", it calls a DNN model (e.g., YOLOv3 for object detection) executed on the GPU to verify that there is indeed an object of interest. It then uses this information to calculate the resource-accurate profile more quickly and efficiently.

[0053] In various aspects, operations 402 and 404 may occur only at the master edge node. However, in other aspects, the other edge nodes may collect data samples from their connected nodes and calculate a resource accuracy profile for each of their connected nodes. In this aspect, the other edge nodes send the resource accuracy profile to the master edge node for storage. At optional operation 406, the resource accuracy profile is stored at the master edge node so that the controller can access it for use in conjunction with Figure 5 The optimization operations described. In addition, resource accuracy profiles can be stored at the edge nodes to which they are connected.

[0054] At operation 408, the edge node receives a request from the next node in the network (such as Figure 3A Node C2 in the edge node 312 receives high-quality data samples (e.g., high-resolution video streams). In various aspects, the next node sends stream samples in response to receiving a control packet from a scheduler in a controller (not shown) on the control plane. At operation 410, the edge node calculates a resource accuracy profile for the next node as described in operation 404. The resource accuracy profile can optionally be stored at the master edge node at operation 412.

[0055] At decision 414, it is determined whether there is an edge node of another node that has not received a sample in this round of sampling. If the answer is "yes", method 400 moves to operation 408 to collect / receive high-quality samples for the next node, and the process returns to decision 414 through operations 408-412 until the answer is "no"-no edge node of the node has not received high-quality samples in this round of sampling. If the answer to decision 414 is "no", method 400 proceeds to optional decision 416 to determine whether the edge node has received any unscheduled samples from any of its connected nodes. As discussed in conjunction with Figure 2, video nodes can be equipped with optional motion detectors and optional video analysis applications, which allow them to detect changes in the scene they are imaging. In this case, the video node can send high-resolution samples to the edge node, even if it is not scheduled to do so. That is, even if it does not receive a control data packet indicating that it sends high-resolution samples on the control plane. In this case, the answer to decision 416 is "yes", and method 400 proceeds to operation 418. At operation 420, the edge node reduces the size of the samples from the reporting node and determines a resource accuracy profile as described in conjunction with operation 404. In optional operation 422, the profile is stored at the master edge node (either directly or after being sent from another edge node), and the method returns to operation 402. If the answer to decision 416 is "no," or the system is not capable of receiving unscheduled video samples, the method returns to operation 402 to begin the loop sampling method 400 again. Although Figure 4 Various aspects of the present invention have been described with respect to the transmission of video streams, but those skilled in the art will appreciate that other types of data and data streams may be practiced using the exemplary systems and methods disclosed herein.

[0056] Figure 5 An example method 500 is illustrated to optimize topology or flow control to maximize the accuracy of data analysis for nodes in a network according to aspects of the present disclosure. In various aspects, the method 500 may be performed by, for example, Figure 2B Executed by optimizer 270 of controller 264. In various aspects, method 500 is divided into two phases: applying independent stream routing 501 (involving operations 502-510) and applying specific bitrate allocation 513 (involving operations 512-516), both of which are intended to improve the overall analysis accuracy of the network.

[0057] The general sequence of operations for method 500 is as follows: Figure 5 Typically, method 500 begins with a start operation 502 and ends with an end operation 520. Method 500 may include more or fewer steps, or may be arranged in a manner similar to that of FIG. Figure 5The method 500 may be performed as a computer-executable instruction set executed by a computer system and encoded or stored on a computer-readable medium. In addition, the method 400 may be performed by gates or circuits associated with a processor, ASIC, FPGA, SOC, or other hardware device. Hereinafter, the method 500 will be referred to in conjunction with Figure 1 Figure 2A , Figure 2B , Figure 3A-3C , Figure 4 , Fig. 6A , Figure 6B , Figure 7 , Fig. 8A ,and Figure 8B The described systems, components, devices, modules, software, data structures, data characteristic representations, signaling diagrams, methods, etc. are explained.

[0058] At optional operation 502, the optimizer generates an initial relay topology (e.g., a routing tree) based on the shortest path (least hops) between each node and the nearest edge node in the network. In the relay topology, not all data nodes are connected to edge nodes. Instead, multiple data nodes are connected to data nodes in a tree structure, which are ultimately connected to edge nodes. The initial relay topology may look like Figure 3A In a topology, each node has at most 2 hops (e.g., connections to other data nodes) to reach edge node 304. For example, node C8 324 has two hops (node ​​C9 326 and node C4 316) to reach edge node 304. Figure 1 As discussed, instructions for which node to connect to which node are sent from the controller to each node via control packets on the control plane. When there are multiple edge nodes distributed on the network, method 500 establishes a set of routing trees with balanced network traffic rooted at each edge node. Operation 502 generates M initial topologies (e.g., routing trees) for M edge nodes using the same shortest path algorithm described above. In order to balance the size of the M routing trees (the number of nodes in each tree), the M edge nodes take turns selecting nodes into their routing trees based on the shortest path metric. Nodes selected by one edge node will not be selected by other nodes.

[0059] At operation 504, path utilization is calculated for each path in the initial topology. In the case of multiple edge nodes, path utilization is calculated for all paths of all topologies (e.g., routing trees) in the network. In various aspects, path utilization is calculated by an optimizer in the controller of the primary edge node. For a node, the amount of data traffic that needs to be transmitted is equal to the sum of its own traffic plus the traffic from all its descendant (e.g., downstream) nodes. As an example, the more downstream nodes connected to a node, the higher the utilization on the upstream path of the node. In various aspects, the utilization of each path should be less than 100%. Load balancing the routes on all links / paths can improve the overall accuracy of the network.

[0060] Decision operation 506 determines whether any paths are over-utilized, e.g., have a utilization greater than 100%. In various aspects, the goal is to reduce bottlenecks in the system to improve the overall accuracy of the system. Return Figure 3A , path 350 is likely overutilized because it transmits video streams from five nodes to edge node 304: node C5 318, node C9 326, node C8 324, node C6 320, and node C7 322. In contrast, link 332 transmits video streams from one node C2 312 to edge node 304. If the answer to decision 506 is "yes," the method proceeds to operation 508 to look for possible rerouting to reduce bottlenecks and improve overall accuracy in the topology. Method 500 alleviates radio utilization bottlenecks by relocating cameras to other paths along the current most utilized path. For example, if Figure 3B As shown, by changing node C7 322 from using link 336 to link 354, the utilization of path 350 is reduced. Node C7 322 is now connected to edge node 304 through nodes C2 312 and C1 310, rather than being connected to edge node 304 through node C4 316. The control plane is used to send instructions to node C7 322 to connect to node C2 312 on link 354, rather than connecting to node C4 on link 336. Alternatively or additionally, Figure 3C As shown, node C6 320 can be moved from link 338 to link 356, which will also cause it to be connected to edge node 304 via link 332, rather than over-utilizing link 350. Again, instructions for rerouting are sent on the control plane via control packets. Rerouting is selected to maximize the accuracy of the analysis (e.g., video detection analysis) rather than throughput.

[0061] Once a re-route is determined at operation 508, method 500 proceeds to operation 510 to update the topology via instructions sent in control packets on the control plane, and then proceeds to decision 506 to determine if any paths are over-utilized in the new topology. If the answer is "yes," method 500 continues to determine another re-route, update the topology via the control plane at operation 510, and test for over-utilization at decision 506 until the answer is "no" or it is not possible to route the network further to reduce over-utilization of any path.

[0062] In a network with multiple edge nodes, operations 506, 508, and 510 are iteratively performed by the master edge node for each routing tree (e.g., relay topology) of each edge node, so that each relay topology for each edge node has no over-utilized paths, or it is not possible to route the network to further reduce any path over-utilization. In this way, the overall accuracy within the network (e.g., the relay network connected to an edge node) and between networks (all paths in the entire network including all edge nodes) is improved.

[0063] If the answer to decision 506 is "no," then method 500 moves from the apply independent stream routing stage 501 to the apply specific bit rate allocation stage 513. Figure 4 512, method 500 proceeds to termination 518. If the answer at decision 512 is "no," method 500 continues to find possible knob adjustments from the resource accuracy profiles of the node(s) to improve the overall accuracy. In other words, throughput can be adjusted for different nodes and different paths to improve the overall accuracy of the entire network. For example, reducing the knob for a node (thereby reducing flow quality, accuracy, and throughput) may improve the overall accuracy of the network (all nodes combined). See below for more information. Fig. 6A and Figure 6B Explain the overall accuracy determination. The method 500 finds a replacement knob for the node(s) at operation 514, updates the knob via the control plane at 516, and returns to decision 514 to determine whether the network is operating at the target overall accuracy. If the answer is still "no," the method 500 continues through loops 514 to 516 to 512 until the answer at decision 512 is "yes."

[0064] In a network with multiple edge nodes, operations 512, 514, and 516 are performed by the master edge node for the entire network, thereby improving the overall accuracy of the entire network. Also in a network with multiple edge nodes, method 500 also takes into account the computing power on each edge node (not shown). That is, each edge node requires sufficient computing power to operate at the incoming rate of data, which is transmitted to the edge node from the data nodes directly connected to the edge node over the relay topology of the edge node.

[0065] Fig. 6A and Figure 6B An example relay topology for determining overall accuracy for a network is illustrated in accordance with aspects of the present disclosure. Fig. 6A and Figure 6B Both illustrate multi-hop relay networks 600 and 630, respectively, each containing four nodes C1 610, node C2 612, node C3 614, and node C4 616, and edge node 604. Solid unidirectional arrows designate active mmWave links as part of the data plane—each labeled with its maximum capacity. Dashed lines illustrate possible but unutilized mmWave links. Capacity (e.g., throughput) for each link is shown in Mbps.

[0066] like Fig. 6A As shown, node C1 610 is connected to edge node 604 via link 620, which has a throughput of 400 Mbps. Node C2 612 is connected to node C1 610 via link 622, which has a throughput of 1000 Mbps. Node C3 614 is connected to edge node 604 via link 628, which has a throughput of 1000 Mbps. Node C4 616 is connected to node C3 614 via link 626, which has a capacity of 200 Mbps. There is a possible but unused link 624 between node C2 612 and node C3 614, which has a capacity of 1000 Mbps. Figure 4 The system has determined the procedures outlined in Fig. 6A The resource accuracy profile for nodes C1-C4 is shown in Table 1:

[0067] Table 1

[0068] Resource Accuracy Profile for Nodes C1-C4

[0069] Knob Index 1 2 3 4 5 6 7 Bit rate (Mbps) 12 24 60 120 210 540 900 Accuracy 10% 35% 60% 77% 88% 95% 99%

[0070] As mentioned above, the higher the knob index, the higher the analysis parameters such as resolution and frame rate. Figure 5The system sets the four nodes at the following knobs, which results in the highest overall accuracy for the entire network (all nodes combined), as shown in Table 2:

[0071] Table 2

[0072]

[0073] In all respects, the overall accuracy of the network can be determined by taking the average accuracy of all nodes, which is Fig. 6A The topology is 79.75%. However, the present disclosure is not limited to taking an average to determine the overall accuracy. Any metric that combines the accuracy of nodes in the network can be used, including, for example, taking the median or average of all accuracies or maximizing the minimum accuracy of the least accurate node.

[0074] With the knobs set according to Table 2, no path is overutilized and the network operates at the target average accuracy of all nodes in the network. For example, link 620 has a capacity of 400 Mbps and transmits data streams from node C1 610 (throughput 120) and node C2 612 (throughput 120), the sum of node C1 610 and node C2 612 is less than the capacity of link 620 (400 Mbps).

[0075] In the example, Fig. 6A A scene change for node C4 616 in may cause a change in the resource accuracy profile for that node. In this example, the scene for node C4 616 has become quiet, with little or no change in the imaged frame. The controller will Figure 4 The new resource accuracy profiles for nodes C1-C4 are shown in Tables 3 and 4:

[0076] Table 3

[0077] Resource Accuracy Profile for Nodes C1-C3

[0078] Knob Index l 2 3 4 5 6 7 Bit rate (Mbps) 12 24 60 120 210 540 900 Accuracy 10% 35% 60% 77% 88% 95% 99%

[0079] Table 4

[0080] Resource Accuracy Profile for Node C4

[0081] Knob Index 1 2 3 4 5 6 7 Bit rate (Mbps) 2 4 10 20 35 90 150 Accuracy 10% 35% 60% 77% 88% 95% 99%

[0082] As shown in the comparison between Table 3 and Table 4, this change in scenario results in a decrease in the bit rate required by the knob of node C4 616 to transmit the data stream. Figure 5In the method 500, in the application specific bit rate allocation phase 513, as shown in Table 5, the system adjusts the knobs for node C1 610 and node C4 616 to maximize the accuracy of the system, so that the average overall accuracy of the system is 85.25%. The system uses the control plane to send instructions to change the knobs for node C1 610 and node C4 614.

[0083] Table 5

[0084]

[0085] As another example, in Figure 6B In the example, due to interference or some other problem, the capacity of link 620 has dropped from 400 Mbps to 200 Mbps, which causes the path to be overutilized. This example assumes that the four nodes C1-C4 have resource accuracy profiles as shown in Table 1. Figure 5 In the process of method 500 in FIG. 5 , in the apply independent flow routing phase 501, the topology of the multi-hop relay network is rerouted to the topology of network 630. Specifically, node C2 612 is rerouted to use link 624 instead of link 622, which reduces congestion on link 620. Figure 1 As shown, the instruction to make node C2 612 use link 624 instead of 622 is sent via a control packet on the control plane. Figure 5 In the method 500 of applying specific bit rate allocation phase 513, the system then reconfigures the knobs for node C1 610, node C2 612, and node C4 616 to maximize the overall accuracy of 74% as shown in Table 6:

[0086] Table 6

[0087]

[0088] Figure 7 The method of detecting that a system according to aspects of the present disclosure may not be operating with the target accuracy is illustrated. Rather than continually probing traffic for throughput estimation (which is expensive), the disclosed technique develops a topology and flow plan for a network and lets it run until a problem occurs. That is, a reactive approach to throughput estimation detects when an existing link has reduced throughput or congestion and then probes new links as illustrated in method 700.

[0089] The general sequence of operations for method 700 is as follows: Figure 7 Typically, method 700 begins with start operation 702 and ends with operation 716. Method 700 may include more or fewer steps, or may be arranged in a manner similar to that of FIG. Figure 7The method 700 may be performed as a computer-executable instruction set executed by a computer system and encoded or stored on a computer-readable medium. In addition, the method 700 may be performed by gates or circuits associated with a processor, ASIC, FPGA, SOC, or other hardware device. Hereinafter, the method 700 will be referred to in conjunction with Figure 1 , Figure 2A , Figure 2B , Figure 3A-3C , Figure 5 , Fig. 6A , Figure 6B , Fig. 8A ,and Figure 8B The described systems, components, devices, modules, software, data structures, data characteristic representations, signaling diagrams, methods, etc. are explained.

[0090] according to Figure 5 Optimization method 500. At operation 702, a controller sends instructions to network nodes on a control plane regarding what links the nodes should use to transmit data streams to edge nodes (e.g., upstream links). In various aspects, the controller bases this information on a topology that it determines will maximize the overall accuracy of the network. In various aspects, the controller sends this information to each node in the network.

[0091] At operation 704, the controller sends, over the control plane, to the network nodes the throughput thresholds at which their upstream data links should operate. In various aspects, the controller sends this information to each node in the network.

[0092] At operation 706 , the node sends the data flow through the upstream link identified by the controller on the control plane in operation 702 .

[0093] At decision 708, the node determines whether the upstream data link is operating at least at the threshold identified in operation 704. If the answer is "yes", the method 700 proceeds to decision 710 to determine whether it has received rerouting instructions from the controller. For example, even if the node's link is operating at least at its threshold, there may be other problems in the overall system that may cause the controller to change the flow / topology or node. If the answer is "no", the method 700 continues back to operation 706 and continues to send its video stream on the upstream link according to the instructions of the controller on the control plane.

[0094] However, if the answer to decision 708 is "no," the node reports the failure to the controller via the control plane at operation 712. At operation 714, the node probes additional alternate upstream links via the data plane. At operation 716, the node sends its link availability, link usage, and throughput information determined from the probe to the controller via the control plane, and waits for instructions on what link to use at operation 702.

[0095] Returning to decision 710, if the answer is "yes," the method moves to operation 702 where the node receives instructions as to what upstream link to use.

[0096] Fig. 8A and Figure 8B A computing device or mobile computing device 800 is illustrated, such as a mobile phone, a smart phone, a wearable computer (e.g., a smart watch), a tablet computer, a laptop computer, etc., with which aspects of the present disclosure can be practiced. In some aspects, a client (e.g., Figure 1 The computing system 105 in the example may be a mobile computing device. Fig. 8A , illustrates one aspect of a mobile computing device 800 for implementing these aspects. In a basic configuration, the mobile computing device 800 is a handheld computer having both input elements and output elements. The mobile computing device 800 typically includes a display 805 and one or more input buttons 810 that allow a user to input information into the mobile computing device 800. The display 805 of the mobile computing device 800 can also be used as an input device (e.g., a touch screen display). The optional side input element 815 (if included) allows further user input. The side input element 815 can be a rotary switch, a button, or any other type of manual input element. In alternative aspects, the mobile computing device 800 can include more or fewer input elements. For example, the display 805 may not be a touch screen in some aspects. In yet another alternative aspect, the mobile computing device 800 is a portable telephone system, such as a cellular phone. The mobile computing device 800 can also include an optional keyboard 835. The optional keyboard 835 can be a physical keyboard or a "soft" keyboard generated on the touch screen display. In various aspects, the output elements include a display 805 for displaying a graphical user interface (GUI), a visual indicator 820 (e.g., a light emitting diode), and / or an audio transducer 825 (e.g., a speaker). In some aspects, the mobile computing device 800 includes a vibration transducer for providing tactile feedback to the user. In yet another aspect, the mobile computing device 800 includes input and / or output ports, such as an audio input (e.g., a microphone jack), an audio output (e.g., a headphone jack), and a video output (e.g., an HDMI port) for sending signals to or receiving signals from external devices.

[0097] Figure 8B 800 is a block diagram illustrating an architecture of one aspect of a computing device, a server (e.g., server 109 or server 104), a mobile computing device, etc. That is, a computing device 800 may include a system (e.g., architecture) 802 to implement some aspects. The system 802 may be implemented as a "smart phone" capable of running one or more applications (e.g., browser, email, calendar, contact manager, messaging client, game, and media client / player). In some aspects, the system 802 is integrated into a computing device, such as an integrated digital assistant (PDA) and a wireless phone.

[0098] One or more application programs 866 may be loaded into the memory 862 and run on or in association with the operating system 864. Examples of application programs include a phone dialer, an email program, a information management (PIM) program, a word processing program, a spreadsheet program, a web browser program, a messaging program, and the like. The system 802 also includes a non-volatile storage area 869 within the memory 862. The non-volatile storage area 869 may be used to store persistent information that should not be lost when the system 802 loses power. The application programs 866 may use and store information in the non-volatile storage area 869, such as emails or other messages used by the email application, and the like. A synchronization application (not shown) also resides on the system 802 and is programmed to interact with a corresponding synchronization application residing on a host computer to keep information stored in the non-volatile storage area 869 synchronized with corresponding information stored on the host computer. It should be understood that other applications may be loaded into the memory 862 and run on the mobile computing device 800 described herein.

[0099] The system 802 has a power source 870, which can be implemented as one or more batteries. The power source 870 can also include an external power source, such as an AC adapter or a powered docking station to supplement or charge the batteries.

[0100] The system 802 may also include a radio interface layer 872 that performs the functions of transmitting and receiving radio communications. The radio interface layer 872 facilitates wireless connections between the system 802 and the "outside world" via a communications operator or service provider. Transmissions to or from the radio interface layer 872 are performed under the control of the operating system 864. In other words, communications received by the radio interface layer 872 may be propagated to the application 866 via the operating system 864, and vice versa.

[0101] The visual indicator 820 can be used to provide visual notifications, and / or the audio interface 874 can be used to generate auditory notifications via the audio transducer 825. In the illustrated configuration, the visual indicator 820 is a light emitting diode (LED), and the audio transducer 825 is a speaker. These devices can be directly coupled to the power supply 870 so that when it is activated, even if the processor 860 and other components may be turned off to save battery power, they will remain on for the duration specified by the notification mechanism. The LED can be programmed to remain on indefinitely until the user takes action to indicate the power-on state of the device. The audio interface 874 is used to provide an auditory signal to the user and receive an auditory signal from the user. For example, in addition to being coupled to the audio transducer 825, the audio interface 874 can also be coupled to a microphone to receive auditory input, such as to facilitate a telephone conversation. As described below, according to various aspects of the present disclosure, the microphone can also be used as an audio sensor to facilitate the control of notifications. The system 802 can also include a video interface 876, which enables an on-board camera 830 to operate to record still images, video streams, etc.

[0102] The mobile computing device 800 implementing the system 802 may have additional features or functionality. For example, the mobile computing device 800 may also include additional data storage devices (removable and / or non-removable), such as magnetic disks, optical disks, or tapes. Such additional storage Figure 8B The non-volatile storage area 869 is illustrated in FIG.

[0103] As described above, data / information generated or captured by the mobile computing device 800 and stored via the system 802 may be stored locally on the mobile computing device 800, or the data may be stored on any number of storage media accessible by the device via the radio interface layer 872 or via a wired connection between the mobile computing device 800 and a separate computing device associated with the mobile computing device 800, such as a server computer in a distributed computing network such as the Internet. It should be understood that such data / information may be accessed via the mobile computing device 800 via the radio interface layer 872 or via a distributed computing network. Similarly, such data / information may be easily transferred between computing devices for storage and use according to well-known data / information transmission and storage means, including electronic mail and collaborative data / information sharing systems.

[0104] As will be understood from the foregoing disclosure, one aspect of the present technology relates to a computer-implemented method for optimizing data analysis accuracy in a multi-hop relay network, the multi-hop relay network comprising a plurality of nodes, including a plurality of data nodes and an edge node. The method includes identifying a plurality of nodes in a relay topology by an edge node, wherein the plurality of nodes in the relay topology communicate with each other via a data plane comprising a high-frequency wireless link, and the data node communicates with the edge node via a control plane comprising a low-frequency wireless link; determining a path utilization for the high-frequency wireless link; when the high-frequency wireless link is overutilized, transmitting a command to change the relay topology from the edge node to the data node via the control plane; determining whether the relay topology is operating at a target accuracy; when it is determined that the relay topology is not operating at the target accuracy, adjusting a data analysis parameter for the node. At least one data node may be a camera. The edge node may be a 5G base station. At least one data node may be a mobile computing device. The high-frequency wireless link may be a millimeter wave link. The low-frequency wireless link may be an 802.11 Wi-Fi link. The target accuracy may include a maximum overall average accuracy for the relay topology. In another example, the target accuracy may include a maximum overall median accuracy for the relay topology. The data analysis parameters may be associated with throughput and accuracy levels. The data analysis parameters may be related to one or more of video resolution and frame rate. In another example, the method further includes receiving a report from a data node via a control plane that an upstream high frequency link is not operating at a threshold level; in response to receiving the report, receiving an identification of an alternative upstream high frequency link operating at a threshold level from the data node via the control plane; and sending an instruction to the data node via the control plane to send data using the alternative upstream high frequency link. In another example, the method further includes determining that a resource accuracy profile for the data node has changed; in response to determining that a resource accuracy profile for the data node has changed: determining a path utilization for the high frequency wireless link; when the high frequency wireless link is overutilized, transmitting a command to change the relay topology to the data node via the control plane; determining whether the relay topology is operating at a target accuracy; when it is determined that the relay topology is not operating at the target accuracy, adjusting the data analysis parameters for the node. In another example, the method further includes transmitting a command to change the relay topology to the data node via the control plane when it is determined that the relay topology is not operating at the target accuracy.

[0105] In another aspect, the present technology relates to a multi-hop relay network, including a first group of data nodes arranged in a first relay topology, wherein the first group of data nodes communicate with each other via a first data plane including a first group of high-speed wireless links, and a master edge node, the master edge node communicating with one or more data nodes in the first group of data nodes via the first data plane, and communicating with the first group of data nodes on a first control plane including a first group of low-speed wireless links. The master edge node includes a processor; a memory storing computer executable instructions, which, when the computer executable instructions are executed by the processor, cause the processor to: determine path utilization for the first group of high-speed wireless links; when a high-speed wireless link in the first group of high-speed wireless links is overutilized, transmit a command to change the first relay topology to a data node in the first group of data nodes via the first control plane; determine whether the first relay topology is operating at a target accuracy; when it is determined that the first relay topology is not operating at the target accuracy, adjust data analysis parameters for the node. In another example, the multi-hop relay network also includes a second group of data nodes arranged in a second relay topology, wherein the second group of data nodes communicate with each other via a second data plane including a second group of high-speed wireless links, and a second edge node, the second edge node communicates with one or more data nodes in the second group of data nodes via the second data plane, communicates with the second group of data nodes on a second control plane including a second group of low-speed wireless links, and communicates with the master edge node on a communication link. The master edge node also includes executable instructions, which, when executed by the processor, cause the processor to: determine path utilization for the second group of high-speed wireless links; when a high-speed wireless link in the second group of high-speed wireless links is overutilized, transmit a command to change the second relay topology to the second edge node via the communication link; determine whether the second relay topology is running at a target accuracy; when it is determined that the second relay topology is not running at the target accuracy, transmit a command to adjust data analysis parameters for nodes in the second group of nodes to the second edge node via the communication link. In another example, the data node is a camera, and the master edge node also includes computer executable instructions that, when executed by a processor, cause the processor to: receive a high-resolution video stream from the camera via a first data plane; reduce the size of the high-resolution video stream into one or more lower-resolution video streams; determine the accuracy and throughput of the high-resolution video stream and the one or more lower-resolution streams; and create a resource accuracy profile for the camera based on the accuracy and throughput of the high-resolution video stream and the one or more lower-resolution streams.

[0106] In another aspect, the present technology relates to a multi-hop relay network, which includes a plurality of cameras arranged in a relay topology and an edge server, wherein the plurality of cameras communicate with each other via a first data plane including a millimeter wave link, the edge server communicates with one or more of the plurality of cameras via the data plane, and communicates with the plurality of cameras on a control plane including a Wi-Fi link. The edge server includes a processor and a memory storing computer executable instructions, which, when executed by the processor, causes the processor to: determine path utilization for the millimeter wave link; transmit a command to change the relay topology to the camera via the control plane when the millimeter wave link is overutilized; determine whether the relay topology is operating at a target accuracy; and transmit a command to adjust video analysis parameters for the camera via the control plane when it is determined that the relay topology is not operating at the target accuracy. In an example, the multi-hop relay network also includes computer executable instructions, which, when executed by the processor, causes the processor to: adjust video analysis parameters for one or more of the plurality of cameras in response to determining that the scene of the camera has changed. In another example, the multi-hop relay network further includes computer executable instructions, which, when executed by the processor, cause the processor to: in response to adjusting the video analysis parameters for one or more cameras of the plurality of cameras, transmit a command to change the throughput for the camera via the control plane to the camera. In yet another example, the multi-hop relay network further includes computer executable instructions, which, when executed by the processor, cause the processor to: in response to determining that the scene for the camera has changed, transmit a command to change the relay topology to the camera via the control plane.

[0107] The phrases "at least one", "one or more", "or" and "and / or" are open-ended expressions that are both conjunctive and separable in operation. For example, "at least one of A, B and C", "at least one of A, B or C", "one or more of A, B and C", "one or more A, B or C", "A, B, and / or C" and "A, B or C" refer to A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B and C together.

[0108] The term "a" or "an" entity refers to one or more of that entity. Therefore, the terms "a" (or "an"), "one or more" and "at least one" can be used interchangeably herein. It is also noted that the terms "including", "comprising" and "having" can be used interchangeably.

[0109] As used herein, the term "automatic" and variations thereof refer to any process or operation, which is generally continuous or semi-continuous, that is performed without substantial human input while the process or operation is being performed. However, a process or operation may be automatic if the input is received prior to the execution of the process or operation, even if the execution of the process or operation uses substantial or insubstantial human input. Human input is considered substantial if such human input affects the manner in which the process or operation is performed. Human input that consents to the execution of the process or operation is not considered "substantial."

[0110] Any of the operations, functions, and actions discussed herein can be performed continuously and automatically.

[0111] The exemplary systems and methods of the present disclosure have been described with respect to computing devices. However, in order to avoid unnecessarily obscuring the present disclosure, the foregoing description omits many known structures and devices. Such omissions should not be construed as limiting the scope of the disclosure claimed. Specific details are set forth to provide an understanding of the present disclosure. However, it should be understood that the present disclosure can be practiced in a variety of ways beyond the specific details set forth herein.

[0112] In addition, although the exemplary aspects illustrated herein show various components of a configured system, certain components of the system may be located in a remote portion of a remote, distributed network, such as a LAN and / or the Internet, or in a dedicated system. Therefore, it should be understood that the components of the system may be combined into one or more devices, such as a server, a communication device, or configured at a specific node of a distributed network, such as an analog and / or digital telecommunications network, a packet switching network, or a circuit switching network. It can be understood from the foregoing description that for reasons of computational efficiency, the components of the system may be arranged at any location within the distributed network of components without affecting the operation of the system.

[0113] In addition, it should be understood that the various links connecting the elements may be wired or wireless links, or any combination thereof, or any other known or later developed (multiple) elements capable of providing and / or transmitting data to or from the connected elements. These wired or wireless links may also be secure links and may be capable of transmitting encrypted information. For example, the transmission medium used as a link may be any suitable electrical signal carrier, including coaxial cable, copper wire, and optical fiber, and may take the form of sound waves or light waves, such as those generated during radio wave and infrared data communications.

[0114] Although the flow charts have been discussed and illustrated with respect to a particular sequence of events, it should be understood that changes, additions, and omissions to this sequence may occur without materially affecting the configurations and operation of the aspects disclosed.

[0115] Many variations and modifications of the disclosure may be used. Some features of the disclosure may be provided without providing other features.

[0116] In another configuration, the system and method of the present disclosure can be combined with a special-purpose computer, a programmed microprocessor or microcontroller, and (multiple) peripheral integrated circuit elements, ASICs or other integrated circuits, digital signal processors, hard-wired electronic or logic circuits, such as discrete element circuits, programmable logic devices or gate arrays, such as PLDs, PLAs, FPGAs, PALs, special-purpose computers, any similar devices, etc. In general, any (multiple) devices or devices capable of implementing the methods illustrated herein can be used to implement various aspects of the present disclosure. Exemplary hardware that can be used for the present disclosure includes computers, handheld devices, phones (e.g., cellular, Internet-enabled, digital, analog, hybrid, etc.) and other hardware known in the art. Some of these devices include processors (e.g., single or multiple microprocessors), memory, non-volatile memory, input devices, and output devices. In addition, alternative software implementations including but not limited to distributed processing or component / object distributed processing, and row processing or virtual machine processing can also be constructed to implement the methods described herein.

[0117] In yet another configuration, the disclosed method can be readily implemented in conjunction with software using an object or object-oriented software development environment that provides portable source code that can be used on a variety of computer or workstation platforms. Alternatively, the disclosed system can be implemented in part or in whole in hardware using standard logic circuits or VLSI designs. Whether software or hardware is used to implement a system according to the present disclosure depends on the speed and / or efficiency requirements of the system, the specific functionality, and the specific software or hardware system or microprocessor or microcomputer system utilized.

[0118] In yet another configuration, the disclosed method may be implemented in part in software, which may be stored on a storage medium and executed on a programmed general-purpose computer in cooperation with a controller and memory, a dedicated computer, a microprocessor, etc. In these cases, the systems and methods of the present disclosure may be implemented as a resource resident on a server or computer workstation, as a program embedded in a computer, such as an applet, Or CGI script, as a routine program embedded in a dedicated measurement system, system component, etc. The system can also be implemented by physically incorporating the system and / or method into a software and / or hardware system.

[0119] Although the present disclosure describes components and functions implemented with reference to specific standards and protocols, the present disclosure is not limited to these standards and protocols. Other similar standards and protocols not mentioned herein exist and are considered to be included in the present disclosure. In addition, the standards and protocols mentioned herein and other similar standards and protocols not mentioned herein are periodically replaced by faster or more effective equivalents with substantially the same functions. Such alternative standards and protocols with the same functions are considered to be equivalents included in the present disclosure.

[0120] In various configurations and aspects, the present disclosure includes components, methods, processes, systems, and / or devices substantially as depicted and described herein, including various combinations, subcombinations, and subsets thereof. Those skilled in the art will understand how to make and use the systems and methods disclosed herein after understanding the present disclosure. In various configurations and aspects, the present disclosure includes providing devices and processes in the absence of items not depicted and / or described herein or in various configurations or aspects thereof, including in the absence of such items that may have been used in previous devices or processes, for example, to improve performance, achieve ease of use, and / or reduce the cost of implementation.

[0121] For example, various aspects of the present disclosure are described above with reference to the block diagrams and / or operational diagrams of the methods, systems, and computer program products according to various aspects of the present disclosure. The functions / actions noted in the boxes may appear in an order other than the order shown in any flow chart. For example, depending on the functions / actions involved, two blocks shown in succession may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order.

[0122] The description and illustration of one or more aspects provided in this application are not intended to limit or restrict the scope of the claimed disclosure in any way. The various aspects, examples, and details provided in this application are considered to be sufficient to convey ownership and enable others to make and use the best mode of the claimed disclosure. For example, the disclosure claimed should not be interpreted as being limited to any aspect, or the details provided in this application. Whether it is shown and described in combination or individually, it is intended to selectively include or omit various features (structures and methods) to produce an embodiment with a specific feature set. Description and illustration of the application have been provided, and those skilled in the art can envision changes, modifications, and alternative aspects that do not deviate from the broader scope of the claimed disclosure within the spirit of the broader aspects of the general inventive concept embodied in this application.

Claims

1. A method for optimizing data analysis accuracy in a multi-hop relay network, the multi-hop relay network comprising a plurality of nodes, the plurality of nodes comprising a plurality of data nodes and edge nodes, the method comprising: identifying, by an edge node, the plurality of nodes in a relay topology, wherein the plurality of nodes in the relay topology communicate with each other via a data plane comprising a high frequency wireless link, and the plurality of data nodes communicate with the edge node via a control plane comprising a low frequency wireless link; determining a path utilization for the high frequency wireless link; When a high frequency wireless link is overutilized, transmitting a command to change the relay topology from the edge node to the data node via the control plane; determining whether the relay topology is operating at a target accuracy by combining the accuracy of the plurality of nodes in the relay topology; and When it is determined that the relay topology is not operating at the target accuracy, a data analysis parameter for a node is adjusted. 2 . The method of claim 1 , wherein at least one of the plurality of data nodes is a camera.

3. The method of claim 1, wherein the edge node is a 5G base station. The method of claim 1 , wherein at least one of the plurality of data nodes is a mobile computing device. The method of claim 1 , wherein the high frequency wireless link is a millimeter wave link.

6. The method of claim 1, wherein the low-frequency wireless link is an 802.11 Wi-Fi link. The method of claim 1 , wherein the target accuracy comprises a maximum overall average accuracy for the relay topology.

8. The method of claim 1, wherein the target accuracy comprises a maximum overall median accuracy for the relay topology.

9. The method of claim 1, wherein the data analysis parameters are associated with throughput and accuracy levels.

10. The method of claim 1, wherein the data analysis parameter is related to one or more of video resolution and frame rate.

11. The method according to claim 1, further comprising: receiving, via the control plane, from a data node a report that an upstream high frequency link is not operating at a threshold level; In response to receiving the report, receiving from the data node via the control plane an identification of an alternative upstream high frequency link operating at the threshold level; as well as An instruction for sending data using the alternative upstream high frequency link is sent to the data node via the control plane.

12. The method according to claim 1, further comprising: Determine that the resource accuracy profile for the data node has changed; In response to determining that the resource precision profile for the data node has changed: determining a path utilization for the high frequency wireless link; When the high frequency wireless link is over-utilized, transmitting a command to change the relay topology to the data node via the control plane; determining whether the relay topology is operating at a target accuracy by combining the accuracy of a plurality of nodes in the relay topology; and When it is determined that the relay topology is not operating at the target accuracy, a data analysis parameter for a node is adjusted.

13. The method according to claim 1, further comprising: When it is determined that the relay topology is not operating at the target accuracy, a command to change the relay topology is transmitted to a data node via the control plane.

14. A multi-hop relay network, comprising: a first set of data nodes arranged in a first relay topology, wherein the first set of data nodes communicate with each other via a first data plane including a first set of high-speed wireless links; A master edge node, the master edge node communicating with one or more data nodes in the first group of data nodes via the first data plane and communicating with the first group of data nodes on a first control plane including a first group of low-speed wireless links, wherein the master edge node comprises: Processor; and A memory storing computer executable instructions, wherein when the computer executable instructions are executed by the processor, the processor: determining a path utilization for the first set of high-speed wireless links; When a high-speed wireless link in the first group of high-speed wireless links is over-utilized, transmitting a command to change the first relay topology to a data node in the first group of data nodes via the first control plane; determining whether the first relay topology operates at a target accuracy by combining the accuracy of the first set of data nodes; and When it is determined that the first relay topology is not operating at the target accuracy, a data analysis parameter for a node is adjusted.

15. The multi-hop relay network according to claim 14, further comprising: a second set of data nodes arranged in a second relay topology, wherein the second set of data nodes communicate with each other via a second data plane including a second set of high-speed wireless links; a second edge node, the second edge node communicating with one or more of the second set of data nodes via the second data plane, communicating with the second set of data nodes over a second control plane including a second set of low-speed wireless links, and communicating with the master edge node over a communication link; The master edge node further includes executable instructions, which, when executed by the processor, cause the processor to: determining a path utilization for the second set of high-speed wireless links; When a high-speed wireless link in the second group of high-speed wireless links is over-utilized, transmitting a command to change the second relay topology to the second edge node via the communication link; determining whether the second relay topology operates at the target accuracy; as well as When it is determined that the second relay topology is not operating at the target accuracy, a command to adjust a data analysis parameter for a node in the second set of nodes is transmitted to the second edge node via the communication link.

16. The multi-hop relay network of claim 14, wherein the data node is a camera, and the master edge node further comprises computer executable instructions which, when executed by the processor, cause the processor to: receiving a high-resolution video stream from the camera via the first data plane; reducing the high resolution video stream to one or more low resolution video streams; determining accuracy and throughput for the high-resolution video stream and the one or more low-resolution video streams; as well as A resource precision profile for the camera is created based on the precision and throughput for the high-resolution video stream and the one or more low-resolution streams.

17. A multi-hop relay network, comprising: a plurality of cameras arranged in a relay topology, wherein the plurality of cameras communicate with each other via a first data plane comprising millimeter wave links; an edge server that communicates with one or more of the plurality of cameras via the data plane and communicates with the plurality of cameras on a control plane that includes a Wi-Fi link, wherein the edge server comprises: Processor; and A memory storing computer executable instructions, which, when executed by the processor, causes the processor to: determining a path utilization for the millimeter wave link; When the millimeter wave link is overutilized, transmitting a command to change the relay topology to the camera via the control plane; determining whether the relay topology is operating at a target accuracy by combining the accuracy of a plurality of nodes in the relay topology; and When it is determined that the relay topology is not operating at the target accuracy, a command is transmitted to a camera via the control plane to adjust a video analytics parameter for the camera.

18. The multi-hop relay network according to claim 17, further comprising computer executable instructions, which, when executed by the processor, cause the processor to: In response to determining that a scene for a camera has changed, video analytics parameters for one or more cameras of the plurality of cameras are adjusted.

19. The multi-hop relay network of claim 17, further comprising computer executable instructions, which, when executed by the processor, cause the processor to: In response to adjusting the video analytics parameters for the one or more cameras of the plurality of cameras, a command to change a throughput for the camera is transmitted to the camera via the control plane.

20. The multi-hop relay network of claim 17, further comprising computer executable instructions, which, when executed by the processor, cause the processor to: In response to determining that a scene for a camera has changed, a command to change the relay topology is transmitted to the camera via the control plane.