Method and system for optimized deployment of analytic models in edge topology

Abstract

Techniques are provided for optimized deployment of analytic models in edge topologies. A description of a multi-tier edge topology having a plurality of nodes, a multimedia stream analysis composition, and performance targets are received, where the multimedia stream analysis composition includes tasks that use analytic models. The analytic models are optimized and clustered to form clusters of optimized analytic models. A representative optimized analytic model is selected from each cluster. A configuration recommendation is determined that indicates a deployment of the tasks and each selected representative optimized analytic model on the plurality of nodes to meet the performance targets. One or more workflows are generated from the configuration recommendation, and the one or more workflows are executed on the plurality of nodes to generate an output of the multimedia stream analysis composition.

Description

TECHNICAL FIELD

[0001] The present invention relates generally to optimized deployment of analytic models in edge topologies. Embodiments of the invention include recommending optimal compression configurations for analytic model composition in multi-tier edge topologies. BACKGROUND

[0002] Edge computing can be described as a distributed computing environment that brings data analytics / processing and data storage closer to where it is needed to improve response times and save bandwidth. With the advent of smart cameras and small single-board computers with camera add-ons, some video analytics capabilities are moving to the edge. Such smart cameras and small single-board computers with camera add-ons can be described as edge devices.

[0003] However, edge devices are typically not powerful enough to run typical video analytics compositions independently on live streams. Compression techniques can help run deep learning models more efficiently on edge devices, but can produce poor results. SUMMARY

[0004] According to one aspect of the invention, a computer-implemented method for optimized deployment of analytic models in edge topologies is provided. The computer-implemented method includes operations. A description of a multi-tier edge topology having a plurality of nodes, a multimedia stream analytics composition executed on the plurality of nodes, and a performance goal is received, where the multimedia stream analytics composition includes tasks that use analytic models. The analytic models are optimized. The optimized analytic models are clustered to form clusters of optimized analytic models. Representative optimized analytic models are selected from each cluster of optimized analytic models. A configuration recommendation is determined that indicates a deployment of the tasks and each selected representative optimized analytic model on the plurality of nodes to meet the performance goal. One or more workflows are generated to execute the tasks of the multimedia stream analytics composition on nodes of different tiers of the multi-tier edge topology. The one or more workflows are executed on the plurality of nodes to generate an output of the multimedia stream analytics composition.

[0005] According to another aspect, a computer program product for optimized deployment of analytic models in an edge topology is provided. The computer program product includes a computer readable storage medium having program code embodied therewith, the program code executable by at least one processor to perform operations. A description of a multi-tiered edge topology having a plurality of nodes, a multimedia stream analytics composition to be executed on the plurality of nodes, and a performance target are received, where the multimedia stream analytics composition includes tasks that use analytic models. The analytic models are optimized. The optimized analytic models are clustered to form clusters of optimized analytic models. Representative optimized analytic models are selected from each cluster of optimized analytic models. A configuration recommendation is determined that indicates deployment of the tasks and each of the selected representative optimized analytic models on the plurality of nodes to meet the performance target. One or more workflows are generated to execute the tasks of the multimedia stream analytics composition on the nodes of different tiers of the multi-tiered edge topology. The one or more workflows are executed on the plurality of nodes to generate an output of the multimedia stream analytics composition.

[0006] According to another aspect, a computer system for optimized deployment of analytic models in an edge topology is provided. The computer system includes one or more processors, one or more computer readable memories, and one or more computer readable tangible storage devices; and program instructions stored on at least one of the one or more computer readable tangible storage devices for execution by at least one of the one or more processors via at least one of the one or more memories to perform operations. A description of a multi-tiered edge topology having a plurality of nodes, a multimedia stream analytics composition to be executed on the plurality of nodes, and a performance target are received, where the multimedia stream analytics composition includes tasks that use analytic models. The analytic models are optimized. The optimized analytic models are clustered to form clusters of optimized analytic models. Representative optimized analytic models are selected from each cluster of optimized analytic models. A configuration recommendation is determined that indicates deployment of the tasks and each of the selected representative optimized analytic models on the plurality of nodes to meet the performance target. One or more workflows are generated to execute the tasks of the multimedia stream analytics composition on the nodes of different tiers of the multi-tiered edge topology. The one or more workflows are executed on the plurality of nodes to generate an output of the multimedia stream analytics composition.

[0007] Embodiments advantageously provide a multi-tiered edge topology that allows different portions of a multimedia stream analytics composition to be run using different model compression settings and different frame rates in each tier of the multi-tiered edge topology. That is, embodiments select what portions of a multimedia stream analytics composition and what compression models should be run where and when.

[0008] According to further embodiments, the performance targets include any combination of accuracy, precision, recall, and throughput. This allows different combinations of performance targets to be used to customize the distribution preference of the tasks of the multimedia stream analytics synthesis.

[0009] According to further embodiments, optimizing the analytics model includes compressing the analytics model. This advantageously allows the analytics model to be compressed to improve efficiency.

[0010] According to more embodiments, the clustering is based on benchmark results, and wherein the benchmark results are generated by benchmarking the optimized analytics model using test data and the plurality of nodes in the multi-tiered edge topology. This advantageously provides a clustering technique based on test data and the plurality of nodes in the multi-tiered edge topology that results in a better distribution of the tasks of the multimedia stream analytics synthesis.

[0011] According to yet further embodiments, one or more of the images and metadata are transmitted between the nodes in the plurality of nodes at different transmission speeds. Allowing for different transmission speeds allows for a better distribution of the tasks of the multimedia stream analytics synthesis.

[0012] According to further embodiments, for each node in the multi-tiered edge topology, a target function is defined based on performance, accuracy, precision, and recall performance targets. Then, for each node, a cumulative target function is determined based on cumulative performance, cumulative accuracy, cumulative precision, and cumulative recall. A total target function for the multi-tiered edge topology is created based on the cumulative target function for each node. This advantageously generates a total target function for the multimedia stream analytics synthesis.

[0013] According to yet further embodiments, the plurality of nodes includes a central server in a cloud infrastructure, wherein the central server stores one of the images and metadata for the multimedia stream analytics synthesis in a data store. This advantageously allows for centralized storage of the images and / or metadata for the multimedia stream analytics synthesis.

[0014] In other embodiments, a software as a service (SaaS) is configured to perform the operations of the method. This advantageously allows the execution of the one or more workflows and the generation of the configuration recommendations via a service. BRIEF DESCRIPTION OF DRAWINGS

[0015] Reference will now be made to the drawings wherein like numerals refer to like parts throughout:

[0016] Figure 1 A multi-tiered edge topology according to certain embodiments is illustrated in a block diagram.

[0017] Figure 2 A multimedia stream analytics synthesis process according to certain embodiments is illustrated.

[0018] Figure 3 A computing environment is shown in a block diagram in accordance with certain embodiments.

[0019] Figure 4A and 4B Operations for generating a workflow are shown in a flowchart in accordance with certain embodiments.

[0020] Figure 5 Operations for optimizing an analytic model are shown in a flowchart in accordance with certain embodiments.

[0021] Figure 6 Operations for benchmarking and clustering are shown in a flowchart in accordance with certain embodiments.

[0022] Figure 7 A 2-dimensional plot of an optimized analytic model is shown in accordance with certain embodiments.

[0023] Figure 8 Operations for additional optimization are shown in a flowchart in accordance with certain embodiments.

[0024] Figure 9 An example of a configuration recommendation is shown in a block diagram in accordance with certain embodiments.

[0025] Figure 10 Operations for executing a workflow are shown in a flowchart in accordance with certain embodiments.

[0026] Figure 11 Operations for layered multimedia stream analytics synthesis execution performed with camera 1, on-premise server at location LI, and central server are shown in a block diagram in accordance with certain embodiments.

[0027] Figure 12 Tasks for layered multimedia stream analytics synthesis at camera 1 are shown in a block diagram in accordance with certain embodiments.

[0028] Figure 13 Tasks for layered synthesis execution at on-premise server at local LI are shown in a block diagram in accordance with certain embodiments.

[0029] Figure 14 Tasks for layered synthesis execution at central server are shown in a block diagram in accordance with certain embodiments.

[0030] Figure 15 Operations for layered multimedia stream analytics synthesis execution performed with camera 3, on-premise server at location L2, and central server are shown in a block diagram in accordance with certain embodiments.

[0031] Figure 16 Tasks for layered multimedia stream analytics synthesis at camera 3 are shown in a block diagram in accordance with certain embodiments.

[0032] Figure 17 Tasks performed by the layered composition are illustrated in a block diagram at an on-prem server of a local L2, according to certain embodiments.

[0033] Figure 18 Tasks performed by the layered composition are illustrated in a block diagram at a central server, according to certain embodiments.

[0034] Figure 19 A compute node is illustrated, according to certain embodiments.

[0035] Figure 20 A cloud computing environment is illustrated, according to certain embodiments.

[0036] Figure 21 An abstraction model layer is illustrated, according to certain embodiments. DETAILED DESCRIPTION

[0037] The description of various embodiments of the present application has been presented for purposes of illustration, but is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0038] Embodiments provide a multi-layered edge topology that allows different model optimization (e.g., compression) settings and different frame rates to be used to run different parts of a multimedia stream analysis composition in each layer of the multi-layered edge topology. Embodiments select what parts of a multimedia stream analysis composition and what optimizations (e.g., compression models) should be run where and when. The multimedia stream includes any combination of video, audio, closed captioning, annotations, and metadata.

[0039] Figure 1 A multi-layered edge topology 100 is illustrated in a block diagram, according to certain embodiments. The multi-layered edge topology 100 includes an edge device layer 120, an on-prem device layer 140, and a cloud infrastructure layer 160. The edge device layer 120 includes cameras 122, 124, 126, 128. Each of the cameras 122, 124, 126, 128 can be a smart camera (e.g., on a smartphone, tablet computer, laptop computer, desktop computer, etc.) or a small single-board computer that includes a graphics processing unit (GPU), CPU, or other chip. A GPU can be described as a specialized electronic circuit designed to rapidly manipulate and process images. Each of the cameras 122, 124, 126, 128 outputs a multimedia stream. For four cameras 122, 124, 126, 128, four multimedia streams are output.

[0040] On-premise device tier 140 includes on-premise server 142 at local Ll (first location) and on-premise server 144 at location L2 (second location).

[0041] Cloud infrastructure tier 160 includes central server 162 (i.e., cloud node) and data store 164 (e.g., database or other storage). Central server 162 stores images and / or metadata for multimedia stream analysis synthesis in the data store. In certain embodiments, the cloud infrastructure tier allows for auto-scaling to add additional central servers to edge topology 100. In certain embodiments, central server 162 does not include a GPU, while in other embodiments, central server 162 includes a GPU.

[0042] Cameras 122, 124 transmit multimedia streams to on-premise server 142 at location Ll at a transmission speed of 100 megabits per second (100 Mb / s). Cameras 126, 128 transmit multimedia streams to on-premise server 144 at location L2 at a transmission speed of 100 megabits per second (100 Mb / s). On-premise servers 142, 144 transmit multimedia streams to central server 162 at a transmission speed of 10 gigabytes per second (10 Gb / s). Transmission speed can also be referred to as frame rate.

[0043] Cameras 122, 124, 126, 128, servers 142, 144, and central server 160 can be described as devices or nodes of multi-tier edge topology 100.

[0044] Figure 2 A multimedia stream analysis synthesis 200 is shown in accordance with certain embodiments. Multimedia stream analysis synthesis 200 can be described as having tasks 210-224, some of which use analysis models. In Figure 2 In general, “object metadata” can include objects and / or metadata.

[0045] Preliminary operations include multimedia stream capture 210 with a camera that outputs a multimedia stream (e.g., of two moving cars) to a multimedia stream file reader 212 of output images. That is, file reader 212 decodes the multimedia stream into a sequence of raw images.

[0046] Image preprocessor 214 receives images, performs pre-processing, and outputs pre-processed images. Pre-processing can include one or more of: adjusting resolution, smoothing, histogram equalization, compensation for automatic gain control effects, etc. In certain embodiments, image preprocessor 214 is a down-sampler that adjusts resolution and outputs images with better resolution.

[0047] The object detector 216 identifies objects and metadata about the objects, and outputs the detected object metadata (e.g., adds a bounding box around each of the two cars). The object detector 216 can be an analysis model. In certain embodiments, the object detector 216 uses You Only Look Once (YOLO) object detection, which implements a convolutional neural network (CNN) for real-time object detection by applying a single neural network to the entire image, dividing the image into regions, and predicting bounding boxes and probabilities for each region.

[0048] The object class filter 218 filters (removes) anything other than objects from the detected object metadata to output filtered object metadata (e.g., only the two cars). The image cropper 220 can crop the images in the filtered object metadata to output cropped object metadata. Cropping can include extracting sub-images from larger images, where each sub-image contains only a car and possibly some space around the car. The object property classifier 222 (e.g., a vehicle color classifier) identifies properties of each object in the cropped object metadata to output classified object metadata (e.g., including the colors of the two cars). The object property classifier 222 can be another analysis model. The object property classifier 222 can classify objects based on the properties.

[0049] The object tracker 224 identifies each object in one or more frames of the classified object metadata to output tracked object metadata (e.g., indicating that the first car is in frames 1-10 of the initial multimedia stream). In certain embodiments, the object tracker 224 is yet another analysis model. In certain embodiments, the object tracker 224 uses General Object Tracking (GOTURN) techniques that utilize a regression network.

[0050] In certain embodiments, there can be additional tasks, such as a track aggregator that receives the tracked object metadata and performs statistical analysis across lifetimes of tracks (where a track describes different objects as the objects move through frames of the multimedia stream) and outputs aggregated track metadata.

[0051] Each analysis model can be a deep learning model (e.g., based on artificial neural networks (e.g., convolutional neural networks (CNNs)), can be an artificial intelligence model, can be an analysis function that performs a function on the multimedia stream, can be an analysis synthesis, etc.

[0052] Figure 3A computing environment is shown in a block diagram, according to certain embodiments. A computing device 300 is coupled to a data store 350. The computing device 300 includes an optimization configuration system 320. The optimization configuration system 320 receives an edge topology description 360 (with hardware specifications), a multimedia stream analysis composition 370 to run on the edge device, and a set of performance goals 380 (e.g., user constraints), and the optimization configuration system 320 outputs one or more configuration recommendations 330. The one or more configuration recommendations 330 indicate how to optimize various analysis models used in the multimedia stream analysis composition 370, where to deploy the analysis models in the edge topology 100, and how the edge tiers 120, 140, 160 will interact. The optimization configuration system 320 also converts the configuration recommendations 330 into one or more workflows 340 (i.e., executable workflows).

[0053] The optimization configuration system 320 performs automatic benchmarking and clustering of optimization models to reduce the number of options to consider. The optimization configuration system 320 provides configuration recommendations 330 for a composition of analysis models of the edge topology 100. Traditional systems, on the other hand, are limited to variations of a single deep learning model distributed between a pair of nodes. The optimization configuration system 320 converts the configuration recommendations 330 into tier-specific workflows 340.

[0054] Figure 4A and 4B Operations for generating a workflow are shown in a flowchart, according to certain embodiments. Control begins in block 400, where the optimization configuration system 320 receives a description of a multi-tier edge topology having a plurality of nodes (i.e., the edge topology description 360) and a hardware specification for each node. For example, a description of the multi-tier edge topology 100 can be received.

[0055] In block 402, the optimization configuration system 320 receives a multimedia stream analysis composition 370 to execute on the plurality of nodes, where the multimedia stream analysis composition 370 has tasks, and where one or more tasks use an analysis model. The multimedia stream analysis composition 200 is an example of the multimedia stream analysis composition 370.

[0056] In block 404, the optimization configuration system 320 receives test data 390 that is later used to test tasks of the multimedia stream analysis composition, where the tasks are distributed in the multi-tier edge topology.

[0057] In block 406, the optimization configuration system 320 receives performance goals, such as accuracy, precision, recall, and throughput (e.g., multimedia stream analysis throughput or performance). The following are performance goals of the examples provided herein:

[0058] Process at least 15 frames per second per each multimedia.

[0059] Up to 10 objects per frame are processed.

[0060] Recall >= 0.6

[0061] Precision >= 0.9

[0062] Precision is prioritized over recall.

[0063] Total latency < 1 s

[0064] In block 408, the optimization configuration system 320 optimizes the analysis models using a variety of techniques and precision. In certain embodiments, the optimization is compression, and the techniques used for compression include, for example, parameter pruning and sharing, low-rank factorization, shift / compact convolution filters, and knowledge distillation. Thus, for embodiments, different types of compression can be used for different types of optimization. The multiple precisions refer to the precision of the analysis model parameters (e.g., reducing floating point precision from 32 bits to 16 bits to save memory and increase efficiency) and the size of the input (“network input dimension,” e.g., the analysis model accepts 320 x 240 images instead of 640 x 480 images). From block 408 ( Figure 4A ), processing continues to block 410 ( Figure 4B ).

[0065] In block 410, the optimization configuration system 320 performs a benchmark test of the optimized analysis models using the test data 390 and the nodes in the multi-layer edge topology.

[0066] In block 412, the optimization configuration system 320 clusters the optimized analysis models based on the results of the benchmark test to form one or more clusters. For example, the optimization configuration system 320 can find that cluster A has high accuracy and slow throughput, while cluster B has low accuracy and fast throughput.

[0067] In block 414, the optimization configuration system 320 performs additional optimization by selecting a representative optimized analysis model from each cluster of optimized analysis models. In certain embodiments, this includes solving a constrained optimization problem (i.e., optimizing a performance target with respect to variables in the presence of performance targets about those variables). In certain embodiments, this includes selecting a different representative optimized analysis model from each cluster of optimized analysis models to determine which representative model best meets the performance targets.

[0068] In block 416, the optimization configuration system 320 provides a configuration recommendation 330 that indicates the deployment of the tasks and each selected representative optimized analysis model on the plurality of nodes to meet the performance targets. In certain embodiments, the configuration recommendation 330 also indicates the transmission speed of the communication between the layers of the multi-layer edge topology.

[0069] In block 418, the optimization configuration system 320 generates one or more workflows from the configuration recommendations to perform tasks of the multimedia stream analysis synthesis in whole or in part on different layers of the multi-layered edge topology, where images and metadata are transmitted between nodes of the layers at different transmission speeds (i.e., frame rates). That is, tasks of the multimedia stream analysis synthesis can be distributed across the multi-layered edge topology (e.g., object detection on one node, object tracking on a different node, etc.), where each selected representative optimization analysis model is used by the tasks.

[0070] In block 420, the optimization configuration system 320 executes the one or more workflows on the nodes in the multi-layered edge topology.

[0071] Figure 5 Operations for optimizing an analysis model according to certain embodiments are illustrated in a flowchart. Control begins in block 500, where the optimization configuration system 320 selects an optimization technique of a plurality of optimization techniques and a precision of a plurality of precisions, starting with a unique first pair of optimization technique and precision. In block 502, the optimization configuration system 320 generates an optimization analysis model based on the selected optimization technique and precision. In block 504, the optimization configuration system 320 adds the optimization analysis model to a list of optimization analysis models for benchmarking and clustering. In block 506, the optimization configuration system 320 determines whether all unique pairs of optimization techniques and precisions have been selected. If so, processing continues to block 500 to select another unique pair, otherwise, processing is complete.

[0072] Figure 6 Operations for benchmarking and clustering according to certain embodiments are illustrated in a flowchart. Control begins in block 600, where the optimization configuration system 320 runs different benchmark tests on various optimization analysis models using the same test data 390. The various optimization analysis models can be analysis models on the list of optimization analysis models. The tests provide performance, accuracy, precision, and recall (four dimensions forming a 4-dimensional space). In block 602, the optimization configuration system 320 clusters the optimization analysis models in the multi-dimensional space to form clusters, each cluster including one or more optimization analysis models. For example, for 4 dimensions, the optimization analysis models are clustered in the 4-dimensional space or in a lower derived dimensional space (e.g., 2-dimensional space such as performance and precision). In block 604, the optimization configuration system 320 selects a representative optimization analysis model for each cluster (e.g., according to a utility function or target edge layer of the multi-layered edge topology). In block 606, the optimization configuration system 320 generates a multi-dimensional plot with each selected representative optimization analysis model. For example, if there are 10 models placed into 4 clusters, 4 representative optimization analysis models are selected for the plot.

[0073] Figure 7A 2-dimensional plot 700 showing optimization of analysis models is shown, according to certain embodiments. The plot 700 has an x-axis for performance and a y-axis for precision. Cluster 710 has low precision and low performance, while cluster 720 has high performance and high precision.

[0074] Figure 8 Operations for additional optimization are shown in a flowchart, according to certain embodiments. Control begins in block 800, where the optimization configuration system 320 defines, for each node in the multi-layered edge topology, a target function based on a performance goal using a combination of accuracy, precision, recall, and throughput. For example, a user can apply a performance goal of recall of at least 0.9. Instead of stating a particular accuracy metric (out of many possible accuracy metrics), the performance goal can state: "define cumulative performance and accuracy for the entire multimedia stream analysis synthesis for each node." Here, the throughput can be speed (e.g., maximum latency), and the accuracy can be how well the analysis synthesis did the work for the respective task (e.g., detect at least 8 out of 10 objects of interest).

[0075] In block 802, for each node, the optimization configuration system 320 defines a cumulative target function based on cumulative accuracy, cumulative precision, cumulative recall, and cumulative throughput. For example, the cumulative throughput can be the sum of the throughput of each analysis model in the multimedia stream analysis synthesis 370. In certain embodiments, the accuracy can be the product of the accuracy of all analysis models in the multimedia stream analysis synthesis 370.

[0076] In block 804, the optimization configuration system 320 creates a total target function for the multi-layered edge topology based on (e.g., by summing) the cumulative target functions for each node. For example, if the target function is precision for each node, the total target function is the sum of the precisions from each node.

[0077] In block 806, the optimization configuration system 320 converts the total target function into an equation. This is done to solve a constrained optimization problem. For example, the equation can be: recall >= 0.6 (i.e., recall is not optimized) and latency < 1 s.

[0078] In block 808, the optimization configuration system 320 maximizes the total target function according to the performance goal (i.e., such that the performance goal is adhered to). There can be a finite number of combinations of optimization analysis models, which is further reduced by clustering and selecting optimization analysis models from each cluster.

[0079] In block 810, the optimization configuration system 320 performs fine-tuning by trying different representative optimization analysis models from each cluster. For example, in Figure 6In the process, the optimization configuration system 320 identifies clusters and selects a representative for each cluster. However, competing representatives may exist in one or more clusters, making the selected representative potentially less than optimal. In box 810, other representatives are selected, and the process is executed... Figure 8 The processing was used to compare the results of different selected representatives used for fine-tuning.

[0080] Figure 9 A block diagram illustrates an example of configuration recommendation 900 according to certain embodiments. Figure 9 There exist optimization analysis models M1 for object detectors, M2 for object attribute classifiers, and M3 for object trackers. Figure 9 In the configuration recommendation 900, it indicates which nodes in each edge layer of the multi-layered edge topology should execute one or more optimization analysis models M1, M2, and M3. Furthermore, in Figure 9 In this system, cameras forward images and metadata to locally deployed servers, which in turn forward metadata without images to a central server. Each camera transmits at a different rate (i.e., frames per second (fps) or frame rate).

[0081] Figure 10 A flowchart illustrates the operations for executing a workflow according to certain embodiments. Control begins at box 1000, where the optimization configuration system 320 generates a workflow for multimedia stream analysis and synthesis at the nodes of each edge layer. Although multimedia stream analysis and synthesis is compatible with many types of multi-layer edge topology architectures, the workflow details the actions performed by each node of each edge layer and the interactions between different edge layers.

[0082] In box 1002, the optimization configuration system 320 executes a workflow to generate output. In some embodiments, this output may be an aggregated list of tracked objects.

[0083] Figure 11 A block diagram illustrates the operation of a hierarchical multimedia stream analysis and synthesis execution 1100, performed using camera 1, a locally deployed server at location L1, and a central server according to certain embodiments. In this example, camera 1 performs some processing on the multimedia stream and sends the output to the locally deployed server at location L1, and the locally deployed server at location L1 performs some processing and sends the output to the central server. The hierarchical multimedia stream analysis and synthesis execution 1100 uses an optimized analysis model for object detectors, object attribute classifiers, and object trackers.

[0084] Figure 12Tasks 1200 of layered multimedia stream analytics composition at camera 1 are illustrated in a block diagram, according to certain embodiments. In this example, camera 1 performs tasks 1200 of multimedia stream analytics composition 1100 from multimedia stream capture by object class filter and outputs filtered object metadata to a pre-line server at location LI. Layered multimedia stream analytics composition performs using optimized analytics models for object detector.

[0085] Figure 13 Tasks 1300 of layered composition execution at an on-premise server at local LI are illustrated in a block diagram, according to certain embodiments. In this example, the on-premise server at location LI receives filtered object metadata, performs tasks 1300 of multimedia stream analytics composition from image cropper by object tracker, and outputs tracked object metadata to a central server. Layered multimedia stream analytics composition performs using optimized analytics models for object attribute classifier and object tracker.

[0086] Figure 14 Tasks 1400 of layered composition execution at a central server are illustrated in a block diagram, according to certain embodiments. In this example, the central server receives tracked object metadata from the on-premise server at location LI and stores tracked object metadata in a database.

[0087] Figure 15 Operations of layered multimedia stream analytics composition execution 1500 performed with camera 3, an on-premise server at location L2, and a central server are illustrated in a block diagram, according to certain embodiments. In this example, camera 3 performs some processing on a multimedia stream and sends output to an on-premise server at location L2, and the on-premise server at location L2 performs some processing and sends output to a central server. Layered multimedia stream analytics composition execution 1500 uses optimized analytics models for object detector, object attribute classifier, and object tracker.

[0088] Figure 16 Tasks 1600 of layered multimedia stream analytics composition at camera 3 are illustrated in a block diagram, according to certain embodiments. In this example, camera 3 performs tasks 1600 of multimedia stream analytics composition 1500 from multimedia stream capture by object attribute classifier and outputs classified object metadata to an on-premise server at location L2. Layered multimedia stream analytics composition performs using optimized analytics models for object detector and object attribute classifier.

[0089] Figure 17A task 1700 of hierarchical composition execution at an on-premise server at a local L2 is shown in a block diagram, according to certain embodiments. In this example, the on-premise server at location L2 receives classified object metadata, performs the task 1700 of multimedia stream analysis composition synthesis of the object tracker, and outputs tracked object metadata to the central server. The hierarchical multimedia stream analysis composition execution uses optimized analysis models for the object tracker.

[0090] Figure 18 A task 1800 of hierarchical composition execution 1800 at a central server is shown in a block diagram, according to certain embodiments. In this example, the central server receives tracked object metadata from the on-premise server at location L2, and stores the tracked object metadata in a database.

[0091] Thus, with embodiments, the optimization configuration system 120 receives an edge topology with hardware specifications, a multimedia stream analysis composition that needs to run on edge devices, and a set of performance goals, and then the optimization configuration system 120 recommends how to optimize the various analysis models involved in that composition, where to deploy them, and how the edge tiers should interact.

[0092] Embodiments distribute deep learning multimedia stream analysis composition tasks among or between multiple edge devices. Embodiments provide optimization of analysis models, which is different from conventional systems that only compress video or metadata.

[0093] Embodiments also allow for the formation of heterogeneous devices into a work team. For example, if a location has legacy cameras that will not be upgraded, those legacy cameras can be part of a work team with newer equipment. Then, the output from those cameras can be used as input to new object tracking analysis models.

[0094] Figure 19 A computing environment 1910 is shown, according to certain embodiments. In certain embodiments, the computing environment is a cloud computing environment. With reference to Figure 19 The computer node 1912 is but one example of a suitable computing node, and is not intended to suggest any limitation as to the scope of use or functionality of embodiments of the application described herein. Regardless, computer node 1912 is capable of being implemented and / or performing any of the functionality set forth hereinabove.

[0095] Computer node 1912 can be a computer system that can operate with many other general-purpose or special-purpose computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations suitable for use with computer node 1912 include, but are not limited to, personal computer systems, server computer systems, thin clients, fat clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems or devices.

[0096] Computer node 1912 can be described in the general context of executable instructions in a computer system, such as program modules executed by the computer system. Typically, program modules can include routines, programs, objects, components, logic, data structures, etc., that perform a specific task or implement a specific abstract data type. Computer node 1912 can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked via a communication network. In a distributed cloud computing environment, program modules can reside in local and remote computer system storage media, including memory storage devices.

[0097] like Figure 19 As shown, computer node 1912 is illustrated as a general-purpose computing device. Components of computer node 1912 may include, but are not limited to, one or more processors or processing units 1916, system memory 1928, and a bus 1918 that couples various system components, including system memory 1928, to one or more processors or processing units 1916.

[0098] Bus 1918 represents one or more of several types of bus architectures, including memory buses or memory controllers, peripheral buses, accelerated graphics ports, and processor or local buses using any of the various bus architectures. By way of example and not limitation, these architectures include the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MCA) bus, the Enhanced ISA (EISA) bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0099] Computer node 1912 typically includes various computer system readable media. Such media can be any available media that can be accessed by computer node 1912, and it includes both volatile and non-volatile media, removable and non-removable media.

[0100] The system memory 1928 can include computer system readable media in the form of volatile memory, such as random-access memory (RAM) 1930 and / or cache memory 1932. Computer node 1912 can further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 1934 can be provided for reading from and writing to non-removable, non-volatile magnetic media (not shown and typically called a "hard disk drive"). Although not explicitly shown, a magnetic disk drive can also be provided for reading from and writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive can be provided for reading from or writing to a removable, non-volatile optical disk (such as a CD-ROM, DVD-ROM or other optical media). Each of these disk drives and their associated computer-readable media can be connected to the system bus 1918 by one or more disk controllers. As will be seen, the system memory 1928 can include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the application.

[0101] By way of example, and not limitation, as embodied in the system memory 1928, the program / utility 1940, having a set (at least one) of program modules 1942, can include an operating system, one or more application programs, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data or some combination thereof, can include an implementation of a networking environment. The program modules 1942 generally carry out the functions and / or methodologies of embodiments of the application as described herein.

[0102] The computer node 1912 can also communicate with one or more external devices 1914 such as a keyboard, a pointing device, a display 1924, etc.; one or more devices that enable a user to interact with the computer node 1912; and / or any devices (e.g., network card, modem, etc.) that enable the computer node 1912 to communicate with one or more other computing devices. Such communication can occur via Input / Output (I / O) interface(s) 1922. Still yet, the computer node 1912 can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet) via network adapter 1920. As depicted, the network adapter 1920 communicates with the other components of the computer node 1912 via the bus 1918. It should be understood that, although not shown, other hardware and / or software components could be used in conjunction with the computer node 1912. Examples include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0103] In certain embodiments, the cameras 122, 124, 126, 128, on-premise servers 142, 144, central server 162, and computing device 300 have the architecture of computer node 1912.

[0104] Cloud embodiments

[0105] It should be understood that while the present disclosure includes detailed descriptions of cloud computing, implementation of the teachings presented herein are not limited to a cloud computing environment. Rather, embodiments of the application are capable of implementation in conjunction with any other type of computing environment now known or later developed.

[0106] Cloud computing is a model of service delivery for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g. networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a provider of the service. This cloud model can include at least five characteristics, at least three service models, and at least four deployment models.

[0107] Features include the following:

[0108] On-demand self-service: cloud consumers can unilaterally provision computing capabilities, such as server time and network storage, as needed automatically without requiring human interaction with the service's provider.

[0109] Broad network access: capabilities are available over a network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0110] Resource pooling: the provider's computing resources are

[0111] Rapid elasticity: capabilities can be rapidly and elastically provisioned, in some cases automatically, to quickly scale out and rapidly scale in. To the consumer, the provider

[0112] Measured service: cloud systems automatically control and optimize resource use by leveraging usage-based pricing models for cloud consumers. Resource usage can be monitored, controlled, and reported providing transparency for both the provider and consumer of the utilized service.

[0113] Service models are as follows:

[0114] Software as a Service (SaaS): the capability provided to the consumer is to use the provider's applications running on a cloud infrastructure. The applications are accessible from various client devices through either a thin client interface, such as a web browser (e.g., web-based e-mail) or a program interface. The consumer does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0115] Platform as a Service (PaaS): the capability provided to the consumer is to deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including networks, servers, operating systems, or storage, but has control over the deployed applications and possibly application hosting environment configurations.

[0116] Infrastructure as a Service (laaS): the capability provided to the consumer is to provision processing, storage, networks, and other fundamental computing resources where the consumer is able to deploy and run arbitrary software, which can include an operating system and applications. The consumer does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, deployed applications, and possibly limited control of select networking components (e.g., host firewalls).

[0117] Deployment models are as follows:

[0118] Private cloud: the cloud infrastructure is operated solely for an organization. It can be managed by the organization or a third party and can exist on-premises or off-premises.

[0119] Community cloud: the cloud infrastructure is shared by several organizations and supports mission-oriented business

[0120] Public cloud: the cloud infrastructure is made available to general public or a large industry group and is owned by an organization selling cloud services.

[0121] Hybrid cloud: the cloud infrastructure is a composition of two or more types of cloud (private, community, or public) that remain unique clouds but are bound together by standardized or proprietary technologies that enable data and application portability.

[0122] Cloud computing environments are service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability. At the heart of cloud computing is the infrastructure of a network of interconnected nodes.

[0123] Now for reference Figure 20 This describes an illustrative cloud computing environment 2050. As shown, the cloud computing environment 2050 includes one or more cloud computing nodes 2010 to which local computing devices used by cloud computing consumers can communicate, such as personal digital assistants (PDAs) or cellular phones 2054A, desktop computers 2054B, laptop computers 2054C, and / or automotive computer systems 2054N. The nodes 2010 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as private clouds, community clouds, public clouds, or hybrid clouds, or combinations thereof, as described above. This allows the cloud computing environment 2050 to provide cloud consumers with infrastructure, platforms, and / or Software-as-a-Service that eliminates the need for them to maintain resources on their local computing devices. It should be understood that... Figure 20 The types of computing devices 2054A-N shown are intended to be illustrative only, and the computing node 2010 and cloud computing environment 2050 can communicate with any type of computing device on any type of network and / or network-addressable connection (e.g., using a web browser).

[0124] Now for reference Figure 21 This demonstrates the 2050 cloud computing environment ( Figure 20 This provides a set of functional abstractions. It should be understood beforehand that... Figure 21 The components, layers, and functions shown are for illustrative purposes only, and embodiments of the invention are not limited thereto. As depicted, the following layers and corresponding functions are provided:

[0125] The hardware and software layer 2160 includes hardware and software components. Examples of hardware components include: a host 2161; a server 2162 based on a RISC (Reduced Instruction Set Computer) architecture; a server 2163; a blade server 2164; a storage device 2165; and network and networking components 2166. In some embodiments, software components include network application server software 2167 and database software 2168.

[0126] The virtualization layer 2170 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual server 2171; virtual storage 2172; virtual network 2173, including virtual private network; virtual application and operating system 2174; and virtual client 2175.

[0127] In one example, management layer 2180 can provide the functions described below. Resource provisioning 2181 provides dynamic procurement of computing resources and other resources that are utilized to perform tasks within the cloud computing environment. Metering and Pricing 2182 provide cost tracking as resources are utilized within the cloud computing environment, and cost can be determined as how to bill or scale an

[0128] Workloads layer 2190 provides examples of functionality for which the cloud computing environment can be utilized. Examples of workloads and functions which can be provided from this layer include: mapping and navigation 2191; software development and lifecycle management 2192; virtual classroom education delivery 2193; data analytics processing 2194; transaction processing 2195; and optimization of deployment of analytics models in edge topology 2196.

[0129] Accordingly, in certain embodiments, software or programs implementing the optimized deployment of analytics models in edge topology according to the embodiments described herein are provided as a service in a cloud environment.

[0130] Additional Embodiment Details

[0131] The present application can be a system, a method, and / or a computer program product at any possible technical detail level of integration. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present application.

[0132] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch cards or

[0133] The computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions into the respective computing / processing device for storage in a computer readable storage medium within the respective computing / processing device.

[0134] Computer readable program instructions for carrying out operations of the present application can be assembly instructions, instruction-set-architecture (ISA) instructions, machine- related instructions, microcode, firmware instructions, state-setting data, configuration data for an integrated circuit, or source code or object code written in any combination of one or more programming languages, including an object-oriented programming language such as Smalltalk, C++ or the like, and a procedural programming language such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0135] Aspects of the present application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0136] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0137] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0138] The computer program product of the second aspect can include a computer readable storage medium. The computer readable storage medium can include instructions. The instructions can include one or both of: instructions for causing a computer to implement a method of the first aspect; and instructions for causing a computer to implement a system of the third aspect.

[0139] The terms "an embodiment," "one or more embodiments," "embodiments," "some embodiments," and "one embodiment" mean "one or more (but not all) embodiments of the invention," unless expressly specified otherwise.

[0140] The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless expressly specified otherwise.

[0141] A list of items joined by "or" means any one of the items or any combination if more than one of the items, unless specified otherwise.

[0142] The terms "a," "an," and "the" mean "one or more," unless expressly specified otherwise.

[0143] Unless specifically stated otherwise, devices that are in communication with each other need not be in continuous communication with each other, unless expressly specified otherwise. In addition, devices that are in communication with each other can communicate directly or through one or more intermediaries.

[0144] Description of embodiments with several components in communication with each other does not imply that all such components are required. To the contrary, a variety of optional components are described to illustrate the wide variety of potential embodiments of the present invention.

[0145] When a single device or article is described herein, it will be readily apparent that more than one device / article (whether or not they cooperate) can be used in place of a single device / article. Similarly, where more than one device or article is described herein (whether or not they cooperate), it will be readily apparent that a single device / article can be used in place of the more than one device or article or a different number of devices / articles can be used instead of the shown number of devices / articles. The functionality and / or the features of a device can be alternatively embodied by one or more other devices which are not explicitly described as having such functionality / features. Thus, other embodiments of the application need not include the device itself.

[0146] The foregoing description of various embodiments of the application has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the application to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. It is intended that the scope of the application be limited not with this detailed description, but rather by the claims appended hereto. The description above, as well as the examples and data, provide a complete description of the manufacture and use of the composition of the application. Since many embodiments of the application can be made without departing from the scope of the application, the application resides in the claims hereinafter appended. The foregoing description provides examples of embodiments of the application, and can be used to modify or adapt other embodiments of the application.

Claims

1. A computer-implemented method for optimizing the deployment of an analysis model in an edge topology, comprising the following operations: The system receives a description of a multi-layer edge topology with multiple nodes, multimedia stream analysis and synthesis performed on the multiple nodes, and performance targets, wherein the multimedia stream analysis and synthesis includes a task using an analysis model, wherein the analysis model includes a neural network. To form a list of optimized analysis models, at least one of the analysis models is optimized by performing the following operations: For each unique pair of optimization techniques and one of the multiple precision rates of the parameters of the analysis model, An optimization analysis model is generated based on the aforementioned optimization techniques and accuracy. as well as Add the optimization analysis model to the list; Benchmark the optimization analysis model in the list using test data and the multiple nodes in the multi-layer edge topology; The optimization analysis models in the list are clustered based on the results of benchmark tests to form clusters of optimization analysis models; Select representative optimization analysis models from the clusters of each optimization analysis model; A configuration recommendation is determined, which indicates the deployment of the task and each selected representative optimization analysis model on the multiple nodes to meet the performance objective; The configuration recommends generating one or more workflows to perform the multimedia stream analysis and synthesis task on nodes at different layers of the multi-layer edge topology; as well as The one or more workflows are executed on the plurality of nodes to generate the output of the multimedia stream analysis synthesis.

2. The computer-implemented method of claim 1, wherein the performance objective includes any combination of accuracy, precision, recall, and throughput.

3. The computer-implemented method according to claim 1 or 2, wherein optimizing the analysis model includes compressing the analysis model.

4. The computer-implemented method according to claim 1 or 2, wherein one or more of the image and metadata are transmitted between the nodes of the plurality of nodes at different transmission speeds.

5. The computer-implemented method according to claim 1 or 2 further includes the following operations: For each node in the multi-layer edge topology, an objective function is defined based on performance targets for performance, accuracy, precision, and recall. For each node, a cumulative objective function is defined based on cumulative performance, cumulative accuracy, cumulative precision, and cumulative recall. as well as The total objective function of the multi-layer edge topology is created based on the cumulative objective function of each node.

6. The computer-implemented method of claim 1 or 2, wherein the plurality of nodes includes a central server in a cloud infrastructure, wherein the central server stores one of the images and metadata synthesized for the multimedia stream analysis in a data storage.

7. The computer-implemented method of claim 1 or 2, wherein Software as a Service (SaaS) is configured to perform the operations of the method.

8. A computer program product comprising a computer-readable storage medium having program code implemented therewith, said program code being executable by at least one processor to perform the following operations: The system receives a description of a multi-layer edge topology with multiple nodes, multimedia stream analysis and synthesis performed on the multiple nodes, and performance targets, wherein the multimedia stream analysis and synthesis includes a task using an analysis model, wherein the analysis model includes a neural network. To form a list of optimized analysis models, at least one of the analysis models is optimized by performing the following operations: For each unique pair of optimization techniques and one of the multiple precision rates of the parameters of the analysis model, An optimization analysis model is generated based on the aforementioned optimization techniques and accuracy. as well as Add the optimization analysis model to the list; Benchmark the optimization analysis model in the list using test data and the multiple nodes in the multi-layer edge topology; The optimization analysis models in the list are clustered based on the results of benchmark tests to form clusters of optimization analysis models; Select representative optimization analysis models from the clusters of each optimization analysis model; A configuration recommendation is determined, which indicates the deployment of the task and each selected representative optimization analysis model on the multiple nodes to meet the performance objective; The configuration recommends generating one or more workflows to perform the multimedia stream analysis and synthesis task on nodes at different layers of the multi-layer edge topology; as well as The one or more workflows are executed on the plurality of nodes to generate the output of the multimedia stream analysis synthesis.

9. The computer program product of claim 8, wherein the performance objective includes any combination of accuracy, precision, recall, and throughput.

10. The computer program product of claim 8 or 9, wherein optimizing the analysis model includes compressing the analysis model.

11. The computer program product of claim 8 or 9, wherein one or more of the image and metadata are transmitted between the nodes of the plurality of nodes at different transmission speeds.

12. The computer program product of claim 8 or 9, wherein the program code is executable by the at least one processor to perform the following further operations: For each node in the multi-layer edge topology, an objective function is defined based on performance targets for performance, accuracy, precision, and recall. For each node, a cumulative objective function is defined based on cumulative performance, cumulative accuracy, cumulative precision, and cumulative recall. as well as The total objective function of the multi-layer edge topology is created based on the cumulative objective function of each node.

13. The computer program product of claim 8 or 9, wherein the plurality of nodes includes a central server in a cloud infrastructure, wherein the central server stores one of the images and metadata synthesized for the multimedia stream analysis in a data storage.

14. The computer program product of claim 8 or 9, wherein the Software as a Service (SaaS) is configured to perform the operations of the computer program product.

15. A computer system for the optimized deployment of an analysis model in an edge topology, comprising: One or more processors, one or more computer-readable storage devices, and one or more computer-readable tangible storage devices; as well as Program instructions, stored on at least one of the one or more computer-readable tangible storage devices, are executed by at least one of the one or more processors via at least one of the one or more memories to perform operations including: The system receives a description of a multi-layer edge topology with multiple nodes, multimedia stream analysis and synthesis performed on the multiple nodes, and performance targets, wherein the multimedia stream analysis and synthesis includes a task using an analysis model, wherein the analysis model includes a neural network. To form a list of optimized analysis models, at least one of the analysis models is optimized by performing the following operations: For each unique pair of optimization techniques and one of the multiple precision rates of the parameters of the analysis model, An optimization analysis model is generated based on the aforementioned optimization techniques and accuracy. as well as Add the optimization analysis model to the list; Benchmark the optimization analysis model in the list using test data and the multiple nodes in the multi-layer edge topology; The optimization analysis models in the list are clustered based on the results of benchmark tests to form clusters of optimization analysis models; Select representative optimization analysis models from the clusters of each optimization analysis model; A configuration recommendation is determined, which indicates the deployment of the task and each selected representative optimization analysis model on the multiple nodes to meet the performance objective; The configuration recommends generating one or more workflows to perform the multimedia stream analysis and synthesis task on nodes at different layers of the multi-layer edge topology; as well as The one or more workflows are executed on the plurality of nodes to generate the output of the multimedia stream analysis synthesis.

16. The computer system of claim 15, wherein the performance objective includes any combination of accuracy, precision, recall, and throughput.

17. The computer system of claim 15 or 16, wherein optimizing the analysis model includes compressing the analysis model.

18. The computer system of claim 15 or 16, wherein one or more of the image and metadata are transmitted between the nodes of the plurality of nodes at different transmission speeds.

19. The computer system according to claim 15 or 16, wherein the operation further comprises: For each node in the multi-layer edge topology, an objective function is defined based on performance targets for performance, accuracy, precision, and recall. For each node, a cumulative objective function is defined based on cumulative performance, cumulative accuracy, cumulative precision, and cumulative recall. as well as The total objective function of the multi-layer edge topology is created based on the cumulative objective function of each node.

20. The computer system of claim 15 or 16, wherein the plurality of nodes includes a central server in a cloud infrastructure, wherein the central server stores one of the images and metadata synthesized for the multimedia stream analysis in a data storage.

21. The computer system of claim 15 or 16, wherein Software as a Service (SaaS) is configured to perform operations of the computer system.

Images