Multi-video semantic collaborative analysis method and system based on federated learning

Abstract

The present application relates to the technical field of video processing, and particularly relates to a multi-video semantic collaborative analysis method and system based on federated learning. Obtain task complexities reported by a plurality of end-side computing devices respectively; based on the plurality of task complexities, divide the plurality of end-side computing devices into K device groups; initialize K semantic analysis models for the K device groups respectively at a server side; based on a federated learning framework, collaboratively perform hierarchical distillation training on the K semantic analysis models by the plurality of end-side computing devices and the server side; and correspondingly distribute the trained K semantic analysis models to each end-side computing device in the K device groups for video semantic analysis. The present application can improve the overall efficiency, accuracy and generalization ability of multi-video semantic collaborative analysis, and optimize the configuration of computing resources.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and in particular to a multi-video semantic collaborative analysis method and system based on federated learning. Background Technology

[0002] Traditional multi-video semantic analysis primarily involves deploying a unified semantic analysis model on each end device, with each device performing its own analysis and then uploading the results. However, this approach fails to consider the differences in tasks across end devices. The unified model is inadequate for high-complexity tasks and wastes resources for low-complexity tasks, resulting in an imbalance in the allocation of computing resources. Furthermore, traditional methods lack inter-device collaboration mechanisms, leading to weak overall analysis generalization ability and inefficient resource allocation. Summary of the Invention

[0003] This invention addresses the technical problems of weak generalization ability and inefficient resource allocation in existing technologies by providing a multi-video semantic collaborative analysis method and system based on federated learning.

[0004] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0005] In a first aspect, the present invention provides a multi-video semantic collaborative analysis method based on federated learning, comprising: obtaining task complexity reported by multiple edge computing devices, wherein the task complexity is dynamically calculated and generated by each edge computing device based on local video data features; dividing the multiple edge computing devices into K device groups based on the multiple task complexities, wherein K is an integer greater than 1; initializing K semantic analysis models for the K device groups on the server side, wherein the structural complexity of the kth semantic analysis model corresponding to the kth device group is positively correlated with the typical task complexity of the kth device group; performing hierarchical distillation training of the K semantic analysis models in collaboration with the multiple edge computing devices and the server side based on a federated learning framework; and distributing the trained K semantic analysis models to each edge computing device in the K device groups for video semantic analysis.

[0006] Optionally, the task complexity reported by multiple edge computing devices is obtained, including: determining the baseline task coefficients of multiple edge computing devices based on the identification task type and the prior category rule matrix; each of the multiple edge computing devices locally collects real-time video data features and performs feature fusion on the real-time video data features; and adjusting the feature fusion results with the baseline task coefficients as weight coefficients to obtain the task complexity of the multiple edge computing devices.

[0007] The method of dividing multiple edge computing devices into K device groups based on multiple task complexities includes: determining the number of groups K based on multiple task complexities and using the elbow method; and using a density clustering algorithm to divide the multiple edge computing devices into K device groups according to the spatial proximity relationship between multiple task complexities.

[0008] Specifically, on the server side, K semantic analysis models are initialized for each of the K device groups. The structural complexity of the kth semantic analysis model corresponding to the kth device group is positively correlated with the typical task complexity of the kth device group. This includes: determining a global mapping model of complexity-model size parameters based on a preset analysis window and historical analysis records of multiple edge computing devices; analyzing and obtaining the K typical task complexities of the K device groups using the elbow method; traversing the K complexity-model size global mapping models, matching and determining the model size parameters corresponding to the K typical task complexities, and configuring the K semantic analysis models accordingly. The model types of the K semantic analysis models are consistent.

[0009] The process of determining a global mapping model for complexity-model size parameters based on a preset analysis window and historical analysis records from multiple edge computing devices includes: extracting raw analysis logs from multiple edge computing devices by traversing them through the preset analysis window; obtaining real-time analysis task requirements from multiple edge computing devices, and selecting analysis records that meet these requirements from the raw analysis logs, outputting them as historical analysis records; and performing regression analysis based on the historical analysis records, using historical task complexity and historical model size parameters as analysis dimensions, to obtain the global mapping model for complexity-model size parameters.

[0010] The method of analyzing and obtaining K typical task complexities for K device groups using the elbow method includes: randomly selecting one of the K device groups as the target group for analysis; obtaining multiple task complexities corresponding to multiple edge computing devices in the target group for analysis, and analyzing and obtaining the task complexity distribution of the target group for analysis, wherein the task complexity distribution uses the task complexity value as the first coordinate and the density of task complexity at the task complexity value as the second coordinate; using the elbow method to obtain the elbow point of the task complexity distribution, and outputting the task complexity value at the elbow point as the typical task complexity; iteratively updating the target group for analysis until the K typical task complexities corresponding to the K device groups are obtained.

[0011] Optionally, based on a federated learning framework, multiple edge computing devices and the server collaborate to perform hierarchical distillation training on the K semantic analysis models, including: instructing multiple edge computing devices to perform local training on the semantic analysis models corresponding to their respective device groups based on local video data, and uploading model updates; using the K device groups as indexes, aggregating the model updates received within the same device group, and updating the corresponding semantic analysis models; using the k-th semantic analysis model as the student model and the (k+1)-th semantic analysis model as the teacher model, performing hierarchical distillation training on the server by traversing the K semantic analysis models.

[0012] The process involves using the k-th semantic analysis model as the student model and the (k+1)-th semantic analysis model as the teacher model. On the server side, hierarchical distillation training is performed by traversing the K semantic analysis models. This includes: defining the k-th semantic analysis model as the student model and the (k+1)-th semantic analysis model as the teacher model, where k is less than or equal to K-1; obtaining the high-order sample input and high-order training output of the (k+1)-th semantic analysis model, and generating corresponding pseudo-low-order sample input based on the high-order sample input; establishing the association between the pseudo-low-order sample input and the high-order training output, and outputting it as the k-th hierarchical distillation sample; iteratively training the k-th semantic analysis model based on the k-th hierarchical distillation sample, calculating the distillation loss, and optimizing the student model based on the distillation loss, wherein the distillation loss is calculated based on the difference between the high-order training output and the student model's output on the k-th hierarchical distillation sample; and iteratively performing hierarchical distillation training on the K semantic analysis models.

[0013] Secondly, this invention provides a multi-video semantic collaborative analysis system based on federated learning, comprising:

[0014] The task complexity acquisition module is used to acquire the task complexity reported by multiple end-side computing devices, wherein the task complexity is dynamically calculated and generated by each end-side computing device based on local video data features;

[0015] The edge device grouping module is used to divide multiple edge computing devices into K device groups based on the complexity of multiple tasks, where K is an integer greater than 1;

[0016] The semantic analysis model initialization module is used to initialize K semantic analysis models for the K device groups on the server side, wherein the structural complexity of the kth semantic analysis model corresponding to the kth device group is positively correlated with the typical task complexity of the kth device group.

[0017] The semantic analysis model distillation module is used to perform hierarchical distillation training on the K semantic analysis models based on a federated learning framework, in coordination with multiple edge computing devices and the server.

[0018] The semantic analysis model distribution module is used to distribute the trained K semantic analysis models to each end-side computing device in the K device groups for video semantic analysis.

[0019] By implementing this invention, it is possible to obtain the task complexity reported by multiple edge computing devices. The task complexity is dynamically calculated and generated by each edge computing device based on local video data features. The task complexity is calculated based on real-time local video data features, which can reflect the difficulty of the video semantic analysis task currently faced by the device in real time and avoid the adaptation deviation caused by using fixed values.

[0020] By implementing this invention, multiple edge computing devices can be divided into K device groups based on the complexity of the multiple tasks. Subsequent model-related operations can be performed on the groups, avoiding the blindness of uniformly processing all devices. Precise measures can be taken according to the characteristics of the groups, thereby improving the overall process efficiency.

[0021] By implementing this invention, it is possible to initialize K semantic analysis models for the K device groups on the server side. The structural complexity of the Kth semantic analysis model corresponding to the Kth device group is positively correlated with the typical task complexity of the Kth device group. The structural complexity of the model is positively correlated with the typical task complexity of the group. Groups with high task complexity correspond to models with more complex structures and stronger capabilities, while groups with low task complexity correspond to models with simple structures. This avoids resource waste and prevents the problem of insufficient model capabilities.

[0022] By implementing this invention, it is possible to achieve hierarchical distillation training of the K semantic analysis models based on a federated learning framework, in collaboration with multiple edge computing devices and the server, so that the models in each group can improve their performance on their own basis. Furthermore, under the federated learning framework, edge devices do not need to upload the original video data, but only upload model updates, which effectively protects the privacy of users' video data.

[0023] By implementing this invention, the trained K semantic analysis models can be distributed to each edge computing device in the K device groups for video semantic analysis. After the models are distributed, the edge devices can use them directly without additional complex configuration, and quickly acquire video semantic analysis capabilities.

[0024] In summary, by implementing this invention, the overall efficiency, accuracy, and generalization ability of multi-video semantic collaborative analysis can be significantly improved, while optimizing the allocation of computing resources and avoiding waste of computing resources. Attached Figure Description

[0025] Figure 1 A flowchart illustrating a multi-video semantic collaborative analysis method based on federated learning provided by this invention;

[0026] Figure 2 This is a schematic diagram of the structure of a multi-video semantic collaborative analysis system based on federated learning provided by the present invention.

[0027] In the attached diagram, the components represented by each number are as follows:

[0028] Task complexity acquisition module 11, end-side device group division module 12, semantic analysis model initialization module 13, semantic analysis model distillation module 14, semantic analysis model distribution module 15. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0031] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.

[0032] Example 1, as Figure 1 As shown, this embodiment of the invention provides a multi-video semantic collaborative analysis method based on federated learning, including:

[0033] S100: Obtain the task complexity reported by multiple end-side computing devices, wherein the task complexity is dynamically calculated and generated by each end-side computing device based on local video data features;

[0034] S200: Based on the complexity of the multiple tasks, divide the multiple edge computing devices into K device groups, where K is an integer greater than 1;

[0035] S300: Initialize K semantic analysis models for the K device groups on the server side, wherein the structural complexity of the kth semantic analysis model corresponding to the kth device group is positively correlated with the typical task complexity of the kth device group;

[0036] S400: Based on the federated learning framework, multiple edge computing devices and the server coordinate to perform hierarchical distillation training on the K semantic analysis models;

[0037] S500: The trained K semantic analysis models are distributed to each end-side computing device in the K device groups for video semantic analysis.

[0038] In step S100 of this application embodiment, the task complexity reported by multiple edge computing devices is obtained, including:

[0039] Based on the identification task type and the prior category rule matrix, the baseline task coefficients of multiple edge computing devices are determined.

[0040] Multiple edge computing devices locally collect real-time video data features and perform feature fusion on the real-time video data features;

[0041] Using the baseline task coefficient as the weight coefficient, the feature fusion results are adjusted respectively to obtain the task complexity of multiple edge computing devices.

[0042] In this embodiment of the application, the purpose of step S100 is to accurately and dynamically quantify the difficulty of the video semantic analysis task currently faced by each end-side computing device through progressive operations such as determining the baseline task coefficient, real-time video feature fusion, and weighted adjustment calculation. This provides objective and suitable core data support for subsequent key steps such as device grouping in S200 and model initialization in S300, and avoids resource mismatch or poor analysis results in subsequent processes due to inaccurate assessment of task complexity.

[0043] The first step is to pre-construct the aforementioned "category rule matrix." This matrix needs to establish a correspondence between "recognition task types" and "baseline task coefficients" based on historical video semantic analysis scenarios. For example, the matrix can clearly define "pedestrian detection" as having a baseline task coefficient of 0.3, "multi-target behavior recognition in complex scenes" as having a baseline task coefficient of 0.8, etc., forming a standardized benchmark for judging task difficulty.

[0044] Then, the server or edge device first determines the type of video semantic analysis task that each edge device needs to perform, such as "vehicle violation recognition at traffic intersections" or "shopping mall pedestrian flow heat map analysis." Based on the identified task type, it matches the data in a pre-built "category rule matrix" to extract the corresponding baseline task coefficients, which set the basic reference standard for calculating the complexity of subsequent tasks.

[0045] The second step involves collecting real-time video data features locally on multiple edge computing devices and then performing feature fusion on these features.

[0046] Optionally, real-time video data to be analyzed can be directly collected from each end device, and multi-dimensional features related to semantic analysis can be extracted. For example, video data features include the "number of targets" in the video frame, such as the number of pedestrians and vehicles in the frame; the "target motion complexity", such as whether the target is moving in a uniform linear motion or an irregular motion; the "scene complexity", such as whether the frame has a single background or a complex background containing multiple elements such as buildings and vegetation; and the "lighting conditions", such as strong light, weak light, and backlight.

[0047] Next, the edge device uses feature fusion algorithms such as weighted fusion and feature stitching to integrate the features of the collected multi-dimensional real-time video data. For example, after standardizing features such as "number of targets", "motion complexity" and "scene complexity", they are merged into a comprehensive feature vector according to preset logic, eliminating the limitations of single features and forming a "feature fusion result" that can comprehensively reflect the characteristics of the current video data.

[0048] The third step is to use the baseline task coefficients as weighting coefficients to adjust the feature fusion results and obtain the task complexity of multiple edge computing devices.

[0049] Optionally, the "baseline task coefficient" determined in the first step can be used as a weight to perform a weighted calculation with the "feature fusion result" obtained in the second step. For example, if the baseline task coefficient of a certain end-side device is 0.6 and the feature fusion result is 0.5 after quantization, then the quantized value of the task complexity of the device, such as 0.3, can be calculated by "baseline task coefficient × feature fusion result" or other preset weighting formulas.

[0050] The above method is used to obtain the task complexity of multiple edge computing devices.

[0051] In step S200 of this application embodiment, based on the multiple task complexities, the multiple edge computing devices are divided into K device groups, including:

[0052] Based on the complexity of the multiple tasks, the number of groups K is determined using the elbow method.

[0053] A density clustering algorithm is used to divide multiple edge computing devices into K device groups based on the spatial proximity relationship between the complexities of multiple tasks.

[0054] In this embodiment, the purpose of step S200 is to group edge computing devices with similar task complexity into one group by scientifically determining the number of groups and accurately executing the device grouping operation. This ensures that the video semantic analysis tasks faced by devices in the same group are of similar difficulty, laying the foundation for subsequent steps S300 to initialize adaptation models for different groups and S400 to carry out efficient collaborative training. This avoids resource waste or insufficient model adaptation caused by mixed device grouping.

[0055] The first step is to determine the number of groups, K.

[0056] First, task complexity data reported by all edge computing devices is collected to form a task complexity dataset.

[0057] Assuming the number of groups increases progressively from 1 (K = 1, 2, 3, n), for each assumed value of K, a pre-defined clustering algorithm, such as K-means, is used to cluster the task complexity dataset. Here, clustering is only used to assist in error calculation and is not the final grouping algorithm. Then, the sum of squared errors (SSE) after each clustering is calculated; this is the sum of the squares of the differences between the task complexity of each device and the complexity of its cluster center. A smaller SSE indicates a smaller difference in task complexity among devices within the same group.

[0058] Then, plot the "K-SSE" curve with the number of groups K on the horizontal axis and the corresponding total sum of squared errors (SSE) on the vertical axis. Observe the trend of the curve: when K is small, as K increases, the SSE decreases very rapidly, and the curve is steep; when K increases to a certain value, further increases in K cause the rate of SSE decrease to slow down significantly, and the curve tends to flatten. The K value corresponding to the inflection point where the curve changes from steep to gentle, resembling an "elbow," is the optimal number of groups. For example, if the curve begins to flatten when K=3, it indicates that dividing the equipment into 3 groups achieves the optimal balance between grouping accuracy and grouping efficiency.

[0059] The second step requires the use of a density clustering algorithm to divide multiple edge computing devices into K device groups based on the spatial proximity relationship between the complexities of multiple tasks.

[0060] First, the task complexity of each edge device needs to be mapped as a data point to a one-dimensional data space. If the task complexity consists of multiple dimensional parameters, it can also be mapped to the corresponding dimensional space to form a set of task complexity data points, with each data point corresponding to an edge device.

[0061] Then, based on the distribution characteristics of the task complexity data, the core parameters of the density clustering algorithm are set. Here, we take the DBSCAN algorithm as an example. The DBSCAN algorithm requires setting two core parameters: ε and MinPts.

[0062] Wherein, ε (neighborhood radius): defines the "neighborhood range" of a data point, that is, other data points less than or equal to ε at a distance from the data point are considered to be in the same "dense region" as the data point. MinPts (minimum number of core points): stipulates that when the number of other data points contained in the ε neighborhood of a data point is greater than or equal to MinPts, the point is determined to be a "core point", representing that its area is a data-dense region.

[0063] Next, all task complexity data points are traversed to identify all "core points" and adjacent core points and their non-core points in their neighborhood are grouped into a "dense cluster".

[0064] For isolated data points that are not assigned to any dense cluster, i.e., data points whose number of samples in the neighborhood is less than MinPts and are not in the neighborhood of any core point, they are assigned to the nearest cluster based on their spatial distance from each dense cluster.

[0065] Through the above steps, K "dense clusters" are finally formed, and each cluster corresponds to a "device group".

[0066] In step S300 of this application embodiment, K semantic analysis models are initialized on the server side for the K device groups respectively. The structural complexity of the k-th semantic analysis model corresponding to the k-th device group is positively correlated with the typical task complexity of the k-th device group, including:

[0067] Based on the preset analysis window and the historical analysis records of multiple edge computing devices, a global mapping model for complexity-model size parameters is determined.

[0068] Combining the elbow method, we analyze and obtain the complexity of K typical tasks for K groups of the aforementioned devices;

[0069] Traverse the K complexity-model-size global mapping models, match and determine the model size parameters corresponding to the K typical task complexities, and configure the K semantic analysis models accordingly, wherein the K semantic analysis models have the same model type.

[0070] In this embodiment of the application, the purpose of step S300 is to initialize a semantic analysis model for each device group whose structural complexity is precisely matched with the task difficulty of the group. This avoids low analysis accuracy due to insufficient model capabilities in groups with complex tasks, and prevents resource waste due to model redundancy in groups with simple tasks. At the same time, it ensures the collaborative compatibility of models of the same type.

[0071] To achieve the above objectives, it is first necessary to determine a global mapping model for complexity-model size parameters based on a preset analysis window and historical analysis records from multiple edge computing devices.

[0072] In step S300 of this application embodiment, based on a preset analysis window and historical analysis records from multiple edge computing devices, a global mapping model for complexity-model size parameters is determined, including:

[0073] Based on a preset analysis window, the raw analysis logs are extracted by traversing multiple edge computing devices.

[0074] The system acquires real-time analysis task requirements from multiple edge computing devices, and uses these real-time analysis task requirements as targets to filter out analysis records that meet the requirements from the original analysis logs, and outputs them as historical analysis records.

[0075] Based on the historical analysis records, regression analysis was performed using historical task complexity and historical model size parameters as analysis dimensions to obtain the global mapping model of complexity-model size parameters.

[0076] The first step is to extract raw analysis logs by traversing multiple edge computing devices based on a preset analysis window.

[0077] Specifically, the preset analysis window can have its boundaries determined based on the timeliness and data representativeness requirements of the video semantic analysis task. The window can be set according to two dimensions: time dimension and data volume dimension.

[0078] For the time dimension, for example, setting the window to "past 30 days" will extract only the analysis data within the last 30 days, ensuring that the data can reflect the actual characteristics of the current device tasks and avoid interference from data that is too old. For the data volume dimension, for example, setting the window to "1000 valid analysis records per device" will ensure that the data volume of a single device is sufficient to support subsequent analysis, while avoiding computational redundancy caused by excessive data volume.

[0079] Then, the server uses the communication interface of the federated learning framework to traverse all participating edge computing devices and extract the "raw analysis logs" from the local storage modules of each device. The logs must contain task-related information, model-related information, and device-related information.

[0080] Among them, the task-related information includes the types of video semantic analysis tasks executed in the past, such as pedestrian detection and vehicle violation identification, as well as information such as task execution time and historical task complexity quantification values;

[0081] Model-related information includes the scale parameters of the semantic analysis model used to perform the task, such as the number of network layers, the number of convolutional kernels, the total number of neurons, and the number of model parameters.

[0082] Device-related information includes the device number on the edge and the device hardware configuration, such as computing power and memory.

[0083] Finally, the raw analysis logs from all edge devices were uploaded to the server's log storage module and categorized by "device number - task execution time" to form a structured raw analysis log dataset, laying the foundation for subsequent filtering operations.

[0084] The second step is to obtain the real-time analysis task requirements of multiple edge computing devices, and with the real-time analysis task requirements as the target, filter the analysis records that meet the requirements from the original analysis log and output them as the historical analysis records.

[0085] First, the server sends a "real-time task requirement collection instruction" to each end-side computing device through the task scheduling module. Each end-side device then provides feedback on its real-time task requirements based on the video data currently to be processed. The feedback includes: core task types, such as "shopping mall pedestrian trajectory analysis" and "highway multi-target tracking"; accuracy requirements, such as "target recognition accuracy ≥ 95%" and "semantic segmentation pixel accuracy ≥ 90%"; and scene features, such as "complex background" and "dynamic lighting".

[0086] Next, log filtering and matching rules need to be formulated based on the core dimensions of real-time task requirements. First, there is task type matching, which requires that the types of historical tasks in the original analysis logs be completely consistent with the types of real-time tasks, such as both being vehicle violation identification. Then, there is accuracy and scenario matching, which requires that the accuracy requirements of historical tasks in the original analysis logs be no lower than the real-time accuracy requirements, and that the similarity between the scenario features of historical tasks and the features of real-time tasks be ≥80%. This scenario feature similarity can be achieved through scenario feature keyword matching or feature vector similarity calculation.

[0087] Based on the above matching rules, the server's data analysis module filters the original analysis log dataset line by line, selecting records that match the rules to form the "historical analysis record" dataset.

[0088] The third step is to perform regression analysis based on the historical analysis records, using historical task complexity and historical model size parameters as the analysis dimensions, to obtain the global mapping model of the complexity-model size parameters.

[0089] First, it is necessary to determine the regression analysis method, specifically by selecting a suitable regression model based on the distribution characteristics of the two dimensions of data.

[0090] If the data shows a linear correlation trend, that is, as observed through a scatter plot, the model size parameters grow approximately linearly when the complexity of historical tasks increases, then a linear regression model, such as multiple linear regression, should be selected.

[0091] If the data exhibits a non-linear correlation trend, such as slow growth of model size parameters when the task complexity is low, but accelerated growth after the complexity exceeds the threshold, choose a non-linear regression model, such as multinomial regression, ridge regression, or gradient boosting regression model based on machine learning.

[0092] Next, define the variables. Specifically, use "historical task complexity" as the independent variable (X) and "historical model size parameter" as the dependent variable (Y).

[0093] Next, the preprocessed historical analysis records were divided into training and validation sets in a 7:3 ratio. The regression model was trained using the training set, and the model hyperparameters were adjusted using the validation set to ensure that the model fit (R²) was ≥0.85, thus avoiding overfitting or underfitting.

[0094] Ultimately, the trained and optimized regression model is determined to be a complexity-model size parameter global mapping model. This model explicitly defines the correspondence between "task complexity → model size parameter" in the form of a mathematical function, which can be directly used for subsequent operations that input typical task complexity and output appropriate model size parameters. The mathematical function form is, for example, Y=0.7X+0.2, or a nonlinear function Y=aX²+bX+c.

[0095] In step S300 of this application embodiment, the elbow method is used to analyze and obtain the K typical task complexities of the K device groups, including:

[0096] One of the K device groups is randomly selected as the target group for analysis;

[0097] The task complexity of multiple edge computing devices in the target analysis group is obtained, and the task complexity distribution of the target analysis group is analyzed and obtained. The task complexity distribution is defined with the task complexity value as the first coordinate and the density of task complexity at the task complexity value as the second coordinate.

[0098] By combining the elbow method, the elbow points of the task complexity distribution are obtained, and the task complexity value at the elbow point is output as the typical task complexity.

[0099] The analysis target group is iteratively updated until the K typical task complexities corresponding to the K device groups are obtained.

[0100] In this embodiment of the application, the purpose of the above steps in step S300 is to extract the "typical task complexity" that can represent the core task difficulty for each device group, so as to avoid the deviation caused by using the mean or extreme value of the complexity of all devices in the group, and to ensure that the model size matched based on the typical value of the task complexity can accurately adapt to the overall task requirements of the group, while ensuring the consistency and rationality of the typical values ​​of the K groups.

[0101] The first step is to randomly select one of the K device groups as the target group for analysis. Specifically, based on the device grouping results completed in S200, a "grouping list" containing K device groups can be compiled. Each group is marked with a unique identifier, such as Group1, Group2...GroupK, and the number of end-side devices in the group is also marked to ensure that no groups are omitted or duplicated.

[0102] Then, a random number generation algorithm is used to select one group from the group list as the "analysis target group".

[0103] The second step is to obtain the task complexity of multiple edge computing devices in the target analysis group and analyze the task complexity distribution of the target analysis group.

[0104] Optionally, based on the identifier of the target group, the task complexity quantification values ​​reported by all edge devices in S100 within that group can be filtered to form an "analysis target group complexity dataset". For example, if Group3 contains 100 edge devices, then 100 corresponding complexity values ​​can be extracted, such as 0.2, 0.3, 0.3, 0.4, etc.

[0105] Next, it is necessary to plot the task complexity distribution curve. The "task complexity value" can be used as the first coordinate (X-axis), with the value range covering the minimum to maximum complexity value in the dataset, and the scale can be divided according to a preset interval, such as 0.05. The "complexity density", that is, the density at the task complexity value, can be used as the second coordinate (Y-axis). The density calculation uses kernel density estimation (KDE) or frequency statistics method.

[0106] Next, calculate the Y-axis density value corresponding to each X-axis scale, and use curve fitting tools such as Python's Matplotlib library to draw the "task complexity-density" distribution curve. The peak area of ​​the curve represents the concentrated range of task complexity for most devices in the group, while the two sides of the curve correspond to the low / high complexity range for a few devices.

[0107] The third step involves using the elbow method to obtain the elbow points of the task complexity distribution and outputting the task complexity value at each elbow point as the typical task complexity. Specifically, based on the task complexity distribution curve obtained in the second step, the focus is on its density decreasing segment. By calculating the rate of change of the curve's slope, the elbow point where the slope becomes gentler is found. The task complexity value corresponding to this point is the typical task complexity of that group.

[0108] The fourth step involves iteratively updating the analysis target groups until the K typical task complexities corresponding to the K device groups are obtained. Specifically, groups that have completed analysis are marked as "analyzed," removed from the list of groups to be analyzed, and their typical values ​​are recorded. Steps one through three are repeated until all K device groups have completed analysis, ultimately yielding the K corresponding typical task complexities, which provide a basis for subsequent model parameter matching.

[0109] In this embodiment of the application, it is also necessary to traverse the K complexity-model size global mapping models, match and determine the model size parameters corresponding to the K typical task complexities, and configure the K semantic analysis models accordingly, wherein the K semantic analysis models have the same model type.

[0110] Optionally, you can call the pre-built "complexity-model size parameter global mapping model" and input the typical task complexity of K device groups in sequence. For example, the typical value for Group1 is 0.7, the typical value for Group2 is 0.4, etc.

[0111] Then, based on the functional relationship of the mapping model, the target model size parameters corresponding to each typical value are calculated, such as the number of parameters, the number of network layers, and the number of convolutional kernels. For example, a typical value of 0.7 corresponds to 0.8 billion parameters and a 16-layer network; a typical value of 0.4 corresponds to 0.3 billion parameters and an 8-layer network, achieving a precise mapping between "typical task complexity and model size".

[0112] Finally, a unified base model type, such as CNN or Transformer, is selected. The base model is then adjusted based on the matched scaling parameters, such as increasing / decreasing the number of layers or adjusting the number of parameters, to initialize a corresponding semantic analysis model for each group. This results in K semantic analysis models, whose structural complexity is positively correlated with the typical task complexity of their respective groups, and whose model types are completely identical, thus meeting the requirements for subsequent collaborative training.

[0113] In step S400 of this application embodiment, based on the federated learning framework, multiple edge computing devices and the server coordinate to perform hierarchical distillation training on the K semantic analysis models, including:

[0114] Each device is instructed to train the semantic analysis model corresponding to its respective device group locally based on local video data, and then upload the model update.

[0115] Using the K device groups as indexes, the model updates received within the same device group are aggregated, and the corresponding semantic analysis model is updated.

[0116] Using the k-th semantic analysis model as the student model and the (k+1)-th semantic analysis model as the teacher model, hierarchical distillation training is performed on the server side by traversing the K semantic analysis models.

[0117] In this embodiment of the application, the purpose of step S400 is to improve the performance of each group semantic analysis model while protecting data privacy, and at the same time realize knowledge transfer and collaborative optimization between models, so that the semantic analysis model of low complexity group can learn from the analysis capabilities of the semantic analysis model of high complexity group, and finally form a semantic analysis model cluster with hierarchical performance and overall adaptability.

[0118] To achieve the above objectives, the first step requires instructing multiple edge computing devices to locally train the semantic analysis model corresponding to their respective device groups based on local video data, and then uploading the model updates. Specifically, the server sends instructions to each edge computing device, specifying its device group and corresponding semantic analysis model; for example, devices in Group 1 receive the initial semantic analysis model for that group.

[0119] Then, each end device uses its own local video data to train and optimize the assigned model without uploading the original data. The model parameters are adjusted through backpropagation to minimize the semantic analysis loss of the local task.

[0120] After training is completed, the edge device only calculates and uploads the "model update amount", such as parameter gradient changes and weight differences, rather than the complete model or original data, to ensure data privacy and security.

[0121] The second step is to use the K device groups as indexes to aggregate the model updates received within the same device group and update the corresponding semantic analysis model.

[0122] The server collects model update data uploaded by all end-devices within a group, such as Group1 to GroupK. Then, it uses federated learning aggregation algorithms, such as FedAvg, to perform a weighted average of the model update data within the same group. The specific weight values ​​can be set according to the data volume or computing power of the end-devices, ultimately generating the "global update data" for that group's models.

[0123] Then, the global update is applied to the semantic analysis model corresponding to the group, the model parameters are updated, and the model iteration on the server side is completed in this round, so that the model can retain the local knowledge of each end side and integrate the common features of the devices in the same group.

[0124] The third step involves using the k-th semantic analysis model as the student model and the (k+1)-th semantic analysis model as the teacher model, and performing hierarchical distillation training on the server side by traversing the K semantic analysis models.

[0125] In step S400 of this embodiment, using the k-th semantic analysis model as the student model and the (k+1)-th semantic analysis model as the teacher model, hierarchical distillation training is performed on the server side by traversing the K semantic analysis models, including:

[0126] Define the kth semantic analysis model as the student model, and define the (k+1)th semantic analysis model as the teacher model, where k is less than or equal to K-1;

[0127] Obtain the high-order sample input and high-order training output of the (k+1)th semantic analysis model, and generate the corresponding pseudo-low-order sample input based on the high-order sample input;

[0128] Establish the correlation between the pseudo-low-order sample input and the high-order training output, and output the k-th stratified distillation sample;

[0129] Based on the k-th stratified distillation sample, the k-th semantic analysis model is iteratively trained, and the distillation loss is calculated. The student model is then optimized based on the distillation loss, wherein the distillation loss is calculated based on the difference between the high-level training output and the student model's output on the k-th stratified distillation sample.

[0130] Iterative hierarchical distillation training is performed by traversing K semantic analysis models.

[0131] In this embodiment, stratified distillation training is performed to enable low-complexity student models to accurately learn the analytical capabilities of high-complexity teacher models. This improves the semantic analysis accuracy of student models without relying on the original data on the edge, while ensuring the effectiveness and adaptability of knowledge transfer between models, ultimately achieving a step-by-step optimization of the performance of all grouped models.

[0132] First, the semantic analysis model corresponding to the kth device group needs to be set as the "student model", and the model corresponding to the (k+1)th group needs to be set as the "teacher model", with k ≤ K-1.

[0133] Next, it is necessary to retrieve the "high-order sample input" of the k+1th teacher model from the server storage. This is the video data features with high complexity used during the training of the teacher model, such as video features with multiple targets and complex backgrounds, as well as the corresponding "high-order training output", which is the accurate semantic analysis result of the teacher model on the high-order sample, such as the target category probability distribution and semantic segmentation mask.

[0134] By simplifying features, such as reducing feature dimensionality and minimizing target detail information, the high-order sample input of the teacher model is transformed into a "pseudo-low-order sample input"—whose complexity matches the complexity of the grouping task corresponding to the k-th student model. This simulates the video data features actually processed by the student model, ensuring that the student model can learn effectively.

[0135] Then, a one-to-one correspondence is established between "pseudo-low-order sample inputs" and "high-order training outputs": each pseudo-low-order sample input is bound to the high-order training output corresponding to its source high-order sample input to form a single "hierarchical distillation sample"; after batch processing all high-order data, they are summarized to form the "kth hierarchical distillation sample set", providing a dedicated dataset for the distillation training of the student model.

[0136] Furthermore, based on the k-th stratified distillation sample, the k-th semantic analysis model needs to be trained iteratively, and the distillation loss needs to be calculated. The student model is then optimized based on the distillation loss, wherein the distillation loss is calculated based on the difference between the high-order training output and the student model's output on the k-th stratified distillation sample.

[0137] For example, the kth stratified distillation sample set is first input into the kth student model, and the student model makes predictions based on the pseudo low-order sample input, outputting the "student prediction result".

[0138] Then, by comparing the student's prediction results with the high-order training output in the distillation samples, the difference between the two is calculated using indicators such as KL divergence and mean square error, and the "distillation loss" is obtained. The smaller the distillation loss value, the closer the student model's prediction results are to the teacher model's accurate output.

[0139] Next, with the goal of minimizing distillation loss, the parameters of the student model are adjusted through backpropagation, and the "training-loss calculation-parameter adjustment" process is executed iteratively until the change in distillation loss is less than the preset threshold for multiple consecutive rounds, thus completing the optimization of the student model.

[0140] Finally, repeat the above steps in order from k to k-1, constructing distillation samples and training and optimizing student models for each group of teacher and student models, until all K-1 groups of teacher and student models have completed distillation training, realizing knowledge transfer and performance improvement of all grouped models.

[0141] In step S500 of this application embodiment, it is also necessary to distribute the trained K semantic analysis models to each end-side computing device in the K device groups for video semantic analysis.

[0142] For example, the server can index the trained K models by device group and match them with the corresponding groups. For instance, the models in group 1 can be distributed to all edge devices in group 1. Through the secure communication channel of the federated learning framework, the model file containing optimized parameters and structure configuration can be distributed to each edge device. After receiving the model, the edge device can load it and use it to process local video data to complete the semantic analysis task.

[0143] By implementing the federated learning-based multi-video semantic collaborative analysis method provided in the embodiments of this application, at least the following can be achieved:

[0144] 1. This ensures that model complexity is positively correlated with task difficulty, avoiding resource waste by high-complexity models on simple tasks and performance deficiencies by low-complexity models on complex tasks, thereby achieving optimized allocation of computing resources.

[0145] 2. Knowledge transfer is achieved through stratified distillation, which improves the overall model's generalization ability and analytical accuracy.

[0146] 3. Under the federated learning framework, the client only uploads model updates and does not leak the original video data.

[0147] Example 2, as Figure 2 As shown, based on the same inventive concept as the federated learning-based multi-video semantic collaborative analysis method provided in Embodiment 1, this embodiment of the invention also provides a federated learning-based multi-video semantic collaborative analysis system, including:

[0148] The task complexity acquisition module 11 is used to acquire the task complexity reported by multiple end-side computing devices, wherein the task complexity is dynamically calculated and generated by each end-side computing device based on local video data features.

[0149] The edge device grouping module 12 is used to divide multiple edge computing devices into K device groups based on the complexity of multiple tasks, where K is an integer greater than 1;

[0150] The semantic analysis model initialization module 13 is used to initialize K semantic analysis models for the K device groups on the server side, wherein the structural complexity of the kth semantic analysis model corresponding to the kth device group is positively correlated with the typical task complexity of the kth device group.

[0151] The semantic analysis model distillation module 14 is used to perform hierarchical distillation training on the K semantic analysis models based on a federated learning framework, in coordination with multiple edge computing devices and the server.

[0152] The semantic analysis model distribution module 15 is used to distribute the trained K semantic analysis models to each end-side computing device in the K device groups for video semantic analysis.

[0153] Furthermore, the task complexity acquisition module 11 includes the following execution steps:

[0154] Based on the identification task type and the prior category rule matrix, the baseline task coefficients of multiple edge computing devices are determined.

[0155] Multiple edge computing devices locally collect real-time video data features and perform feature fusion on the real-time video data features;

[0156] Using the baseline task coefficient as the weight coefficient, the feature fusion results are adjusted respectively to obtain the task complexity of multiple edge computing devices.

[0157] Furthermore, the end-side device group partitioning module 12 includes the following execution steps:

[0158] Based on the complexity of the multiple tasks, the number of groups K is determined using the elbow method.

[0159] A density clustering algorithm is used to divide multiple edge computing devices into K device groups based on the spatial proximity relationship between the complexities of multiple tasks.

[0160] Furthermore, the semantic analysis model initialization module 13 includes the following execution steps:

[0161] Based on the preset analysis window and the historical analysis records of multiple edge computing devices, a global mapping model for complexity-model size parameters is determined.

[0162] Combining the elbow method, we analyze and obtain the complexity of K typical tasks for K groups of the aforementioned devices;

[0163] Traverse the K complexity-model-size global mapping models, match and determine the model size parameters corresponding to the K typical task complexities, and configure the K semantic analysis models accordingly, wherein the K semantic analysis models have the same model type.

[0164] Among them, based on a preset analysis window and historical analysis records from multiple edge computing devices, a global mapping model for complexity-model size parameters is determined, including:

[0165] Based on a preset analysis window, the raw analysis logs are extracted by traversing multiple edge computing devices.

[0166] The system acquires real-time analysis task requirements from multiple edge computing devices, and uses these real-time analysis task requirements as targets to filter out analysis records that meet the requirements from the original analysis logs, and outputs them as historical analysis records.

[0167] Based on the historical analysis records, regression analysis was performed using historical task complexity and historical model size parameters as analysis dimensions to obtain the global mapping model of complexity-model size parameters.

[0168] Among them, combining the elbow method, the complexity of K typical tasks for obtaining K groups of the aforementioned devices is analyzed, including:

[0169] One of the K device groups is randomly selected as the target group for analysis;

[0170] The task complexity of multiple edge computing devices in the target analysis group is obtained, and the task complexity distribution of the target analysis group is analyzed and obtained. The task complexity distribution is defined with the task complexity value as the first coordinate and the density of task complexity at the task complexity value as the second coordinate.

[0171] By combining the elbow method, the elbow points of the task complexity distribution are obtained, and the task complexity value at the elbow point is output as the typical task complexity.

[0172] The analysis target group is iteratively updated until the K typical task complexities corresponding to the K device groups are obtained.

[0173] Furthermore, the semantic analysis model distillation module 14 includes the following execution steps:

[0174] Each device is instructed to train the semantic analysis model corresponding to its respective device group locally based on local video data, and then upload the model update.

[0175] Using the K device groups as indexes, the model updates received within the same device group are aggregated, and the corresponding semantic analysis model is updated.

[0176] Using the k-th semantic analysis model as the student model and the (k+1)-th semantic analysis model as the teacher model, hierarchical distillation training is performed on the server side by traversing the K semantic analysis models.

[0177] Specifically, the k-th semantic analysis model is used as the student model, and the (k+1)-th semantic analysis model is used as the teacher model. Hierarchical distillation training is performed on the server side by traversing the K semantic analysis models, including:

[0178] Define the kth semantic analysis model as the student model, and define the (k+1)th semantic analysis model as the teacher model, where k is less than or equal to K-1;

[0179] Obtain the high-order sample input and high-order training output of the (k+1)th semantic analysis model, and generate the corresponding pseudo-low-order sample input based on the high-order sample input;

[0180] Establish the correlation between the pseudo-low-order sample input and the high-order training output, and output the k-th stratified distillation sample;

[0181] Based on the k-th stratified distillation sample, the k-th semantic analysis model is iteratively trained, and the distillation loss is calculated. The student model is then optimized based on the distillation loss, wherein the distillation loss is calculated based on the difference between the high-level training output and the student model's output on the k-th stratified distillation sample.

[0182] Iterative hierarchical distillation training is performed by traversing K semantic analysis models.

[0183] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0184] Those skilled in the art will understand that embodiments of the present invention can provide methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0185] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0186] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0187] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0188] Although preferred embodiments of the invention have been described, those skilled in the art, once they have learned the basic inventive concept, can make other changes and modifications to these embodiments.

[0189] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of this invention and its equivalents, this invention also intends to include these modifications and variations.

Claims

1. A multi-video semantic collaborative analysis method based on federated learning, characterized in that, include: The task complexity reported by multiple edge computing devices is obtained, wherein the task complexity is dynamically calculated and generated by each edge computing device based on local video data features; Based on the complexity of the tasks, the multiple edge computing devices are divided into K device groups, where K is an integer greater than 1; On the server side, K semantic analysis models are initialized for the K device groups respectively, wherein the structural complexity of the kth semantic analysis model corresponding to the kth device group is positively correlated with the typical task complexity of the kth device group; Based on the federated learning framework, multiple edge computing devices and the server are coordinated to perform hierarchical distillation training on the K semantic analysis models. The trained K semantic analysis models are distributed to each end-side computing device in the K device groups for video semantic analysis.

2. The multi-video semantic collaborative analysis method based on federated learning as described in claim 1, characterized in that, Obtain the task complexity reported by multiple edge computing devices, including: Based on the identification task type and the prior category rule matrix, the baseline task coefficients of multiple edge computing devices are determined. Multiple edge computing devices locally collect real-time video data features, and perform feature fusion on the real-time video data features; Using the baseline task coefficient as the weight coefficient, the feature fusion results are adjusted respectively to obtain the task complexity of multiple edge computing devices.

3. The multi-video semantic collaborative analysis method based on federated learning as described in claim 2, characterized in that, Based on the aforementioned task complexities, multiple edge computing devices are divided into K device groups, including: Based on the complexity of the multiple tasks, the number of groups K is determined using the elbow method. A density clustering algorithm is used to divide multiple edge computing devices into K device groups based on the spatial proximity relationship between the complexities of multiple tasks.

4. The multi-video semantic collaborative analysis method based on federated learning as described in claim 3, characterized in that, On the server side, K semantic analysis models are initialized for each of the K device groups. The structural complexity of the k-th semantic analysis model corresponding to the k-th device group is positively correlated with the typical task complexity of the k-th device group, including: Based on the preset analysis window and the historical analysis records of multiple edge computing devices, a global mapping model for complexity-model size parameters is determined. Combining the elbow method, we analyze and obtain the complexity of K typical tasks for K groups of the aforementioned devices; Traverse the K complexity-model-size global mapping models, match and determine the model size parameters corresponding to the K typical task complexities, and configure the K semantic analysis models accordingly, wherein the K semantic analysis models have the same model type.

5. The multi-video semantic collaborative analysis method based on federated learning as described in claim 4, characterized in that, Based on a preset analysis window and historical analysis records from multiple edge computing devices, a global mapping model for complexity-model size parameters is determined, including: Based on a preset analysis window, the raw analysis logs are extracted by traversing multiple edge computing devices. The system acquires real-time analysis task requirements from multiple edge computing devices, and uses these real-time analysis task requirements as targets to filter out analysis records that meet the requirements from the original analysis logs, and outputs them as historical analysis records. Based on the historical analysis records, regression analysis was performed using historical task complexity and historical model size parameters as analysis dimensions to obtain the global mapping model of complexity-model size parameters.

6. The multi-video semantic collaborative analysis method based on federated learning as described in claim 5, characterized in that, Combining the elbow method, we analyze the complexity of K typical tasks for obtaining K groups of the aforementioned devices, including: One of the K device groups is randomly selected as the target group for analysis; The task complexity of multiple edge computing devices in the target analysis group is obtained, and the task complexity distribution of the target analysis group is analyzed and obtained. The task complexity distribution is defined with the task complexity value as the first coordinate and the density of task complexity at the task complexity value as the second coordinate. By combining the elbow method, the elbow points of the task complexity distribution are obtained, and the task complexity value at the elbow point is output as the typical task complexity. The analysis target group is iteratively updated until the K typical task complexities corresponding to the K device groups are obtained.

7. The multi-video semantic collaborative analysis method based on federated learning as described in claim 1, characterized in that, Based on a federated learning framework, multiple edge computing devices and the server collaborate to perform hierarchical distillation training on the K semantic analysis models, including: Each device is instructed to train the semantic analysis model corresponding to its device group locally based on local video data, and then upload the model update. Using the K device groups as indexes, the model updates received within the same device group are aggregated, and the corresponding semantic analysis model is updated. Using the k-th semantic analysis model as the student model and the (k+1)-th semantic analysis model as the teacher model, hierarchical distillation training is performed on the server side by traversing the K semantic analysis models.

8. The multi-video semantic collaborative analysis method based on federated learning as described in claim 7, characterized in that, Using the k-th semantic analysis model as the student model and the (k+1)-th semantic analysis model as the teacher model, hierarchical distillation training is performed on the server side by traversing the K semantic analysis models, including: Define the kth semantic analysis model as the student model, and define the (k+1)th semantic analysis model as the teacher model, where k is less than or equal to K-1; Obtain the high-order sample input and high-order training output of the (k+1)th semantic analysis model, and generate the corresponding pseudo-low-order sample input based on the high-order sample input; Establish the correlation between the pseudo-low-order sample input and the high-order training output, and output the k-th stratified distillation sample; Based on the k-th stratified distillation sample, the k-th semantic analysis model is iteratively trained, and the distillation loss is calculated. The student model is then optimized based on the distillation loss, wherein the distillation loss is calculated based on the difference between the high-level training output and the student model's output on the k-th stratified distillation sample. Iterative hierarchical distillation training is performed by traversing K semantic analysis models.

9. A multi-video semantic collaborative analysis system based on federated learning, characterized in that, To implement the federated learning-based multi-video semantic collaborative analysis method according to any one of claims 1-8, it includes: The task complexity acquisition module is used to acquire the task complexity reported by multiple end-side computing devices, wherein the task complexity is dynamically calculated and generated by each end-side computing device based on local video data features; The edge device grouping module is used to divide multiple edge computing devices into K device groups based on the complexity of multiple tasks, where K is an integer greater than 1; The semantic analysis model initialization module is used to initialize K semantic analysis models for the K device groups on the server side, wherein the structural complexity of the kth semantic analysis model corresponding to the kth device group is positively correlated with the typical task complexity of the kth device group. The semantic analysis model distillation module is used to perform hierarchical distillation training on the K semantic analysis models based on a federated learning framework, in coordination with multiple edge computing devices and the server. The semantic analysis model distribution module is used to distribute the trained K semantic analysis models to each end-side computing device in the K device groups for video semantic analysis.

Images