Intelligent monitoring processing methods, devices, equipment, and media based on transfer learning

Abstract

This application relates to a monitoring intelligent processing method, apparatus, device, and medium based on transfer learning. The method includes: acquiring monitoring data via IoT communication connected to monitoring devices; preprocessing the data to obtain initial monitoring data; performing data analysis on the initial monitoring data to obtain labeled business segments and non-business segments; generating a decision scheme and executing the decision scheme when triggered, while sending the business segments and non-business segments to an integrated intelligent monitoring processing center; training an integrated model based on the business segments and non-business segments to obtain a target integrated model and target monitoring feature data; performing transfer learning based on the target monitoring feature data to generate multiple target lightweight models, and deploying these multiple target lightweight models to the connected devices. This application improves the intelligence and efficiency of monitoring intelligent processing, meets the requirements of real-time monitoring, and reduces the cost of monitoring video storage.

Description

Technical Field

[0001] This application relates to the field of Internet of Things (IoT) technology, and in particular to a monitoring intelligent processing method, device, equipment, and medium based on transfer learning. Background Technology

[0002] With the development of the Internet of Things and the popularization of enterprise private domains, more and more intersections, workplaces, schools, and warehouses are being equipped with surveillance equipment, which simultaneously transmits video to the cloud. The purpose of installing surveillance is to better protect our property and provide video evidence to reconstruct the scene in case of disputes. Surveillance systems play an increasingly important role in production and daily life, becoming an indispensable security barrier. The need for video surveillance in public areas such as banks, supermarkets, shopping malls, shops, factories, schools, residential areas, internet cafes, and public transportation (hereinafter collectively referred to as video storage locations) is self-evident. The crime-solving rate of public security organs across the country has also skyrocketed with the assistance of video surveillance. With the construction of safe cities, surveillance systems are increasingly integrating into our lives and playing their role.

[0003] As such scenarios become increasingly common, the amount of surveillance video also grows. Existing behavior recognition methods often require complex image processing and calculations on these videos, resulting in high computational costs, low accuracy, and slow response times, failing to meet the requirements of real-time monitoring. On the other hand, the amount of video data that needs to be stored is increasing. Video storage providers either can only store videos from the past 7 days or need to incur higher costs by leasing larger cloud resources or even purchasing hardware for data transfer. Given these high costs or the inability to store sufficient data from the past, scenario-based video processing is a good approach. This method not only reduces storage costs but also optimizes the dataset for model learning, speeds up the process of identifying anomalies for administrators, and strengthens the risk protection capabilities of alarm systems. Summary of the Invention

[0004] The purpose of this application is to propose a monitoring intelligent processing method, device, equipment, and medium based on transfer learning, so as to improve the intelligence and efficiency of monitoring intelligent processing, thereby meeting the requirements of real-time monitoring and reducing the cost of monitoring video storage.

[0005] To address the aforementioned technical problems, embodiments of this application provide a monitoring intelligent processing method based on transfer learning, comprising:

[0006] Monitoring data is obtained by connecting to monitoring devices via IoT communication, and the monitoring data is preprocessed to obtain initial monitoring data;

[0007] The initial monitoring data is analyzed to obtain tagged business segments and non-business segments;

[0008] A decision scheme is generated based on the business segment, and when the decision scheme is triggered, the decision scheme is executed, and the business segment and the non-business segment are sent to the intensive intelligent monitoring and processing center at preset time intervals.

[0009] The intensive model is trained based on the business segments and the non-business segments to obtain the target intensive model and target monitoring feature data;

[0010] Based on the target monitoring feature data, transfer learning is performed to generate multiple target lightweight models, and these multiple target lightweight models are deployed to downstream devices.

[0011] To address the aforementioned technical problems, embodiments of this application provide a monitoring intelligent processing device based on transfer learning, comprising:

[0012] The monitoring data acquisition unit is used to acquire monitoring data by connecting to monitoring devices via Internet of Things communication, and to preprocess the monitoring data to obtain initial monitoring data;

[0013] The data analysis unit is used to perform data analysis on the initial monitoring data to obtain business segments and non-business segments with tags;

[0014] The decision scheme execution unit is used to generate a decision scheme based on the business segment, and execute the decision scheme when it is triggered, and send the business segment and the non-business segment to the integrated intelligent monitoring and processing center at a preset time interval.

[0015] The intensive model training unit is used to train the intensive model based on the business segment and the non-business segment to obtain the target intensive model and target monitoring feature data;

[0016] The lightweight model deployment unit is used to perform transfer learning based on the target monitoring feature data, generate multiple target lightweight models, and deploy the multiple target lightweight models to the downstream devices.

[0017] To solve the above-mentioned technical problems, one technical solution adopted by the present invention is to provide an electronic device, including one or more processors; and a memory for storing one or more programs, such that the one or more processors implement the monitoring intelligent processing method based on transfer learning as described in any one of the above-mentioned methods.

[0018] To solve the above-mentioned technical problems, one technical solution adopted by the present invention is: a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the monitoring intelligent processing method based on transfer learning described above.

[0019] This invention provides a method, apparatus, device, and medium for intelligent monitoring processing based on transfer learning. The method includes: acquiring monitoring data via IoT communication connected to a monitoring device and preprocessing the monitoring data to obtain initial monitoring data; performing data analysis on the initial monitoring data to obtain labeled business segments and non-business segments; generating a decision scheme based on the business segments, and executing the decision scheme when triggered, while sending the business segments and non-business segments to an intensive intelligent monitoring processing center at preset time intervals; training an intensive model based on the business segments and non-business segments to obtain a target intensive model and target monitoring feature data; performing transfer learning based on the target monitoring feature data to generate multiple target lightweight models, and deploying the multiple target lightweight models to the connected device. This invention achieves swarm intelligence by analyzing monitoring data and generating decision schemes for real-time monitoring data processing. Simultaneously, by training the intensive model and performing transfer learning to generate multiple target lightweight models and deploying them to the connected device, it realizes the ability of collective intelligence, thereby improving the intelligence and efficiency of intelligent monitoring processing, meeting the requirements of real-time monitoring, and reducing the cost of monitoring video storage. Attached Figure Description

[0020] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the implementation of the intelligent monitoring processing method based on transfer learning provided in this application embodiment;

[0022] Figure 2 This is a flowchart illustrating the implementation of a sub-process in the intelligent monitoring processing method based on transfer learning provided in this application embodiment;

[0023] Figure 3 This is a flowchart illustrating the implementation of a sub-process in the intelligent monitoring processing method based on transfer learning provided in this application embodiment;

[0024] Figure 4 This is a flowchart illustrating the implementation of a sub-process in the intelligent monitoring processing method based on transfer learning provided in this application embodiment;

[0025] Figure 5 This is a flowchart illustrating the implementation of a sub-process in the intelligent monitoring processing method based on transfer learning provided in this application embodiment;

[0026] Figure 6 This is a flowchart illustrating the implementation of a sub-process in the intelligent monitoring processing method based on transfer learning provided in this application embodiment;

[0027] Figure 7 This is a schematic diagram of a monitoring intelligent processing device based on transfer learning provided in an embodiment of this application;

[0028] Figure 8 This is a schematic diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0030] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0031] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0032] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0033] It should be noted that the intelligent monitoring processing method based on transfer learning provided in this application is generally executed by an electronic device, and correspondingly, the intelligent monitoring processing device based on transfer learning is generally configured in the electronic device.

[0034] Please see Figure 1 , Figure 1 This paper illustrates a specific implementation of a monitoring intelligent processing method based on transfer learning.

[0035] It should be noted that if substantially the same result is obtained, the method of this invention is not based on... Figure 1 Limited to the order of the processes shown, this method includes the following steps:

[0036] S1: Obtain monitoring data by connecting to monitoring equipment via IoT communication, and preprocess the monitoring data to obtain initial monitoring data.

[0037] In this embodiment, monitoring data is acquired in real time via a downstream monitoring device connected through an Internet of Things (IoT) network. For example, in a security monitoring system, a wireless camera is used as the monitoring device, transmitting video data to a central control server via a network. The downstream device preprocesses the monitoring data, including data cleaning, noise reduction, and standardization, to improve the accuracy and efficiency of subsequent processing. For example, for video data, image denoising and enhancement processing can be performed to remove background noise and adjust image brightness and contrast.

[0038] S2: Perform data analysis on the initial monitoring data to obtain tagged business segments and non-business segments.

[0039] In this embodiment of the application, data analysis is performed on the initial monitoring data to extract the same features such as video or other data, and to obtain business segments and non-business segments with tags.

[0040] Please see Figure 2 , Figure 2 A specific implementation of step S2 is shown below:

[0041] S21: Extract features from the initial monitoring data to obtain target feature data.

[0042] S22: Based on a preset time period, the target feature data is cropped to obtain cropped video data.

[0043] S23: Label the cropped video data based on the business type to obtain the business segment, and label the cropped video data based on the non-business type to obtain the non-business segment.

[0044] In this embodiment, feature extraction is performed on the initial monitoring data to obtain target feature data, wherein the extracted features are video or other data-related features. Then, the target feature data is cropped based on a preset time period to extract video data for the corresponding time period, resulting in cropped video data. The cropped video data is labeled according to the service requirements of the downstream device to obtain service segments, and non-service segments are labeled accordingly. The preset time period is set according to actual conditions and is not limited here.

[0045] S3: Generate a decision scheme based on the business segment, and execute the decision scheme when it is triggered, and send the business segment and the non-business segment to the integrated intelligent monitoring and processing center at preset time intervals.

[0046] In this embodiment, the downstream device formulates corresponding decision-making schemes based on data analysis results. For example, it may trigger an alarm, send a notification, or perform a specific operation. If it's a segment that needs to be used periodically, it may trigger an upload. The decision-making process for business segments is optimized and customized according to specific application scenarios. Corresponding operations are implemented on monitoring equipment or other related equipment, such as activating protection mechanisms, adjusting monitoring parameters, or sending control signals. Business segments and non-business segments are uniformly stored locally and uploaded to the centralized intelligent monitoring and processing center daily or weekly.

[0047] S4: Train the intensive model based on the business segment and the non-business segment to obtain the target intensive model and target monitoring feature data.

[0048] In this embodiment of the application, in the intensive intelligent monitoring and processing center, the intensive model is trained according to business segments and non-business segments to obtain the target intensive model and target monitoring feature data.

[0049] Please see Figure 3 , Figure 3 A specific implementation of step S4 is shown below:

[0050] S41: Merge the business segments and the non-business segments to obtain a dataset that includes videos of different categories, wherein each video in the dataset is labeled with a corresponding category label.

[0051] S42: Perform frame-by-frame processing on each video in the video set to obtain frame-by-frame video data.

[0052] S43: The segmented video data is cropped to obtain cropped video data, and the cropped video data is normalized to obtain training data.

[0053] S44: Train the intensive model based on the training data to obtain the target intensive model and the target monitoring feature data.

[0054] In this embodiment, business segments and non-business segments are integrated into a dataset containing videos of different categories, where each video is labeled with a corresponding category tag. Each video in the dataset is split into small consecutive frames, resulting in frame-segmented video data. For example, one frame is sampled per second, thus converting the video into an image sequence, facilitating subsequent input into a convolutional neural network model. The video data is then cropped to the location of the accident, resulting in cropped video data. Location, environment, and time tags are recorded in this cropped video data. The cropped video is then normalized to a value range between 0 and 1, yielding training data. Finally, the intensive model is trained based on the training data to obtain the target intensive model and target monitoring feature data.

[0055] Please see Figure 4 , Figure 4 A specific implementation of step S44 is shown below:

[0056] S441: The motion vector information between frames in the training data is extracted using the optical flow method.

[0057] S442: Use the motion vector information and the image frames in the training data as target training data, and input the target training data into the VGG model for model training to obtain the target simplification model.

[0058] S443: Generate a test dataset based on the non-business segment, evaluate the target intensive model using the test dataset, and after the target intensive model is evaluated and labeled, obtain the target training data of the current iteration and use the target training data of the current iteration as the target monitoring feature data.

[0059] In this embodiment, VGG is used as the model basis. Since video consists of a series of frames, video sequences need to be processed. Therefore, this embodiment employs optical flow to extract motion vector information between frames. Optical flow can obtain motion vector information by calculating pixel changes between adjacent frames. This motion vector information can be used as part of the input sequence and input into the VGG model along with the image frames. Then, the target training data is input into the VGG model for model training to obtain the target-focused model. A test dataset is then generated based on non-business segments, and the target-focused model is evaluated using the test dataset. After the target-focused model is evaluated and labeled, the target training data for the current iteration is obtained and used as the target monitoring feature data.

[0060] Specifically, convolutional layers, pooling layers, and fully connected layers are added to the VGG model, and appropriate activation functions (ReLU) and regularization methods (Dropout) are selected. The output layer uses softmax as the activation function, suitable for the multi-class classification problem in this embodiment. The specific model training process involves inputting the prepared target training data into the VGG model for training. Cross-entropy loss and optimization algorithms (e.g., the Adam optimizer) are used to minimize the difference between the predicted output and the true label. Simultaneously, a portion of the target training data can be divided into a validation set, which is used to monitor the model's training progress and adjust hyperparameters, thereby obtaining the target-optimized model. A test dataset is then generated based on non-business segments. The target-optimized model is evaluated using this test dataset to calculate its accuracy on the test data. If the accuracy reaches a set value, the target-optimized model is considered successfully evaluated. Therefore, after the target-optimized model is evaluated, the target training data for the current iteration is obtained and used as the target monitoring feature data.

[0061] S5: Based on the target monitoring feature data, perform transfer learning to generate multiple target lightweight models, and deploy the multiple target lightweight models to the downstream devices.

[0062] Please see Figure 5 , Figure 5 A specific implementation of step S5 is shown below:

[0063] S51: Select one category from the target monitoring feature data as the target category, and merge the remaining categories into non-target categories.

[0064] Specifically, select one category from the target monitoring feature data as the target category (e.g., vehicle-to-human collision is a specific target type), and then merge all other categories into a single non-target category. It should be noted that specific categories can be selected for processing based on the actual situation; for example, vehicle-to-human collision can be considered a specific category.

[0065] S52: Obtain the target monitoring feature data corresponding to the target category as target category data, and obtain the target monitoring data corresponding to the non-target category as non-target category data.

[0066] S53: Modify the target intensive model to obtain the initial target intensive model.

[0067] Specifically, the model structure is modified according to the requirements of the target task. Using transfer learning, some layers or feature extractors of the intensive model are retained, and the classification layer is replaced to adapt to new target and non-target class data.

[0068] S54: Fine-tune the initial target aggregation model using the target category data and the non-target category data, and evaluate the fine-tuned initial target aggregation model to obtain multiple target lightweight models.

[0069] Specifically, the target-oriented lightweight model is fine-tuned using both target and non-target class data. During fine-tuning, the model attempts to freeze most of its layers, training only newly added or modified classification layers to optimize for both target and non-target classes. Different layers in the VGG model are frozen for different scenarios to achieve the best results. The fine-tuned model is evaluated using a test dataset to ensure it performs well in distinguishing between target and non-target classes. If the accuracy improvement is significantly higher than the current downlink device model, the lightweight transfer learning training is considered successful.

[0070] Furthermore, a specific implementation method following step S54 is provided: the target lightweight model is compressed using a preset model compression technique, wherein the preset model compression technique is any one of pruning technique, quantization technique, and model distillation technique.

[0071] Specifically, if the size of the target lightweight model is still too large, model compression techniques, such as pruning, quantization, or model distillation, are used to reduce the size of the model and the computational resource requirements.

[0072] In a specific embodiment, preprocessed data belonging to the broad category of traffic accident data is divided into two subcategories: vehicle-to-person collisions and "other" categories. The "other" subcategories include vehicle-to-vehicle collisions, running red lights, etc. Common sense tells us that these subcategories share features such as vehicle movement and sudden changes in the movement of other objects. The network layers of the intensive model are frozen, and only the subsequent fully connected layers are trained. The pre-trained network is used as a feature extractor for the new task. This allows for more efficient feature extraction and differentiation based on the vehicle-to-person collision subcategory, which commonly recognizes a sudden displacement of a person's coordinates. Furthermore, based on this feature, the system can be configured with granularity. For example, if the displacement is less than one meter, only one ambulance needs to be prepared; if the displacement is greater than a certain number of meters or multiple displacements occur, multiple ambulances can be dispatched, and whether blood bags are needed, etc. This kind of optimization in business scenarios has extremely high value. Furthermore, depending on the different devices, for example, the performance of a device monitoring speeding is certainly not as high as that of a terminal device that takes photos and uploads them while passing by. Therefore, in the transfer learning of speeding, the embodiments of this application can reduce the number of fully connected layers and use model distillation to reduce the CPU and memory required for the actual operation and analysis of the model. While not affecting the capabilities of the speeding monitoring camera itself, it is equipped with the collective intelligence capabilities of the downstream devices, which greatly improves the efficiency of hardware and software utilization.

[0073] S55: Deploy multiple of the target lightweight models to the downstream device.

[0074] Please see Figure 6 , Figure 6 A specific implementation of step S55 is shown below:

[0075] S551: Obtain the downstream device and convert the format of the multiple target lightweight models to obtain multiple converted target lightweight models.

[0076] S552: Using a preset inference framework, multiple converted target lightweight models are deployed in the downstream device respectively.

[0077] In this embodiment, a suitable hardware device is selected as the downlink device based on the model's computational requirements. If real-time processing is required, a GPU or a dedicated deep learning accelerator (TensorRT) can be considered. The target lightweight model is converted to a deployment-appropriate format, such as ONNX or TensorFlow Lite. Format conversion reduces the model size and achieves better performance. During model deployment, the model can be deployed to an embedded system or mobile device (downlink device) and accelerated using a pre-defined inference framework (such as TensorRT, Core ML, or Android NN). Furthermore, based on the frequency of incidents, rapid model iteration is performed during low-frequency periods. Packet transmission can be attempted first to ensure network connectivity before iteration. If successful, a replacement is used; otherwise, the original configuration is maintained to ensure normal capability utilization.

[0078] In this embodiment, monitoring data is acquired via IoT communication connected to a monitoring device. The monitoring data is preprocessed to obtain initial monitoring data. Data analysis is performed on the initial monitoring data to obtain tagged business segments and non-business segments. A decision-making scheme is generated based on the business segments, and when the decision-making scheme is triggered, it is executed. The business segments and non-business segments are sent to an intensive intelligent monitoring processing center at preset time intervals. An intensive model is trained based on the business segments and non-business segments to obtain a target intensive model and target monitoring feature data. Transfer learning is performed based on the target monitoring feature data to generate multiple target lightweight models, which are then deployed to the connected devices. This embodiment of the invention analyzes monitoring data and generates decision-making schemes for real-time monitoring data processing. Simultaneously, by training the intensive model and performing transfer learning to generate multiple target lightweight models and deploying them to the connected devices, it achieves swarm intelligence capabilities, thereby improving the intelligence and efficiency of intelligent monitoring processing, meeting the requirements of real-time monitoring, and reducing the cost of monitoring video storage.

[0079] Furthermore, by using the built-in model processing mechanism of the video surveillance terminal to first identify and process the video content that needs attention, a collective intelligent monitoring capability is formed. In contrast, the most traditional monitoring system mainly uses cameras to capture images or videos in real time and relies on manual monitoring. Due to the vast monitoring area and limited human resources, only a few areas can be monitored, and comprehensive monitoring is not possible. Moreover, due to fatigue and instability issues in manual monitoring, the monitoring quality cannot be guaranteed. Current monitoring systems mainly use simple behavior recognition models to identify common monitored behaviors based on fixed datasets, and all data is classified in a single model, which often deviates from the actual physical scene. For example, in traffic scenarios, multiple downstream devices such as speeding, accident monitoring, and roadside photography are often located in the same place. Due to the limited accuracy and response time of existing monitoring algorithms, when multiple accidents occur simultaneously with high concurrency, such as multiple situations where a speeding car crashes and flees, this processing method will assess the severity of the accident and send a police pursuit order. Based on the collision state and vehicle type, it will assess whether an ambulance or fire department order is needed. Existing models, when used with edge computing, may not be able to detect potential security vulnerabilities and issue all corresponding alarms in a timely manner; they can only trigger alarms from the police or hospitals on-site. Because many business scenarios already have well-established recognition models, this method requires less computational resources to achieve higher accuracy and alarm and processing methods that better meet specific needs.

[0080] Furthermore, compared to commonly used neural network models, this embodiment employs an optical flow technique based on the VGG model, which is more suitable for video stream analysis. By perceiving and analyzing the behavior of monitored targets within the monitoring area, it can more accurately and efficiently identify abnormal behaviors and continuously monitor target objects in a comprehensive manner, thereby greatly improving monitoring effectiveness and security. Simultaneously, this embodiment can automatically determine abnormal situations and issue alarm signals in real time, avoiding the problems of manual omissions or delays, and improving monitoring efficiency and reliability. Setting the granularity of anomalies allows for the recording of certain unconventional actions, greatly saving storage space and making subsequent accountability faster and more convenient. Moreover, this embodiment distributes system resources to each edge device, saving significant manpower for labeling and utilizing the hardware resources of the edge devices for data preprocessing, enabling the intensive model to rapidly iterate and respond in different scenarios (labels). Because various processing power terminals exist in the market, the embodiments of this application can select the appropriate terminal according to the corresponding needs and perform corresponding distillation on the model to solve the problems of resource consumption for large-scale cloud traffic transmission and network resources. High accuracy can be achieved in a short time when accessing different terminals. The lightweight model obtained by distilling based on device performance using transfer learning is deployed to the device to achieve swarm intelligence, ensuring the practicality of this application.

[0081] Please refer to Figure 7 As a response to the above Figure 1 The implementation of the method shown in this application provides an embodiment of a monitoring intelligent processing device based on transfer learning. This device embodiment is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0082] like Figure 7 As shown, the intelligent monitoring processing device based on transfer learning in this embodiment includes: a monitoring data acquisition unit 61, a data analysis unit 62, a decision-making scheme execution unit 63, an intensive model training unit 64, and a lightweight model deployment unit 65, wherein:

[0083] The monitoring data acquisition unit 61 is used to acquire monitoring data by connecting to the monitoring device through Internet of Things communication, and to preprocess the monitoring data to obtain initial monitoring data.

[0084] Data analysis unit 62 is used to perform data analysis on the initial monitoring data to obtain business segments and non-business segments with tags;

[0085] The decision scheme execution unit 63 is used to generate a decision scheme based on the business segment, and execute the decision scheme when it is triggered, and send the business segment and the non-business segment to the integrated intelligent monitoring and processing center at a preset time interval.

[0086] The intensive model training unit 64 is used to train the intensive model based on the business segment and the non-business segment to obtain the target intensive model and target monitoring feature data;

[0087] The lightweight model deployment unit 65 is used to perform transfer learning based on the target monitoring feature data, generate multiple target lightweight models, and deploy the multiple target lightweight models to the downstream devices.

[0088] Furthermore, the intensive model training unit 64 includes:

[0089] A video segment merging unit is used to merge the business segments and the non-business segments to obtain a dataset including videos of different categories, wherein each video in the dataset is labeled with a corresponding category label;

[0090] The frame-segmentation processing unit is used to perform frame-segmentation processing on each video in the video set to obtain frame-segmented video data.

[0091] The data cropping unit is used to crop the framed video data to obtain cropped video data, and to normalize the cropped video data to obtain training data.

[0092] The model training unit is used to train the intensive model based on the training data to obtain the target intensive model and the target monitoring feature data.

[0093] Furthermore, the model training unit includes:

[0094] A motion vector information extraction unit is used to extract motion vector information between frames in the training data using optical flow.

[0095] The target ensemble model generation unit is used to take the motion vector information and the image frames in the training data as target training data, and input the target training data into the VGG model for model training to obtain the target ensemble model;

[0096] The monitoring feature data acquisition unit is used to generate a test dataset based on the non-business segment, evaluate the target intensive model using the test dataset, and when the target intensive model evaluation meets the standard, acquire the target training data of the current iteration and use the target training data of the current iteration as the target monitoring feature data.

[0097] Furthermore, the lightweight model deployment unit 65 includes:

[0098] The category selection unit is used to select one category from the target monitoring feature data as the target category and merge the remaining categories into non-target categories.

[0099] The category data acquisition unit is used to acquire target monitoring feature data corresponding to the target category as target category data, and to acquire target monitoring data corresponding to the non-target category as non-target category data;

[0100] The model modification unit is used to modify the target intensive model to obtain an initial target intensive model.

[0101] The model fine-tuning unit is used to fine-tune the initial target ensemble model with the target category data and the non-target category data, and to evaluate the fine-tuned initial target ensemble model to obtain multiple target lightweight models.

[0102] The model deployment unit is used to deploy multiple target lightweight models to downstream devices.

[0103] Furthermore, the model fine-tuning unit also includes:

[0104] The model compression unit is used to compress the target lightweight model using a preset model compression technique, wherein the preset model compression technique is any one of pruning technique, quantization technique, and model distillation technique.

[0105] Furthermore, the model deployment unit includes:

[0106] The format conversion unit is used to acquire the downstream device and convert the formats of the multiple target lightweight models to obtain multiple converted target lightweight models;

[0107] The target lightweight model deployment unit is used to deploy multiple converted target lightweight models in the downstream device using a preset inference framework.

[0108] Furthermore, the data analysis unit 62 includes:

[0109] A feature extraction unit is used to extract features from the initial monitoring data to obtain target feature data;

[0110] The cropped video data generation unit is used to crop the target feature data based on a preset time period to obtain cropped video data.

[0111] The data annotation unit is used to annotate the cropped video data based on the business type to obtain the business segment, and to annotate the cropped video data based on the non-business type to obtain the non-business segment.

[0112] To address the aforementioned technical problems, embodiments of this application also provide an electronic device. Please refer to [link / reference needed] for details. Figure 8 , Figure 8 This is a basic structural block diagram of the electronic device in this embodiment.

[0113] Electronic device 7 includes a memory 71, a processor 72, and a network interface 73 that are interconnected via a system bus. It should be noted that only electronic device 7 with three components—memory 71, processor 72, and network interface 73—is shown in the figure; however, it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the electronic device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0114] Electronic devices can be computing devices such as desktop computers, laptops, PDAs, and cloud servers. Electronic devices can interact with users through methods such as keyboards, mice, remote controls, touchpads, or voice-activated devices.

[0115] The memory 71 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 71 may be an internal storage unit of the electronic device 7, such as the hard disk or memory of the electronic device 7. In other embodiments, the memory 71 may also be an external storage device of the electronic device 7, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 7. Of course, the memory 71 may also include both internal storage units and external storage devices of the electronic device 7. In this embodiment, the memory 71 is typically used to store the operating system and various application software installed on the electronic device 7, such as the program code of a monitoring intelligent processing method based on transfer learning. In addition, the memory 71 may also be used to temporarily store various types of data that have been output or will be output.

[0116] In some embodiments, processor 72 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. This processor 72 is typically used to control the overall operation of electronic device 7. In this embodiment, processor 72 is used to run program code stored in memory 71 or process data, for example, to run the program code of the aforementioned transfer learning-based intelligent monitoring processing method to implement various embodiments of the transfer learning-based intelligent monitoring processing method.

[0117] The network interface 73 may include a wireless network interface or a wired network interface, which is typically used to establish a communication connection between the electronic device 7 and other electronic devices.

[0118] This application also provides another embodiment, namely, providing a computer-readable storage medium storing a computer program that can be executed by at least one processor to cause the at least one processor to perform the steps of the above-described monitoring intelligent processing method based on transfer learning.

[0119] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of this application.

[0120] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A monitoring intelligent processing method based on transfer learning, characterized in that, include: The monitoring video data is obtained by connecting to the monitoring equipment through IoT communication, and the monitoring video data is preprocessed to obtain the initial monitoring data; The initial monitoring data is analyzed to obtain tagged business segments and non-business segments; A decision scheme is generated based on the business segment, and when the decision scheme is triggered, the decision scheme is executed, and the business segment and the non-business segment are sent to the intensive intelligent monitoring and processing center at preset time intervals. In the intensive intelligent monitoring and processing center, the intensive model is trained based on the business segments and the non-business segments to obtain the target intensive model and target monitoring feature data; Based on the target monitoring feature data, transfer learning is performed to generate multiple target lightweight models, and the multiple target lightweight models are deployed to the downstream monitoring devices; The step of performing transfer learning based on the target monitoring feature data to generate multiple lightweight target models, and deploying the multiple lightweight target models to the downstream monitoring devices, includes: Select one category from the target monitoring feature data as the target category, and merge the remaining categories into non-target categories; Obtain the target monitoring feature data corresponding to the target category as target category data, and obtain the target monitoring data corresponding to the non-target category as non-target category data; The target intensive model is modified to obtain the initial target intensive model; The initial target aggregation model is fine-tuned using the target category data and the non-target category data, and the fine-tuned initial target aggregation model is evaluated to obtain multiple target lightweight models; Multiple lightweight target models are deployed to the downstream monitoring device.

2. The intelligent monitoring processing method based on transfer learning according to claim 1, characterized in that, The process of training the intensive model based on the business segments and the non-business segments to obtain the target intensive model and target monitoring feature data includes: The business segments and the non-business segments are merged to obtain a dataset that includes videos of different categories, wherein each video in the dataset is labeled with a corresponding category label; Each video in the video set is processed into frames to obtain the framed video data. The segmented video data is cropped to obtain cropped video data, and the cropped video data is normalized to obtain training data. The intensive model is trained based on the training data to obtain the target intensive model and the target monitoring feature data.

3. The intelligent monitoring processing method based on transfer learning according to claim 2, characterized in that, In the intensive intelligent monitoring and processing center, the intensive model is trained based on the training data to obtain the target intensive model and the target monitoring feature data, including: In the intensive intelligent monitoring and processing center, the optical flow method is used to extract motion vector information between frames in the training data; The motion vector information and the image frames in the training data are used as target training data, and the target training data is input into the VGG model for model training to obtain the target condensed model; A test dataset is generated based on the non-business segments. The target intensive model is evaluated using the test dataset. Once the target intensive model evaluation meets the standard, the target training data for the current iteration is obtained and used as the target monitoring feature data.

4. The intelligent monitoring processing method based on transfer learning according to claim 1, characterized in that, After fine-tuning the initial target-based model using the target category data and the non-target category data, and evaluating the fine-tuned initial target-based model to obtain multiple target-lightweight models, the method further includes: The target lightweight model is compressed using a preset model compression technique, wherein the preset model compression technique is any one of pruning technique, quantization technique, and model distillation technique.

5. The intelligent monitoring processing method based on transfer learning according to claim 1, characterized in that, The step of deploying multiple target lightweight models to the downstream monitoring device includes: The downstream monitoring device is acquired, and the multiple target lightweight models are converted to different formats to obtain multiple converted target lightweight models. Using a pre-defined inference framework, multiple lightweight models of the transformed target are deployed in the downstream monitoring devices.

6. The intelligent monitoring processing method based on transfer learning according to any one of claims 1 to 5, characterized in that, The step of performing data analysis on the initial monitoring data to obtain tagged business segments and non-business segments includes: Feature extraction is performed on the initial monitoring data to obtain target feature data; The target feature data is cropped based on a preset time period to obtain cropped video data. The cropped video data is labeled based on the business type to obtain the business segment, and the cropped video data is labeled based on the non-business type to obtain the non-business segment.

7. A monitoring intelligent processing device based on transfer learning, characterized in that, include: The monitoring data acquisition unit is used to acquire monitoring video data by connecting to monitoring equipment via Internet of Things communication, and to preprocess the monitoring video data to obtain initial monitoring data; The data analysis unit is used to perform data analysis on the initial monitoring data to obtain business segments and non-business segments with tags; The decision scheme execution unit is used to generate a decision scheme based on the business segment, and execute the decision scheme when it is triggered, and send the business segment and the non-business segment to the integrated intelligent monitoring and processing center at a preset time interval. An intensive model training unit is used to train an intensive model based on the business segments and the non-business segments in the intensive intelligent monitoring and processing center to obtain a target intensive model and target monitoring feature data. The lightweight model deployment unit is used to perform transfer learning based on the target monitoring feature data, generate multiple target lightweight models, and deploy the multiple target lightweight models to the downstream monitoring devices. The lightweight model deployment unit includes: The category selection unit is used to select one category from the target monitoring feature data as the target category and merge the remaining categories into non-target categories. The category data acquisition unit is used to acquire target monitoring feature data corresponding to the target category as target category data, and to acquire target monitoring data corresponding to the non-target category as non-target category data; The model modification unit is used to modify the target intensive model to obtain an initial target intensive model. The model fine-tuning unit is used to fine-tune the initial target ensemble model with the target category data and the non-target category data, and to evaluate the fine-tuned initial target ensemble model to obtain multiple target lightweight models. The model deployment unit is used to deploy multiple target lightweight models to the downstream monitoring device.

8. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the monitoring intelligent processing method based on transfer learning as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the intelligent monitoring processing method based on transfer learning as described in any one of claims 1 to 6.

Images