Model management method and device, computer equipment and storage medium

Abstract

The embodiment of the invention belongs to the technical field of detection models, and relates to a model management method and device, computer equipment and a storage medium, the method comprises the following steps: receiving real-time video data collected by camera equipment arranged at a side end, the side end having been deployed with an initial lightweight model; performing adaptive sampling operation on the real-time video data to obtain an adaptive sampling result; sending the real-time adaptive sampling result to a cloud teacher model for online labeling operation to obtain an online labeling result; and performing adaptive training operation on the initial lightweight model according to the online labeling result to obtain an optimized lightweight model. According to the method, through knowledge distillation, a large model with high accuracy is used as a teacher model at a cloud end, so that a lightweight model on an edge device continuously learns from the large model, and the accuracy of real-time video object detection is improved.

Description

Technical Field

[0001] The present application relates to the field of detection model technology, and in particular to a model management method, apparatus, computer equipment and storage medium. Background Art

[0002] Real-time video object detection has a wide range of application scenarios, including intelligent traffic monitoring and autonomous driving. Real-time video object detection prefers edge devices for reasoning because timely feedback is required. Edge computing does not require expensive network bandwidth to stream video to the cloud, which can solve the problem of high transmission latency.

[0003] Rapid advances in deep neural networks have driven the development of real-time video inference, but due to limited computing resources on edge devices, only DNNs with fewer weights and shallower architectures can be deployed. Lightweight DNNs are easily affected by data drift, that is, the real-time video content changes dramatically. Models trained offline are difficult to achieve ideal accuracy in a variety of environments, where video scenes change over time, such as crowd density, weather, and lighting.

[0004] It can be seen that the lightweight DNN model has poor adaptability and only performs well in limited scenarios. When the scene changes, objects of the same category will appear visually different, leading to the problem of data drift. Summary of the invention

[0005] The purpose of the embodiments of the present application is to propose a model management method, apparatus, computer device and storage medium to solve the problem of poor adaptability of traditional lightweight DNN models.

[0006] In order to solve the above technical problems, the embodiment of the present application provides a model updating method for real-time object detection, which adopts the following technical solution:

[0007] Receiving real-time video data collected by a camera device disposed at an edge, wherein the edge has deployed an initial lightweight model;

[0008] Performing an adaptive sampling operation on the real-time video data to obtain an adaptive sampling result;

[0009] The real-time adaptive sampling result is sent to the cloud teacher model for online annotation operation to obtain an online annotation result;

[0010] The initial lightweight model is adaptively trained according to the online annotation result to obtain an optimized lightweight model.

[0011] Furthermore, the step of performing an adaptive sampling operation on the real-time video data to obtain an adaptive sampling result specifically includes the following steps:

[0012] The real-time video data is sampled according to a sampling strategy of content similarity to obtain the adaptive sampling result.

[0013] Furthermore, the step of performing an adaptive training operation on the initial lightweight model according to the online annotation result to obtain an optimized lightweight model specifically includes the following steps:

[0014] Calling a lightweight model similar to the initial lightweight model, and performing a model optimization operation on the similar lightweight model according to the online annotation result to obtain an optimized lightweight model;

[0015] The optimized lightweight model is redeployed to the edge to replace the lightweight model, thereby obtaining the optimized lightweight model.

[0016] Furthermore, the step of performing an adaptive training operation on the initial lightweight model according to the online annotation result to obtain an optimized lightweight model specifically includes the following steps:

[0017] Deploying a federated learning framework on the edge;

[0018] The initial lightweight model is federated trained according to the federated learning framework and the online annotation results to obtain the optimized lightweight model.

[0019] Furthermore, after the step of performing adaptive training on the initial lightweight model according to the online annotation result to obtain an optimized lightweight model, the following steps are also included:

[0020] Receiving the latest video data collected by the camera device;

[0021] The latest video data is transmitted to the optimized lightweight model for object detection operation to obtain real-time object detection results.

[0022] Furthermore, after the step of receiving the latest video data collected by the camera device, the following steps are also included:

[0023] Performing a complexity determination operation on the latest video data according to a preset complexity threshold to obtain a complexity determination result;

[0024] If the complexity judgment result exceeds the complexity threshold, the latest video data is sent to the cloud teacher model for object detection operation to obtain a real-time object detection result;

[0025] The real-time object detection result is sent to the real-time edge for display.

[0026] In order to solve the above technical problems, the embodiment of the present application further provides a model updating device for real-time object detection, which adopts the following technical solution:

[0027] A video acquisition module, used to receive real-time video data collected by a camera device arranged at an edge, wherein the edge has deployed an initial lightweight model;

[0028] An adaptive sampling module, used for performing an adaptive sampling operation on the real-time video data to obtain an adaptive sampling result;

[0029] An online annotation module is used to send the real-time adaptive sampling result to the cloud teacher model for online annotation operation to obtain an online annotation result;

[0030] The adaptive training module is used to perform adaptive training operations on the lightweight model according to the online annotation results to obtain an optimized lightweight model.

[0031] Furthermore, the adaptive training module includes:

[0032] A model optimization submodule, used for calling a second lightweight model that is the same as the lightweight model, and performing a model optimization operation on the second lightweight model according to the online annotation result to obtain an optimized second lightweight model;

[0033] The model deployment submodule is used to redeploy the optimized second lightweight model to the edge to replace the lightweight model.

[0034] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0035] The invention comprises a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the model updating method applied to real-time object detection as described above are implemented.

[0036] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0037] The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the model updating method applied to real-time object detection as described above are implemented.

[0038] The present application provides a model update method for real-time object detection, including: receiving real-time video data collected by a camera device set at the edge, wherein the edge has deployed an initial lightweight model; performing an adaptive sampling operation on the real-time video data to obtain an adaptive sampling result; sending the real-time adaptive sampling result to a cloud teacher model for online annotation operation to obtain an online annotation result; performing an adaptive training operation on the initial lightweight model according to the online annotation result to obtain an optimized lightweight model. Compared with the prior art, the present application uses a high-accuracy large model as a teacher model in the cloud through knowledge distillation, so that the lightweight model on the edge device continuously learns from the large model, thereby improving the accuracy of real-time video object detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0041] Figure 2 is a flow chart of an implementation of a model updating method for real-time object detection provided by an embodiment of the present application;

[0042] Figure 3 is a structural diagram of a model updating device for real-time object detection provided by an embodiment of the present application;

[0043] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as understood by a person skilled in the art to which this application belongs; the terms used herein in the specification of the application are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of this application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0045] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0046] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0047] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0048] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0049] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to the laptop computer 1011, tablet computer 1012 or mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0050] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0051] It should be noted that the model updating method for real-time object detection provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the model updating device for real-time object detection is generally arranged in a server / terminal device.

[0052] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0053] Continue to refer Figure 2 , shows a flow chart of an embodiment of a model updating method for real-time object detection according to the present application. The model updating method for real-time object detection comprises: step S201, step S202, step S203, step S204, step S205, step S206 and step S207.

[0054] In step S201, real-time video data collected by a camera device disposed at an edge is received, wherein an initial lightweight model has been deployed at the edge.

[0055] In the embodiment of the present application, edge refers to the devices or nodes at the edge of the network, which can directly interact with users or the physical world. In this scenario, edge refers to the place where the camera equipment is deployed, which is responsible for collecting video data in real time.

[0056] In the embodiment of the present application, the video data collected by the camera device is real-time, which means that the data is captured and transmitted continuously.

[0057] In the embodiment of the present application, an initial lightweight model has been deployed on the edge. A lightweight model refers to a machine learning model that can still run efficiently under limited computing resources (such as processing power, memory, etc.). This model is used for preliminary video analysis and object detection.

[0058] In step S202, an adaptive sampling operation is performed on the real-time video data to obtain an adaptive sampling result.

[0059] In the embodiment of the present application, the adaptive sampling operation is a process of selectively extracting key frames or segments from the original video data according to the dynamic changes of the video content or specific needs. This operation can reduce the amount of data transmitted to the cloud while retaining enough information for subsequent analysis.

[0060] In the embodiment of the present application, the adaptive sampling result refers to the video data obtained after such operation and has been filtered or compressed.

[0061] In step S203, the real-time adaptive sampling result is sent to the cloud teacher model for online annotation operation to obtain the online annotation result.

[0062] In an embodiment of the present application, the cloud teacher model is a more complex and accurate but computationally expensive machine learning model deployed on a cloud server.

[0063] In the embodiment of the present application, the online annotation operation refers to the cloud teacher model analyzing the received adaptive sampling results and generating corresponding annotation data. The annotation data includes information such as object recognition and behavior classification.

[0064] In the embodiment of the present application, the online annotation result is the analysis and understanding of the video data by the cloud-based teacher model, which is used for subsequent model training.

[0065] In step S204, an adaptive training operation is performed on the initial lightweight model according to the online annotation result to obtain an optimized lightweight model.

[0066] In the embodiment of the present application, the adaptive training operation refers to adjusting and optimizing the initial lightweight model on the edge according to the online annotation results returned by the cloud. This training is real-time or near real-time, which means that the model can be quickly updated according to new data.

[0067] In the embodiment of the present application, the optimized lightweight model is a model that has been trained to improve performance while maintaining its lightweight. It can more accurately identify objects or behaviors in the video while still being able to run efficiently on the edge device.

[0068] In the embodiment of the present application, the edge is responsible for data collection and preliminary processing, the cloud is responsible for advanced analysis and annotation, and the edge model is continuously optimized based on the feedback from the cloud. This architecture helps to continuously improve the intelligence level of the system while maintaining high efficiency and real-time performance.

[0069] In an embodiment of the present application, a model update method for real-time object detection is provided, including: receiving real-time video data collected by a camera device arranged at an edge, wherein an initial lightweight model has been deployed at the edge; performing an adaptive sampling operation on the real-time video data to obtain an adaptive sampling result; sending the real-time adaptive sampling result to a cloud teacher model for online annotation operation to obtain an online annotation result; performing an adaptive training operation on the initial lightweight model according to the online annotation result to obtain an optimized lightweight model. Compared with the prior art, the present application uses a large model with high accuracy as a teacher model in the cloud through knowledge distillation, so that the lightweight model on the edge device continuously learns from the large model, thereby improving the accuracy of real-time video object detection.

[0070] In some optional implementations of the embodiments of the present application, the above step of performing an adaptive sampling operation on the real-time video data to obtain an adaptive sampling result specifically includes the following steps:

[0071] The real-time video data is sampled according to the sampling strategy of content similarity to obtain an adaptive sampling result.

[0072] In the embodiment of the present application, the sampling strategy of content similarity refers to the process of selecting part of the data from the original data set as samples according to certain rules or algorithms based on the similarity between two or more video clips in visual features (such as color, texture, etc.) and time sequence. Specifically:

[0073] (1) Video segmentation: real-time video data is segmented into multiple smaller video segments for subsequent processing. The basis for segmentation can be time length, shot switching, etc.

[0074] (2) Feature extraction: Extract features from each video clip to obtain a feature vector that can represent the visual content of the clip. Commonly used feature extraction methods include color histogram, texture features, edge features, etc.

[0075] (3) Similarity calculation: Based on the extracted feature vectors, the similarity between different video clips is calculated; the similarity calculation method can be Euclidean distance, cosine similarity, Manhattan distance, etc.

[0076] (4) Sampling decision: Based on the similarity calculation results, decide which video clips to sample; the sampling strategy can be to select the clip with the highest similarity, select the most representative clip, or screen according to a preset threshold.

[0077] (5) Adaptive adjustment: The sampling strategy is adaptively adjusted according to the changes in real-time video data. For example, when the video content changes rapidly, the sampling frequency can be increased; when the video content is relatively stable, the sampling frequency can be reduced.

[0078] In an embodiment of the present application, a sampling strategy based on content similarity is adopted for the massive video data collected by the edge device to automatically screen out representative video clips.

[0079] In some optional implementations of the embodiments of the present application, the above-mentioned step of performing an adaptive training operation on the initial lightweight model according to the online annotation results to obtain an optimized lightweight model specifically includes the following steps:

[0080] Calling a lightweight model similar to the initial lightweight model, and performing model optimization operations on the similar lightweight model according to the online annotation results to obtain an optimized lightweight model;

[0081] The optimized lightweight model is redeployed to the edge to replace the lightweight model, thereby obtaining the optimized lightweight model.

[0082] In an embodiment of the present application, since the initial lightweight model cannot meet the needs of all scenarios, or needs to be adjusted as the application scenarios change, it is necessary to call a lightweight model similar to the initial model as the optimization starting point. The selection of similar models can be based on multiple dimensions such as model structure, performance, and application areas.

[0083] In an embodiment of the present application, the performance of similar lightweight models is evaluated based on the online annotation results. Specifically, the model is optimized using the online annotation results through methods such as transfer learning, incremental learning or retraining.

[0084] In the embodiment of the present application, the optimization goal may be to improve the prediction accuracy and robustness of the model or to reduce the complexity of the model.

[0085] In the embodiment of the present application, after the optimization operation, a lightweight model with improved performance is obtained, and the optimized model has higher prediction accuracy, stronger generalization ability or lower computational complexity.

[0086] In an embodiment of the present application, the optimized lightweight model is redeployed to the edge device to replace the original lightweight model, wherein the deployment process involves conversion of the model format, adjustment of parameters, and integration with other system components.

[0087] In an embodiment of the present application, by calling a model similar to the initial lightweight model and performing model optimization operations based on the online annotation results, an optimized lightweight model can be obtained and redeployed to the edge to replace the original model, thereby achieving continuous updating and optimization of the model.

[0088] In some optional implementations of the embodiments of the present application, the above-mentioned step of performing an adaptive training operation on the initial lightweight model according to the online annotation results to obtain an optimized lightweight model specifically includes the following steps:

[0089] Deploy the federated learning framework on the edge;

[0090] The initial lightweight model is federated trained according to the federated learning framework and the online annotation results to obtain an optimized lightweight model.

[0091] In an embodiment of the present application, a federated learning framework is deployed on the edge device, and the lightweight video processing model is federated trained using the online annotation results. By exchanging and aggregating model parameters between multiple edge devices, continuous optimization of model performance is achieved while ensuring data privacy and security.

[0092] In some optional implementations of the embodiments of the present application, after the step of performing an adaptive training operation on the initial lightweight model according to the online annotation result to obtain the optimized lightweight model, the following steps are also included:

[0093] Receive the latest video data collected by the camera device;

[0094] The latest video data is transferred to the optimized lightweight model for object detection operations to obtain real-time object detection results.

[0095] In some optional implementations of the embodiments of the present application, after the step of receiving the latest video data collected by the camera device, the following steps are also included:

[0096] Performing a complexity judgment operation on the latest video data according to a preset complexity threshold to obtain a complexity judgment result;

[0097] If the complexity judgment result exceeds the complexity threshold, the latest video data is sent to the cloud teacher model for object detection to obtain real-time object detection results;

[0098] Send real-time object detection results to the real-time edge for display.

[0099] In an embodiment of the present application, the complexity judgment operation refers to preprocessing the latest video data, extracting key frames or features, and performing complexity evaluation on the extracted key frames or features according to a preset complexity threshold, wherein the complexity evaluation method can be based on statistics of the amount of calculation, complexity analysis based on image features, etc., to obtain a complexity judgment result and determine whether the video data exceeds the complexity threshold.

[0100] In an embodiment of the present application, if the complexity judgment result exceeds the complexity threshold, it means that the current video data is difficult to process and needs to be processed with the help of a cloud-based teacher model. The latest video data is sent to the cloud server, and the teacher model is used to perform object detection operations. The cloud-based teacher model is based on a deep learning algorithm to perform efficient and accurate object detection on video data, identify objects in the video and their locations, and obtain real-time object detection results, including object category, location, confidence and other information.

[0101] In an embodiment of the present application, by making a complexity judgment on the latest video data according to a preset complexity threshold, and selecting whether to use the cloud-based teacher model for object detection based on the judgment result, and finally sending the detection result to the real-time edge for display, efficient and accurate object detection can be achieved, and the real-time requirements in practical applications can be met.

[0102] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0103] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0104] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0105] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0106] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a model updating device for real-time object detection. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0107] like Figure 3 As shown, the model updating device 200 applied to real-time object detection in the embodiment of the present application includes:

[0108] The video acquisition module 210 is used to receive real-time video data collected by a camera device arranged at an edge, wherein an initial lightweight model has been deployed at the edge;

[0109] The adaptive sampling module 220 is used to perform an adaptive sampling operation on the real-time video data to obtain an adaptive sampling result;

[0110] An online annotation module 230 is used to send the real-time adaptive sampling results to the cloud teacher model for online annotation operations to obtain online annotation results;

[0111] The adaptive training module 240 is used to perform adaptive training operations on the lightweight model according to the online annotation results to obtain an optimized lightweight model.

[0112] In an embodiment of the present application, a model updating device 200 for real-time object detection is provided, including: a video acquisition module 210, for receiving real-time video data collected by a camera device arranged at an edge, wherein an initial lightweight model has been deployed at the edge; an adaptive sampling module 220, for performing adaptive sampling operations on real-time video data to obtain adaptive sampling results; an online annotation module 230, for sending the real-time adaptive sampling results to a cloud teacher model for online annotation operations to obtain online annotation results; an adaptive training module 240, for performing adaptive training operations on the lightweight model according to the online annotation results to obtain an optimized lightweight model. Compared with the prior art, the present application uses a large model with high accuracy as a teacher model in the cloud through knowledge distillation, so that the lightweight model on the edge device continuously learns from the large model, thereby improving the accuracy of real-time video object detection.

[0113] In some optional implementations of the embodiments of the present application, the above-mentioned adaptive training module includes:

[0114] A model optimization submodule, used for calling a second lightweight model that is the same as the lightweight model, and performing a model optimization operation on the second lightweight model according to the online annotation result to obtain an optimized second lightweight model;

[0115] The model deployment submodule is used to redeploy the optimized second lightweight model to the edge to replace the lightweight model.

[0116] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of a computer device according to an embodiment of the present application.

[0117] The computer device 300 includes a memory 310, a processor 320, and a network interface 330 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 300 having components 310-330, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0118] The computer device may be a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with the user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0119] The memory 310 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 310 can be an internal storage unit of the computer device 300, such as a hard disk or memory of the computer device 300. In other embodiments, the memory 310 can also be an external storage device of the computer device 300, such as a plug-in hard disk equipped on the computer device 300, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc. Of course, the memory 310 can also include both the internal storage unit of the computer device 300 and its external storage device. In the embodiment of the present application, the memory 310 is used to store the operating system and various application software installed on the computer device 300, such as computer-readable instructions for the model updating method applied to real-time object detection. In addition, the memory 310 can also be used to temporarily store various data that have been output or are to be output.

[0120] The processor 320 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 320 is used to control the overall operation of the computer device 300. In the embodiment of the present application, the processor 320 is used to run the computer-readable instructions stored in the memory 310 or process data, such as running the computer-readable instructions of the model update method applied to real-time object detection.

[0121] The network interface 330 may include a wireless network interface or a wired network interface, and the network interface 330 is used to establish a communication connection between the computer device 300 and other electronic devices.

[0122] The computer device provided in this application uses a high-accuracy large model as a teacher model in the cloud through knowledge distillation, so that the lightweight model on the edge device continuously learns from the large model, thereby improving the accuracy of real-time video object detection.

[0123] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the model updating method applied to real-time object detection as described above.

[0124] The computer-readable storage medium provided in this application uses a high-accuracy large model as a teacher model in the cloud through knowledge distillation, so that the lightweight model on the edge device continuously learns from the large model, thereby improving the accuracy of real-time video object detection.

[0125] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0126] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A model updating method for real-time object detection, characterized in that: The steps include: Receiving real-time video data collected by a camera device disposed at an edge, wherein the edge has deployed an initial lightweight model; Performing an adaptive sampling operation on the real-time video data to obtain an adaptive sampling result; The real-time adaptive sampling result is sent to the cloud teacher model for online annotation operation to obtain an online annotation result; The initial lightweight model is adaptively trained according to the online annotation result to obtain an optimized lightweight model.

2. The model updating method for real-time object detection according to claim 1, characterized in that: The step of performing an adaptive sampling operation on the real-time video data to obtain an adaptive sampling result specifically includes the following steps: The real-time video data is sampled according to a sampling strategy of content similarity to obtain the adaptive sampling result.

3. The model updating method for real-time object detection according to claim 1, characterized in that: The step of performing an adaptive training operation on the initial lightweight model according to the online annotation result to obtain an optimized lightweight model specifically includes the following steps: Calling a lightweight model similar to the initial lightweight model, and performing a model optimization operation on the similar lightweight model according to the online annotation result to obtain an optimized lightweight model; The optimized lightweight model is redeployed to the edge to replace the lightweight model, thereby obtaining the optimized lightweight model.

4. The model updating method for real-time object detection according to claim 1, characterized in that: The step of performing an adaptive training operation on the initial lightweight model according to the online annotation result to obtain an optimized lightweight model specifically includes the following steps: Deploying a federated learning framework on the edge; The initial lightweight model is federated trained according to the federated learning framework and the online annotation results to obtain the optimized lightweight model.

5. The model updating method for real-time object detection according to claim 1, characterized in that: After the step of performing an adaptive training operation on the initial lightweight model according to the online annotation result to obtain an optimized lightweight model, the following steps are also included: Receiving the latest video data collected by the camera device; The latest video data is transmitted to the optimized lightweight model for object detection operation to obtain real-time object detection results.

6. The model updating method for real-time object detection according to claim 5, characterized in that: After the step of receiving the latest video data collected by the camera device, the method further includes the following steps: Performing a complexity determination operation on the latest video data according to a preset complexity threshold to obtain a complexity determination result; If the complexity judgment result exceeds the complexity threshold, the latest video data is sent to the cloud teacher model for object detection operation to obtain a real-time object detection result; The real-time object detection result is sent to the real-time edge for display.

7. A model updating device for real-time object detection, characterized in that: include: A video acquisition module, used to receive real-time video data collected by a camera device arranged at an edge, wherein the edge has deployed an initial lightweight model; An adaptive sampling module, used for performing an adaptive sampling operation on the real-time video data to obtain an adaptive sampling result; An online annotation module is used to send the real-time adaptive sampling result to the cloud teacher model for online annotation operation to obtain an online annotation result; The adaptive training module is used to perform adaptive training operations on the lightweight model according to the online annotation results to obtain an optimized lightweight model.

8. The model updating device for real-time object detection according to claim 7, characterized in that: The adaptive training module comprises: A model optimization submodule, used for calling a second lightweight model that is the same as the lightweight model, and performing a model optimization operation on the second lightweight model according to the online annotation result to obtain an optimized second lightweight model; The model deployment submodule is used to redeploy the optimized second lightweight model to the edge to replace the lightweight model.

9. A computer device comprising a memory and a processor, characterized in that: The memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the model updating method for real-time object detection according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the model updating method for real-time object detection according to any one of claims 1 to 6 are implemented.

Images