Incremental Training Method and System for Image Classification Model Based on Kafka Message Queue

Abstract

The present application discloses an incremental training method and system for an image classification model based on a Kafka message queue. The incremental training method for the image classification model based on the Kafka message queue includes: collecting incremental data through the Kafka message queue; obtaining a model that has been trained; and performing incremental training on the model that has been trained with the incremental data. In view of the problem that incremental learning algorithms are separated from the production environment, the present application provides a system and method for incremental training of an image classification model with the help of a message queue system, combines the incremental learning solution with a stream processing platform, and can effectively simplify the process of incremental model training and deployment.

Description

Technical Field

[0001] This application relates to the technical field of model training, and specifically relates to an incremental training method for an image classification model based on a Kafka message queue and an incremental training system for an image classification model based on a Kafka message queue. Background Art

[0002] With the development of computer technology, classification models based on artificial neural networks have made important progress in multiple fields. Image classification is an important field of image processing, which classifies different types of images according to the semantic information of the images. Currently, the existing classification model training mechanisms often adopt static methods, training with a large amount of data to solve a fixed task. The research on artificial neural networks also mainly focuses on static tasks. Usually, random data is used to ensure compliance with the independent and identically distributed conditions, and the performance is greatly improved by revisiting the training data in multiple epochs. However, the following problems exist in the current real-world image classification tasks:

[0003] High-speed and high-capacity information constantly floods the learning system, and new data and new types often become gradually available over time. Continuous incremental data brings storage bottlenecks, and some data cannot be stored for a long time for privacy protection and other purposes. Old data gradually becomes unavailable over time.

[0004] Although deep learning models have high trainability and generalization ability, they also have a problem, namely catastrophic forgetting: forgetting the data learned previously when learning new data. The occurrence of this phenomenon may lead to a decrease in the accuracy of the model and may also affect the generalization ability of the model. The research field for this problem is incremental learning. Common methods for incremental learning include: 1) replay-based incremental learning, which slows down the forgetting of the model by replaying some past data; 2) regularization-based incremental learning, which restricts the update of the model through an additional loss function. Although these methods perform well in many simulations, they still rely too much on the simulated incremental environment and lack integration with the production scenario, which greatly limits the application and deployment of continuous learning models.

[0005] Therefore, it is hoped that there is a technical solution to solve or at least alleviate the problem that the existing continuous learning methods cannot adapt to the existing production environment. Summary of the Invention

[0006] The purpose of the present invention is to provide an incremental training method for an image classification model based on a Kafka message queue to at least alleviate one of the above technical problems.

[0007] One aspect of the present invention provides an incremental training method for an image classification model based on a Kafka message queue. The incremental training method for the image classification model based on the Kafka message queue includes:

[0008] Collect incremental data through the Kafka message queue;

[0009] Obtain a trained model;

[0010] Perform incremental training on the trained model with the incremental data.

[0011] Optionally, the incremental training method for the image classification model based on the Kafka message queue further includes:

[0012] Evaluate the model after incremental training to verify the accuracy and generalization ability of the model.

[0013] Optionally, the collecting of incremental data through the Kafka message queue includes:

[0014] The user creates a Kafka image message producer at the image data source and produces one or more classes of labeled image data to the Kafka server cluster;

[0015] Consume the image data into the OpenCV working area, query the corresponding preprocessing format for this category, and perform image processing according to the preprocessing format to obtain the processed image data;

[0016] Produce the image data to the Topic corresponding to the label category according to the label of the processed image data.

[0017] Optionally, the performing of incremental training on the trained model with the incremental data includes:

[0018] Initialize the environment, use a Kafka consumer to consume the training environment configuration in the configured Topic, initialize the replay area space in the memory area according to the configuration, and use a Kafka consumer to consume the replay area data into the replay area in the memory area;

[0019] Load the model data, and use a Kafka consumer to consume the trained model from the corresponding Kafka Topic into the memory area;

[0020] Load the incremental data, select the Topic where the incremental data is located, define the training data offset, and consume the incremental data into the memory area as training data; mix the replay area data with the incremental data as training data;

[0021] Incrementally train the model, and perform incremental training on the trained model with the mixed training data;

[0022] Update playback zone data, and obtain playback configuration items from management data of playback zone data;

[0023] Save the status, update the configuration items, and create a new configuration topic to obtain the incrementally trained model.

[0024] Optionally, the process of evaluating the incrementally trained model includes:

[0025] Collect image data for evaluation independent of training data;

[0026] Preprocessing the collected image data for evaluation according to the system preset format;

[0027] The consumption model topic obtains the incrementally trained model and the representative data playback topic obtains representative data. The weighted nearest class classifier is used, and the weighted mean of the stored image features is used as the category representation of the class. The distance between the features obtained by the model and the representative data features of the test image is calculated, and the classification of the test data can be verified.

[0028] Optionally, the loading of model data and using a Kafka consumer to consume the trained model from a corresponding Topic in Kafka to a memory area includes:

[0029] Use Kafka consumers to consume the trained convolutional neural network model from the old Kafka model topic to the memory area. The trained convolutional neural network model includes a feature extractor F and a linear fully connected layer G. The feature extractor F converts the input data x i Map it into the feature space and get the feature vector of the image: z i =F(x i ), and the linear fully connected layer G is used as a classifier to obtain the similarity metric logits of each category during training, and the pytorch loading model method is used to initialize the existing model as the old model.

[0030] Optionally, the incremental training model, using the mixed training data to incrementally train the trained model includes:

[0031] The model is incrementally trained using the mixed training data, and multiple loss functions are preset during the model training process to guide the model update.

[0032] The present application also provides an image classification model incremental training system based on a Kafka message queue, and the image classification model incremental training system based on a Kafka message queue includes:

[0033] An incremental data collection device, which is used to collect incremental data through a Kafka message queue;

[0034] A model acquisition device, which is used to acquire a trained model;

[0035] An incremental model training device, which is used to perform incremental training on a trained model with incremental data.

[0036] Optionally, the image classification model incremental training system based on the Kafka message queue further includes:

[0037] An incremental model deployment device, which is used to verify the model that has undergone incremental training.

[0038] Beneficial effects

[0039] In view of the problem that the incremental learning algorithm is separated from the production environment, this application provides a system and method for incremental training of an image classification model with the help of a message queue system, combines the incremental learning solution with a stream processing platform, and can effectively simplify the process of incremental model training and deployment. Description of the drawings

[0040] Figure 1 It is a schematic flowchart of an image classification model incremental training method based on the Kafka message queue according to an embodiment of this application.

[0041] Figure 2 It is a schematic diagram of an electronic device capable of implementing the image classification model incremental training method based on the Kafka message queue according to an embodiment of this application.

[0042] Figure 3 It is a schematic diagram of an image classification model incremental training system based on the Kafka message queue according to an embodiment of this application. Detailed implementation manners

[0043] To make the objectives, technical solutions, and advantages of the present application more clear, the following will describe in more detail the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. In the drawings, the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions from beginning to end. The described embodiments are some but not all of the embodiments of the present application. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present application, and should not be construed as a limitation to the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts belong to the scope of protection of the present application. The following will explain the embodiments of the present application in detail with reference to the accompanying drawings.

[0044] Figure 1 It is a schematic flowchart of an incremental training method for an image classification model based on a Kafka message queue according to an embodiment of the present application.

[0045] As Figure 1 shown, the incremental training method for an image classification model based on a Kafka message queue includes:

[0046] Collect incremental data through the Kafka message queue;

[0047] Obtain a model that has been trained;

[0048] Perform incremental training on the trained model with the incremental data.

[0049] In view of the problem that the incremental learning algorithm is separated from the production environment, the present application provides a system and method for incremental training of an image classification model with the help of a message queue system, combines the incremental learning solution with a stream processing platform, and can effectively simplify the process of incremental model training and deployment.

[0050] In this embodiment, the incremental training method for an image classification model based on a Kafka message queue further includes:

[0051] Evaluate the model that has undergone incremental training.

[0052] In this embodiment, the collecting of incremental data through the Kafka message queue includes:

[0053] The user creates a Kafka image message producer at the image data source and produces one or more classes of labeled image data to the Kafka server cluster;

[0054] Consume the image data into the OpenCV working area, query the corresponding preprocessing format for this category, and perform image processing according to the preprocessing format to obtain the processed image data;

[0055] Produce the image data to the Topic corresponding to the label category according to the label of the processed image data.

[0056] In this embodiment, the incremental training of the already trained model by the incremental data includes:

[0057] Initialize the environment, use the Kafka consumer to consume the training environment configuration in the configured Topic, initialize the replay area space in the memory area according to the configuration, and use the Kafka consumer to consume the replay area data to the replay area in the memory area;

[0058] Load the model data, and use the Kafka consumer to consume the already trained model from the corresponding Topic in Kafka to the memory area;

[0059] Load the incremental data, select the Topic where the incremental data is located, define the training data offset, and consume the incremental data to the memory area as training data; mix the replay area data and the incremental data as training data;

[0060] Incrementally train the model, and use the mixed training data to perform incremental training on the already trained model;

[0061] Update the replay area data, and obtain the replay configuration items from the management data of the replay area data;

[0062] Save the state, update the configuration items, create a new configuration Topic, so as to obtain the model after incremental training.

[0063] In this embodiment, the verification of the model after incremental training includes:

[0064] Collect the image data for evaluation;

[0065] Preprocess the collected image data for evaluation according to the system preset format;

[0066] Consume the model Topic to obtain the model after incremental training and the representative data replay Topic to obtain the representative data. Use the weighted nearest class classifier, and use the weighted mean of the stored image features as the class representation of this class. Calculate the distance between the features obtained by the test image through the model and the representative data features, and then the classification verification of the test data can be performed.

[0067] In this embodiment, the loading of the model data, and using the Kafka consumer to consume the already trained model from the corresponding Topic in Kafka to the memory area includes:

[0068] Use a Kafka consumer to consume the pre-trained convolutional neural network model from the Kafka old model Topic into the memory area. The pre-trained convolutional neural network model includes a feature extractor F and a linear fully connected layer G. The feature extractor F maps the input data x i into the feature space to obtain the feature vector of the image: z i = F(x i ), and the linear fully connected layer G serves as a classifier to obtain the similarity metric logits for each class during training. Use the pytorch model loading method to initialize the existing model as the old model.

[0069] In this embodiment, for the incremental training model, using the mixed training data to perform incremental training on the pre-trained model includes:

[0070] Use the mixed training data to perform incremental training on the model, and guide the update of the model by presetting multiple loss functions during the model training process.

[0071] The following further elaborates on this application by way of example. It can be understood that this example does not constitute any limitation to this application.

[0072] 1. Data incremental collection

[0073] (1) The user creates a Kafka image message producer at the image data source and produces one or more types of labeled image data to the Kafka server cluster.

[0074] (2) Consume the image data into the OpenCV working area, query the corresponding preprocessing format for this category, and perform image processing.

[0075] (3) According to the label of one or more types of image data, produce the image data to the Topic corresponding to the category.

[0076] 2. Model incremental training

[0077] The model training system runs on a distributed Kubernetes cluster. The main working process of the model incremental training system is as follows:

[0078] (1) Environment initialization. Use a Kafka consumer to consume the training environment configuration from the configuration Topic, initialize the replay area space in the memory area according to the configuration, and use a Kafka consumer to consume the replay area data into the memory area replay area.

[0079] (2) Model data loading: Use a Kafka consumer to consume the existing convolutional neural network model from the corresponding Kafka Topic into the memory area, or create a new model.

[0080] (3) Training data loading: Select the Topic where the incremental data is located and define the training data offset. Consume the incremental data into the memory area as training data; Mix the data in the replay area with the incremental data as training data.

[0081] (4) Incremental model training: Use the mixed training data to perform incremental training on the model. In this embodiment, by customizing multiple loss functions to guide the update of the model, the purpose of slowing down catastrophic forgetting is achieved.

[0082] (5) Replay area data update: Obtain the replay configuration items from the management data of the replay area data. If the incremental data is a new category, perform a removal operation on the replay data according to the configuration item and add the new data to the idle replay area; If the incremental data is an old category, perform random replacement and update on the replay area data. Update the replay area data configuration item.

[0083] (6) Status saving: Update the configuration item, create a new configuration Topic, and use a Kafka producer to produce the configuration to the new configuration Topic. Create a new Kafka data replay Topic and use a Kafka producer to produce the memory replay area data to the new data replay Topic. Create a new model Topic and use a Kafka producer to produce the trained model to the new model Topic.

[0084] 3. Classification model evaluation and deployment

[0085] (1) Data collection and preprocessing: Collect data containing new type images or old type images and preprocess the data according to the system preset format.

[0086] (2) Consume the model Topic to obtain the latest trained model and the representative data replay Topic to obtain representative data. Use a weighted nearest class classifier and use the weighted mean of the stored image features as the class representation of this class. Calculate the distance between the features obtained by the test image through the model and the representative data features to classify the test data.

[0087] In this embodiment, define a model M and a sequence data set composed of N tasks where D = (X n , Y n ) represents the data set arriving at the nth stage, and this data set contains C n categories. The method includes:

[0088] Incremental data collection

[0089] The Kafka message queue cluster runs on the Kubernetes cluster. Incremental image data is incrementally produced as a stream to the Kafka message queue. This data includes the original image data and the corresponding supervised labels. The incremental data will be written into the Topic corresponding to this category according to the preset data processing format. The data incremental collection process will go through the following steps:

[0090] (1) The user creates a Kafka image message producer at the image data source and produces one or more categories of labeled image data to the Kafka server cluster.

[0091] (2) Consume the image data to the OpenCV working area, query the corresponding preprocessing format for this category, and perform image processing.

[0092] (3) According to the labels of one or more categories of image data, produce the image data to the Topic corresponding to the category.

[0093] 2. Incremental model training

[0094] The model training system runs on the distributed Kubernetes cluster. The main workflow of the model incremental training system is as follows:

[0095] (1) Environment initialization. Use the Kafka consumer to consume the training environment configuration in the configured Topic, initialize the replay area space in the memory area according to the configuration, and use the Kafka consumer to consume the replay area data to the replay area in the memory area.

[0096] (2) Model data loading. Use the Kafka consumer to consume the existing convolutional neural network model M from the Kafka old model Topic to the memory area, or create a new model M. Model M consists of two parts, a feature extractor F and a linear fully connected layer G. Among them, the feature extractor F maps the input data x i to the feature space to obtain the feature vector of the picture: z i = F(x i ), and the linear fully connected layer G serves as a classifier to obtain the similarity measure logits of each category during training. Use the pytorch model loading method to initialize the existing model as the old model;

[0097] (3) Training data loading. Select the Topic where the incremental data is located and define the training data offset. Consume the incremental data to the memory area as training data; mix the replay area data and the incremental data as training data.

[0098] (4) Model incremental training, using the mixed training data to perform incremental training on the model. In this embodiment, multiple preset loss functions can be used during the model training process to guide the update of the model, so that the model can continuously improve its accuracy during training. For example, add the cross-entropy loss function and construct an additional knowledge distillation loss function.

[0099] (5) Replay area data update, obtain the replay configuration items from the management data of the replay area data. If the incremental data is a new category, perform a removal operation on the replay data according to the configuration items and add the new data to the idle replay area; if the incremental data is an old category, perform random replacement and update on the replay area data. Update the replay area data configuration items.

[0100] (6) Status saving, update the configuration items, create a new configuration Topic, and use the Kafka producer to produce the configuration to the new configuration Topic. Create a new Kafka data replay Topic, and use the Kafka producer to produce the in-memory replay area data to the new data replay Topic. Create a new model Topic, and use the Kafka producer to produce the trained model to the new model Topic.

[0101] 3. Classification model evaluation and deployment

[0102] (1) Data collection and preprocessing, collect data containing new type images or old type images, and preprocess the data according to the system preset format.

[0103] (2) Consume the model Topic to obtain the trained model, and consume the Topic corresponding to the stored images to obtain the stored images. Use the weighted nearest class classifier, and use the weighted mean of the stored image features as the class representation of this class. Calculate the distance between the features obtained by the test image through the model and the representative data features, and then the test data can be classified. To predict the label of a new sample x, it is necessary to use the trained model to obtain the mean μ of all prototypes c .

[0104]

[0105] where (x i , y i ) represents the i-th sample in the dataset, and this formula realizes the calculation of the mean of the features of the n c samples belonging to the c-th class of the stored data.

[0106]

[0107] where λ c is the preset mean feature weight, and y * is the predicted label.

[0108] Among them, the above functions run in parallel and do not affect each other.

[0109] For example, illustrate the incremental process of 5 classification tasks:

[0110] For a complete image dataset with ten categories, such as mnist handwritten digit recognition. This dataset can be divided into 5 tasks, [[0,1],[2,3],[4,5],[6,7],[8,9]].

[0111] Environment description: In the environment of data increment, data of different categories are incremented. For example, at time t0, two categories [0,1] come in ---- with an interval of two days ---- [2,3] ---- with an interval of one day ---- [4,5] ---- with an interval of one year ---- [6,7].........

[0112] Task description: Suppose we need to train a ten-class handwritten digit model, but we don't want to wait until all the data arrives and then use "all" the training data for training. This is usually not good. One reason is that as the data accumulates more and more, the cost of training and storing the data will become higher and higher, and we also cannot guarantee that all the data collected in the past will still be visible in the future.

[0113] The method of this application can achieve the class incremental process: Use [0,1] to iteratively train a "binary classification model" for multiple layers (multiple iterations can be performed) ---->>>[Incremental process (data of [2,3] can also be used)] ---->>>Incremental training (either by using representative data replay * using a small part of the representative samples of [0,1], or not using the data in [0,1] at all) ----->>>Get a four-classification model ----->>>And so on.

[0114] The class incremental process is the common assumed incremental process in the current "continuous learning" field. However, the academic community is separated from the production environment and needs to use a simulated environment to simulate the incremental process. Our core innovation is "implementing this incremental process with Kafka and integrating steps such as data processing, model training sets, and model evaluation into a complete system".

[0115] The present application also provides an incremental training system for an image classification model based on a Kafka message queue. The incremental training system for the image classification model based on the Kafka message queue includes an incremental data collection device, a model acquisition device, an incremental model training device, and an incremental model deployment device. The incremental data collection device is used to collect incremental data through the Kafka message queue; the model acquisition device is used to acquire a model that has been trained; the incremental model training device is used to perform incremental training on the trained model with the incremental data; and the incremental model deployment device is used to verify the model that has undergone incremental training.

[0116] The present invention proposes an incremental training system and method for an image classification model based on Kafka. This method can effectively collect and converge multi-source homogeneous data, improve the accuracy of the incremental classification model based on the old model and old knowledge, and provide a classification prediction method for the newly trained model.

[0117] See Figure 3 , a schematic diagram of an image classification system based on Kafka according to an embodiment of the present invention. The system includes:

[0118] An incremental data collection device 21, which is used to, when new image data of the same category or different categories is produced, push the labeled image data to the Kafka message queue in real time; and determine whether the image data of the current category meets a preset data format. If it meets the format, directly write the original image data into the Topic corresponding to the picture category. If it does not meet the format, configure a preset image preprocessing format and use OpenCV to process the image data, and write it into the Topic corresponding to the current picture category.

[0119] An incremental model training device 22, which is used to build a new convolutional neural network or load an old convolutional neural network; when a user submits a new model training task, load the incremental data in the stream processing system into the training memory area, determine whether there is a sampling strategy. If there is a sampling strategy, produce the incremental data to the corresponding Topic in the Kafka message queue according to the sampling strategy. Load the replay data, determine whether there is replay data. If there is replay data, consume the replay data from the Kafka message queue into the training memory area and mix it with the incremental data; perform an incremental training process according to the training configuration preset by the user; and after the training is completed, save the training status to the Kafka message queue.

[0120] An incremental model deployment device 23, which is used to synchronously or periodically verify the accuracy of the data incremental image classification model; and optionally, Kafka is in a publish / subscribe mode to consume and use the model in the deployment environment.

[0121] Preferably, the incremental data collection device 21 runs on a Kubernetes distributed cluster, which is composed of multiple servers.

[0122] It can be understood that the above description of the method also applies equally to the description of the device.

[0123] The present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, it implements the above image classification model incremental training method based on the Kafka message queue.

[0124] The present application also provides a computer-readable storage medium storing a computer program, which when executed by a processor can implement the above image classification model incremental training method based on the Kafka message queue.

[0125] Figure 2 It is an exemplary structural diagram of an electronic device capable of implementing the image classification model incremental training method based on the Kafka message queue provided by an embodiment of the present application.

[0126] As Figure 2 shown, the electronic device includes an input device 501, an input interface 502, a central processing unit 503, a memory 504, an output interface 505, and an output device 506. Among them, the input interface 502, the central processing unit 503, the memory 504, and the output interface 505 are connected to each other through a bus 507. The input device 501 and the output device 506 are respectively connected to the bus 507 through the input interface 502 and the output interface 505, and then connected to other components of the electronic device. Specifically, the input device 504 receives input information from the outside and transmits the input information to the central processing unit 503 through the input interface 502; the central processing unit 503 processes the input information based on the computer-executable instructions stored in the memory 504 to generate output information, temporarily or permanently stores the output information in the memory 504, and then transmits the output information to the output device 506 through the output interface 505; the output device 506 outputs the output information to the outside of the electronic device for the user to use.

[0127] That is to say, Figure 2 the electronic device shown can also be implemented as including: a memory storing computer-executable instructions; and one or more processors that can implement the image classification model incremental training method based on the Kafka message queue in combination with Figure 1 the description.

[0128] In one embodiment,Figure 2 The electronic device shown can be implemented to include: a memory 504 configured to store executable program code; and one or more processors 503 configured to run the executable program code stored in the memory 504 to execute the method for incremental training of an image classification model based on a Kafka message queue in the above embodiments.

[0129] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0130] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM), and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

[0131] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology for storing information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0132] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0133] In addition, it is obvious that the word "including" does not exclude other units or steps. The multiple units, modules, or devices stated in the apparatus claims can also be implemented by one unit or a general device through software or hardware.

[0134] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code, and the module, the segment of a program, or the part of code includes one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive marked blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or overall flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0135] In this embodiment, the so-called processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor may also be any conventional processor, etc.

[0136] The memory can be used to store computer programs and / or modules. By running or executing the computer programs and / or modules stored in the memory, and by invoking the data stored in the memory, the processor realizes various functions of the device / terminal equipment. The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created according to the use of the mobile phone (such as audio data, phone book, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0137] In this embodiment, if the modules / units integrated in the device / terminal device are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above-described embodiment methods of the present invention, it can also be completed by a computer program instructing relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of patent practice. Although the present application is disclosed above with preferred embodiments, it is not actually used to limit the present application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the protection scope of the present application should be determined by the scope defined by the claims of the present application.

[0138] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, system, or computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0139] In addition, obviously, the word "including" does not exclude other units or steps. The multiple units, modules, or devices stated in the device claims can also be implemented by one unit or a general device through software or hardware.

[0140] Although the present invention has been described in detail above with general descriptions and specific embodiments, based on the present invention, some modifications or improvements can be made, which are obvious to those skilled in the art. Therefore, these modifications or improvements made without departing from the spirit of the present invention all fall within the scope of protection required by the present invention.

Claims

1. An incremental training method for an image classification model based on Kafka message queue, characterized in that, The incremental training method for an image classification model based on a Kafka message queue includes: Collect incremental data through a Kafka message queue; Obtain a model that has been trained; Perform incremental training on the trained model using the incremental data; The incremental training method for an image classification model based on a Kafka message queue further includes: Evaluate the model that has undergone incremental training; The step of collecting incremental data through a Kafka message queue includes: The user creates a Kafka image message producer at the image data source and produces one or more classes of labeled image data to the Kafka server cluster; Consume the image data into the OpenCV working area, query the corresponding preprocessing format for this category, and perform image processing according to the preprocessing format to obtain the processed image data; Produce the image data to the Topic corresponding to the label category according to the label of the processed image data; The step of performing incremental training on the trained model using the incremental data includes: Initialize the environment, use a Kafka consumer to consume the training environment configuration in the configured Topic, initialize the replay area space in the memory area according to the configuration, and use a Kafka consumer to consume the replay area data into the replay area in the memory area; Load the model data, and use a Kafka consumer to consume the trained model from the corresponding Topic in Kafka into the memory area; Load the incremental data, select the Topic where the incremental data is located, define the training data offset, and consume the incremental data into the memory area as training data; mix the replay area data with the incremental data as training data; Incrementally train the model, and use the mixed training data to perform incremental training on the trained model; Update the replay area data, and obtain the replay configuration items from the management data of the replay area data; Save the state, update the configuration items, and create a new configuration Topic to obtain the model that has undergone incremental training.

2. The method for incremental training of an image classification model based on a Kafka message queue according to claim 1, wherein The step of evaluating the model that has undergone incremental training includes: Collect image data for evaluation; Preprocess the collected image data for evaluation according to the system preset format; Consume the model Topic to obtain the model that has undergone incremental training and the representative data replay Topic to obtain the representative data. Use a weighted nearest class classifier, use the weighted mean of the stored image features as the category representation of this class, and calculate the distance between the features obtained by the test image through the model and the representative data features to perform classification verification on the test data.

3. The method for incremental training of an image classification model based on a Kafka message queue according to claim 2, wherein The step of loading the model data, and using a Kafka consumer to consume the trained model from the corresponding Topic in Kafka into the memory area includes: Use a Kafka consumer to consume the pre-trained convolutional neural network model from the Kafka old model Topic into the memory area. The pre-trained convolutional neural network model includes a feature extractor F and a linear fully-connected layer G. The feature extractor F maps the input data x i to the feature space to obtain the feature vector of the image: z i = F(x i ), and the linear fully-connected layer G serves as a classifier to obtain the similarity measures logits for each class during training. Initialize the existing model as the old model using the pytorch model loading method.

4. The method for incremental training of an image classification model based on a Kafka message queue according to claim 3, wherein The step of incrementally training the model, and using the mixed training data to perform incremental training on the trained model includes: Use the mixed training data to perform incremental training on the model, and guide the update of the model through a preset variety of loss functions during the model training process so that the model can continuously improve its accuracy during the training process.

5. An incremental training system for an image classification model based on Kafka message queue, characterized in that, The incremental training system for an image classification model based on a Kafka message queue includes: Incremental data collection device, which is used to collect incremental data through the Kafka message queue; Model acquisition device, which is used to acquire a trained model; Incremental model training device, which is used to perform incremental training on a trained model with incremental data; The image classification model incremental training system based on the Kafka message queue further includes: Incremental model deployment device, which is used to evaluate the model that has undergone incremental training; The collection of incremental data through the Kafka message queue includes: The user creates a Kafka image message producer in the image data source and produces one or more types of labeled image data to the Kafka server cluster; Consume the image data into the OpenCV working area, query the corresponding preprocessing format for this category, and perform image processing according to the preprocessing format to obtain the processed image data; Produce the image data into the Topic corresponding to the label category according to the label of the processed image data; The incremental training of a trained model with incremental data includes: Initialize the environment, use a Kafka consumer to consume the training environment configuration in the configured Topic, initialize the replay area space in the memory area according to the configuration, and use a Kafka consumer to consume the replay area data into the memory area replay area; Load model data, and use a Kafka consumer to consume the trained model from the corresponding Topic in Kafka into the memory area; Load incremental data, select the Topic where the incremental data is located, define the training data offset, and consume the incremental data into the memory area as training data; mix the replay area data and the incremental data as training data; Incrementally train the model, and use the mixed training data to perform incremental training on the trained model; Update the replay area data, and obtain the replay configuration items from the management data of the replay area data; Save the state, update the configuration items, and create a new configuration Topic to obtain the model that has undergone incremental training.

Images