An Online Incremental Update Method and Device for a Remote Sensing Image Object Classification Model

Abstract

The present invention discloses an online incremental update method and device for a remote sensing image object classification model. The method includes: obtaining a remote sensing image object classification model and a historical data set; using the historical data set to process the remote sensing image object classification model to obtain a first optimized model and a training data label set; using measured remote sensing image data and the training data label set to perform iterative incremental processing on the first optimized model to obtain an optimized remote sensing image object classification model. The present invention proposes an online incremental update method and device for a remote sensing image object classification model. An online data selection is performed by using a data increment control model based on sample importance ranking. The remote sensing image object classification model uses new data for continuous learning to acquire new knowledge, solving the problems of low iterative optimization efficiency and high training calculation cost of the existing intelligent positioning model for remote sensing image objects.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and remote sensing data processing, and particularly to an online incremental update method and device for a remote sensing image object classification model. Background Art

[0002] Remote sensing earth observation is an important means to obtain information about surface features. Currently, with the rapid development of remote sensing technology, the amount of earth observation data has increased exponentially. Using artificial intelligence technology to process massive remote sensing data has become the mainstream technology in current remote sensing applications. Among them, remote sensing image object localization and classification technology based on deep learning has been widely applied in multiple fields. However, the remote sensing database is continuously accumulated and updated, and surface features are also constantly evolving. This requires the remote sensing image object localization and classification model to have the ability to continuously optimize to adapt to new data, and at the same time maintain the "memory" of the "old" knowledge of historical data. Training the model regularly with historical data and new data can solve this problem. However, as the training data continues to increase, the cost of training from scratch keeps rising, bringing a huge computational burden to practical applications. Incremental learning allows the model to continuously learn new knowledge using new data without having to train the entire model from scratch, which is an important idea to solve the above problems. The present invention aims at the remote sensing object localization application system in business operation, and proposes an online incremental update method and device for a remote sensing image object classification model. By using a data increment control model based on sample importance ranking, it can continuously learn new knowledge using new data without having to train the entire model from scratch, improving the learning efficiency and saving the training cost. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide an online incremental update method and device for a remote sensing image object classification model, so as to solve the problems of low iterative optimization efficiency and high training calculation cost of the existing remote sensing image object intelligent localization model based on newly acquired observation data.

[0004] To solve the above technical problem, in the first aspect of the embodiments of the present invention, an online incremental update method for a remote sensing image object classification model is disclosed, and the method includes:

[0005] S1, obtaining a remote sensing image object classification model and a historical data set;

[0006] S2, using the historical data set to process the remote sensing image object classification model to obtain a first optimized model and a training data label set;

[0007] S3, using the measured remote sensing image data and the training data label set to perform iterative incremental processing on the first optimized model to obtain an optimized remote sensing image object classification model.

[0008] As an alternative implementation, in the first aspect of the embodiments of the present invention, processing the remote sensing image object classification model using the historical data set to obtain a first optimized model and a training data label set includes:

[0009] S21, dividing the historical data set according to a preset ratio to obtain a historical data training set and a historical data test set;

[0010] Set the number of training times value to 1;

[0011] S22, using the remote sensing image object classification model to process the historical data training set to obtain first image object classification information;

[0012] S23, using the first image object classification information and the historical data training set label information to update the parameters of the remote sensing image object classification model;

[0013] S24, determining whether the number of training times value is greater than a preset training times threshold to obtain a training times determination result;

[0014] Increase the number of training times value by 1;

[0015] When the training times determination result is yes, execute S25;

[0016] When the training times determination result is no, execute S22;

[0017] S25, using the remote sensing image object classification model to process the historical data test set to obtain second image object classification information;

[0018] S26, comparing the second image object classification information with the historical data test set label information to obtain prediction accuracy information;

[0019] S27, determining whether the prediction accuracy information is greater than a preset accuracy threshold to obtain an accuracy discrimination result;

[0020] When the accuracy discrimination result is no, execute S21;

[0021] If the accuracy discrimination result is yes, complete the training process of the remote sensing image object classification model to obtain a first optimized model, and execute S28;

[0022] S28, performing a fusion process on the historical data training set label information and the historical data test set label information to obtain a training data label set.

[0023] As an alternative implementation, in the first aspect of the embodiments of the present invention, the iterative incremental processing of the first optimization model by using the measured remote sensing image data and the training data label set to obtain the optimized remote sensing image object classification model includes:

[0024] S31, set the number of iterations, and set the number of incremental processing times to 1;

[0025] S32, obtain the measured remote sensing image data set;

[0026] S33, use the first optimization model to process the measured remote sensing image data set to obtain the first classification and positioning data set and the second classification and positioning data set;

[0027] S34, process the historical data set, the first classification and positioning data set, and the second classification and positioning data set to obtain an incremental training data set;

[0028] S35, use the incremental training data set and the training data label set to process the first optimization model to obtain a second optimization model and an incremental training label set;

[0029] S36, perform fusion processing on the incremental training label set and the training data label set to obtain a historical label set;

[0030] Increase the number of incremental processing times by 1;

[0031] S37, determine whether the number of incremental processing times is greater than the number of iterations to obtain an iteration judgment result;

[0032] When the iteration judgment result is negative, update the incremental training data set to the historical data set, update the historical label set to the training data label set, and execute S32;

[0033] When the iteration judgment result is positive, update the second optimization model to the optimized remote sensing image object classification model.

[0034] As an alternative implementation, in the first aspect of the embodiments of the present invention, the processing of the historical data set, the first classification and positioning data set, and the second classification and positioning data set to obtain an incremental training data set includes:

[0035] S341, perform fusion processing on the historical data set and the third classification and positioning data set to obtain a fourth classification and positioning data set;

[0036] S342, process the fourth classification and positioning data set based on the first data increment control model to obtain a fifth classification and positioning data set;

[0037] S343. Process the first type of positioning dataset based on the second data increment control model to obtain a sixth classified positioning dataset;

[0038] S344. Perform a fusion process on the fifth classified positioning dataset and the sixth classified positioning dataset to obtain an incremental training dataset.

[0039] As an optional implementation manner, in the first aspect of the embodiments of the present invention, the expression of the first data increment control model is:

[0040] Rs(i) = UC(i) + IA(i)

[0041]

[0042] IA{img i} = 1 - AP50(TL(img i ), PR(img i ))

[0043] Among them, Rs(i) represents the sorting score of the i-th remote sensing image; UC(i) represents the uncertainty score of the i-th remote sensing image; IA(i) represents the inaccuracy score of the i-th remote sensing image; i represents the index of the remote sensing image; uc{img i} represents the uncertainty of the i-th remote sensing image; R(img i ) represents the transformed image of the i-th remote sensing image rotated by 180°; uc{R(img i ) represents the uncertainty of the transformed image; img i represents the i-th remote sensing image; is the confidence of the j-th predicted object box in the i-th remote sensing image; N represents the total number of predicted object placement boxes; n represents the index of the predicted object placement box; IA{img i} represents the difference between the predicted box and the true box of the i-th remote sensing image; TL(img i ) is the true label in the i-th remote sensing image, and PR(img i ) is the predicted value in the i-th remote sensing image; AP50() represents performing an average precision calculation.

[0044] As an optional implementation manner, in the first aspect of the embodiments of the present invention, the process of using the incremental training dataset and the training data label set to process the first optimization model to obtain a second optimization model and an incremental training label set includes:

[0045] S351. Obtain the incremental training dataset;

[0046] The incremental training data set includes a number of incremental training data; the incremental training data is historical data, or first classification and positioning data, or second classification and positioning data;

[0047] S352. Process the incremental training data set based on the training data label set to obtain an incremental training label set;

[0048] S353. Process the first optimization model using the incremental training data set based on the incremental training label set to obtain a second optimization model.

[0049] As an optional implementation manner, in the first aspect of the embodiments of the present invention, the process of processing the incremental training data set based on the training data label set to obtain an incremental training label set includes:

[0050] S3521. Traverse the incremental training data set to obtain all incremental training data;

[0051] S3522. Parse and process any one of the incremental training data to obtain data type information;

[0052] S3523. Perform matching processing on the training data label set based on the data type information to obtain incremental training label information;

[0053] S3524. Combine all the incremental training label information in sequence to obtain an incremental training label set.

[0054] The second aspect of the embodiments of the present invention discloses an online incremental update device for a remote sensing image object classification model, and the device includes:

[0055] An acquisition module, configured to acquire a remote sensing image object classification model and a historical data set;

[0056] A first processing module, configured to process the remote sensing image object classification model using the historical data set to obtain a first optimization model;

[0057] A second processing module, configured to perform iterative incremental processing on the first optimization model using measured remote sensing image data to obtain an optimized remote sensing image object classification model.

[0058] The third aspect of the present invention discloses another online incremental update device for a remote sensing image object classification model, and the device includes:

[0059] A memory storing executable program code;

[0060] A processor coupled to the memory;

[0061] The processor calls the executable program code stored in the memory and executes some or all of the steps in the method for online incremental update of the remote sensing image object classification model disclosed in the first aspect of the embodiments of the present invention.

[0062] The fourth aspect of the present invention discloses a computer-readable storage medium storing computer instructions, which are used to execute some or all of the steps in the method for online incremental update of the remote sensing image object classification model disclosed in the first aspect of the embodiments of the present invention when the computer instructions are called.

[0063] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0064] In the embodiments of the present invention, an online data selection is performed by using a data increment control model based on sample importance ranking, and the remote sensing image object classification model continuously learns with new data to acquire new knowledge, solving the problems of low iteration optimization efficiency and high training calculation cost of the existing intelligent positioning model for remote sensing image objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0066] Figure 1 is a schematic diagram of the scenario of an online incremental update system for a remote sensing image object classification model disclosed in the embodiments of the present invention;

[0067] Figure 2 is a schematic flowchart of a method for online incremental update of a remote sensing image object classification model disclosed in the embodiments of the present invention;

[0068] Figure 3 is a schematic structural diagram of an online incremental update device for a remote sensing image object classification model disclosed in the embodiments of the present invention;

[0069] Figure 4 is a schematic structural diagram of another online incremental update device for a remote sensing image object classification model disclosed in the embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0070] To enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0071] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products or devices.

[0072] The mention of "embodiment" in this article means that the specific features, structures or characteristics described in connection with the embodiment may be included in at least one embodiment of the present invention. The appearance of this phrase in various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0073] In this application, the word "exemplary" is used to mean "serving as an example, illustration, or description". Any embodiment described as "exemplary" in this application is not necessarily to be construed as more preferred or more advantageous than other embodiments. In order to enable any person skilled in the art to implement and use this application, the following description is given. In the following description, details are set forth for the purpose of explanation. It should be understood that those of ordinary skill in the art can recognize that this application can be implemented without using these specific details. In other instances, well-known structures and processes will not be described in detail so as not to obscure the description of this application with unnecessary details. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope that conforms to the principles and features disclosed in this application.

[0074] It should be noted that since the method of the embodiment of this application is executed in a computer device, the processing objects of each computer device exist in the form of data or information. For example, time, which is actually time information. It can be understood that in subsequent embodiments, if dimensions, quantities, positions, etc. are mentioned, they are all corresponding data existences for the computer device to process, and specific details will not be elaborated here.

[0075] It should be noted that a brief introduction to the artificial intelligence-related technologies that may be involved in this application is provided. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a way similar to human intelligence. Artificial intelligence also studies the design principles and implementation methods of various intelligent machines to enable the machines to have the functions of perception, reasoning, and decision-making.

[0076] Artificial intelligence technology is an interdisciplinary subject that involves a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. The software technologies of artificial intelligence mainly include several major directions such as computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0077] Computer Vision Technology (CV) Computer vision is a science that studies how to enable machines to "see". Further, it refers to using cameras and computers to replace human eyes to identify and measure targets, etc., which is machine vision, and further perform graphic processing to make the computer process the images into images that are more suitable for human eyes to observe or be transmitted to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes technologies such as image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, etc., and also includes common biometric recognition technologies such as face recognition and fingerprint recognition.

[0078] Single-modal information is data of only one type, such as one of the data information of text, image, audio, video, electromagnetic signal, etc. Multi-modal information is data information that includes at least two types of single-modal information. Further, multi-modal information is applicable to complex tasks that require integrating multiple information sources, such as sentiment analysis, robot interaction, and autonomous driving. By integrating information of multiple modalities, higher performance and accuracy can usually be achieved in tasks.

[0079] Large models refer to artificial neural network models with a very large number of parameters. In the field of artificial intelligence, large models generally refer to models with hundreds of millions to trillions of parameters. The models usually need to be trained on large-scale datasets and require a large amount of computing resources for optimization and adjustment. Large models are typically used to solve complex natural language processing, computer vision, speech recognition, and other tasks. Generative AI is a type of AI that can create new content and ideas, including conversations, stories, images, videos, and music. In the embodiments of this application, the large model can be large language models such as ChatGPT, BERT, XLNet, Zhipu Model, Claude, Moonshot AI Model, ChatGLM Model, Qianyitongwen Model, MiniMax Model, Spark Model, Llama Model, 360GPT Model, Qwen Model, Baichuan Model, Lark Model, vivoLM Model, and Wenxin Yiyan, and the embodiments of this application do not make any limitations.

[0080] The embodiments of this application provide a method, device, computer device, and computer-readable storage medium for online incremental update of a remote sensing image object classification model, which will be described in detail below.

[0081] Please refer to Figure 1 , Figure 1 , which is a schematic diagram of the application scenario of the online incremental update system of the remote sensing image object classification model in the remote sensing object positioning system. The remote sensing object positioning system may include a computer device 100, and the computer device 100 is integrated with an online incremental update device for the remote sensing image object classification model, such as Figure 1 the computer device in

[0082] In the embodiments of this application, the computer device 100 may be an independent server or a server network or server cluster composed of servers. For example, the computer device 100 described in the embodiments of this application includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. Among them, the cloud server is composed of a large number of computers or network servers based on cloud computing.

[0083] It can be understood that the computer device 100 used in the embodiments of this application may be a device that includes both receiving and transmitting hardware, that is, a device with receiving and transmitting hardware capable of performing two-way communication on a two-way communication link. Such devices may include: cellular or other communication devices, which have a single-line display or a multi-line display or cellular or other communication devices without a multi-line display. Specifically, the computer device 100 may be a desktop terminal or a mobile terminal, and the computer device 100 may specifically also be one of a mobile phone, a tablet computer, a laptop computer, etc.

[0084] Those skilled in the art can understand that Figure 1 the application environment shown is only one application scenario of the solution of this application, and does not constitute a limitation on the application scenario of the solution of this application. Other application environments may also include more or fewer computer devices than Figure 1 shown, for example Figure 1 only 1 computer device is shown in. It can be understood that the system may also include one or more other services, which are not specifically limited here.

[0085] In addition, as Figure 1 shown, the remote sensing object positioning system may also include a memory 200 for storing simulation data, such as historical data and label data, etc.

[0086] It should be noted that Figure 1 the scene schematic diagram of the remote sensing object positioning system shown is only an example. The multi-sample simulation control system and scene for air-ground game described in the embodiments of this application are for more clearly explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art know that with the evolution of the simulation control management system and the emergence of new business scenarios, the technical solutions provided by the embodiments of this application are equally applicable to similar technical problems.

[0087] The present invention discloses a method and device for online incremental update of a remote sensing image object classification model. An online data selection is performed by using a data increment control model based on sample importance ranking. The remote sensing image object classification model uses new data for continuous learning to obtain new knowledge, and solves the problems of low iteration optimization efficiency and high training calculation cost of the existing remote sensing image object intelligent positioning model. The following will be described in detail respectively.

[0088] Embodiment 1

[0089] Please refer to Figure 2 , Figure 2 which is a flowchart of a method for online incremental update of a remote sensing image object classification model disclosed in an embodiment of the present invention. Among them, Figure 2 the method for online incremental update of the remote sensing image object classification model described is applied to a remote sensing object positioning application system, such as a local server or a cloud server for remote sensing object positioning system management, etc., which is not limited in the embodiments of the present invention. As Figure 2 shown, the online incremental update of the multi-remote sensing image object classification model may include the following operations:

[0090] S1, obtaining a remote sensing image object classification model and a historical data set;

[0091] It should be noted that in this embodiment, the remote sensing image object classification model is the YOLOv5 model;

[0092] S2. Use the historical data set to process the remote sensing image object classification model to obtain a first optimized model and a training data label set;

[0093] It should be noted that the first optimized model refers to the first optimized model obtained by optimizing the remote sensing image object classification model;

[0094] S3. Use the measured remote sensing image data and the training data label set to perform iterative incremental processing on the first optimized model to obtain a remote sensing image object classification optimized model.

[0095] It can be seen that in the method for online incremental update of the remote sensing image object classification model described in the embodiments of the present invention, a data incremental control model based on sample importance ranking is used for online data selection. The remote sensing image object classification model uses new data for continuous learning to acquire new knowledge, improving the iterative optimization efficiency of the remote sensing image object classification model and reducing the training calculation cost.

[0096] In an optional embodiment, in the above step S2, the using the historical data set to process the remote sensing image object classification model to obtain a first optimized model and a training data label set includes:

[0097] S21. Divide the historical data set according to a preset ratio to obtain a historical data training set and a historical data test set;

[0098] Set the training times value to 1;

[0099] S22. Use the remote sensing image object classification model to process the historical data training set to obtain first image object classification information;

[0100] S23. Use the first image object classification information and the historical data training set label information to update the parameters of the remote sensing image object classification model;

[0101] S24. Judge whether the training times value is greater than a preset training times threshold to obtain a training times judgment result;

[0102] Increase the training times value by 1;

[0103] When the training times judgment result is yes, execute S25;

[0104] When the training times judgment result is no, execute S22;

[0105] S25. Use the remote sensing image object classification model to process the historical data test set to obtain second image object classification information;

[0106] S26. Compare the second image object classification information with the historical data test set label information to obtain prediction accuracy information;

[0107] S27. Determine whether the prediction accuracy information is greater than a preset accuracy threshold to obtain an accuracy discrimination result;

[0108] When the accuracy discrimination result is negative, execute S21;

[0109] If the accuracy discrimination result is positive, complete the training process of the remote sensing image object classification model to obtain a first optimized model, and execute S28;

[0110] S28. Perform a fusion process on the historical data training set label information and the historical data test set label information to obtain a training data label set.

[0111] It can be seen that when implementing the online incremental update method of the remote sensing image object classification model described in the embodiments of the present invention, a data increment control model based on sample importance ranking is used for online data selection. The remote sensing image object classification model uses new data to continuously learn and acquire new knowledge, improving the iterative optimization efficiency of the remote sensing image object classification model and reducing the training calculation cost.

[0112] In an alternative embodiment, in the above step S23, the use of the first image object classification information and the historical data training set label information to update the parameters of the remote sensing image object classification model includes:

[0113] Based on a parameter update model, use the first image object classification information and the historical data training set label information to update the parameters of the remote sensing image object classification model;

[0114] The expression of the parameter update model is:

[0115]

[0116] θ←θ + v;

[0117] In the formula, x (i) is the i-th remote sensing image of the historical data training set, y (i) is the label information of the i-th remote sensing image object of the historical data training set, v is the update speed, θ is the parameter of the remote sensing image object classification model, η2 is the initial parameter learning rate, α is the momentum parameter, f(·) is the model calculation function, and l() is the first image object classification information;

[0118] It should be noted that in this embodiment, the initial parameter learning rate η2 is set to 0.001.

[0119] It can be seen that when implementing the online incremental update method for the remote sensing image object classification model described in the embodiments of the present invention, a data increment control model based on sample importance ranking is used for online data selection. The remote sensing image object classification model utilizes new data for continuous learning to acquire new knowledge, improving the iterative optimization efficiency of the remote sensing image object classification model and reducing the training calculation cost.

[0120] In an optional embodiment, in step S28 above, the fusion processing of the historical data training set label information and the historical data test set label information to obtain a training data label set includes:

[0121] S281, traverse the historical data training set to obtain all historical training data;

[0122] S282, perform parsing processing on any one of the historical training data to obtain training data type information and training data label information;

[0123] S283, traverse the historical data test set to obtain all historical test data;

[0124] S284, perform parsing processing on any one of the historical test data to obtain test data type information and test data label information;

[0125] S285, perform deduplication processing on the training data type information, the training data label information, the test data type information, and the test data label information to obtain a training data label set;

[0126] It should be noted that the deduplication processing means only retaining one for multiple data of the same type and label.

[0127] It can be seen that when implementing the online incremental update method for the remote sensing image object classification model described in the embodiments of the present invention, a data increment control model based on sample importance ranking is used for online data selection. The remote sensing image object classification model utilizes new data for iterative update processing to obtain an optimized model for remote sensing image object classification, improving the iterative optimization efficiency of the remote sensing image object classification model and reducing the training calculation cost.

[0128] In an optional embodiment, in step S3 above, the iterative incremental processing of the first optimized model using the measured remote sensing image data and the training data label set to obtain an optimized model for remote sensing image object classification includes:

[0129] S31. Set the number of iterations and set the number of incremental processing times to 1;

[0130] S32. Obtain the measured remote sensing image dataset;

[0131] S33. Use the first optimization model to process the measured remote sensing image dataset to obtain a first classification and positioning dataset and a second classification and positioning dataset;

[0132] It should be noted that the first classification and positioning dataset includes several first classification and positioning data; the first classification and positioning data represents the measured remote sensing image data with positioning classification errors, missed detections, and false alarms;

[0133] It should be noted that the second classification and positioning dataset includes several second classification and positioning data; the second classification and positioning data represents the measured remote sensing image data with accurate positioning classification;

[0134] It should be noted that the measured remote sensing image dataset is the combination of the first classification and positioning dataset and the second classification and positioning dataset;

[0135] S34. Process the historical dataset, the first classification and positioning dataset, and the second classification and positioning dataset to obtain an incremental training dataset;

[0136] S35. Use the incremental training dataset and the training data label set to process the first optimization model to obtain a second optimization model and an incremental training label set;

[0137] It should be noted that the second optimization model represents the second remote sensing image object classification optimization model obtained by iteratively processing the first optimization model;

[0138] S36. Perform a fusion process on the incremental training label set and the training data label set to obtain a historical label set;

[0139] Increase the incremental processing times by 1;

[0140] S37. Determine whether the incremental processing times are greater than the number of iterations to obtain an iteration judgment result;

[0141] When the iteration judgment result is no, update the incremental training dataset to the historical dataset, update the historical label set to the training data label set, and execute S32;

[0142] When the iteration judgment result is yes, update the second optimization model to the remote sensing image object classification optimization model.

[0143] It can be seen that by implementing the online incremental update method for the remote sensing image object classification model described in the embodiments of the present invention, an online data selection is performed using a data incremental control model based on sample importance ranking. The remote sensing image object classification model is iteratively updated using new data to obtain an optimized remote sensing image object classification model, which improves the iterative optimization efficiency of the remote sensing image object classification model and reduces the training calculation cost.

[0144] In an alternative embodiment, in the above step S34, the processing of the historical data set, the first classification and positioning data set, and the second classification and positioning data set to obtain an incremental training data set includes:

[0145] S341, fusing the historical data set and the third classification and positioning data set to obtain a fourth classification and positioning data set;

[0146] It should be noted that the fusing process means combining the historical data set and the third classification and positioning data set in sequence.

[0147] S342, processing the fourth classification and positioning data set based on a first data incremental control model to obtain a fifth classification and positioning data set;

[0148] S343, processing the first type of positioning data set based on a second data incremental control model to obtain a sixth classification and positioning data set;

[0149] S344, fusing the fifth classification and positioning data set and the sixth classification and positioning data set to obtain an incremental training data set;

[0150] It should be noted that the fusing process means combining the fifth classification and positioning data set and the sixth classification and positioning data set in sequence.

[0151] It can be seen that by implementing the online incremental update method for the remote sensing image object classification model described in the embodiments of the present invention, an online data selection is performed using a data incremental control model based on sample importance ranking, and the remote sensing image object classification model can be iteratively updated online, which improves the iterative optimization efficiency of the remote sensing image object classification model and reduces the training calculation cost.

[0152] In another alternative embodiment, in the above step S342, the expression of the first data incremental control model is:

[0153] Rs(i) = UC(i) + IA(i)

[0154]

[0155] IA{imgi} = 1 - AP50(TL(img i ), PR(img i ))

[0156] Where Rs(i) represents the sorting score of the i-th remote sensing image; UC(i) represents the uncertainty score of the i-th remote sensing image; IA(i) represents the inaccuracy score of the i-th remote sensing image; i represents the index of the remote sensing image; uc{img i} represents the uncertainty of the i-th remote sensing image; R(img i ) represents the transformed image after rotating the i-th remote sensing image by 180°; uc{R(img i ) represents the uncertainty of the transformed image; img i represents the i-th remote sensing image; is the confidence of the j-th predicted object box in the i-th remote sensing image; N represents the total number of predicted object placement boxes; n represents the index of the predicted object placement box; IA{img i} represents the difference between the predicted box and the ground truth box of the i-th remote sensing image; TL(img i ) is the ground truth label in the i-th remote sensing image, and PR(img i ) is the predicted value in the i-th remote sensing image; AP50() represents the calculation of average precision;

[0157] It should be noted that the first data increment control model is used to obtain the sorting score of the remote sensing image. In this embodiment, the uncertainty and inaccuracy sorting scores are used to measure the importance of instance data, and the remote sensing measured data is sampled at uniform intervals based on the importance sorting, which can reflect the samples of the existing learning experience of the remote sensing image object classification model. When re-learning, the more stable the model can be maintained. The prediction result of the remote sensing image object classification model for the "old" knowledge instance data is based on the existing learning experience. Therefore, in the incremental training dataset, the more diverse the prediction results, that is, the "old" knowledge instances with greater prediction result differences, can better reflect the existing learning experience of the remote sensing image object classification model and are conducive to memory retention. In this embodiment, the uncertainty score and inaccuracy score sorting are used to measure the data importance, and the more diverse the sorting score values, the higher the importance of maintaining the model stability in incremental training.

[0158] It can be seen that by implementing the online incremental update method for the remote sensing image object classification model described in the embodiments of the present invention, the importance of instances is measured by sorting the uncertainty score and the inaccuracy score, and the first data increment control model is used to select data online. The remote sensing image object classification model uses new data for iterative update processing to obtain an optimized remote sensing image object classification model, which improves the iterative optimization efficiency of the remote sensing image object classification model and reduces the training calculation cost.

[0159] In another optional embodiment, in the above step S342, the expression of the second data increment control model is:

[0160]

[0161] Among them, A i represents the uncertainty of the i-th remote sensing image; A 2i represents the uncertainty of the transformed image after rotating the i-th remote sensing image by 180°; A 1i represents the uncertainty of the transformed image; i represents the index of the processed remote sensing image; B j is the confidence of the j-th predicted object box in the i-th remote sensing image; N represents the total number of predicted object placement boxes; n represents the index of the predicted object placement box.

[0162] It should be noted that the second data increment control model is used to obtain the uncertainty ranking score of the remote sensing image, and the uncertainty ranking score is used to measure the importance of the remote sensing image data and select the remote sensing image data with high importance to enter the incremental training sample set.

[0163] Exemplarily, the second data increment is the measured remote sensing image data with positioning classification errors, missed detections, and false alarms. After the remote sensing image object classification model predicts a remote sensing image data, it outputs the prediction results at the object box level (the class confidence scores and position coordinates of all object boxes). For a predicted object box, the highest among all class confidence scores is the confidence of the remote sensing image object. The lower this value, the more uncertain the prediction result of the remote sensing image object classification model for this object box.

[0164] It can be seen that by implementing the online incremental update method for the remote sensing image object classification model described in the embodiments of the present invention, the first data increment control model is used to select data online. The remote sensing image object classification model uses new data for iterative update processing to obtain an optimized remote sensing image object classification model, which improves the iterative optimization efficiency of the remote sensing image object classification model and reduces the training calculation cost.

[0165] In an optional embodiment, in the above step S35, the process of using the incremental training data set and the training data label set to process the first optimized model to obtain a second optimized model includes:

[0166] S351, obtain the incremental training data set;

[0167] The incremental training data set includes a number of incremental training data; the incremental training data is historical data or first classification and positioning data or second classification and positioning data;

[0168] S352, process the incremental training data set based on the training data label set to obtain an incremental training label set;

[0169] S353, based on the incremental training label set, use the incremental training data set to process the first optimized model to obtain a second optimized model.

[0170] It can be seen that in the method for online incremental update of the remote sensing image object classification model described in the embodiments of the present invention, a data increment control model based on sample importance ranking is used for online data selection. The remote sensing image object classification model uses new data for continuous learning to obtain new knowledge, improving the iterative optimization efficiency of the remote sensing image object classification model and reducing the training calculation cost.

[0171] In an optional embodiment, in the above step S352, the process of processing the incremental training data set based on the training data label set to obtain an incremental training label set includes:

[0172] S3521, traverse the incremental training data set to obtain all incremental training data;

[0173] S3522, perform parsing processing on any one of the incremental training data to obtain data type information;

[0174] S3523, perform matching processing on the training data label set based on the data type information to obtain incremental training label information;

[0175] S3524, combine all the incremental training label information in sequence to obtain an incremental training label set.

[0176] It can be seen that in the method for online incremental update of the remote sensing image object classification model described in the embodiments of the present invention, a data increment control model based on sample importance ranking is used for online data selection. The remote sensing image object classification model uses new data for continuous learning to obtain new knowledge, improving the iterative optimization efficiency of the remote sensing image object classification model and reducing the training calculation cost.

[0177] In an optional embodiment, in the above step S353, processing the first optimization model with the incremental training data set based on the incremental training label set to obtain a second optimization model includes:

[0178] S3531, traversing the incremental training data set to obtain all incremental training data;

[0179] S3532, inputting any one of the incremental training data into the first optimization model to obtain third image object classification information;

[0180] S3533, comparing the third image object classification information with the incremental training label set to obtain classification accuracy information;

[0181] S3534, determining whether the classification accuracy information is greater than a preset accuracy threshold to obtain a classification accuracy discrimination result;

[0182] When the classification accuracy discrimination result is negative, execute S3532;

[0183] If the classification accuracy discrimination result is positive, obtain the second optimization model.

[0184] It can be seen that when implementing the remote sensing image object classification model online incremental update method described in the embodiments of the present invention, a data increment control model based on sample importance ranking is used for online data selection. The remote sensing image object classification model uses new data for continuous learning to acquire new knowledge, improving the iterative optimization efficiency of the remote sensing image object classification model and reducing the training calculation cost.

[0185] Embodiment 2

[0186] Please refer to Figure 3 , Figure 3 which is a schematic structural diagram of an online incremental update device for a remote sensing image object classification model disclosed in the embodiments of the present invention. Among them, Figure 3 the described device can be applied to a remote sensing object positioning system, such as a local server or a cloud server for remote sensing object positioning, etc., which is not limited in the embodiments of the present invention. As Figure 3 shown, the device may include:

[0187] An acquisition module 101, configured to acquire a remote sensing image object classification model and a historical data set;

[0188] A first processing module 102, configured to process the remote sensing image object classification model with the historical data set to obtain a first optimization model;

[0189] The second processing module 103 is configured to perform iterative incremental processing on the first optimization model by using the measured remote sensing image data, so as to obtain an optimized remote sensing image object classification model.

[0190] It can be seen that by implementing the multi-satellite collaborative three-dimensional observation method described in the embodiments of the present invention, the limitations and errors existing in single-satellite observation are overcome, the difficult-to-understand spatial information is visualized and made intuitive, and the efficiency of data analysis and processing is improved.

[0191] Embodiment III

[0192] Please refer to Figure 4 , Figure 4 which is a schematic structural diagram of another on-line incremental update device for a remote sensing image object classification model disclosed in the embodiments of the present invention. Among them, Figure 4 the described device can be applied to a satellite observation system, such as a local server or a cloud server for satellite observation, etc., which is not limited in the embodiments of the present invention. As Figure 4 shown, the device may include:

[0193] A memory 201 storing executable program code;

[0194] A processor 202 coupled to the memory;

[0195] The processor 202 calls the executable program code stored in the memory 201 to execute the steps in the on-line incremental update method for the remote sensing image object classification model described in Embodiment I.

[0196] Embodiment IV

[0197] The embodiments of the present invention disclose a computer-readable storage medium, which stores a computer program for electronic data exchange. Among them, the computer program enables a computer to execute the steps in the on-line incremental update method for the remote sensing image object classification model described in Embodiment I.

[0198] The device embodiments described above are only illustrative. The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place, or may be distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0199] Through the specific descriptions of the above embodiments, those skilled in the art can clearly understand that each implementation can be achieved by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, and the storage medium includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc memories, magnetic disk memories, tape memories, or any other computer-readable medium that can be used to carry or store data.

[0200] Finally, it should be noted that: The remote sensing image object classification model online incremental update method and device disclosed in the embodiments of the present invention only disclose the preferred embodiments of the present invention, which are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An online incremental update method for a remote sensing image object classification model, characterized in that The method includes: S1. Obtain a remote sensing image object classification model and a historical data set; S2. Use the historical data set to process the remote sensing image object classification model to obtain a first optimized model and a training data label set; S3. Use the measured remote sensing image data and the training data label set to perform iterative incremental processing on the first optimized model to obtain a remote sensing image object classification optimized model; Among them, the step of using the measured remote sensing image data and the training data label set to perform iterative incremental processing on the first optimized model to obtain a remote sensing image object classification optimized model includes: S31. Set the number of iterations, and set the number of iterative processing times to 1; S32. Obtain a measured remote sensing image data set; S33. Use the first optimized model to process the measured remote sensing image data set to obtain a first classification and positioning data set and a second classification and positioning data set; S34. Process the historical data set, the first classification and positioning data set, and the second classification and positioning data set to obtain an incremental training data set; S35. Use the incremental training data set and the training data label set to process the first optimized model to obtain a second optimized model and an incremental training label set; S36. Perform fusion processing on the incremental training label set and the training data label set to obtain a historical label set; Increase the number of iterative processing times by 1; S37. Judge whether the number of iterative processing times is greater than the number of iterations to obtain an iterative judgment result; When the iterative judgment result is negative, update the historical label set to the training data label set, and execute S32; When the iterative judgment result is positive, update the second optimized model to the remote sensing image object classification optimized model; Among them, the step of processing the historical data set, the first classification and positioning data set, and the second classification and positioning data set to obtain an incremental training data set includes: S341. Perform fusion processing on the historical data set and a third classification and positioning data set to obtain a fourth classification and positioning data set; S342. Based on a first data increment control model, process the fourth classification and positioning data set to obtain a fifth classification and positioning data set; S343. Based on a second data increment control model, process the first classification and positioning data set to obtain a sixth classification and positioning data set; S344. Perform fusion processing on the fifth classification and positioning data set and the sixth classification and positioning data set to obtain an incremental training data set.

2. The method for online incremental update of the remote sensing image object classification model according to claim 1, wherein The step of using the historical data set to process the remote sensing image object classification model to obtain a first optimized model and a training data label set includes: S21. Divide the historical data set according to a preset ratio to obtain a historical data training set and a historical data test set; Set the value of the number of training times to 1; S22. Use the remote sensing image object classification model to process the historical data training set to obtain first image object classification information; S23. Use the first image object classification information and the historical data training set label information to update the parameters of the remote sensing image object classification model; S24. Determine whether the training times value is greater than a preset training times threshold to obtain a training times judgment result; Increment the training times value by 1; When the training times judgment result is yes, execute S25; When the training times judgment result is no, execute S22; S25. Use the remote sensing image object classification model to process the historical data test set to obtain second image object classification information; S26. Compare the second image object classification information with the historical data test set label information to obtain a prediction accuracy rate information; S27. Determine whether the prediction accuracy rate information is greater than a preset accuracy rate threshold to obtain an accuracy rate discrimination result; When the accuracy rate discrimination result is no, execute S21; If the accuracy rate discrimination result is yes, obtain a first optimized model and execute S28; S28. Perform a fusion process on the historical data training set label information and the historical data test set label information to obtain a training data label set.

3. The method for online incremental update of the remote sensing image object classification model according to claim 1, wherein The expression of the first data increment control model is: Rs(i) = UC(i) + IA(i) IA{img i} = 1 - AP50(TL(img i ), PR(img i )) Among them, Rs(i) represents the sorting score of the i-th remote sensing image; UC(i) represents the uncertainty score of the i-th remote sensing image; IA(i) represents the inaccuracy score of the i-th remote sensing image; i represents the index of the remote sensing image; uc{img i} represents the uncertainty of the i-th remote sensing image; R(img i ) represents the transformed image after the i-th remote sensing image is rotated 180°; uc{R(img i ) represents the uncertainty of the transformed image; img i represents the i-th remote sensing image; is the confidence of the j-th predicted object box in the i-th remote sensing image; N represents the total number of predicted object placement boxes; n represents the index of the predicted object placement box; IA{img i} represents the difference between the predicted box and the ground truth box of the i-th remote sensing image; TL(img i ) is the ground truth label in the i-th remote sensing image, and PR(img i ) is the predicted value in the i-th remote sensing image; AP50() represents the calculation of the average precision.

4. The method for online incremental update of the remote sensing image object classification model according to claim 1, characterized in that, The process of using the incremental training data set and the training data label set to process the first optimized model to obtain a second optimized model and an incremental training label set includes: S351. Obtain the incremental training data set; The incremental training data set includes several pieces of incremental training data; the incremental training data is historical data or first classification and positioning data or second classification and positioning data; S352. Process the incremental training data set based on the training data label set to obtain an incremental training label set; S353. Based on the incremental training label set, use the incremental training data set to process the first optimized model to obtain a second optimized model.

5. The method for online incremental update of the remote sensing image object classification model according to claim 4, wherein The process of processing the incremental training data set based on the training data label set to obtain an incremental training label set includes: S3521. Traverse the incremental training data set to obtain all the incremental training data; S3522. Perform parsing processing on any one of the incremental training data to obtain data type information; S3523. Based on the data type information, perform matching processing on the training data label set to obtain incremental training label information; S3524. Combine all the incremental training label information in sequence to obtain an incremental training label set.

6. A device for online incremental updating of a remote sensing image object classification model, which is used to execute the online incremental updating method of the remote sensing image object classification model according to any one of claims 1-5, characterized in that, The device includes: An acquisition module for acquiring a remote sensing image object classification model and a historical data set; A first processing module for processing the remote sensing image object classification model using the historical data set to obtain a first optimized model; A second processing module for performing iterative incremental processing on the first optimized model using measured remote sensing image data to obtain a remote sensing image object classification optimized model.

7. An online incremental update device for a remote sensing image object classification model, characterized in that The device includes: A memory storing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory and executes the remote sensing image object classification model online incremental update method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions which, when called, are used to execute the method for online incremental update of the remote sensing image object classification model according to any one of claims 1-5.

Images