Online incremental updating method and device for object classification model of remote sensing image

Abstract

The invention discloses an online incremental updating method and device for a remote sensing image object classification model. The method comprises the steps that the remote sensing image object classification model and a historical data set are acquired; processing the remote sensing image object classification model by using the historical data set to obtain a first optimization model and a training data label set; and utilizing actually measured remote sensing image data and the training data label set to perform iterative increment processing on the first optimization model to obtain a remote sensing image object classification optimization model. The invention provides an online incremental updating method and device for a remote sensing image object classification model, and the method comprises the steps: carrying out the online data selection through employing a data incremental control model based on the sorting of the importance degrees of samples, and carrying out the continuous learning of a remote sensing image object classification model through employing new data, and obtaining new knowledge. The problems that an existing remote sensing image object intelligent positioning model is low in iterative optimization efficiency and high in training calculation cost are solved.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and remote sensing data processing, and in particular to an online incremental updating method and device for a remote sensing image object classification model. Background Art

[0002] Remote sensing earth observation is an important means of obtaining information about surface objects. At present, with the rapid development of remote sensing technology, earth observation data is increasing exponentially. Using artificial intelligence technology to process massive remote sensing data has become the mainstream technology of current remote sensing applications. Among them, remote sensing image object positioning and classification technology based on deep learning has been widely used in many fields. However, remote sensing databases are continuously accumulated and updated, and surface objects are also constantly developing and changing. This requires the remote sensing image object positioning and classification model to have the ability to continuously optimize to adapt to new data, while also maintaining the "memory" of the "old" knowledge of historical data. This problem can be solved by regularly training the model with historical data and new data, but as the training data continues to increase, the cost of training from scratch continues to increase, which brings a huge computational burden to practical applications. Incremental learning allows the model to continuously learn and acquire new knowledge using new data without having to train the entire model from scratch, which is an important idea to solve the above problems. The present invention is aimed at a business operation remote sensing object positioning application system, and proposes an online incremental update method and device for a remote sensing image object classification model. A data incremental control model based on sample importance sorting is adopted, and new data can be used for continuous learning to acquire new knowledge without having to train the entire model from scratch, thereby improving learning efficiency and saving training costs. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a method and device for online incremental updating of a remote sensing image object classification model, so as to solve the problems of low iterative optimization efficiency and high training calculation cost of the existing remote sensing image object intelligent positioning model based on newly acquired observation data.

[0004] In order to solve the above technical problems, the first aspect of the embodiment of the present invention discloses an online incremental update method for a remote sensing image object classification model, the method comprising:

[0005] S1, obtain remote sensing image object classification model and historical data set;

[0006] S2, using the historical data set, processing the remote sensing image object classification model to obtain a first optimization model and a training data label set;

[0007] S3, using the measured remote sensing image data and the training data label set, iteratively and incrementally process the first optimization model to obtain a remote sensing image object classification optimization model.

[0008] As an optional implementation, in the first aspect of the embodiment of the present invention, the use of the historical data set to process the remote sensing image object classification model to obtain a first optimization model and a training data label set includes:

[0009] S21, dividing the historical data set according to a preset ratio to obtain a historical data training set and a historical data test set;

[0010] Set the number of training times to 1;

[0011] S22, using the remote sensing image object classification model, processing the historical data training set to obtain first image object classification information;

[0012] S23, using the first image object classification information and historical data training set label information, updating parameters of the remote sensing image object classification model;

[0013] S24, determining whether the training times value is greater than a preset training times threshold, and obtaining a training times determination result;

[0014] Increase the training times by 1;

[0015] When the result of the training number determination is yes, executing S25;

[0016] When the result of the training number determination is no, executing S22;

[0017] S25, using the remote sensing image object classification model, processing the historical data test set to obtain second image object classification information;

[0018] S26, comparing the second image object classification information with the historical data test set label information to obtain prediction accuracy information;

[0019] S27, determining whether the prediction accuracy information is greater than a preset accuracy threshold, and obtaining an accuracy determination result;

[0020] When the accuracy determination result is no, executing S21;

[0021] If the accuracy determination result is yes, the training process of the remote sensing image object classification model is completed to obtain a first optimization model, and S28 is executed;

[0022] S28, fusing the historical data training set label information and the historical data test set label information to obtain a training data label set.

[0023] As an optional implementation, in the first aspect of the embodiment of the present invention, the use of the measured remote sensing image data and the training data label set to iteratively increment the first optimization model to obtain the remote sensing image object classification optimization model includes:

[0024] S31, set the number of iterations, and set the number of incremental processing times to 1;

[0025] S32, obtaining a measured remote sensing image dataset;

[0026] S33, using the first optimization model, processing the measured remote sensing image data set to obtain a first classification positioning data set and a second classification positioning data set;

[0027] S34, processing the historical data set, the first classification and positioning data set, and the second classification and positioning data set to obtain an incremental training data set;

[0028] S35, S35, using the incremental training data set and the training data label set, processing the first optimization model to obtain a second optimization model and an incremental training label set;

[0029] S36, fusing the incremental training label set with the training data label set to obtain a historical label set;

[0030] Increase the number of incremental processing times by 1;

[0031] S37, determining whether the number of incremental processing times is greater than the number of iterations, and obtaining an iteration determination result;

[0032] When the iterative judgment result is no, the incremental training data set is updated to the historical data set, the historical label set is updated to the training data label set, and S32 is executed;

[0033] When the iterative judgment result is yes, the second optimization model is updated to a remote sensing image object classification optimization model.

[0034] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the processing of the historical data set, the first classification and positioning data set, and the second classification and positioning data set to obtain an incremental training data set includes:

[0035] S341, fusing the historical data set and the third classification positioning data set to obtain a fourth classification positioning data set;

[0036] S342, processing the fourth classification positioning data set based on the first data increment control model to obtain a fifth classification positioning data set;

[0037] S343, processing the first category positioning data set based on the second data increment control model to obtain a sixth category positioning data set;

[0038] S344, fusing the fifth classification positioning data set and the sixth classification positioning data set to obtain an incremental training data set.

[0039] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the first data increment control model expression is:

[0040] Rs(i)=UC(i)+IA(i)

[0041]

[0042] IA{img i}=1-AP50(TL(img i ), PR(img i ))

[0043] Where Rs(i) represents the ranking score of the i-th remote sensing image; UC(i) represents the uncertainty score of the i-th remote sensing image; IA(i) represents the inaccuracy score of the i-th remote sensing image; i represents the index of the remote sensing image; uc{img i} represents the uncertainty of the i-th remote sensing image; R(img i ) represents the transformed image after the i-th remote sensing image is rotated 180°; uc{R(img i ) represents the uncertainty of the transformed image; img i represents the i-th remote sensing image; is the confidence of the jth predicted object frame in the i-th remote sensing image; N represents the total number of predicted object frames; n represents the index of the predicted object frame; IA{img i} represents the difference between the predicted box and the real box of the i-th remote sensing image; TL(img i ) is the true label in the i-th remote sensing image, PR(img i ) is the predicted value in the i-th remote sensing image; AP50() indicates the average precision calculation.

[0044] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the using the incremental training data set and the training data label set to process the first optimization model to obtain the second optimization model and the incremental training label set includes:

[0045] S351, obtaining the incremental training data set;

[0046] The incremental training data set includes a plurality of incremental training data; the incremental training data is historical data or first classification positioning data or second classification positioning data;

[0047] S352, based on the training data label set, processing the incremental training data set to obtain an incremental training label set;

[0048] S353: Based on the incremental training label set, the first optimization model is processed using the incremental training data set to obtain a second optimization model.

[0049] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the processing of the incremental training data set based on the training data label set to obtain the incremental training label set includes:

[0050] S3521, traverse the incremental training data set to obtain all incremental training data;

[0051] S3522, parsing and processing any of the incremental training data to obtain data type information;

[0052] S3523, performing matching processing on the training data label set based on the data type information to obtain incremental training label information;

[0053] S3524: All the incremental training label information is combined in order to obtain an incremental training label set.

[0054] A second aspect of an embodiment of the present invention discloses an online incremental update device for a remote sensing image object classification model, the device comprising:

[0055] An acquisition module is used to obtain remote sensing image object classification models and historical data sets;

[0056] A first processing module, used to process the remote sensing image object classification model using the historical data set to obtain a first optimization model;

[0057] The second processing module is used to use the measured remote sensing image data to perform iterative incremental processing on the first optimization model to obtain a remote sensing image object classification optimization model.

[0058] A third aspect of the present invention discloses another online incremental update device for a remote sensing image object classification model, the device comprising:

[0059] A memory storing executable program code;

[0060] a processor coupled to the memory;

[0061] The processor calls the executable program code stored in the memory to execute part or all of the steps in the online incremental update method of the remote sensing image object classification model disclosed in the first aspect of the embodiment of the present invention.

[0062] The fourth aspect of the present invention discloses a computer-readable storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute part or all of the steps in the online incremental update method of the remote sensing image object classification model disclosed in the first aspect of an embodiment of the present invention.

[0063] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0064] In an embodiment of the present invention, a data increment control model based on sample importance sorting is used for online data selection, and the remote sensing image object classification model uses new data for continuous learning to acquire new knowledge, thereby solving the problems of low iterative optimization efficiency and high training calculation cost of the existing remote sensing image object intelligent positioning model. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0066] Figure 1 It is a scene schematic diagram of an online incremental update system of a remote sensing image object classification model disclosed in an embodiment of the present invention;

[0067] Figure 2 It is a flowchart of an online incremental updating method of a remote sensing image object classification model disclosed in an embodiment of the present invention;

[0068] Figure 3 It is a structural schematic diagram of an online incremental updating device for a remote sensing image object classification model disclosed in an embodiment of the present invention;

[0069] Figure 4 It is a structural schematic diagram of another online incremental updating device for a remote sensing image object classification model disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0070] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or equipment.

[0072] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0073] In this application, the word "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any technician in the field to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0074] It should be noted that since the method of the embodiment of the present application is executed in a computer device, the processing objects of each computer device exist in the form of data or information. For example, time is actually time information. It can be understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data for processing by the computer device. The details will not be repeated here.

[0075] It should be noted that the artificial intelligence related technologies that may be involved in this application are briefly described. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0076] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0077] Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further processing graphics so that the computer processing becomes an image that is more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0078] Unimodal information is data of only one type, such as text, image, audio, video, electromagnetic signal, etc. Multimodal information is data that includes at least two types of unimodal information. Furthermore, multimodal information is suitable for complex tasks that require the integration of multiple information sources, such as sentiment analysis, robot interaction, autonomous driving, etc. By integrating information from multiple modalities, higher performance and accuracy can usually be achieved on the task.

[0079] A large model refers to an artificial neural network model with a very large number of parameters. In the field of artificial intelligence, a large model generally refers to a model with hundreds of millions to trillions of parameters. Models usually need to be trained on large-scale data sets and require a large amount of computing resources to be optimized and adjusted. Large models are often used to solve complex tasks such as natural language processing, computer vision, and speech recognition. Generative AI is an AI that can create new content and ideas, including conversations, stories, images, videos, and music. In an embodiment of the present application, the large model may be ChatGPT, BERT, XLNet, Zhipu model, Claude, Moonshot AI model, ChatGLM model, Qianyi Tongwen model, MiniMax model, Spark model, Llama model, 360GPT model, Qwen model, Baichuan model, Skylark model, vivoLM model, and Wenxin Yiyan scale language models, which are not limited in the embodiments of the present application.

[0080] The embodiments of the present application provide a method, apparatus, computer device, and computer-readable storage medium for online incremental updating of a remote sensing image object classification model, which are described in detail below.

[0081] See also Figure 1 , Figure 1 Schematic diagram of an application scenario of the online incremental update system of the remote sensing image object classification model provided by the embodiment of the present application in the remote sensing object positioning system. The remote sensing object positioning system may include a computer device 100, in which the online incremental update device of the remote sensing image object classification model is integrated, such as Figure 1 Computer equipment in.

[0082] In the embodiment of the present application, the computer device 100 may be an independent server, or a server network or server cluster composed of servers. For example, the computer device 100 described in the embodiment of the present application includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets or a cloud server composed of a plurality of servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0083] It is understandable that the computer device 100 used in the embodiments of the present application may be a device including both receiving and transmitting hardware, that is, a device having receiving and transmitting hardware capable of performing two-way communication on a two-way communication link. Such a device may include: a cellular or other communication device having a single-line display or a multi-line display or a cellular or other communication device without a multi-line display. The specific computer device 100 may be a desktop terminal or a mobile terminal, and the computer device 100 may also be one of a mobile phone, a tablet computer, a laptop computer, etc.

[0084] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only one application scenario of the present application solution and does not constitute a limitation on the application scenario of the present application solution. Other application environments may also include Figure 1 More or less computer equipment as shown in Figure 1 Only one computer device is shown in the figure. It can be understood that the system can also include one or more other services, which are not limited here.

[0085] In addition, if Figure 1 As shown, the remote sensing object positioning system may further include a memory 200 for storing simulation data, such as historical data and label data.

[0086] It should be noted that Figure 1 The scene diagram of the remote sensing object positioning system shown is only an example. The multi-sample simulation control system and scene for air-ground game described in the embodiment of the present application are for more clearly illustrating the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. Ordinary technicians in this field can know that with the evolution of the simulation control management system and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0087] The present invention discloses an online incremental update method and device for a remote sensing image object classification model, which uses a data incremental control model based on sample importance ranking to select online data. The remote sensing image object classification model uses new data for continuous learning to acquire new knowledge, solving the problems of low iterative optimization efficiency and high training calculation cost of the existing remote sensing image object intelligent positioning model. The following are detailed descriptions.

[0088] Embodiment 1

[0089] See also Figure 2 , Figure 2 1 is a flow chart of an online incremental update method for a remote sensing image object classification model disclosed in an embodiment of the present invention. Figure 2 The described online incremental update method of the remote sensing image object classification model is applied to a remote sensing object positioning application system, such as a local server or cloud server for remote sensing object positioning system management, and the embodiments of the present invention do not limit this. Figure 2 As shown, the online incremental update of the multi-remote sensing image object classification model may include the following operations:

[0090] S1, obtain remote sensing image object classification model and historical data set;

[0091] It should be noted that, in this embodiment, the remote sensing image object classification model is a YOLOv5 model;

[0092] S2, using the historical data set, processing the remote sensing image object classification model to obtain a first optimization model and a training data label set;

[0093] It should be noted that the first optimization model refers to a first optimization model obtained by optimizing the remote sensing image object classification model;

[0094] S3, using the measured remote sensing image data and the training data label set, iteratively and incrementally process the first optimization model to obtain a remote sensing image object classification optimization model.

[0095] It can be seen that the online incremental update method of the remote sensing image object classification model described in the embodiment of the present invention adopts a data increment control model based on sample importance sorting for online data selection. The remote sensing image object classification model uses new data for continuous learning to acquire new knowledge, thereby improving the iterative optimization efficiency of the remote sensing image object classification model and reducing the training calculation cost.

[0096] In an optional embodiment, in the above step S2, the use of the historical data set to process the remote sensing image object classification model to obtain a first optimization model and a training data label set includes:

[0097] S21, dividing the historical data set according to a preset ratio to obtain a historical data training set and a historical data test set;

[0098] Set the number of training times to 1;

[0099] S22, using the remote sensing image object classification model, processing the historical data training set to obtain first image object classification information;

[0100] S23, using the first image object classification information and historical data training set label information, updating parameters of the remote sensing image object classification model;

[0101] S24, determining whether the training times value is greater than a preset training times threshold, and obtaining a training times determination result;

[0102] Increase the training times by 1;

[0103] When the result of the training number determination is yes, executing S25;

[0104] When the result of the training number determination is no, executing S22;

[0105] S25, using the remote sensing image object classification model, processing the historical data test set to obtain second image object classification information;

[0106] S26, comparing the second image object classification information with the historical data test set label information to obtain prediction accuracy information;

[0107] S27, determining whether the prediction accuracy information is greater than a preset accuracy threshold, and obtaining an accuracy determination result;

[0108] When the accuracy determination result is no, executing S21;

[0109] If the accuracy determination result is yes, the training process of the remote sensing image object classification model is completed to obtain a first optimization model, and S28 is executed;

[0110] S28, fusing the historical data training set label information and the historical data test set label information to obtain a training data label set.

[0111] It can be seen that the online incremental update method of the remote sensing image object classification model described in the embodiment of the present invention adopts a data increment control model based on sample importance sorting for online data selection. The remote sensing image object classification model uses new data for continuous learning to acquire new knowledge, thereby improving the iterative optimization efficiency of the remote sensing image object classification model and reducing the training calculation cost.

[0112] In an optional embodiment, in the above step S23, the use of the first image object classification information and the historical data training set label information to update the parameters of the remote sensing image object classification model includes:

[0113] Based on the parameter update model, the remote sensing image object classification model is updated with the first image object classification information and the historical data training set label information;

[0114] The parameter update model expression is:

[0115]

[0116] θ←θ+v;

[0117] In the formula, x (i) is the i-th remote sensing image in the historical data training set, y (i) is the label information of the object in the ith remote sensing image of the historical data training set, v is the update speed, θ is the parameter of the remote sensing image object classification model, η 2 is the initial parameter learning rate, α is the momentum parameter, f(·) is the model calculation function, and l() is the object classification information of the first image;

[0118] It should be noted that, in this embodiment, the initial parameter learning rate η 2 Set to 0.001.

[0119] It can be seen that the online incremental update method of the remote sensing image object classification model described in the embodiment of the present invention adopts a data increment control model based on sample importance sorting for online data selection. The remote sensing image object classification model uses new data for continuous learning to acquire new knowledge, thereby improving the iterative optimization efficiency of the remote sensing image object classification model and reducing the training calculation cost.

[0120] In an optional embodiment, in the above step S28, the fusing of the historical data training set label information and the historical data test set label information to obtain the training data label set includes:

[0121] S281, traversing the historical data training set to obtain all historical training data;

[0122] S282, parsing any of the historical training data to obtain training data type information and training data label information;

[0123] S283, traversing the historical data test set to obtain all historical test data;

[0124] S284, parsing any of the historical test data to obtain test data type information and test data label information;

[0125] S285, performing deduplication processing on the training data type information, the training data label information, the test data type information, and the test data label information to obtain a training data label set;

[0126] It should be noted that the deduplication process means that only one of multiple identical data types and data labels is retained.

[0127] It can be seen that the online incremental update method of the remote sensing image object classification model described in the embodiment of the present invention adopts a data increment control model based on sample importance sorting for online data selection. The remote sensing image object classification model uses new data for iterative update processing to obtain a remote sensing image object classification optimization model, which improves the iterative optimization efficiency of the remote sensing image object classification model and reduces the training calculation cost.

[0128] In an optional embodiment, in the above step S3, the use of the measured remote sensing image data and the training data label set to iteratively increment the first optimization model to obtain the remote sensing image object classification optimization model includes:

[0129] S31, set the number of iterations, and set the number of incremental processing times to 1;

[0130] S32, obtaining a measured remote sensing image dataset;

[0131] S33, using the first optimization model, processing the measured remote sensing image data set to obtain a first classification positioning data set and a second classification positioning data set;

[0132] It should be noted that the first classification positioning data set includes a plurality of first classification positioning data; the first classification positioning data represents the measured remote sensing image data in which positioning classification errors, missed detections and false alarms occur;

[0133] It should be noted that the second classification positioning data set includes a plurality of second classification positioning data; the second classification positioning data represents the measured remote sensing image data with accurate positioning classification;

[0134] It should be noted that the measured remote sensing image dataset is a collection of the first classification positioning dataset and the second classification positioning dataset;

[0135] S34, processing the historical data set, the first classification and positioning data set, and the second classification and positioning data set to obtain an incremental training data set;

[0136] S35, using the incremental training data set and the training data label set, processing the first optimization model to obtain a second optimization model and an incremental training label set;

[0137] It should be noted that the second optimization model refers to a second remote sensing image object classification optimization model obtained by iteratively processing the first optimization model;

[0138] S36, fusing the incremental training label set with the training data label set to obtain a historical label set;

[0139] Increase the number of incremental processing times by 1;

[0140] S37, determining whether the number of incremental processing times is greater than the number of iterations, and obtaining an iteration determination result;

[0141] When the iterative judgment result is no, the incremental training data set is updated to the historical data set, the historical label set is updated to the training data label set, and S32 is executed;

[0142] When the iterative judgment result is yes, the second optimization model is updated to a remote sensing image object classification optimization model.

[0143] It can be seen that the online incremental update method of the remote sensing image object classification model described in the embodiment of the present invention adopts a data increment control model based on sample importance sorting for online data selection. The remote sensing image object classification model uses new data for iterative update processing to obtain a remote sensing image object classification optimization model, which improves the iterative optimization efficiency of the remote sensing image object classification model and reduces the training calculation cost.

[0144] In an optional embodiment, in the above step S34, the processing of the historical data set, the first classification and positioning data set, and the second classification and positioning data set to obtain an incremental training data set includes:

[0145] S341, fusing the historical data set and the third classification positioning data set to obtain a fourth classification positioning data set;

[0146] It should be noted that the fusion processing means combining the historical data set and the third classification positioning data set in a sequential order;

[0147] S342, processing the fourth classification positioning data set based on the first data increment control model to obtain a fifth classification positioning data set;

[0148] S343, based on the second data increment control model, processing the first category positioning data set to obtain a sixth category positioning data set;

[0149] S344, fusing the fifth classification positioning data set and the sixth classification positioning data set to obtain an incremental training data set;

[0150] It should be noted that the fusion processing means combining the fifth classification positioning data set and the sixth classification positioning data set in a sequential order.

[0151] It can be seen that the online incremental update method of the remote sensing image object classification model described in the embodiment of the present invention adopts a data increment control model based on sample importance sorting for online data selection, which can perform online iterative update processing on the remote sensing image object classification model, thereby improving the iterative optimization efficiency of the remote sensing image object classification model and reducing the training calculation cost.

[0152] In another optional embodiment, in the above step S342, the first data increment control model expression is:

[0153] Rs(i)=UC(i)+IA(i)

[0154]

[0155] IA{imgi}=1-AP50(TL(img i ), PR(img i ))

[0156] Where Rs(i) represents the ranking score of the i-th remote sensing image; UC(i) represents the uncertainty score of the i-th remote sensing image; IA(i) represents the inaccuracy score of the i-th remote sensing image; i represents the index of the remote sensing image; uc{img i} represents the uncertainty of the i-th remote sensing image; R(img i ) represents the transformed image after the i-th remote sensing image is rotated 180°; uc{R(img i ) represents the uncertainty of the transformed image; img i represents the i-th remote sensing image; is the confidence of the jth predicted object frame in the i-th remote sensing image; N represents the total number of predicted object frames; n represents the index of the predicted object frame; IA{img i} represents the difference between the predicted box and the real box of the i-th remote sensing image; TL(img i ) is the true label in the i-th remote sensing image, PR(img i ) is the predicted value in the i-th remote sensing image; AP50() indicates the average precision calculation;

[0157] It should be noted that the first data increment control model is used to obtain the ranking score of the remote sensing image. This embodiment measures the importance of the instance data based on the uncertainty and inaccuracy ranking scores, and samples the remote sensing measured data at uniform intervals based on the importance ranking, which can reflect the samples of the existing learning experience of the remote sensing image object classification model, and the stability of the model can be maintained during re-learning. The prediction results of the remote sensing image object classification model for the "old" knowledge instance data are guided by the existing learning experience. Therefore, in the incremental training data set, the more diverse the prediction results, and the "old" knowledge instances with greater differences in prediction results, the more they can reflect the existing learning experience of the remote sensing image object classification model, which is conducive to the retention of memory. In this embodiment, the uncertainty score and inaccuracy score ranking are used to measure the importance of data. The more diverse the ranking score values, the higher the importance of maintaining model stability in incremental training.

[0158] It can be seen that the online incremental update method of the remote sensing image object classification model described in the embodiment of the present invention is implemented, the uncertainty score and the inaccuracy score ranking are used to measure the importance of the instance, and the first data incremental control model is used to select data online. The remote sensing image object classification model is iteratively updated using new data to obtain the remote sensing image object classification optimization model, which improves the iterative optimization efficiency of the remote sensing image object classification model and reduces the training calculation cost.

[0159] In another optional embodiment, in the above step S342, the second data increment control model expression is:

[0160]

[0161] Among them, A i represents the uncertainty of the i-th remote sensing image; A 2i A represents the uncertainty of the transformed image after the i-th remote sensing image is rotated 180°; 1i represents the uncertainty of the transformed image; i represents the processed remote sensing image index; B j is the confidence of the j-th predicted object frame in the i-th remote sensing image; N represents the total number of predicted object frames; n represents the index of the predicted object frame.

[0162] It should be noted that the second data increment control model is used to obtain the uncertainty ranking score of the remote sensing image, and the uncertainty ranking score is used to measure the importance of remote sensing image data and select remote sensing image data with high importance to enter the incremental training sample set.

[0163] Exemplarily, the second data increment is measured remote sensing image data with positioning classification errors, missed detections and false alarms. After the remote sensing image object classification model predicts a piece of remote sensing image data, it outputs the prediction results at the object frame level (category confidence scores and position coordinates of all object frames). For a predicted object frame, the highest of all category confidence scores is the remote sensing image object confidence. The lower the value, the more uncertain the prediction result of the remote sensing image object classification model for the object frame.

[0164] It can be seen that the online incremental update method of the remote sensing image object classification model described in the embodiment of the present invention adopts the first data incremental control model to select data online, and the remote sensing image object classification model uses new data to perform iterative update processing to obtain the remote sensing image object classification optimization model, thereby improving the iterative optimization efficiency of the remote sensing image object classification model and reducing the training calculation cost.

[0165] In an optional embodiment, in the above step S35, the use of the incremental training data set and the training data label set to process the first optimization model to obtain the second optimization model includes:

[0166] S351, obtaining the incremental training data set;

[0167] The incremental training data set includes a plurality of incremental training data; the incremental training data is historical data or first classification positioning data or second classification positioning data;

[0168] S352, based on the training data label set, processing the incremental training data set to obtain an incremental training label set;

[0169] S353: Based on the incremental training label set, the first optimization model is processed using the incremental training data set to obtain a second optimization model.

[0170] It can be seen that the online incremental update method of the remote sensing image object classification model described in the embodiment of the present invention adopts a data increment control model based on sample importance sorting for online data selection. The remote sensing image object classification model uses new data for continuous learning to acquire new knowledge, thereby improving the iterative optimization efficiency of the remote sensing image object classification model and reducing the training calculation cost.

[0171] In an optional embodiment, in the above step S352, the processing of the incremental training data set based on the training data label set to obtain the incremental training label set includes:

[0172] S3521, traverse the incremental training data set to obtain all incremental training data;

[0173] S3522, parsing and processing any of the incremental training data to obtain data type information;

[0174] S3523, performing matching processing on the training data label set based on the data type information to obtain incremental training label information;

[0175] S3524: All the incremental training label information is combined in order to obtain an incremental training label set.

[0176] It can be seen that the online incremental update method of the remote sensing image object classification model described in the embodiment of the present invention adopts a data increment control model based on sample importance sorting for online data selection. The remote sensing image object classification model uses new data for continuous learning to acquire new knowledge, thereby improving the iterative optimization efficiency of the remote sensing image object classification model and reducing the training calculation cost.

[0177] In an optional embodiment, in the above step S353, the first optimization model is processed based on the incremental training label set using the incremental training data set to obtain the second optimization model, including:

[0178] S3531, traverse the incremental training data set to obtain all incremental training data;

[0179] S3532, inputting any of the incremental training data into the first optimization model to obtain third image object classification information;

[0180] S3533, comparing the third image object classification information and the incremental training label set to obtain classification accuracy information;

[0181] S3534, determining whether the classification accuracy information is greater than a preset accuracy threshold, and obtaining a classification accuracy determination result;

[0182] When the classification accuracy determination result is no, executing S3532;

[0183] If the classification accuracy judgment result is yes, a second optimization model is obtained.

[0184] It can be seen that the online incremental update method of the remote sensing image object classification model described in the embodiment of the present invention adopts a data increment control model based on sample importance sorting for online data selection. The remote sensing image object classification model uses new data for continuous learning to acquire new knowledge, thereby improving the iterative optimization efficiency of the remote sensing image object classification model and reducing the training calculation cost.

[0185] Embodiment 2

[0186] See also Figure 3 , Figure 3 : is a schematic diagram of the structure of an online incremental update device for a remote sensing image object classification model disclosed in an embodiment of the present invention. Figure 3 The described device can be applied to a remote sensing object positioning system, such as a local server or a cloud server for remote sensing object positioning, and the embodiments of the present invention are not limited thereto. Figure 3 As shown, the device may include:

[0187] An acquisition module 101 is used to acquire a remote sensing image object classification model and a historical data set;

[0188] A first processing module 102 is used to process the remote sensing image object classification model using the historical data set to obtain a first optimization model;

[0189] The second processing module 103 is used to perform iterative incremental processing on the first optimization model using the measured remote sensing image data to obtain a remote sensing image object classification optimization model.

[0190] It can be seen that the implementation of the multi-satellite collaborative three-dimensional observation method described in the embodiment of the present invention overcomes the limitations and errors of single satellite observation, visualizes and intuitively makes difficult-to-understand spatial information, and improves the efficiency of data analysis and processing.

[0191] Embodiment 3

[0192] See also Figure 4 , Figure 4 : is a structural diagram of another online incremental updating device for remote sensing image object classification model disclosed in an embodiment of the present invention. Figure 4 The described device can be applied to a satellite observation system, such as a local server or a cloud server for satellite observation, and the embodiments of the present invention are not limited thereto. Figure 4 As shown, the device may include:

[0193] A memory 201 storing executable program codes;

[0194] a processor 202 coupled to the memory;

[0195] The processor 202 calls the executable program code stored in the memory 201 to execute the steps in the online incremental update method of the remote sensing image object classification model described in the first embodiment.

[0196] Embodiment 4

[0197] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the online incremental update method of a remote sensing image object classification model described in the first embodiment.

[0198] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0199] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, a magnetic disk storage, a magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0200] Finally, it should be noted that the online incremental update method and device for the remote sensing image object classification model disclosed in the embodiment of the present invention only discloses the preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for online incremental updating of a remote sensing image object classification model, characterized in that: The method comprises: S1, obtain remote sensing image object classification model and historical data set; S2, using the historical data set, processing the remote sensing image object classification model to obtain a first optimization model and a training data label set; S3, using the measured remote sensing image data and the training data label set, iteratively and incrementally process the first optimization model to obtain a remote sensing image object classification optimization model.

2. The online incremental update method of the remote sensing image object classification model according to claim 1 is characterized in that: The method of using the historical data set to process the remote sensing image object classification model to obtain a first optimization model and a training data label set includes: S21, dividing the historical data set according to a preset ratio to obtain a historical data training set and a historical data test set; Set the number of training times to 1; S22, using the remote sensing image object classification model, processing the historical data training set to obtain first image object classification information; S23, using the first image object classification information and historical data training set label information, updating parameters of the remote sensing image object classification model; S24, determining whether the training times value is greater than a preset training times threshold, and obtaining a training times determination result; Increase the training times by 1; When the result of the training number determination is yes, executing S25; When the result of the training number determination is no, executing S22; S25, using the remote sensing image object classification model, processing the historical data test set to obtain second image object classification information; S26, comparing the second image object classification information with the historical data test set label information to obtain prediction accuracy information; S27, determining whether the prediction accuracy information is greater than a preset accuracy threshold, and obtaining an accuracy determination result; When the accuracy determination result is no, executing S21; If the accuracy determination result is yes, a first optimization model is obtained and S28 is executed; S28, fusing the historical data training set label information and the historical data test set label information to obtain a training data label set.

3. The online incremental update method of the remote sensing image object classification model according to claim 1 is characterized in that: The method of using the measured remote sensing image data and the training data label set to iteratively increment the first optimization model to obtain a remote sensing image object classification optimization model includes: S31, setting the number of iterations, setting the number of iterative processing times to 1; S32, obtaining a measured remote sensing image dataset; S33, using the first optimization model, processing the measured remote sensing image data set to obtain a first classification positioning data set and a second classification positioning data set; S34, processing the historical data set, the first classification and positioning data set, and the second classification and positioning data set to obtain an incremental training data set; S35, using the incremental training data set and the training data label set, processing the first optimization model to obtain a second optimization model and an incremental training label set; S36, fusing the incremental training label set with the training data label set to obtain a historical label set; Increase the number of iterations by 1; S37, judging whether the number of iterative processing times is greater than the number of iterations, and obtaining an iterative judgment result; When the iterative judgment result is no, the historical label set is updated to the training data label set, and S32 is executed; When the iterative judgment result is yes, the second optimization model is updated to a remote sensing image object classification optimization model.

4. The online incremental update method of the remote sensing image object classification model according to claim 3 is characterized in that: The processing of the historical data set, the first classification and positioning data set, and the second classification and positioning data set to obtain an incremental training data set includes: S341, fusing the historical data set and the third classification positioning data set to obtain a fourth classification positioning data set; S342, processing the fourth classification positioning data set based on the first data increment control model to obtain a fifth classification positioning data set; S343, processing the first category positioning data set based on the second data increment control model to obtain a sixth category positioning data set; S344, fusing the fifth classification positioning data set and the sixth classification positioning data set to obtain an incremental training data set.

5. The online incremental update method of the remote sensing image object classification model according to claim 4 is characterized in that: The first data increment control model expression is: Rs(i)=UC(i)+IA(i) IA{img i }=1-AP50(TL(img i ),PR(img i )) Where Rs(i) represents the ranking score of the i-th remote sensing image; UC(i) represents the uncertainty score of the i-th remote sensing image; IA(i) represents the inaccuracy score of the i-th remote sensing image; i represents the index of the remote sensing image; uc{img i } represents the uncertainty of the i-th remote sensing image; R(img i ) represents the transformed image after the i-th remote sensing image is rotated 180°; uc{R(img i ) represents the uncertainty of the transformed image; img i represents the i-th remote sensing image; is the confidence of the jth predicted object frame in the i-th remote sensing image; N represents the total number of predicted object frames; n represents the index of the predicted object frame; IA{img i } represents the difference between the predicted box and the real box of the i-th remote sensing image; TL(img i ) is the true label in the i-th remote sensing image, PR(img i ) is the predicted value in the i-th remote sensing image; AP50() indicates the average precision calculation.

6. The online incremental update method of the remote sensing image object classification model according to claim 3 is characterized in that: The step of processing the first optimization model by using the incremental training data set and the training data label set to obtain a second optimization model and an incremental training label set includes: S351, obtaining the incremental training data set; The incremental training data set includes a plurality of incremental training data; the incremental training data is historical data or first classification positioning data or second classification positioning data; S352, based on the training data label set, processing the incremental training data set to obtain an incremental training label set; S353: Based on the incremental training label set, the first optimization model is processed using the incremental training data set to obtain a second optimization model.

7. The online incremental update method of the remote sensing image object classification model according to claim 6 is characterized in that: The step of processing the incremental training data set based on the training data label set to obtain an incremental training label set includes: S3521, traverse the incremental training data set to obtain all incremental training data; S3522, parsing and processing any of the incremental training data to obtain data type information; S3523, performing matching processing on the training data label set based on the data type information to obtain incremental training label information; S3524: All the incremental training label information is combined in order to obtain an incremental training label set.

8. An online incremental update device for a remote sensing image object classification model, characterized in that: The device comprises: An acquisition module is used to obtain remote sensing image object classification models and historical data sets; A first processing module, used to process the remote sensing image object classification model using the historical data set to obtain a first optimization model; The second processing module is used to use the measured remote sensing image data to perform iterative incremental processing on the first optimization model to obtain a remote sensing image object classification optimization model.

9. An online incremental update device for remote sensing image object classification model, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the online incremental update method of the remote sensing image object classification model as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which, when called, are used to execute the online incremental update method for the remote sensing image object classification model as described in any one of claims 1-7.

Images