Adaptive optimization method and system for onboard remote sensing image processing based on resource constraints

Abstract

The present invention relates to the field of adaptive image processing technology, and in particular to an adaptive optimization method for on-board remote sensing image processing based on resource constraints, which aims to improve on-board processing efficiency. The method of the present invention comprises: obtaining task information containing target product information uploaded by a ground system; determining a first processing flow according to the task information; the first processing flow comprises a plurality of steps for producing the target product, each step corresponds to an algorithm subcategory, and each algorithm subcategory has an optional identifier or a mandatory identifier; specifying a corresponding algorithm module for each algorithm subcategory in the first processing flow according to an algorithm configuration table to obtain a second processing flow; optimizing the second processing flow according to the optional identifier and the mandatory identifier, so that the estimated values ​​of each constraint item of the optimized processing flow are all less than or equal to the corresponding upper limit value; and executing the optimized processing flow to generate the target product. The method of the present invention not only improves on-board processing efficiency, but also takes into account the accuracy of information extraction.

Description

Technical Field

[0001] The present invention relates to the technical field of adaptive image processing, and in particular to a method and system for adaptive optimization of on-board remote sensing image processing based on resource constraints. Background Art

[0002] In recent years, remote sensing imaging satellites have seen increasingly higher resolutions and shorter revisit cycles, playing an increasingly important role in areas such as land management, urban management, ecology, and emergency disaster relief. The current conventional remote sensing imaging satellite application model involves acquiring image data in orbit, transmitting it downlink to the ground via satellite-to-ground links, and then processing it to produce various data processing and information extraction products, which are ultimately applied in various specialized fields. However, with the rapid growth of onboard data, the conflict between the massive amount of onboard data and the satellite-to-ground data transmission links has become increasingly prominent. Furthermore, traditional satellite application processes are complex and cumbersome, making them difficult to meet the application requirements of time-sensitive tasks such as fire monitoring and target search.

[0003] On-orbit processing of satellite remote sensing images is an effective way to solve the above problems and has developed rapidly in recent years. However, the computing power and memory size of the on-board hardware cannot fully meet the needs of complex processing algorithms, resulting in a high probability of execution failure, which seriously affects the efficiency of on-board processing. Summary of the Invention

[0004] In order to solve the above problems in the prior art, the present invention proposes an adaptive optimization method and system for on-board remote sensing image processing based on resource constraints, which improves on-board processing efficiency.

[0005] In a first aspect of the present invention, a method for adaptive optimization of on-board remote sensing image processing based on resource constraints is proposed. The method is applicable to an on-board intelligent processing system, and the method comprises:

[0006] Obtaining task information uploaded by the ground system, wherein the task information includes: information of target products to be produced;

[0007] Determine a first processing flow according to the task information; the first processing flow includes multiple steps for producing the target product, each step corresponds to an algorithm subcategory, and each algorithm subcategory is set with an optional flag or a mandatory flag;

[0008] According to the algorithm configuration table, specify a corresponding algorithm module for each algorithm subcategory in the first processing flow, thereby obtaining a second processing flow;

[0009] Optimizing the second processing flow according to the optional flag and the mandatory flag so that the estimated value of each constraint item of the optimized processing flow is less than or equal to the corresponding upper limit value;

[0010] The optimized processing flow is executed to generate the target product.

[0011] Preferably, the step of generating the algorithm configuration table includes:

[0012] Deploy a simulation environment in the ground system that is identical to the onboard software and hardware;

[0013] producing the target product in the simulation environment using a set of images of standard sizes;

[0014] Obtain configuration item information of each algorithm module in the production process and store it in the algorithm configuration table;

[0015] in,

[0016] The configuration item information includes: the name of each algorithm module, the version number, the algorithm subcategory to which it belongs, the preferred evaluation index, one or more replacement modules, and the test value of each constraint item of each algorithm module in the simulation environment.

[0017] Preferably, the step of “optimizing the second processing flow according to the optional flag and the mandatory flag so that the estimated value of each constraint item of the optimized processing flow is less than or equal to the corresponding upper limit value” includes:

[0018] Calculating estimated values ​​of each constraint item of the second processing flow according to the algorithm configuration table;

[0019] Calculating first ratios of the estimated value of each constraint item to the corresponding upper limit value respectively;

[0020] If the first ratio of any constraint item in the second processing flow is greater than 1, the algorithm module in the second processing flow is deleted and / or replaced according to the optional flag or the mandatory flag.

[0021] Preferably, the step of "calculating estimated values ​​of each constraint item of the second processing flow according to the algorithm configuration table" includes:

[0022] Obtaining the test value of each constraint item of each algorithm module in the second processing flow from the algorithm configuration table;

[0023] According to the standard size and the size of the image to be processed, the data volume coefficient is calculated according to the following formula:

[0024]

[0025] Wherein, W and H are the width and height of the image to be processed respectively; W std and H std are the width and height of the standard size image respectively;

[0026] Based on the data volume coefficient and the test value of each constraint item of each algorithm module, the correction value of each constraint item of the algorithm module is calculated according to the following formula:

[0027] c′ i (m)=s*c i (m)

[0028] Wherein, m and i are the serial numbers of the algorithm modules and the serial numbers of the constraints in the second processing flow respectively; m = 1, 2, ..., M; i = 1, 2, ..., N; M and N are the total number of algorithm modules and the total number of constraints in the second processing flow respectively; c′ i (m) and c i (m) are respectively the correction value and the test value of the i-th constraint item of the m-th algorithm module;

[0029] The estimated value of each cumulative constraint item of the second processing flow is calculated according to the following formula:

[0030]

[0031] Among them, c′ j (m) is the correction value of the j-th constraint item of the m-th algorithm module, and the j-th constraint item is the cumulative constraint item; is the estimated value of the j-th constraint item of the second processing flow;

[0032] The estimated value of each extremum constraint item of the second processing flow is calculated according to the following formula:

[0033]

[0034] Among them, c′ k (m) is the correction value of the k-th constraint item of the m-th algorithm module, and the k-th constraint item is the extreme value constraint item; is the estimated value of the kth constraint item of the second processing flow.

[0035] Preferably, the step of “if the first ratio of any constraint item of the second processing flow is greater than 1, deleting and / or replacing the algorithm module in the second processing flow according to the optional flag or the mandatory flag” includes:

[0036] Determining, according to the optional identifier and the mandatory identifier, an optional algorithm module and a mandatory algorithm module in the second processing flow;

[0037] If the first ratio of a constraint item of the second processing flow is greater than 1, and the constraint item is the cumulative constraint item, selectively deleting the optional algorithm module and / or selectively replacing the required algorithm module;

[0038] If the ratio of a constraint item in the second processing flow is greater than 1, and the constraint item is the extreme constraint item, then calculate the second ratio of the correction value of the constraint item to the upper limit value for each algorithm module and each replacement module, delete all the optional algorithm modules whose second ratio is greater than 1, and replace all the required algorithm modules whose second ratio is greater than 1.

[0039] Preferably, the step of “if the first ratio of a constraint item of the second processing flow is greater than 1, and the constraint item is the cumulative constraint item, selectively deleting the optional algorithm module and / or selectively replacing the required algorithm module” includes:

[0040] If the first ratio of a constraint item in the second processing flow is greater than 1, and the constraint item is the cumulative constraint item, then the excess value of the constraint item is calculated:

[0041]

[0042] Wherein, j is the sequence number of the constraint item; and are the excess value, the estimated value and the upper limit value of the constraint item respectively;

[0043] Arranging the correction values ​​of the constraint items of all the optional algorithm modules in descending order to obtain a first sequence;

[0044] Comparing the correction values ​​in the first sequence with the excess value in order from the front to the back, and determining whether there is a correction value in the first sequence that is greater than or equal to the excess value;

[0045] If so, the optional algorithm module corresponding to the last correction value in the first sequence that is greater than or equal to the excess value is deleted from the second processing flow; otherwise, the correction values ​​in the first sequence are accumulated from the beginning to the end, and the optional algorithm module corresponding to each accumulated correction value is deleted from the second processing flow until the accumulated sum is greater than or equal to the excess value;

[0046] If the optional algorithm module does not exist in the second processing flow, or the first ratio is still greater than 1 after all the optional algorithm modules are deleted from the second processing flow, the required algorithm module in the second processing flow is selectively replaced and the first ratio is recalculated until the replacement is stopped when the first ratio is less than 1.

[0047] Preferably, the step of “selectively replacing the required algorithm modules in the second processing flow and recalculating the first ratio until the first ratio is less than 1 and stopping the replacement” includes:

[0048] Arrange the replacement modules corresponding to all the required algorithm modules in descending order according to the preferred evaluation index to obtain a second sequence;

[0049] Determining whether there is a replacement module in the second sequence that meets the first replacement condition;

[0050] If so, the first replacement module in the second sequence that satisfies the first replacement condition is used to replace the corresponding mandatory algorithm module in the second processing flow; otherwise, the replacement module that satisfies the second replacement condition is found from the front to the back in the second sequence, and the corresponding mandatory algorithm module in the second processing flow is replaced with the found replacement module, and the first ratio is recalculated and replaced with the replacement module that satisfies the second replacement condition until the first ratio is less than or equal to 1.

[0051] in,

[0052] Each of the mandatory algorithm modules corresponds to one or more replacement modules;

[0053] The first replacement condition is: the correction value of the replacement module is smaller than the correction value of the corresponding mandatory algorithm module, and the absolute value of the difference between the correction value of the replacement module and the correction value of the mandatory algorithm module is greater than the excess value;

[0054] The second replacement condition is: the correction value of the replacement module is smaller than the correction value of the corresponding mandatory algorithm module.

[0055] Preferably, the step of “optimizing the second processing flow according to the optional flag and the mandatory flag so that the estimated value of each constraint item of the optimized processing flow is less than or equal to the corresponding upper limit value” further includes:

[0056] Determine whether the estimated values ​​of each constraint item of the current second processing flow are all less than the corresponding upper limit value multiplied by the preset first percentage; if so,

[0057] Obtaining a preset process corresponding to the product subcategory to which the target product belongs, and specifying a corresponding algorithm module for each algorithm subcategory in the preset process, thereby obtaining a third processing process;

[0058] Constructing a first set using the optional algorithm module in the third processing flow, constructing a second set using the optional algorithm module in the current second processing flow, and calculating the difference between the first set and the second set;

[0059] sorting the optional algorithm modules in the difference set according to the running order of each module in the third processing flow to obtain a third sequence;

[0060] respectively calculating the correction value of each constraint item of each optional algorithm module in the third sequence;

[0061] In the third sequence, one optional algorithm module is selected each time from the front to the back. If adding the selected optional algorithm module to the current second processing flow causes the new estimated value of each constraint item in the second processing flow to be less than the corresponding upper limit value multiplied by a preset second percentage, then the selected optional algorithm module is added to the current second processing flow;

[0062] The steps of selecting the optional algorithm module and adding the optional algorithm module according to the new estimated value are repeated until each optional algorithm module in the third sequence is traversed.

[0063] Preferably, the target product information includes: the name and / or ID of the target product;

[0064] If the task information further includes an initial processing flow, the step of "determining a first processing flow according to the task information" includes: using the initial processing flow as the first processing flow;

[0065] Otherwise, the step of “determining a first processing flow according to the task information” includes: selecting a corresponding preset flow as the first processing flow according to the product subcategory to which the target product belongs;

[0066] The remote sensing image products in the onboard intelligent processing system are divided into product categories according to the degree of processing, and each product category is further divided into product subcategories according to the processing purpose, technical characteristics and application requirements;

[0067] The algorithms in the onboard intelligent processing system are divided into algorithm categories according to their position and function in the processing flow, and each algorithm category is subdivided into algorithm subcategories according to functions and application requirements;

[0068] The constraints include: memory consumption and production time;

[0069] The mission information also includes: the total operating time of the on-board intelligent processing system;

[0070] Before “optimizing the second processing flow according to the optional identifier and the mandatory identifier”, the method further includes:

[0071] Using the current free memory size of the onboard intelligent processing system as the upper limit of the memory consumption;

[0072] The upper limit value of the production time is obtained by subtracting the time that the on-board intelligent processing system has experienced since startup from the total operation time.

[0073] In a second aspect of the present invention, an adaptive optimization system for on-board remote sensing image processing based on resource constraints is proposed, which is a computer program that executes the method described above.

[0074] The present invention has the following beneficial effects:

[0075] The present invention pre-sets optional and mandatory flags in the processing flow. The onboard intelligent processing system then optimizes the process flow based on these optional and mandatory flags, ensuring that the estimated values ​​of each constraint item in the optimized process flow are less than or equal to the corresponding upper limit. Finally, the optimized process flow is used to generate the target product. By automatically adapting to onboard resource constraints, the present invention reduces the probability of process execution failure, thereby improving onboard processing efficiency. It also maximizes the use of existing onboard resources while ensuring accurate information extraction.

[0076] A simulation environment identical to the onboard hardware and software was pre-deployed in the ground system, using a standard-sized image set for production. This allowed the test values ​​of each constraint corresponding to each algorithm module, as well as the optimal evaluation metrics for each replacement module, to be stored in the algorithm configuration table. Onboard, the test values ​​of each constraint in the algorithm configuration table were converted to corrected values ​​based on the size of the image to be processed. This was then used to calculate estimated values ​​for each constraint in the entire second processing flow, providing an accurate basis for subsequent process optimization.

[0077] During optimization, if the estimated value of any constraint item in the second processing flow is found to be greater than the corresponding upper limit, the optional algorithm modules in the second processing flow are gradually deleted based on whether they are optional or required, and the required algorithm modules are replaced if necessary. This optimization method gradually adapts to the onboard resource constraints, ensuring that the final processing flow can be successfully executed on the satellite without eliminating too many modules at once.

[0078] If it is estimated that there will be relatively abundant resources when the optimized processing flow is running on the satellite, some optional algorithm modules can be appropriately added to fully utilize the on-board resources and improve the quality of the target product as much as possible.

[0079] In the present invention, the upper limit value of each constraint item is obtained in real time by the on-board intelligent processing system, rather than the upper limit value being directly given by the ground system. The upper limit value obtained by this method is more accurate, further ensuring that the optimized process can be smoothly executed on the satellite without being cut too much.

[0080] In summary, the adaptive image processing method of the present invention can not only greatly reduce the probability of execution failure, but also make full use of on-board resources, thereby improving on-board processing efficiency, while taking into account the accuracy of information extraction to ensure the quality of the target product. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 1 is a schematic diagram of the main steps of Example 1 of the adaptive optimization method for on-board remote sensing image processing based on resource constraints in the present invention;

[0082] Figure 2 This is a schematic diagram of the main steps of Example 2 of the adaptive optimization method for on-board remote sensing image processing based on resource constraints in the present invention. DETAILED DESCRIPTION

[0083] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0084] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0085] It should be noted that, in the description of the present invention, the terms "first" and "second" are merely for the convenience of description, and do not indicate or imply the relative importance of the devices, elements or parameters, and therefore should not be understood as limiting the present invention. In addition, the term "and / or" in the present invention is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this document, unless otherwise specified, generally indicates that the associated objects are in an "or" relationship.

[0086] Remote sensing imagery products within the onboard intelligent processing system are divided into two product categories based on the degree of processing: data products and information products. Each product category can be further divided into several subcategories based on processing objectives, technical characteristics, and application requirements. For example, data products can be subdivided into radiometric correction products, geometric correction products, slicing products, band computation products, and cloud-based judgment products. Information products can be further subdivided into products for aircraft, ship, and oil tank inspections, semantic segmentation of buildings and roads, and building change monitoring products.

[0087] In the onboard intelligent processing system, algorithm categories are divided according to the position and role of each algorithm in the processing flow. Each algorithm category can be further divided into several algorithm subcategories according to function and application requirements. The algorithm categories in the embodiment of the present invention include: preprocessing algorithms, data product production algorithms, and information extraction algorithms. Among them, preprocessing algorithms include subcategories such as radial distortion processing, band registration, intra-slice color grading, inter-slice color grading, panchromatic MTF (Modulation Transfer Function) processing, multispectral MTF processing, and RPC (Rational Polynomial Camera) correction; data product production algorithms include subcategories such as radiation correction, geometric correction, slicing processing, band calculation, cloud judgment, reflectivity update, and auxiliary data monitoring; information extraction algorithms include: pre-processing algorithm subcategories such as image fusion and morphological processing; subcategories such as vehicle, ship, aircraft, oil tank, stadium, pond detection, semantic segmentation and change detection of buildings and roads; and post-processing subcategories such as pattern morphology processing and vector boundary processing.

[0088] To extract information on board a satellite, it is actually necessary to migrate the entire pre-processing, data processing, information extraction and other processes of the original ground processing system to the onboard environment, which also has the characteristics of a long processing flow and many algorithm links. Therefore, in order to reduce the demand of onboard algorithms for the computing power and memory of the hardware platform, it is also necessary to start from the overall processing flow and conduct comprehensive processing on the many algorithm modules involved. Taking into account the real-time changes in computing power and memory resources on board, the processing flow must be able to obtain the above resource status and adjust accordingly when optimizing the process in order to achieve the best results. Therefore, studying the technology of dynamically optimizing the overall processing flow based on the real-time computing power and memory status on board has important practical significance and application value in terms of maximizing processing efficiency and ensuring stability while fully utilizing the real-time resources on board.

[0089] Figure 1 This is a schematic diagram of the main steps of the first embodiment of the adaptive optimization method for on-board remote sensing image processing based on resource constraints in the present invention. The method of this embodiment is applicable to on-board intelligent processing systems, such as Figure 1 As shown, the processing method of this embodiment includes steps A10-A40:

[0090] Step A10: Obtain the mission information uploaded by the ground system.

[0091] The task information includes: information about the target product to be produced, such as the name and / or ID of the target product.

[0092] Step A20: Determine the first processing flow according to the task information.

[0093] Among them, the first processing flow includes multiple steps required to produce the target product, each step corresponds to an algorithm subcategory, and each algorithm subcategory is set with an optional flag or a mandatory flag, so that in the subsequent optimization process, this flag can be used to determine whether the algorithm subcategory can be deleted.

[0094] Step A20 may specifically include steps A21-A22:

[0095] Step A21: If the task information also includes an initial processing flow, the initial processing flow is used as the first processing flow.

[0096] The initial processing flow here can be an ideal process pre-set by R&D personnel based on the specific characteristics and usage requirements of the target product, and it also identifies which algorithm subcategories are optional and which are required. However, since the specific algorithm modules to be used for implementation are not specified, the initial processing flow cannot be directly executed.

[0097] Step A21: If the task information does not include an initial processing flow, a corresponding preset flow is selected as the first processing flow according to the product subcategory to which the target product belongs.

[0098] Based on the characteristics of each product subcategory, R&D personnel have pre-set a preset process for each product subcategory. This preset process covers as many algorithm subcategories as possible that can be used in production, and clearly identifies which algorithm subcategories are optional and which are required. This preset process can be uploaded before the method of this embodiment is run, or it can be uploaded together with the processing software that executes the method of this embodiment.

[0099] Step A30: According to the algorithm configuration table, a corresponding algorithm module is assigned to each algorithm subcategory in the first processing flow, thereby obtaining a second processing flow.

[0100] The algorithm configuration table can be generated according to the following method: (1) deploying a simulation environment in the ground system that is exactly the same as the onboard hardware and software; (2) using a standard-sized image set to produce the target product in the simulation environment; (3) obtaining the configuration item information of each algorithm module in the production process and storing it in the algorithm configuration table.

[0101] The configuration item information includes: the name and version number of each algorithm module, the algorithm subcategory to which it belongs, the preferred evaluation metric (the evaluation metric can be determined based on the algorithm module's purpose; for example, for an information extraction algorithm module, the metric can be accuracy, recall, etc.), one or more replacement modules (which can functionally replace the original module), and the test values ​​of each constraint item for each algorithm module in a simulated environment (i.e., the values ​​obtained when testing production using a standard-sized image set). The configuration table can also include the CPU / GPU frequency. However, because the CPU / GPU frequency, once set, cannot be changed during the optimization process, it is not discussed as a constraint item in this embodiment.

[0102] The algorithm module and the algorithm configuration table may be uploaded together with the processing software for executing the method of this embodiment, or may be uploaded at an earlier time.

[0103] Since a specific algorithm module is specified, the second processing flow is already an executable flow. However, due to the limitation of hardware resources on the satellite, the flow may not be executed smoothly, so it needs to be optimized in the following steps.

[0104] Step A40: Optimize the second processing flow according to the optional flags and the mandatory flags, so that the estimated value of each constraint item of the optimized processing flow is less than or equal to the corresponding upper limit value.

[0105] Among them, constraints include: memory consumption and production time, etc.

[0106] Step A40 may specifically include steps A41-A43:

[0107] Step A41: Calculate estimated values ​​of each constraint item of the second processing flow according to the algorithm configuration table.

[0108] This step may specifically include steps A411-A415:

[0109] Step A411: Obtain the test value of each constraint item of each algorithm module in the second processing flow from the algorithm configuration table.

[0110] Step A412: Calculate the data volume coefficient s according to the standard size and the size of the image to be processed using the following formula (1):

[0111]

[0112] Where W and H are the width and height of the image to be processed respectively; W std and H std are the width and height of a standard size image, respectively.

[0113] Step A413: Based on the data volume coefficient and the test values ​​of the constraints of each algorithm module, the correction values ​​of the constraints of the algorithm module are calculated according to the method shown in the following formula (2):

[0114] c′ i (m)=s*c i (m) (2)

[0115] Where m and i are the sequence numbers of the algorithm modules and the constraints in the second processing flow, respectively; m = 1, 2, ..., M; i = 1, 2, ..., N; M and N are the total number of algorithm modules and the total number of constraints in the second processing flow, respectively; c′ i (m) and c i (m) are the corrected value and test value of the i-th constraint item of the m-th algorithm module respectively.

[0116] Step A414: Calculate the estimated value of each cumulative constraint item in the second processing flow according to the method shown in the following formula (3):

[0117]

[0118] Among them, c′ j (m) is the correction value of the j-th constraint item of the m-th algorithm module, and the j-th constraint item is a cumulative constraint item; is the estimated value of the j-th constraint item in the second processing flow.

[0119] For example, the production time is a cumulative constraint. By accumulating the production time of each algorithm module (the time required to run the algorithm module during the production process) according to the above formula (3), we can obtain the estimated production time required for the entire second processing flow to produce the target product.

[0120] Step A415: Calculate the estimated value of each extreme value constraint item in the second processing flow according to the method shown in the following formula (4):

[0121]

[0122] Among them, c′ k (m) is the modified value of the k-th constraint item of the m-th algorithm module, and the k-th constraint item is an extreme value constraint item; is the estimated value of the kth constraint item in the second processing flow.

[0123] For example, memory consumption is an extreme value constraint item. The maximum value of the memory consumption of all algorithm modules in the second processing flow is the estimated value of the memory consumption of the entire second processing flow.

[0124] Step A42: Calculate the first ratio of the estimated value of each constraint item to the corresponding upper limit value.

[0125] If the first ratio of a constraint item is greater than 1, it means that the constraint item exceeds the corresponding upper limit value, which will cause the second processing flow to fail to operate normally.

[0126] Step A43: If the first ratio of any constraint item in the second processing flow is greater than 1, the algorithm module in the second processing flow is deleted and / or replaced according to the optional flag or the mandatory flag.

[0127] This step may specifically include A431-A433:

[0128] Step A431: Determine the optional algorithm module and the mandatory algorithm module in the second processing flow according to the optional identifier and the mandatory identifier.

[0129] Step A432: If the first ratio of a constraint item in the second processing flow is greater than 1, and the constraint item is an additive constraint item, the optional algorithm module is selectively deleted and / or the required algorithm module is selectively replaced.

[0130] This step may specifically include steps A4321-A4325:

[0131] Step A4321: If the first ratio of a constraint item in the second processing flow is greater than 1, and the constraint item is a cumulative constraint item, the excess value of the constraint item is calculated according to the method shown in formula (5):

[0132]

[0133] Wherein, j is the sequence number of the constraint item; and are the exceedance value, estimated value and upper limit value of the constraint item respectively.

[0134] Step A4322: Arrange the correction values ​​of the constraint items of all optional algorithm modules in descending order to obtain a first sequence.

[0135] Step A4323: Compare each correction value in the first sequence with the excess value in order from the front to the back, and determine whether there is a correction value in the first sequence that is greater than or equal to the excess value.

[0136] Step A4324: If there is a correction value in the first sequence that is greater than or equal to the excess value, the optional algorithm module corresponding to the last correction value in the first sequence that is greater than or equal to the excess value is deleted from the second processing flow; otherwise, the correction values ​​in the first sequence are accumulated from the front to the back, and the optional algorithm module corresponding to each accumulated correction value is deleted from the second processing flow until the accumulated sum is greater than or equal to the excess value.

[0137] In this step, if there are correction values ​​in the first sequence that are greater than or equal to the excess value, theoretically, deleting any optional algorithm module corresponding to any correction value greater than or equal to the excess value can ensure that the constraint item in the updated second processing flow no longer exceeds the upper limit. In this embodiment, the optional algorithm module corresponding to the last correction value in the first sequence that is greater than or equal to the excess value is deleted to minimize the impact on production results, because algorithm modules with larger correction values ​​are computationally complex and generally have a greater impact.

[0138] If there is no correction value greater than or equal to the excess value in the first sequence, then simply deleting one algorithm module will not solve the problem. At this time, the method of deleting from the front to the back is adopted, and calculation is performed while deleting until the cumulative sum of the correction values ​​of the deleted modules is greater than or equal to the excess value, indicating that the first ratio of the constraint item in the updated second processing flow is less than or equal to 1, and the deletion operation can be stopped.

[0139] Step A4325: If there is no optional algorithm module in the second processing flow, or if the first ratio is still greater than 1 after all optional algorithm modules are deleted from the second processing flow, the required algorithm modules in the second processing flow are selectively replaced and the first ratio is recalculated until the first ratio is less than 1. This step may specifically include the following steps (1)-(3):

[0140] (1) Arrange the replacement modules corresponding to all the required algorithm modules in descending order according to the preferred evaluation index to obtain the second sequence.

[0141] (2) Determine whether there is a replacement module in the second sequence that meets the first replacement condition.

[0142] Each mandatory algorithm module may correspond to one or more replacement modules. The first replacement condition is: the correction value of the replacement module is smaller than the correction value of the corresponding mandatory algorithm module, and the absolute value of the difference between the correction value of the replacement module and the correction value of the mandatory algorithm module is greater than the excess value.

[0143] (3) If it exists, the first replacement module in the second sequence that meets the first replacement condition is used to replace the corresponding required algorithm module in the second processing flow; otherwise, the replacement module that meets the second replacement condition is found from the front to the back in the second sequence, the corresponding required algorithm module in the second processing flow is replaced with the replacement module found, and the first ratio is recalculated and replaced with the replacement module that meets the second replacement condition, and the replacement is stopped until the first ratio is less than or equal to 1.

[0144] The second replacement condition is that the correction value of the replacement module is smaller than the correction value of the corresponding mandatory algorithm module.

[0145] Theoretically, the problem can be solved by replacing the corresponding required algorithm module with any replacement module that meets the first replacement condition. The reason for selecting the first replacement module that meets this condition from the second sequence is to use the replacement module with the best evaluation index as much as possible.

[0146] If there is no replacement module that meets the first replacement condition, then multiple mandatory modules need to be replaced. At this time, according to the second replacement condition, find modules with better indicators from the front to the back in the second sequence to replace the corresponding mandatory modules.

[0147] Step A433: If the ratio of a constraint item in the second processing flow is greater than 1, and the constraint item is an extreme constraint item, then calculate the second ratio of the correction value of the constraint item to the upper limit value of each algorithm module and each replacement module, delete all optional algorithm modules with a second ratio greater than 1, and replace all required algorithm modules with a second ratio greater than 1 with corresponding replacement modules with a second ratio less than or equal to 1.

[0148] Step A50: Execute the optimized processing flow to generate the target product.

[0149] In a preferred embodiment, after step A43, the following steps may also be included:

[0150] Step A44: determine whether the estimated values ​​of the constraints of the current second processing flow are all less than the corresponding upper limit value multiplied by the preset first percentage; if so, appropriately add optional algorithm modules to improve the quality of the target product.

[0151] Specifically, the step of adding an optional algorithm module may include the following steps (1)-(5):

[0152] (1) Obtain a preset process corresponding to the product subcategory to which the target product belongs, and specify a corresponding algorithm module for each algorithm subcategory in the preset process, thereby obtaining a third processing process.

[0153] (2) Construct a first set using the optional algorithm module in the third processing flow, construct a second set using the optional algorithm module in the current second processing flow, and calculate the difference between the first set and the second set.

[0154] The purpose of this step is to find out the optional algorithm modules that are not used in the current second processing flow.

[0155] (3) Sort the optional algorithm modules in the difference set according to the running order of each module in the third processing flow to obtain a third sequence.

[0156] (4) Calculate the correction value of each constraint item of each optional algorithm module in the third sequence respectively.

[0157] (5) In the third sequence, one optional algorithm module is selected each time from the front to the back. If, after adding the selected optional algorithm module to the current second processing flow, the new estimated value of each constraint item in the second processing flow is less than the corresponding upper limit value multiplied by the preset second percentage, then the selected optional algorithm module is added to the current second processing flow.

[0158] In this step, the method for calculating the estimated values ​​of each constraint item in the second processing flow after adding the optional algorithm module can refer to formulas (3)-(4) in the above embodiment 1.

[0159] (6) Repeat step (5) until all optional algorithm modules in the third sequence are traversed. The second processing flow obtained at this time is the final optimized processing flow, in which the estimated value of each constraint item in the processing flow satisfies the conditions of being greater than or equal to the corresponding upper limit value multiplied by a preset first percentage (such as 50%), and less than the corresponding upper limit value multiplied by a preset second percentage (such as 80%).

[0160] Figure 2 This is a schematic diagram of the main steps of the second embodiment of the adaptive optimization method for satellite remote sensing image processing based on resource constraints in the present invention. Figure 2 As shown, the processing method of this embodiment includes steps B10-B60:

[0161] Step B10: Obtain the mission information uploaded by the ground system.

[0162] Mission information includes: information on the target products that need to be produced and the total operating time of the on-board intelligent processing system.

[0163] Step B20: Determine the first processing flow according to the task information.

[0164] Step B30: According to the algorithm configuration table, a corresponding algorithm module is assigned to each algorithm subcategory in the first processing flow, thereby obtaining a second processing flow.

[0165] Step B40: Obtain the upper limit value of each constraint item.

[0166] Among them, the constraints include: memory consumption and production time.

[0167] Specifically, the current free memory size of the on-board intelligent processing system is used as the upper limit of memory consumption; the time that the on-board intelligent processing system has experienced since startup is subtracted from the total running time to obtain the upper limit of the production time.

[0168] Step B50: Optimize the second processing flow according to the optional flags and the mandatory flags, so that the estimated value of each constraint item of the optimized processing flow is less than or equal to the corresponding upper limit value.

[0169] Step B60: Execute the optimized process flow to generate the target product.

[0170] Although the various steps in the above embodiment are described in the above-mentioned order, those skilled in the art will understand that in order to achieve the effect of this embodiment, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in a reverse order. These simple changes are within the scope of protection of the present invention.

[0171] Based on the above method embodiment, the present invention further provides an embodiment of an adaptive optimization system for satellite remote sensing image processing based on resource constraints. The system of this embodiment executes a computer program of the above method.

[0172] Those skilled in the art should be able to appreciate that the method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of electronic hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0173] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is clearly not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent modifications or substitutions to the relevant technical features, and the technical solutions after such modifications or substitutions will fall within the scope of protection of the present invention.

Claims

1. A resource-constrained adaptive optimization method for onboard remote sensing image processing, characterized in that: The method is applicable to an on-board intelligent processing system, and the method comprises: Obtaining task information uploaded by the ground system, wherein the task information includes: information of target products to be produced; Determine a first processing flow according to the task information; the first processing flow includes multiple steps for producing the target product, each step corresponds to an algorithm subcategory, and each algorithm subcategory is set with an optional flag or a mandatory flag; According to the algorithm configuration table, specify a corresponding algorithm module for each algorithm subcategory in the first processing flow, thereby obtaining a second processing flow; Optimizing the second processing flow according to the optional flag and the mandatory flag so that the estimated value of each constraint item of the optimized processing flow is less than or equal to the corresponding upper limit value; Executing the optimized processing flow to generate the target product; The steps of generating the algorithm configuration table include: Deploy a simulation environment in the ground system that is identical to the onboard software and hardware; producing the target product in the simulation environment using a set of images of standard sizes; Obtain configuration item information of each algorithm module in the production process and store it in the algorithm configuration table; in, The configuration item information includes: the name of each algorithm module, the version number, the algorithm subcategory to which it belongs, the preferred evaluation index, one or more replacement modules, and the test value of each constraint item of each algorithm module in the simulation environment.

2. The adaptive optimization method for satellite remote sensing image processing based on resource constraints according to claim 1, characterized in that: The step of “optimizing the second processing flow according to the optional flag and the mandatory flag so that the estimated value of each constraint item of the optimized processing flow is less than or equal to the corresponding upper limit value” includes: Calculating estimated values ​​of each constraint item of the second processing flow according to the algorithm configuration table; Calculating first ratios of the estimated value of each constraint item to the corresponding upper limit value respectively; If the first ratio of any constraint item in the second processing flow is greater than 1, the algorithm module in the second processing flow is deleted and / or replaced according to the optional flag or the mandatory flag.

3. The adaptive optimization method for satellite remote sensing image processing based on resource constraints according to claim 2, characterized in that: The step of "calculating estimated values ​​of each constraint item of the second processing flow according to the algorithm configuration table" includes: Obtaining the test value of each constraint item of each algorithm module in the second processing flow from the algorithm configuration table; According to the standard size and the size of the image to be processed, the data volume coefficient is calculated according to the following formula: ； in, and are the width and height of the image to be processed respectively; and are the width and height of the standard size image respectively; Based on the data volume coefficient and the test value of each constraint item of each algorithm module, the correction value of each constraint item of the algorithm module is calculated according to the following formula: ； in, and are the sequence numbers of the algorithm modules and the constraints in the second processing flow respectively; ; ; and are the total number of algorithm modules and the total number of constraints in the second processing flow respectively; and Respectively The first algorithm module the modified value and the test value of each constraint item; The estimated value of each cumulative constraint item of the second processing flow is calculated according to the following formula: ； in, For the The first algorithm module The correction value of the constraint item, and the constraint items are the cumulative constraint items; The second processing flow the estimated values ​​of the constraints; The estimated value of each extremum constraint item of the second processing flow is calculated according to the following formula: ； in, For the The first algorithm module The correction value of the constraint item, and the The constraint item is the extreme value constraint item; The second processing flow The estimated value of the constraint terms.

4. The adaptive optimization method for satellite remote sensing image processing based on resource constraints according to claim 3 is characterized in that: The step of “if the first ratio of any constraint item of the second processing flow is greater than 1, deleting and / or replacing the algorithm module in the second processing flow according to the optional flag or the mandatory flag” includes: Determining, according to the optional identifier and the mandatory identifier, an optional algorithm module and a mandatory algorithm module in the second processing flow; If the first ratio of a constraint item of the second processing flow is greater than 1, and the constraint item is the cumulative constraint item, selectively deleting the optional algorithm module and / or selectively replacing the required algorithm module; If the ratio of a constraint item in the second processing flow is greater than 1, and the constraint item is the extreme constraint item, then calculate the second ratio of the correction value of the constraint item to the upper limit value for each algorithm module and each replacement module, delete all the optional algorithm modules whose second ratio is greater than 1, and replace all the required algorithm modules whose second ratio is greater than 1.

5. The adaptive optimization method for on-board remote sensing image processing based on resource constraints according to claim 4, characterized in that: The step of “if the first ratio of a constraint item of the second processing flow is greater than 1, and the constraint item is the cumulative constraint item, selectively deleting the optional algorithm module and / or selectively replacing the required algorithm module” includes: If the first ratio of a constraint item in the second processing flow is greater than 1, and the constraint item is the cumulative constraint item, then the excess value of the constraint item is calculated: ； in, is the sequence number of the constraint item; 、 and are the excess value, the estimated value and the upper limit value of the constraint item respectively; Arranging the correction values ​​of the constraint items of all the optional algorithm modules in descending order to obtain a first sequence; Comparing the correction values ​​in the first sequence with the excess value in order from the front to the back, and determining whether there is a correction value in the first sequence that is greater than or equal to the excess value; If so, the optional algorithm module corresponding to the last correction value in the first sequence that is greater than or equal to the excess value is deleted from the second processing flow; otherwise, the correction values ​​in the first sequence are accumulated from the beginning to the end, and the optional algorithm module corresponding to each accumulated correction value is deleted from the second processing flow until the accumulated sum is greater than or equal to the excess value; If the optional algorithm module does not exist in the second processing flow, or the first ratio is still greater than 1 after all the optional algorithm modules are deleted from the second processing flow, the required algorithm module in the second processing flow is selectively replaced and the first ratio is recalculated until the replacement is stopped when the first ratio is less than 1.

6. The adaptive optimization method for satellite remote sensing image processing based on resource constraints according to claim 5, characterized in that: The step of “selectively replacing the required algorithm module in the second processing flow and recalculating the first ratio until the first ratio is less than 1 and stopping the replacement” includes: Arrange the replacement modules corresponding to all the required algorithm modules in descending order according to the preferred evaluation index to obtain a second sequence; Determining whether there is a replacement module in the second sequence that meets the first replacement condition; If so, the first replacement module in the second sequence that satisfies the first replacement condition is used to replace the corresponding mandatory algorithm module in the second processing flow; otherwise, the replacement module that satisfies the second replacement condition is found from the front to the back in the second sequence, and the corresponding mandatory algorithm module in the second processing flow is replaced with the found replacement module, and the first ratio is recalculated and replaced with the replacement module that satisfies the second replacement condition until the first ratio is less than or equal to 1. in, Each of the mandatory algorithm modules corresponds to one or more replacement modules; The first replacement condition is: the correction value of the replacement module is smaller than the correction value of the corresponding mandatory algorithm module, and the absolute value of the difference between the correction value of the replacement module and the correction value of the mandatory algorithm module is greater than the excess value; The second replacement condition is: the correction value of the replacement module is smaller than the correction value of the corresponding mandatory algorithm module.

7. The adaptive optimization method for satellite remote sensing image processing based on resource constraints according to claim 4, characterized in that: The step of “optimizing the second processing flow according to the optional flag and the mandatory flag so that the estimated value of each constraint item of the optimized processing flow is less than or equal to the corresponding upper limit value” further includes: Determine whether the estimated values ​​of each constraint item of the current second processing flow are all less than the corresponding upper limit value multiplied by the preset first percentage; if so, Obtaining a preset process corresponding to the product subcategory to which the target product belongs, and specifying a corresponding algorithm module for each algorithm subcategory in the preset process, thereby obtaining a third processing process; Constructing a first set using the optional algorithm module in the third processing flow, constructing a second set using the optional algorithm module in the current second processing flow, and calculating the difference between the first set and the second set; sorting the optional algorithm modules in the difference set according to the running order of each module in the third processing flow to obtain a third sequence; respectively calculating the correction value of each constraint item of each optional algorithm module in the third sequence; In the third sequence, one optional algorithm module is selected each time from the front to the back. If adding the selected optional algorithm module to the current second processing flow causes the new estimated value of each constraint item in the second processing flow to be less than the corresponding upper limit value multiplied by a preset second percentage, then the selected optional algorithm module is added to the current second processing flow; The steps of selecting the optional algorithm module and adding the optional algorithm module according to the new estimated value are repeated until each optional algorithm module in the third sequence is traversed.

8. The adaptive optimization method for satellite remote sensing image processing based on resource constraints according to claim 1, characterized in that: The target product information includes: the name and / or ID of the target product; If the task information further includes an initial processing flow, the step of "determining a first processing flow according to the task information" includes: using the initial processing flow as the first processing flow; Otherwise, the step of "determining a first processing flow according to the task information" includes: selecting a corresponding preset flow as the first processing flow according to the product subcategory to which the target product belongs; The remote sensing image products in the onboard intelligent processing system are divided into product categories according to the degree of processing, and each product category is further divided into product subcategories according to the processing purpose, technical characteristics and application requirements; The algorithms in the onboard intelligent processing system are divided into algorithm categories according to their position and function in the processing flow, and each algorithm category is subdivided into algorithm subcategories according to functions and application requirements; The constraints include: memory consumption and production time; The mission information also includes: the total operating time of the on-board intelligent processing system; Before "optimizing the second processing flow according to the optional identifier and the mandatory identifier", the method further includes: Using the current free memory size of the onboard intelligent processing system as the upper limit of the memory consumption; The upper limit value of the production time is obtained by subtracting the time that the on-board intelligent processing system has experienced since startup from the total operation time.

9. An adaptive optimization system for satellite remote sensing image processing based on resource constraints, characterized in that: The system executes a computer program of the method according to any one of claims 1 to 8.

Images