A method for evaluating the on-orbit performance of remote sensing satellite engineering

Abstract

A method for evaluating the on-orbit performance of a remote sensing satellite project belongs to the field of aerospace engineering consulting and evaluation. The implementation method of the present invention is: by investigating relevant departments such as satellite development units, ground operation units, and data application units, and combining the technical characteristics of each satellite model, the evaluation index is optimized and iterated, and an index system consisting of four first-level indicators, namely, satellite-ground integrated data acquisition efficiency, primary product processing and distribution efficiency, data application service effect, and data application service benefit, and several second-level indicators, is constructed based on the satellite on-orbit operation data, and a "qualitative + quantitative" performance evaluation method matching the index system is constructed. The "qualitative + quantitative" method is used to improve the objectivity and accuracy of satellite engineering performance evaluation through cross-domain, cross-industry, and cross-departmental investigation, calculation, statistics, and evaluation. According to the quantitative scoring results of the second-level indicators, a satellite capability radar chart is drawn to realize the digital explicit evaluation of the performance of the evaluated satellite.

Description

Technical Field

[0001] The invention relates to a satellite engineering on-orbit performance evaluation method, belonging to the field of aerospace engineering consulting and evaluation. Background Art

[0002] With the comprehensive formation of the earth observation system composed of land, meteorological and ocean satellites, the number and quality of my country's remote sensing satellites have gradually reached the world's advanced level. The satellite remote sensing system has formed a comprehensive and efficient global observation and data acquisition capability with reasonable configuration of high, medium and low resolutions and optimized combination of multiple observation methods, and the level of data autonomy has been significantly improved. Satellite remote sensing applications have covered more than 20 key industries such as natural resources, ecological environment, agriculture, emergency response, housing and construction, transportation, and water conservancy.

[0003] Satellite engineering is a complex large-scale system engineering. Due to the large number of participating research units, various user departments, and a long industrial chain (see Appendix Figure 1 ), there are still some outstanding problems in the process of satellite in-orbit operation and application services: for example, the ground system collaborative operation mechanism is not perfect, the product quality consistency is not good, and the business application effect is not significant. Satellite engineering performance evaluation involves cross-domain, cross-industry and cross-departmental collaboration such as government investment, satellite development, ground receiving station network construction, public welfare data processing, and user needs. At present, there is no mature quantitative evaluation method in China. The establishment of a satellite engineering performance evaluation system that combines quantitative and qualitative methods is of great significance to improving my country's aerospace governance level, promoting the establishment of a closed-loop assessment mechanism of "construction, use and management", and promoting the high-quality development of civil space infrastructure. Summary of the invention

[0004] In order to solve the problem that some existing remote sensing satellites have low on-orbit operation efficiency, but there is no quantitative method to evaluate their application efficiency and find out the capability shortcomings and causes, the purpose of the present invention is to provide an on-orbit efficiency evaluation method for remote sensing satellite engineering. This method can improve the objectivity and accuracy of satellite engineering efficiency evaluation by establishing a satellite engineering on-orbit efficiency evaluation index system and an on-orbit efficiency evaluation method that combines "qualitative + quantitative".

[0005] The objective of the present invention is achieved through the following technical solutions:

[0006] The invention discloses a method for evaluating the on-orbit performance of a remote sensing satellite project, comprising the following steps:

[0007] Step 1: Through investigations of satellite development units, ground operation units, data application units and other relevant departments, and in combination with the technical characteristics of each satellite model, we carry out optimization and iteration of evaluation indicators, and construct an indicator system consisting of four first-level indicators, namely, satellite-ground integrated data acquisition efficiency, primary product processing and distribution efficiency, data application service effect, and data application service benefit, and several second-level indicators.

[0008] The number of secondary indicators is selected according to the technical characteristics of the satellite being evaluated. The secondary indicators of satellite-ground integrated data acquisition efficiency include satellite available efficiency, payload working efficiency, satellite operation and control efficiency, satellite-ground data transmission efficiency, and ground reception efficiency. The secondary indicators of primary product processing and distribution efficiency include primary product processing efficiency, primary product data quality, and primary product acquisition satisfaction. The secondary indicators of data application service effect include the production of industry-specific products, the effect of supporting industry business, the completion of application services, and the effect of serving major national strategies and needs. The secondary indicators of data application service benefits include data self-sufficiency rate, scientific and technological innovation and driving role, and economic and social benefits.

[0009] Step 2: For the first-level indicator of satellite-ground integrated data acquisition efficiency constructed in step 1, collect the daily imaging plan files of the evaluated satellite payload, extract the payload imaging date, and count the total number of days the payload actually carries out missions. If the number of days actually carrying out payload missions meets the evaluation standard, 100 points will be awarded; if it does not meet the evaluation standard, the ratio will be mapped to the range of [0,100], which is the score of the second-level indicator of the available efficiency of the evaluated satellite.

[0010] Step 3: For the first-level indicator of satellite-ground integrated data acquisition efficiency constructed in step 1, the total imaging time of the camera payload throughout the year is counted, the average daily power-on time throughout the year is calculated, and the matching with the overall development requirements is evaluated. If the evaluation standard is met, it will be scored 100 points; if the evaluation standard is not met, the ratio will be mapped to the range of [0,100], which is the score of the second-level indicator of the satellite payload working efficiency being evaluated.

[0011] Step 4: For the first-level indicator of satellite-ground integrated data acquisition efficiency constructed in step 1, check the satellite mission planning records submitted to the satellite operation and control base, and evaluate the timeliness and matching of the overall development requirements in normal mode and emergency mode. If the evaluation standard is met, it will be scored 100 points; if the evaluation standard is not met, the ratio will be mapped to the range of [0,100], which is the score of the second-level indicator of the satellite operation and control efficiency being evaluated.

[0012] Step 5: For the first-level indicator of satellite-ground integrated data acquisition efficiency constructed in step 1, the on-board data transmission time and the matching of the resource utilization rate of the civil ground receiving station network with the total research and development requirements during the evaluation period are statistically analyzed. If the evaluation standard is met, it will be scored 100 points; if the evaluation standard is not met, the ratio will be mapped to the range of [0,100], which is the score of the second-level indicator of satellite-ground data transmission efficiency.

[0013] Step 6: For the first-level indicator of satellite-ground integrated data acquisition efficiency constructed in step 1, quantitative statistics are made on the ground receiving station's receiving mission success rate, the received data error rate, and the matching of the original data transmission timeliness with the overall research and development requirements. If the evaluation standard is met, it will be scored 100 points; if the evaluation standard is not met, the ratio will be mapped to the range of [0,100], and the three scores will be weighted with weights of 0.3, 0.3, and 0.4 respectively, which is the second-level indicator score of ground receiving efficiency.

[0014] The ground receiving station's receiving mission success rate is the ratio of the total number of successfully received orbits to the total number of orbits.

[0015] The received data bit error rate is the ratio of the number of orbits whose bit error rate does not meet the requirements among the number of orbits successfully received by the ground receiving station network.

[0016] The original data transmission timeliness is the time taken for the ground receiving station network to completely receive the satellite original data and then transmit the data to the data processing center.

[0017] The weights of 0.3, 0.3, and 0.4 are formulated using the analytic hierarchy process (AHP) and confirmed by industry experts.

[0018] Step 7: For the first-level indicator of primary product processing and distribution efficiency constructed in step 1, query the production and distribution records of satellite data, count the time from the complete reception of the original data transmitted from the ground station to the production of the primary product and completion of archiving, and the matching of this with the overall research and development requirements. The score is scored in stages, with 100 points for all qualified, 95 points for 95% to 100%, 90 points for 90% to 95%, and 80 points for 80% to 90%. This is the score for the second-level indicator of primary product processing efficiency.

[0019] Step 8: For the primary product processing and distribution efficiency of the first-level indicator constructed in step 1, a third-party professional organization will independently conduct primary product quality inspection, which includes selecting quality elements and inspection items, confirming sample data, quality inspection, and calculating scores. The inspection items are weighted according to the weights of radiation quality 0.2, geometric quality 0.4, spatial quality 0.2, and data file validity 0.1, which is the score of the secondary indicator of primary product quality.

[0020] The weights of 0.2, 0.4, 0.2, and 0.1 are constructed using the analytic hierarchy process (AHP) and confirmed by industry experts.

[0021] The inspection items include radiation quality, geometric quality, space quality, and data file validity.

[0022] The radiation quality includes relative radiation accuracy, absolute radiation accuracy, integrated sidelobe ratio, and peak sidelobe ratio; the geometric quality includes positioning accuracy and interferometric baseline length; the spatial quality includes imaging width and spatial resolution; and the data file validity includes file integrity and data appearance quality. The above evaluation indicators are all commonly used evaluation indicators for primary products in the remote sensing industry.

[0023] Step 9: For the primary product processing and distribution efficiency of the first-level indicator constructed in step 1, the satellite data application service departments are invited to score the satisfaction of the timeliness, quality, and acquisition method of obtaining primary products by using the method of on-site spot check + questionnaire survey. The application service departments include the Land Satellite Remote Sensing Application Center of the Ministry of Natural Resources and the Satellite Environment Application Center of the Ministry of Ecology and Environment.

[0024] The on-site spot check + questionnaire survey method refers to the assessment unit going to the satellite data application service department for on-site spot checks and surveys, and conducting questionnaire surveys on the quantity of primary products of the assessed satellite, the satisfaction degree of the quantity of primary products, the frequency of demand for primary products, the timeliness of obtaining primary products, the quality of primary products, etc. The satellite data application service department will make quantitative scores based on the actual business situation of the unit and against the overall research and development requirements.

[0025] Step 10: Based on the first-level indicator data application service effect constructed in step 1, a third-party professional organization will conduct quality inspection on the types and precision of special products developed and produced by various industries, and compare them with the overall development requirements. Scores are given based on the matching situation, which is the second-level indicator score for the production of industry special products.

[0026] Step 11: For the application service effect of the first-level indicator data constructed in step 1, a questionnaire survey + qualitative evaluation method is used to invite satellite data application service departments (such as the Land Satellite Remote Sensing Application Center of the Ministry of Natural Resources, the Satellite Environment Application Center of the Ministry of Ecology and Environment, etc.) to fill in the "Survey Form on the Application Service Effect of the Evaluated Satellite Data Products". The respondents are requested to describe in detail the industry business, tasks and major national strategies supported by the evaluated satellite data, the specific work carried out and the role played in them.

[0027] The “Survey Form for Application and Service Effects of Evaluated Satellite Data Products” is a questionnaire designed by the evaluation department for the evaluation of the satellite’s support for industry business and its effect in serving major national strategies, and is mainly a qualitative evaluation.

[0028] Step 12: Based on the first-level indicator data application service effect constructed in Step 1, a questionnaire survey is conducted to invite the counterpart departments of satellite data application services (such as the relevant business departments of the Ministry of Natural Resources and the Ministry of Ecology and Environment) to fill out the "Satisfaction Survey Questionnaire for the Evaluated Satellite Application Services". The questionnaires are collected and sorted out, and the scores are calculated.

[0029] The "Satellite Application Service Satisfaction Questionnaire" is a quantitative evaluation designed by the evaluation department for the satisfaction of the satellites being evaluated with the satellite data products, service timeliness, service process, service personnel, response speed, technical consultation and service, submission method, etc. provided by the application service department. The scoring method is designed with a gradient of 100 points for very satisfied, 0 points for very dissatisfied, and 10 points for very dissatisfied. The results of all questionnaires are counted to form the secondary indicator score of application service satisfaction.

[0030] Step 13: For the first-level indicator data application service benefits constructed in Step 1, invite satellite data application service departments (such as the Land Satellite Remote Sensing Application Center of the Ministry of Natural Resources, the Satellite Environment Application Center of the Ministry of Ecology and Environment, etc.) to fill out the "Survey Form on the Benefits of Evaluated Satellite Application Services" to conduct a qualitative assessment of the benefits of satellite application services.

[0031] The "Evaluated Satellite Application Service Benefits Questionnaire" is a questionnaire designed for the evaluated satellite data self-sufficiency rate, scientific and technological innovation and driving role, and the economic and social benefits generated by satellite data application services. It is mainly a qualitative evaluation.

[0032] Step 14: Based on the quantitative scores of each secondary indicator obtained in steps 2 to 13, draw a satellite capability radar chart to intuitively express the performance shortcomings and advantages of the evaluated satellite.

[0033] Step 15: Based on the qualitative evaluation results of each secondary indicator obtained in steps 2 to 13, comprehensively evaluate the performance of the satellite being evaluated to obtain the on-orbit performance evaluation results of the remote sensing satellite project.

[0034] It also includes step 16: Based on the on-orbit performance evaluation results of the remote sensing satellite project obtained in step 15, with the identified deficiencies in the satellite's on-orbit operation as the focus, optimize the operation control mode of the on-orbit and active satellites, provide support and comprehensive quantitative optimization for the design of subsequent satellite development indicators, help improve the application efficiency of my country's remote sensing satellites, improve the cross-domain, cross-industry and cross-departmental coordination efficiency of remote sensing satellite projects, and solve related engineering problems.

[0035] Beneficial effects:

[0036] 1. The present invention discloses an on-orbit performance evaluation method for remote sensing satellite engineering. By investigating relevant departments such as satellite development units, ground operation units, and data application units, and combining the technical characteristics of each satellite model, the evaluation index optimization and iteration are carried out. Based on the on-orbit operation data of the satellite, an index system consisting of four first-level indicators, namely, satellite-ground integrated data acquisition efficiency, primary product processing and distribution efficiency, data application service effect, and data application service benefit, and several second-level indicators is constructed, and a "qualitative + quantitative" performance evaluation method matching the index system is constructed. The "qualitative + quantitative" method is adopted to improve the objectivity and accuracy of satellite engineering performance evaluation through cross-domain, cross-industry, and cross-departmental investigation, calculation, statistics, and evaluation.

[0037] 2. The invention discloses a method for evaluating the on-orbit performance of remote sensing satellite engineering. Based on the on-orbit operation data of satellites and the application requirements of remote sensing satellite engineering, the invention analyzes and condenses the key elements for evaluating the on-orbit performance of remote sensing satellite engineering, constructs a mapping relationship between classified, layered, and graded indicators and specific scores based on the key elements, and designs a systematic and comprehensive satellite engineering primary, secondary, and tertiary indicator system and a series of questionnaires. According to the indicator system and the results of the series of questionnaires, a systematic and objective on-orbit performance evaluation of satellite engineering is achieved. The performance evaluation method of the invention can also be applied to other space infrastructure such as communications and navigation satellite engineering.

[0038] 3. The present invention discloses an on-orbit performance evaluation method for remote sensing satellite engineering. According to the quantitative scoring results of the secondary indicators, a satellite capability radar chart is drawn, which can intuitively express the performance shortcomings and advantages of the evaluated satellite, realize the digital display evaluation of the evaluated satellite performance, optimize the operation control mode of in-orbit satellites in service, provide support and comprehensive quantitative optimization for the design of subsequent satellite development indicators, and help improve the application efficiency of my country's remote sensing satellites.

[0039] 4. The invention discloses a method for evaluating the on-orbit performance of a remote sensing satellite project. On the basis of achieving the above three beneficial effects, it helps to establish a closed-loop assessment mechanism of "build-use-manage", and improve the cross-domain, cross-industry and cross-departmental coordination efficiency of remote sensing satellite projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a schematic diagram of the remote sensing satellite engineering industry chain;

[0041] Figure 2 This is a diagram of the remote sensing satellite engineering effectiveness evaluation index system;

[0042] Figure 3 This is a radar chart of a remote sensing satellite's capabilities. DETAILED DESCRIPTION

[0043] Combined with Figure 1, taking a remote sensing satellite in orbit as an example, the present invention is further explained.

[0044] Example 1

[0045] like Figure 1 As shown, the present embodiment discloses a method for evaluating the on-orbit performance of a remote sensing satellite project, and the specific implementation steps are as follows:

[0046] Step 1: Combine the technical characteristics of a certain type of remote sensing satellite and Figure 2 An indicator system consisting of four first-level indicators, namely, satellite-ground integrated data acquisition efficiency, primary product processing and distribution efficiency, data application service effect, and data application service benefits, and 10 second-level indicators was selected from a relatively comprehensive second-level indicator system library.

[0047] Step 2: For the secondary indicator satellite available performance constructed in step 1, collect the daily imaging plan files of the satellite payload to be evaluated, extract the payload imaging date, and count the total number of days the payload actually carries out the mission. If the number of days the payload mission is actually carried out meets the evaluation criteria, 100 points will be awarded.

[0048] Step 3: For the secondary indicator payload working efficiency constructed in step 1, the total imaging time of the camera payload throughout the year was counted, the average daily power-on time throughout the year was calculated, and the matching with the overall development requirements was evaluated. If the evaluation standard was not met, the ratio was mapped to [0,100] and the final score was 42.5 points.

[0049] Step 4: For the secondary indicator satellite operation and control efficiency constructed in step 1, check the satellite mission planning records submitted to the satellite operation and control base, and evaluate the timeliness and matching of the overall development requirements in normal mode and emergency mode. If the evaluation standards are met, 100 points will be awarded.

[0050] Step 5: For the secondary indicator satellite-to-ground data transmission efficiency constructed in step 1, the satellite data transmission time during the evaluation period and the matching of the resource utilization rate of the civil ground receiving station network with the overall research and development requirements were statistically analyzed. If the evaluation standard was not met, the ratio was mapped to [0,100] and a score of 41.3 was obtained.

[0051] Step 6: For the secondary indicator ground reception efficiency constructed in step 1, quantitatively count the ground receiving station reception mission success rate, reception data bit error rate, and the original data transmission timeliness and the matching of the overall research and development requirements. If the evaluation standard is not met, the ratio is mapped to [0,100] and the three scores are weighted with weights of 0.3, 0.3, and 0.4 respectively, and the score is 98.5.

[0052] Step 7: For the secondary indicator primary product processing efficiency constructed in step 1, query the production and distribution records of satellite data, count the time from the complete reception of the original data transmitted from the ground station to the production of the primary product and completion of archiving, and the matching of this with the overall research and development requirements. The score is scored in stages, with 100 points for all qualified, 95 points for 95% to 100%, 90 points for 90% to 95%, and 80 points for 80% to 90%, for a total of 95 points.

[0053] Step 8: For the secondary indicator primary product quality constructed in step 1, a third-party professional organization independently conducts primary product quality inspection, including the selection of quality elements and inspection items, confirmation of sample data, quality inspection, and calculation of scores. The inspection items are weighted according to the weights of radiation quality 0.2, geometric quality 0.4, spatial quality 0.2, and data file validity 0.1, and the score is 99.12.

[0054] Step 9: For the secondary indicator of primary product acquisition satisfaction constructed in step 1, a questionnaire survey was conducted to invite satellite data application service departments (such as the Land Satellite Remote Sensing Application Center of the Ministry of Natural Resources, the Satellite Environment Application Center of the Ministry of Ecology and Environment, etc.) to score the satisfaction level of the timeliness, quality, and acquisition method of primary products. After statistical calculation, the score was 80.6.

[0055] Step 10: Based on the secondary indicator industry-specific product production status constructed in Step 1, a third-party professional organization will conduct quality inspections on the types and precision of the industry-specific products developed and produced, and compare them with the overall development requirements. 88 points will be awarded based on the matching situation.

[0056] Step 11: For the application service effect of the secondary indicator data constructed in step 1, a questionnaire survey and qualitative evaluation are used to invite satellite data application service departments (such as the Land Satellite Remote Sensing Application Center of the Ministry of Natural Resources, the Satellite Environment Application Center of the Ministry of Ecology and Environment, etc.) to fill in the "Survey Form on the Application Service Effect of a Certain Satellite Data Product". Please describe in detail the industry business, tasks and major national strategies supported by the satellite data being evaluated, the specific work carried out, and the role played in them.

[0057] Step 12: For the secondary indicator data application service satisfaction constructed in step 1, a questionnaire survey was conducted to invite the counterpart departments of satellite data application services (such as the relevant business departments of the Ministry of Natural Resources and the Ministry of Ecology and Environment) to fill out the "Satisfaction Survey Questionnaire for a Certain Satellite Application Service". After collecting the questionnaires, a score of 95.4 was obtained.

[0058] Step 13: For the first-level indicator data application service benefits constructed in step 1, invite satellite data application service departments (such as the National Land Satellite Remote Sensing Application Center of the Ministry of Natural Resources, the Satellite Environment Application Center of the Ministry of Ecology and Environment, etc.) to fill out the "Satellite Application Service Benefit Survey Form" to conduct a qualitative assessment of the satellite application service benefits.

[0059] Step 14: Based on the quantitative scores of each secondary indicator obtained from Steps 2 to 13, draw the following Figure 3 The radar chart of a satellite's capabilities shown intuitively expresses the performance shortcomings and advantages of the evaluated satellite.

[0060] Step 15: Based on the qualitative evaluation results of each secondary indicator obtained in steps 2 to 13, comprehensively summarize the performance of the satellite being evaluated.

[0061] Step 16: Based on the on-orbit performance evaluation results of the remote sensing satellite project obtained in step 15, it is identified that the shortcomings of the satellite's on-orbit operation are poor payload working efficiency and satellite-to-ground data transmission efficiency. It is recommended that the satellite payload startup time be specially designed, and the subsequent satellite development payload startup indicators be comprehensively and quantitatively optimized to improve the data transmission efficiency from both the satellite and the ground.

[0062] Embodiment 2:

[0063] like Figure 1 As shown, the present embodiment discloses a method for evaluating the on-orbit performance of a remote sensing satellite project, and the specific implementation steps are as follows:

[0064] Step 1: Collect the daily imaging plan files of the satellite payload during the evaluation period, extract the satellite payload imaging date, and calculate the total number of days the satellite payload actually carried out the mission. The number of days the satellite payload was turned on to perform the mission is 365 days, and the evaluation period is one year, so the satellite usable performance index is 100 points.

[0065] Step 2: The total imaging time of the satellite payload throughout the year is counted, and the average daily power-on time for the whole year is calculated to be 52.3 minutes. The general requirements for satellite development clearly state that the average daily power-on time of the satellite payload should be 123 minutes. If the evaluation standard is not met, the ratio is mapped to [0,100], and the final payload work efficiency index is 42.5 points.

[0066] Step 3: Check the mission planning records submitted to the satellite control center, extract the number of tasks submitted to the control center under normal conditions, and the difference between the satellite operation and control instruction injection time and the controllable arc time under emergency conditions. The general development requirements are that the number of tasks submitted to the control center under normal conditions should be no less than once a day, and that the injection instructions should be submitted to the control center at least 0.5 hours in advance under emergency conditions. According to statistics, the satellite under evaluation has tasks submitted every day, and there were 2 emergency tasks throughout the year, both of which were injected within 10 minutes. All of them met the evaluation standards, so the satellite operation and control efficiency index scored 100 points.

[0067] Step 4: Based on the satellite's on-orbit operation and control data, the on-board data transmission time and the ground receiving station network resource utilization rate during the evaluation period are statistically analyzed. The general development requirements stipulate that the average daily on-board data transmission time should be 200 minutes; considering the risk of mechanical wear caused by the uninterrupted operation of the ground receiving station, the evaluation standard requires that the ground receiving station network resource utilization rate exceeds 70%, which is considered to be in full load operation, that is, full marks. If the evaluation standard is not met, the ratio will be mapped to between [0,100]. According to statistics, the average daily transmission time of the satellite payload was 85.3 minutes, which was 42.65 points, and the ground receiving station network utilization rate was 28%, which was 40 points. After the two were equally weighted, the satellite-to-ground data transmission efficiency index scored 41.3 points.

[0068] Step 5: Query the satellite data receiving system and count the ground receiving station's receiving mission success rate, received data error rate, and original data transmission time. Compare these three indicators with the overall development requirements. If the requirements are met, 100 points will be awarded. If the standards are not met, the ratio will be mapped to [0,100]. Then, the three scores will be weighted by 0.3, 0.3, and 0.4 respectively, and the final ground receiving efficiency index is 98.5 points.

[0069] Step 6: Define the five indicators obtained in steps 1 to 5 as secondary indicators 1 to 5 respectively. The comprehensive score of the five indicators can quantitatively describe the satellite-to-ground data acquisition situation of the evaluated satellite, and the overall score is defined as primary indicator 1: satellite-to-ground integrated data acquisition efficiency.

[0070] Step 7: Query the production and distribution records of satellite data, and count the time from the complete reception of the original data transmitted by the ground station to the production of primary products and the completion of archiving. According to the standards stipulated in the general requirements for development, the data is scored as 100 points if all qualified, 95% to 100% qualified as 95 points, 90% to 95% as 90 points, and 80% to 90% as 80 points. If 97% of the data of the satellite is qualified, the primary product processing efficiency index will be 95 points.

[0071] Step 8: A third-party professional organization shall independently conduct the primary product quality inspection of the satellite being evaluated, including four processes: selecting quality elements, defining inspection item E, confirming sample data, and conducting quality inspection. The score of each inspection item is calculated based on the general requirement standard for satellite development as D, according to the scoring criteria of E≤D as 100 points; D<E≤1.2D as 85 points; 1.2D<E≤1.5D as 60 points; E>1.5D as 0 points. After each inspection item is weighted according to the weights of 0.2 for radiation quality, 0.4 for geometric quality, 0.2 for space quality, and 0.1 for data file validity, the primary product quality index of the satellite being evaluated scored 99.12 points.

[0072] Step 9: For the application service departments of satellite data (the application service departments of the evaluated satellite in this embodiment are the Land Satellite Remote Sensing Application Center of the Ministry of Natural Resources and the Satellite Environment Application Center of the Ministry of Ecology and Environment), based on the data in their business systems, the satisfaction degree of the timeliness, quality and acquisition method of obtaining the primary products of satellite data is scored on a percentage basis. After the three are equally weighted, the primary product acquisition satisfaction index of the evaluated satellite is 80.6 points.

[0073] Step 10: Define the three indicators obtained in steps 7 to 9 as secondary indicators 6 to 8 respectively. The comprehensive score of the three indicators can quantitatively describe the primary product processing and distribution of the evaluated satellite, and the overall score is defined as primary indicator 2: primary product processing and distribution efficiency.

[0074] Step 11: A third-party professional organization will conduct quality inspection on the types and accuracy of special products developed and produced by various industries using the evaluated satellite data, and compare them with the requirement D in the general development requirements. The quality inspection result E is based on the evaluation criteria of full score when E≤D, 85 points when D<E≤1.2D, 60 points when 1.2D<E≤1.5D, and 0 points when E>1.5D. The evaluated satellite's industry special product production index scored 88 points.

[0075] Step 12: For the application service departments of satellite data (the application service departments of the evaluated satellite in this embodiment are the Land Satellite Remote Sensing Application Center of the Ministry of Natural Resources and the Satellite Environment Application Center of the Ministry of Ecology and Environment), based on the data in their business systems, statistical surveys are conducted on the industry business supported by the evaluated satellite data to form a list of the supported industry business effects of the evaluated satellite.

[0076] Step 13: Conduct a survey on the corresponding superior departments of the satellite data application service department (the relevant business departments of the Ministry of Natural Resources and the Ministry of Ecology and Environment in this embodiment), and count the task completion of the four factors of satellite data product service timeliness, response speed, technical consultation, and submission method provided by the application service department. According to the evaluation criteria, 100% completion is the full score. If the evaluation criteria are not met, the ratio is mapped to [0, 100]. After the scores of the four factors of the evaluated satellite are equally weighted and averaged, the application service satisfaction score is 95.4 points.

[0077] Step 14: For the application service departments of satellite data (the application service departments of the evaluated satellite in this embodiment are the Land Satellite Remote Sensing Application Center of the Ministry of Natural Resources and the Satellite Environment Application Center of the Ministry of Ecology and Environment), based on the data in their business systems, statistical surveys are conducted on the names of the major national strategies supported and served by the evaluated satellite data to form a list of the effects of the evaluated satellite in supporting and serving the major national strategies.

[0078] Step 15: Define the four indicators obtained in steps 10 to 14 as secondary indicators 9 to 12 respectively. The comprehensive situation of the four indicators can describe the data product application service effect of the evaluated satellite, which is generally defined as primary indicator 3: data product application service effect.

[0079] Step 16: For the application service departments of satellite data (the application service departments of the satellites under evaluation in this embodiment are the Land Satellite Remote Sensing Application Center of the Ministry of Natural Resources and the Satellite Environment Application Center of the Ministry of Ecology and Environment), based on the data in their business systems, statistics are collected on "the proportion of domestic satellites in the current type of satellite data images" and "the proportion of this satellite in the current type of satellite data images". Through mean analysis, the proportion of domestic satellites in the same type of satellite data images of the evaluated satellite is 67.5%, of which the data of the evaluated satellite accounts for 47.5%, which is the data self-sufficiency rate indicator.

[0080] Step 17: Count the number of scientific research projects, academic papers, standards and specifications, patents based on the satellite remote sensing data / products being evaluated, as well as the number of national, provincial, ministerial and other awards obtained, which are indicators of scientific and technological innovation and driving effect.

[0081] Step 18: Based on the public data of the National Bureau of Statistics, collect and investigate the direct and indirect economic benefits generated by satellite data products or services. The satellite data under evaluation has been used in 70 units during the evaluation period, saving each unit more than 100 million yuan in fiscal expenditure and bringing indirect economic benefits of more than 24 million yuan, which is the economic and social benefit indicator.

[0082] Step 19: Define the three indicators obtained in steps 16 to 18 as secondary indicators 13 to 15 respectively. The comprehensive situation of the three indicators can describe the data product application service benefits of the evaluated satellite, and the overall definition is first-level indicator 4: data product application service benefits.

[0083] Step 20: Based on the first-level indicator 1-star-ground integrated data acquisition efficiency obtained in step 6, the first-level indicator 2-primary product processing and distribution efficiency obtained in step 10, the first-level indicator 3-data product application service effect obtained in step 15, and the first-level indicator 4-star-ground integrated data acquisition efficiency obtained in step 19, the following can be obtained: Figure 2 The on-orbit performance index system of the evaluated satellite is shown in FIG.

[0084] Step 21: Based on the quantitative scores of each secondary indicator obtained from Steps 1 to 19, draw the following Figure 3 The radar chart of the satellite's capabilities shown is used to intuitively express the satellite's performance shortcomings and advantages.

[0085] Step 22: Obtain the evaluation results based on Steps 1 to 21, and comprehensively summarize the on-orbit performance of the satellite being evaluated.

[0086] The specific description above further illustrates the purpose, technical solutions and beneficial effects of the invention in detail. It should be understood that the above is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for evaluating the on-orbit performance of a remote sensing satellite project, characterized in that: The following steps are included: Step 1: Through investigation of satellite development units, ground operation units, and data application units, and in combination with the technical characteristics of each satellite model, we conduct the optimization and iteration of evaluation indicators, and construct an indicator system consisting of four first-level indicators, namely, satellite-ground integrated data acquisition efficiency, primary product processing and distribution efficiency, data application service effect, and data application service benefit, and several second-level indicators; the number of the second-level indicators is selected according to the technical characteristics of the satellite being evaluated; Step 2: For the first-level indicator of satellite-ground integrated data acquisition efficiency constructed in step 1, collect the daily imaging plan files of the satellite payload to be evaluated, extract the payload imaging date, and calculate the total number of days the payload actually carried out the mission. If the number of days the payload mission was actually carried out meets the evaluation standard, 100 points will be awarded; If the evaluation criteria are not met, the ratio is mapped to the range [0,100], which is the score of the secondary indicator of the satellite's available effectiveness; Step 3: For the first-level indicator of satellite-ground integrated data acquisition efficiency constructed in step 1, the total imaging time of the camera payload throughout the year is counted, the average daily power-on time throughout the year is calculated, and the matching with the overall development requirements is evaluated. If the evaluation standard is met, it will be scored 100 points; if the evaluation standard is not met, the ratio will be mapped to the range of [0,100], which is the score of the second-level indicator of the satellite payload working efficiency being evaluated; Step 4: For the first-level indicator of satellite-ground integrated data acquisition efficiency constructed in step 1, check the satellite mission planning records submitted to the satellite operation and control base, and evaluate the timeliness and matching of the overall development requirements in the normal mode and emergency mode. If the evaluation standard is met, it will be scored 100 points; if the evaluation standard is not met, the ratio will be mapped to the range of [0,100], which is the score of the second-level indicator of the satellite operation and control efficiency being evaluated; Step 5: For the first-level indicator of satellite-ground integrated data acquisition efficiency constructed in step 1, the satellite data transmission time and the matching of the resource utilization rate of the civil ground receiving station network with the total research and development requirements during the evaluation period are statistically analyzed. If the evaluation standard is met, it will be scored 100 points; if the evaluation standard is not met, the ratio will be mapped to the range of [0,100], which is the score of the second-level indicator of satellite-ground data transmission efficiency; Step 6: For the first-level indicator of satellite-ground integrated data acquisition efficiency constructed in step 1, quantitative statistics are made on the ground receiving station's receiving mission success rate, received data bit error rate, and the matching of the original data transmission timeliness with the overall development requirements. If the evaluation standard is met, it will be scored 100 points; if the evaluation standard is not met, the ratio will be mapped to [0,100], and the three scores will be weighted with weights of 0.3, 0.3, and 0.4 respectively, which is the ground receiving efficiency second-level indicator score; The ground receiving station receiving mission success rate is the ratio of the total number of successfully received orbits to the total number of orbits; The ratio of the number of orbits whose bit error rate does not meet the requirement among the number of orbits successfully received by the ground receiving station network of the received data bit error rate; The original data transmission timeliness is the time taken by the ground receiving station network from the complete reception of the satellite original data to the data transmission to the data processing center; The weights of 0.3, 0.3, and 0.4 were formulated using the analytic hierarchy process (AHP) and confirmed by industry experts; Step 7: For the primary product processing and distribution efficiency of the first-level indicator constructed in step 1, query the production and distribution records of satellite data, count the time from the complete reception of the original data transmitted by the ground station to the production of the primary product and the completion of archiving, and the matching of the overall research and development requirements. The score is scored in stages with 100 points for all qualified, 95% to 100% for 95 points, 90% to 95% for 90 points, and 80% to 90% for 80 points. This is the score of the secondary indicator of primary product processing efficiency. Step 8: For the primary product processing and distribution efficiency of the first-level indicator constructed in step 1, a third-party professional organization shall independently conduct primary product quality inspection. The primary product quality inspection includes selecting quality elements and inspection items, confirming sample data, quality inspection, and calculating scores. The inspection items are weighted according to the weights of radiation quality 0.2, geometric quality 0.4, spatial quality 0.2, and data file validity 0.1, which is the score of the secondary indicator of primary product quality. The weights of 0.2, 0.4, 0.2, and 0.1 are constructed using the analytic hierarchy process (AHP) and confirmed by industry experts; The inspection items include radiation quality, geometric quality, spatial quality, and data file validity; The radiation quality includes relative radiation accuracy, absolute radiation accuracy, integrated sidelobe ratio, peak sidelobe ratio; the geometric quality includes positioning accuracy and interferometric baseline length; the spatial quality includes imaging width and spatial resolution; the data file validity includes file integrity and data apparent quality; the above evaluation indicators are all commonly used evaluation indicators for primary products in the remote sensing industry; Step 9: For the primary product processing and distribution efficiency of the first-level indicator constructed in step 1, the satellite data application service departments are invited to score the timeliness, quality, and satisfaction of the acquisition method of primary products through on-site spot checks + questionnaire surveys; the application service departments include the Land Satellite Remote Sensing Application Center of the Ministry of Natural Resources and the Satellite Environment Application Center of the Ministry of Ecology and Environment; The on-site spot check + questionnaire survey method refers to the assessment unit going to the satellite data application service department for on-site spot checks and surveys, and conducting questionnaire surveys on the quantity of primary products of the assessed satellite, the satisfaction degree of the quantity of primary products, the frequency of demand for primary products, the timeliness of obtaining primary products, and the satisfaction degree of the quality of primary products. The satellite data application service department will make quantitative scores based on the actual business situation of the unit and in accordance with the overall research and development requirements; Step 10: Based on the application service effect of the first-level indicator data constructed in step 1, a third-party professional organization will conduct quality inspections on the types and precision of the special products developed and produced by various industries, and compare them with the overall development requirements. Scores are given based on the matching situation, which is the second-level indicator score for the production of industry special products; Step 11: For the first-level indicator data application service effect constructed in step 1, use the method of questionnaire survey + qualitative evaluation to invite the satellite data application service departments to fill in the "Evaluated Satellite Data Product Application Service Effect Questionnaire"; the respondents are asked to describe in detail the industry business, tasks and major national strategies supported by the evaluated satellite data, the specific work carried out and the role played in them; The “Survey Form for Application and Service Effect of the Evaluated Satellite Data Products” is a questionnaire designed by the evaluation department for the evaluation of the satellite’s support for industry business and its effect in serving the country’s major strategies. It is mainly a qualitative evaluation. Step 12: Based on the first-level indicator data application service effect constructed in step 1, a questionnaire survey is conducted to invite the counterpart departments of satellite data application services to fill in the "Satisfaction Survey Questionnaire for the Satellite Application Services", and the scores are calculated after the questionnaires are collected and sorted; The "Satellite Application Service Satisfaction Questionnaire" is a quantitative evaluation designed by the evaluation department for the satisfaction of the satellite being evaluated with the satellite data products, service timeliness, service process, service personnel, response speed, technical consultation and service, and submission method provided by the application service department. The scoring method is designed with a gradient of 100 points for very satisfied, 0 points for very dissatisfied, and 10 points for very dissatisfied. The results of all questionnaires are counted to form the secondary indicator score of application service satisfaction. Step 13: In view of the first-level indicator data application service benefits constructed in step 1, invite the satellite data application service departments to fill in the "Evaluated Satellite Application Service Benefit Survey Form" to conduct a qualitative assessment of the satellite application service benefits; The "Evaluated Satellite Application Service Benefit Survey Form" is a questionnaire designed for evaluating the satellite data self-sufficiency rate, scientific and technological innovation and driving role, and the economic and social benefits generated by satellite data application services. It is mainly a qualitative evaluation. Step 14: Based on the quantitative scores of each secondary indicator obtained from Steps 2 to 13, draw a satellite capability radar chart to intuitively express the performance shortcomings and advantages of the evaluated satellite; Step 15: Based on the qualitative evaluation results of each secondary indicator obtained in steps 2 to 13, comprehensively evaluate the performance of the satellite being evaluated to obtain the on-orbit performance evaluation results of the remote sensing satellite project.

2. The method for evaluating the on-orbit performance of a remote sensing satellite project according to claim 1, wherein: The secondary indicators of satellite-ground integrated data acquisition efficiency include satellite available efficiency, payload working efficiency, satellite operation and control efficiency, satellite-ground data transmission efficiency, and ground reception efficiency; the secondary indicators of primary product processing and distribution efficiency include primary product processing efficiency, primary product data quality, and primary product acquisition satisfaction; the secondary indicators of data application service effect include the production status of industry-specific products, the effect of supporting industry business, the completion of application services, and the effect of serving major national strategies and needs; the secondary indicators of data application service benefits include data self-sufficiency rate, scientific and technological innovation and its driving role, and economic and social benefits.

3. A remote sensing satellite engineering on-orbit performance evaluation method as claimed in claim 1 or 2, characterized in that: It also includes step 16: based on the on-orbit performance evaluation results of the remote sensing satellite project obtained in step 15, with the identified deficiencies in the satellite's on-orbit operation as the focus, optimize the operation control mode of the on-orbit and active satellites, provide support and comprehensive quantitative optimization for the design of subsequent satellite development indicators, improve the application efficiency of remote sensing satellites, and enhance the cross-domain, cross-industry and cross-departmental coordination efficiency of remote sensing satellite projects.

Images