A marine apparent optical measurement data integration method, device, equipment and medium

Abstract

The application discloses a marine apparent optical measurement data integration method, device and equipment and a medium, and relates to the field of marine data processing. The method comprises the following steps: acquiring a marine apparent optical measurement time sequence; the marine apparent optical measurement time sequence comprises a plurality of measurement times and measurement data corresponding to each measurement time; dividing the marine apparent optical measurement time sequence into a plurality of data groups according to the measurement times; calculating a representative spectrum of each data group respectively, and performing quality assessment on each data group respectively to obtain a quality identifier of each data group; and outputting the representative spectrum and the quality identifier of each data group to a specified file. The application improves the processing efficiency and accuracy of massive marine apparent optical measurement data.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of ocean data processing, and in particular to a method and device for integrating ocean apparent optical measurement data, and a medium. BACKGROUND

[0002] Ocean apparent optical measurement data is the core basic data for satellite ocean remote sensing calibration and verification. In the prior art, the processing of the time series of ocean apparent optical measurement data is mostly limited to directly obtaining original data, without grouping and integrating the continuous time series data according to the measurement time, resulting in redundancy and disorder of the data. Moreover, there is a lack of representative spectrum calculation for the grouped data, which cannot extract key optical information, and there is no data quality evaluation, so that the effectiveness of the data cannot be judged and the accuracy cannot be guaranteed when used for satellite calibration or verification, which cannot meet the actual application requirements of accurate correction of satellite remote sensing data. SUMMARY

[0003] The purpose of the present application is to provide a method and device for integrating ocean apparent optical measurement data, which can improve the processing efficiency and accuracy of massive ocean apparent optical measurement data.

[0004] To achieve the above purpose, the present application provides the following solutions.

[0005] In a first aspect, the present application provides a method for integrating ocean apparent optical measurement data, comprising:

[0006] obtaining a time series of ocean apparent optical measurement data; the time series of ocean apparent optical measurement data comprises a plurality of measurement times and measurement data corresponding to each measurement time;

[0007] dividing the time series of ocean apparent optical measurement data into a plurality of data groups according to the measurement time;

[0008] calculating the representative spectrum of each data group respectively, and performing quality evaluation on each data group respectively to obtain the quality identifier of each data group;

[0009] outputting the representative spectrum and the quality identifier of each data group to a designated file.

[0010] In an embodiment, the measurement data includes geographic location, environmental parameters, remote sensing reflectance data of each waveband, total radiation data of each waveband, sky radiation data of each waveband, and solar irradiance data of each waveband.

[0011] In an embodiment, the marine apparent optical measurement time sequence is divided into a plurality of data groups according to the measurement time, specifically comprising: calculating the time difference between adjacent measurement times in the marine apparent optical measurement time sequence; taking the measurement data with a time difference greater than a preset time interval threshold as a grouping boundary; and dividing the marine apparent optical measurement time sequence into a plurality of data groups based on the grouping boundary.

[0012] In an embodiment, a representative spectrum of each data group is calculated respectively, specifically comprising: calculating the coefficient of variation of the measurement data in a specified waveband range in each data group respectively; for any data group, if the coefficient of variation corresponding to the data group is less than a set variation threshold, a representative spectrum of the data group is calculated according to the measurement data in the data group; if the coefficient of variation corresponding to the data group is greater than or equal to the set variation threshold, the measurement data with the largest deviation in the data group is removed until the data group meets a set condition, and a representative spectrum of the data group is calculated according to the measurement data in the data group when the set condition is met; the set condition is that the coefficient of variation is less than the set variation threshold or the data amount in the data group is 2.

[0013] In an embodiment, the coefficient of variation comprises a remote sensing reflectivity coefficient of variation, a total radiation coefficient of variation, a sky light radiation coefficient of variation, and a solar irradiance coefficient of variation; and the representative spectrum comprises representative remote sensing reflectivity data, representative total radiation data, representative sky light radiation data, and representative solar irradiance data.

[0014] In an embodiment, the measurement data with the largest deviation in the data group is the measurement data with the largest distance from the average value of the data group.

[0015] In an embodiment, quality evaluation is performed on each data group respectively to obtain a quality identifier of each data group, specifically comprising: determining the number of measurements of each data group according to the measurement time in each data group respectively; for any data group, if the coefficient of variation corresponding to the data group is less than a set variation threshold, the quality identifier of the data group is 0; if the number of measurements of the data group is less than a set number threshold, the quality identifier of the data group is 1; and if the coefficient of variation corresponding to the data group is less than a set variation threshold, the quality identifier of the data group is 2.

[0016] In a second aspect, the present application provides a marine apparent optical measurement data integration device, comprising:

[0017] a data acquisition module configured to acquire a marine apparent optical measurement time sequence; the marine apparent optical measurement time sequence comprises a plurality of measurement times and measurement data corresponding to each measurement time;

[0018] a data division module, configured to divide the marine apparent optical measurement time sequence into a plurality of data groups according to measurement time;

[0019] a quality assessment module, configured to calculate a representative spectrum of each data group respectively, and perform quality assessment on each data group respectively to obtain a quality identifier of each data group;

[0020] a result output module, configured to output the representative spectrum and the quality identifier of each data group to a designated file.

[0021] In a third aspect, the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the marine apparent optical measurement data integration method.

[0022] In a fourth aspect, the present application provides a computer readable storage medium, having a computer program stored thereon, wherein the computer program is executable on a processor to implement the marine apparent optical measurement data integration method.

[0023] According to the embodiments provided in the present application, the present application has the following technical effects: the present application provides a marine apparent optical measurement data integration method, device, equipment and medium, which can automatically complete the whole process of data reading, grouping, representative spectrum calculation, quality assessment and result output, realizes automatic and rapid processing of massive apparent optical measurement data, improves the processing efficiency and accuracy of massive marine apparent optical measurement data, and the finally output representative spectrum and quality identifier can be directly used for satellite calibration or verification, which significantly improves the satellite remote sensing data correction accuracy and application efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0025] Figure 1 FIG. 1 is a diagram of an application environment of a marine apparent optical measurement data integration method according to an embodiment of the present application.

[0026] Figure 2 FIG. 2 is a flowchart of a marine apparent optical measurement data integration method according to an embodiment of the present application.

[0027] Figure 3 FIG. 3 is a functional module diagram of a marine apparent optical measurement data integration device according to an embodiment of the present application.

[0028] Figure 4 Fig. 1 is a structural schematic diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0030] In order to make the above objectives, characteristics and advantages of the present application more apparent, further specific embodiments of the present application will be described in detail below with reference to the drawings and specific embodiments.

[0031] The ocean apparent optical measurement data integration method provided by the embodiments of the present application can be applied in an application environment as shown in Figure 1 The terminal 101 communicates with the server 102 through a network. The data storage system can store data required to be processed by the server 102. The data storage system can be separately arranged, integrated on the server 102, placed on the cloud or other servers. The terminal 101 can send the ocean apparent optical measurement time series to the server 102. After receiving the ocean apparent optical measurement time series, the server 102 divides the ocean apparent optical measurement time series into multiple data groups according to the measurement time; calculates the representative spectrum of each data group respectively, and respectively performs quality assessment on each data group to obtain the quality identifier of each data group; and outputs the representative spectrum and the quality identifier of each data group to a designated file. The server 102 can feed back the obtained designated file to the terminal 101. In addition, in some embodiments, the ocean apparent optical measurement data integration method can also be realized by the server 102 or the terminal 101 alone.

[0032] The terminal 101 can be, but is not limited to, various desktop computers, notebook computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle-mounted device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The server 102 can be realized by an independent server or a server cluster composed of multiple servers, and can also be a cloud server.

[0033] In an exemplary embodiment, as shown in Figure 2As shown, a marine apparent optical measurement data integration method is provided, which is executed by a computer device, specifically, can be executed by a terminal or a server, or both. In the embodiments of the present application, the method is applied to Figure 1 The server 102 in the method 100 is taken as an example for illustration, including the following steps 201 to 204.

[0034] In step 201, a marine apparent optical measurement time series is acquired.

[0035] The marine apparent optical measurement time series includes a plurality of measurement times and measurement data corresponding to each measurement time. The measurement data includes geographic location, environmental parameters, remote sensing reflectance data of each wave band, total radiation data of each wave band, sky light radiation data of each wave band, and solar irradiance data of each wave band.

[0036] In step 202, the marine apparent optical measurement time series is divided into a plurality of data groups according to the measurement times.

[0037] In a specific application example, the marine apparent optical measurement time series is divided into a plurality of groups according to a preset time interval threshold, ensuring that the measurement data in each group is collected within a relatively concentrated time range. Step 202 includes the following steps 21 to 23.

[0038] In step 21, the time difference between adjacent measurement times in the marine apparent optical measurement time series is calculated. The measurement times are converted into timestamp format before calculating the time difference.

[0039] In step 22, the measurement data with a time difference greater than the preset time interval threshold is taken as a grouping boundary.

[0040] In step 23, the marine apparent optical measurement time series is divided into a plurality of data groups based on the grouping boundary. Each data group contains a plurality of continuous measurement data within a time period.

[0041] In step 203, the representative spectrum of each data group is calculated, and the quality of each data group is evaluated to obtain a quality identifier of each data group.

[0042] In a specific application example, for each data group, the remote sensing reflectance data, total radiation data, sky light radiation data, and solar irradiance data are evaluated for quality, and the representative spectrum is calculated. Step 203 includes the following steps 31 to 34.

[0043] In step 31, the coefficient of variation of the measurement data in each data group within a specified wave band range is calculated: ; wherein, is the coefficient of variation, a standard deviation of the measurement data in the data set, an average value or a median value of the measurement data in the data set.

[0044] Specifically, the coefficient of variation includes a remote sensing reflectance coefficient of variation, a total radiation coefficient of variation, a sky light radiation coefficient of variation, and a solar irradiance coefficient of variation. That is, the remote sensing reflectance coefficient of variation, the total radiation coefficient of variation, the sky light radiation coefficient of variation, and the solar irradiance coefficient of variation of each data set are calculated respectively. In the coefficient of variation formula, corresponding to a remote sensing reflectance standard deviation, a total radiation standard deviation, a sky light radiation standard deviation, and a solar irradiance standard deviation in the data set respectively. In the coefficient of variation formula, corresponding to a remote sensing reflectance average value or a median value, a total radiation average value or a median value, a sky light radiation average value or a median value, and a solar irradiance average value or a median value in the data set respectively.

[0045] Step 32, for any data set, if the coefficient of variation corresponding to the data set is less than a set variation threshold, a representative spectrum of the data set is calculated according to the measurement data in the data set.

[0046] Specifically, the representative spectrum includes representative remote sensing reflectance data, representative total radiation data, representative sky light radiation data, and representative solar irradiance data. The representative remote sensing reflectance data is an average value or a median value of the remote sensing reflectance data in the data set, the representative total radiation data is an average value or a median value of the total radiation data in the data set, the representative sky light radiation data is an average value or a median value of the sky light radiation data in the data set, and the representative solar irradiance data is an average value or a median value of the solar irradiance data in the data set.

[0047] Step 33, if the coefficient of variation corresponding to the data set is greater than or equal to the set variation threshold, the most deviated measurement data in the data set is removed until the data set meets a set condition, and a representative spectrum of the data set is calculated according to the measurement data in the data set when the set condition is met. The set condition is that the coefficient of variation is less than the set variation threshold or the data amount in the data set is 2.

[0048] Among them, the most deviated measurement data in the data set is the measurement data farthest from the average value of the data set. For example, a data set represents measurement data within 10 minutes, and 100 remote sensing reflectances are collected within 10 minutes. The coefficient of variation of the 100 remote sensing reflectances collected within 10 minutes is calculated, and the data farthest from the average value in the 100 remote sensing reflectances is removed.

[0049] Step 34, the number of measurements of each data set is determined according to the measurement time in each data set respectively.

[0050] Step 35, for any data set, if the coefficient of variation corresponding to the data set is less than a set variation threshold, the quality identification of the data set is 0. If the number of measurements of the data set is less than a set number threshold, the quality identification of the data set is 1. If the coefficient of variation corresponding to the data set is less than a set variation threshold, the quality identification of the data set is 2.

[0051] Step 204, output the representative spectrum and quality identification of each data set to a designated file. The satellite can be calibrated or verified based on the representative spectrum and quality identification in the file.

[0052] Specifically, the representative spectrum output to the designated file includes representative remote sensing reflectance data of the complete waveband, representative remote sensing reflectance data of the designated waveband, representative total radiation data of the designated waveband, representative sky radiation data of the designated waveband, and representative solar irradiance data of the designated waveband.

[0053] The present application automatically completes the whole process of data reading, grouping, outlier elimination, representative spectrum calculation, quality evaluation, and result output, realizes automatic and rapid processing of massive apparent optical measurement data, and ensures the reliability of the representative spectrum and improves the accuracy of data processing through coefficient of variation calculation and outlier elimination. The parameters such as time interval threshold, coefficient of variation threshold, and waveband range can be customized to obtain the representative spectrum of a specific time and meet the needs of different scenarios.

[0054] The present application also provides an application scenario of the marine apparent optical measurement data integration method. Specifically, the marine apparent optical measurement data integration method provided in the embodiment can be applied in the calibration scenario of a dedicated marine color satellite. In the calibration scenario of the dedicated marine color satellite, the long-time sequence of massive data of the dedicated marine color satellite verification field is grouped, representative spectrum is calculated, quality is evaluated, and results are output. Finally, the dedicated marine color satellite is calibrated or verified according to the output results, thereby improving the correction accuracy and application efficiency of the dedicated marine color satellite remote sensing data.

[0055] Based on the same inventive concept, the present embodiment also provides a marine apparent optical measurement data integration device for implementing the marine apparent optical measurement data integration method described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, and therefore the specific limitations in one or more marine apparent optical measurement data integration device embodiments provided below can refer to the limitations of the marine apparent optical measurement data integration method described above, which will not be repeated here.

[0056] In one exemplary embodiment, as Figure 3As shown, a marine apparent optical measurement data integration device is provided, comprising a data acquisition module 301, a data division module 302, a quality assessment module 303 and a result output module 304.

[0057] The data acquisition module 301 is configured to acquire a marine apparent optical measurement time sequence. The marine apparent optical measurement time sequence comprises a plurality of measurement times and measurement data corresponding to each measurement time.

[0058] The data division module 302 is configured to divide the marine apparent optical measurement time sequence into a plurality of data groups according to the measurement times.

[0059] The quality assessment module 303 is configured to calculate a representative spectrum of each data group respectively, and perform quality assessment on each data group respectively to obtain a quality identifier of each data group.

[0060] The result output module 304 is configured to output the representative spectrum and the quality identifier of each data group to a designated file.

[0061] In an exemplary embodiment, a computer device can be provided, which can be a server or a terminal. An internal structure diagram of the computer device can be as shown in Figure 4 The computer device comprises a processor, a memory, an input / output interface (I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is configured to store a marine apparent optical measurement time sequence. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through network connection. The computer program is executed by the processor to implement a marine apparent optical measurement data integration method.

[0062] Those skilled in the art can understand that Figure 4 the structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can comprise more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0063] In an exemplary embodiment, a computer readable storage medium storing a computer program is provided, the computer program being executed by a processor to implement the steps in the above method embodiments.

[0064] In an exemplary embodiment, a computer program product is provided, comprising a computer program being executed by a processor to implement the steps in the above method embodiments.

[0065] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0066] In the present application, all actions of obtaining signals, information or data are performed under the premise of complying with the corresponding data protection regulations and policies of the country where the device is located, and under the premise of obtaining authorization from the owner of the corresponding device.

[0067] Those skilled in the art can understand that all or part of the processes in the above method embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above method embodiments. In the embodiments provided by the present application, any reference to a memory, database or other medium can include at least one of a non-volatile and volatile memory. The non-volatile memory can include a read-only memory (ROM), a magnetic tape, a floppy disk, a flash memory, an optical storage, a high-density embedded non-volatile memory, a resistive memory (ReRAM), a magnetoresistive random access memory (MRAM), a ferroelectric memory (FRAM), a phase change memory (PCM), a graphene memory, etc. The volatile memory can include a random access memory (RAM) or an external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0068] The database involved in each of the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a blockchain, and the like, without being limited thereto. The processor involved in each of the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, and the like, without being limited thereto.

[0069] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, but it should be considered that any combination of the technical features is within the scope of the present disclosure, as long as there is no contradiction.

[0070] The principles and implementation manners of the present application are described by using specific examples herein, and the above embodiments are only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation manners and application ranges can be changed according to the idea of the present application. In summary, the content of the present description should not be understood as a limitation of the present application.

Claims

1. A method for integrating marine surface observatory measurement data, characterized in that, The method comprises: acquiring a marine apparent optical measurement time sequence; the marine apparent optical measurement time sequence comprises a plurality of measurement times and measurement data corresponding to each measurement time; dividing the marine apparent optical measurement time sequence into a plurality of data groups according to the measurement times; calculating a representative spectrum of each data group respectively, and performing quality assessment on each data group respectively to obtain a quality identifier of each data group; outputting the representative spectrum and the quality identifier of each data group to a designated file; wherein, the representative spectrum of each data group is calculated respectively, specifically comprising: calculating the coefficient of variation of the measurement data in a designated waveband range in each data group respectively; the coefficient of variation comprises a remote sensing reflectivity coefficient of variation, a total radiation coefficient of variation, a sky light radiation coefficient of variation and a solar irradiance coefficient of variation; for any data group, if the coefficient of variation corresponding to the data group is less than a set variation threshold, the representative spectrum of the data group is calculated according to the measurement data in the data group; the representative spectrum comprises representative remote sensing reflectivity data, representative total radiation data, representative sky light radiation data and representative solar irradiance data; if the coefficient of variation corresponding to the data group is greater than or equal to the set variation threshold, the measurement data with the largest deviation in the data group is removed until the data group meets the set condition, and the representative spectrum of the data group is calculated according to the measurement data in the data group when the data group meets the set condition; the set condition is that the coefficient of variation is less than the set variation threshold or the data amount in the data group is 2; wherein, the quality assessment on each data group is performed respectively to obtain the quality identifier of each data group, specifically comprising: determining the number of measurements of each data group according to the measurement time in each data group; for any data group, if the coefficient of variation corresponding to the data group is less than the set variation threshold, the quality identifier of the data group is 0; if the number of measurements of the data group is less than a set number threshold, the quality identifier of the data group is 1; if the coefficient of variation corresponding to the data group is less than the set variation threshold, the quality identifier of the data group is 2.

2. The method of ocean apparent optical measurement data integration of claim 1, wherein, The measurement data comprises geographical position, environmental parameters, remote sensing reflectivity data of each waveband, total radiation data of each waveband, sky light radiation data of each waveband and solar irradiance data of each waveband.

3. The method of ocean apparent optical measurement data integration of claim 1, wherein, The marine apparent optical measurement time sequence is divided into a plurality of data groups according to the measurement times, specifically comprising: calculating the time difference between adjacent measurement times in the marine apparent optical measurement time sequence; taking the measurement data with a time difference greater than a preset time interval threshold as a grouping boundary; dividing the marine apparent optical measurement time sequence into a plurality of data groups based on the grouping boundary.

4. The method of ocean apparent optical measurement data integration of claim 1, wherein, The measurement data with the largest deviation in the data group is the measurement data with the largest distance from the average value of the data group.

5. An ocean apparent optical measurement data integration apparatus, characterized by, The device performs the marine apparent optical measurement data integration method in any one of claims 1-4, and the device comprises: a data acquisition module, configured to acquire a marine apparent optical measurement time sequence; the marine apparent optical measurement time sequence comprises a plurality of measurement times and measurement data corresponding to each measurement time; a data division module, configured to divide the ocean apparent optical measurement time series into a plurality of data groups according to measurement time; a quality assessment module, configured to calculate a representative spectrum of each data group respectively, and to perform quality assessment on each data group respectively to obtain a quality identification of each data group; a result output module, configured to output the representative spectrum and the quality identification of each data group to a designated file.

6. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the ocean apparent optical measurement data integration method of any one of claims 1-4.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the ocean apparent optical measurement data integration method of any one of claims 1-4.

Images