Ocean apparent optical measurement data integration method, device, equipment and medium

Abstract

The invention discloses a marine apparent optical measurement data integration method, device and equipment and a medium, and relates to the field of marine data processing, and the method comprises the steps: obtaining a marine apparent optical measurement time sequence; the ocean apparent optical measurement time sequence comprises a plurality of measurement times and measurement data corresponding to each measurement time; dividing the ocean apparent optical measurement time sequence into a plurality of data groups according to measurement time; respectively calculating a representative spectrum of each data group, and respectively carrying out quality evaluation on each data group to obtain a quality identifier of each data group; and outputting the representative spectrum and the quality identifier of each data group to a specified file. According to the invention, the processing efficiency and precision of mass ocean apparent optical measurement data are improved.

Description

Technical Field

[0001] This application relates to the field of marine data processing, and in particular to a method, apparatus, equipment and medium for integrating marine surface geometry data. Background Technology

[0002] Oceanographic data is the core foundation for satellite ocean remote sensing calibration and verification. Current technologies for processing oceanographic time series measurements largely rely on directly acquiring raw data without grouping and integrating continuous time series data according to measurement time. This results in data redundancy and disorder. Furthermore, the lack of representative spectral calculations for grouped data makes it impossible to extract key optical information. Additionally, the absence of data quality assessment makes it difficult to determine data validity and guarantee accuracy for subsequent satellite calibration or verification, failing to meet the practical application requirements for precise correction of satellite remote sensing data. Summary of the Invention

[0003] The purpose of this application is to provide a method, apparatus, equipment, and medium for integrating marine geophysical measurement data, which can improve the processing efficiency and accuracy of massive marine geophysical measurement data.

[0004] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a method for integrating marine surface geophysical measurement data, including: Obtain a time series of oceanographic measurements; the oceanographic measurement time series includes multiple measurement times and the measurement data corresponding to each measurement time. Based on the measurement time, the marine surface astronomical measurement time series is divided into multiple data groups; Calculate the representative spectrum for each data set, and perform a quality assessment for each data set to obtain a quality label for each data set. Output the representative spectra and quality labels for each data set to the specified file.

[0005] In one embodiment, the measurement data includes geographical location, environmental parameters, remote sensing reflectance data for each band, total radiation data for each band, sky radiation data for each band, and solar irradiance data for each band.

[0006] In one embodiment, the oceanographic measurement time series is divided into multiple data groups according to the measurement time, specifically including: calculating the time difference between adjacent measurement times in the oceanographic measurement time series; using measurement data with a time difference greater than a preset time interval threshold as grouping boundaries; and dividing the oceanographic measurement time series into multiple data groups based on the grouping boundaries.

[0007] In one embodiment, the representative spectrum of each data group is calculated, specifically including: calculating the coefficient of variation of the measured data in each data group within a specified band range; for any data group, if the coefficient of variation corresponding to the data group is less than a set variation threshold, then the representative spectrum of the data group is calculated based on the measured data in the data group; if the coefficient of variation corresponding to the data group is greater than or equal to the set variation threshold, then the measured data with the largest deviation in the data group is removed until the data group meets a set condition, and then the representative spectrum of the data group is calculated based on the measured data in the data group when the set condition is met; the set condition is that the coefficient of variation is less than the set variation threshold or the amount of data in the data group is 2.

[0008] In one embodiment, the coefficient of variation includes the coefficient of variation of remote sensing reflectance, the coefficient of variation of total radiation, the coefficient of variation of sky radiation, and the coefficient of variation of solar irradiance; the representative spectrum includes representative remote sensing reflectance data, representative total radiation data, representative sky radiation data, and representative solar irradiance data.

[0009] In one embodiment, the measurement data with the largest deviation in the data set is the measurement data that is furthest from the average value of the data set.

[0010] In one embodiment, quality assessment is performed on each data set to obtain a quality identifier for each data set. Specifically, this includes: determining the number of measurements for each data set based on the measurement time in each data set; for any data set, if the coefficient of variation corresponding to the data set is less than a set variation threshold, the quality identifier of the data set is 0; if the number of measurements for the data set is less than a set number threshold, the quality identifier of the data set is 1; if the coefficient of variation corresponding to the data set is less than a set variation threshold, the quality identifier of the data set is 2.

[0011] Secondly, this application provides a marine surface geophysical measurement data integration device, comprising: The data acquisition module is used to acquire the oceanographic measurement time series; the oceanographic measurement time series includes multiple measurement times and the measurement data corresponding to each measurement time; The data partitioning module is used to divide the oceanographic measurement time series into multiple data groups according to the measurement time. The quality assessment module is used to calculate the representative spectrum of each data group and perform a quality assessment on each data group to obtain a quality label for each data group. The results output module is used to output the representative spectra and quality labels of each data set to a specified file.

[0012] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described method for integrating marine surface observatory measurement data.

[0013] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method for integrating marine surface observatory measurement data.

[0014] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a method, apparatus, equipment and medium for integrating marine epigeometry data, which can automatically complete the entire process of data reading, grouping, representative spectrum calculation, quality assessment and result output, realize automated and rapid processing of massive epigeometry data, improve the processing efficiency and accuracy of massive marine epigeometry data, and the final output of representative spectrum and quality label can directly serve satellite calibration or verification, significantly improving the accuracy and application efficiency of satellite remote sensing data correction. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is an application environment diagram of a marine surface observatory measurement data integration method according to an embodiment of this application.

[0017] Figure 2 This is a flowchart illustrating a method for integrating marine surface observatory measurement data, provided as an embodiment of this application.

[0018] Figure 3 This is a schematic diagram of the functional modules of a marine surface aerometry measurement data integration device provided in an embodiment of this application.

[0019] Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0021] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] The marine surface observatory data integration method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 101 communicates with server 102 via a network. A data storage system can store the data that server 102 needs to process. The data storage system can be set up independently, integrated into server 102, or placed in the cloud or on another server. Terminal 101 can send the oceanographic measurement time series to server 102. After receiving the oceanographic measurement time series, server 102 divides the oceanographic measurement time series into multiple data groups according to the measurement time; calculates the representative spectrum of each data group, and performs a quality assessment on each data group to obtain a quality identifier for each data group; and outputs the representative spectrum and quality identifier of each data group to a designated file. Server 102 can then send the obtained designated file back to terminal 101. Furthermore, in some embodiments, the oceanographic measurement data integration method can also be implemented independently by server 102 or terminal 101.

[0023] The terminal 101 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 102 can be implemented using a standalone server or a server cluster composed of multiple servers, or it can be a cloud server.

[0024] In one exemplary embodiment, such as Figure 2 As shown, a method for integrating marine surface observatory measurement data is provided. This method is executed by computer equipment, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 Taking server 102 as an example, the explanation includes the following steps 201 to 204.

[0025] Step 201: Obtain the time series of oceanographic measurements.

[0026] The marine surface spectral measurement time series includes multiple measurement times and the corresponding measurement data for each measurement time. The measurement data includes geographical location, environmental parameters, remote sensing reflectance data for each band, total radiation data for each band, sky radiation data for each band, and solar irradiance data for each band.

[0027] Step 202: Divide the marine visual astronomical measurement time series into multiple data groups according to the measurement time.

[0028] In a specific application example, the oceanographic measurement time series is divided into multiple groups according to a preset time interval threshold, ensuring that the measurement data within each group are collected within a relatively concentrated time range. Step 202 includes steps 21 to 23.

[0029] Step 21: Calculate the time difference between adjacent measurement times in the marine visual survey time series. Before calculating the time difference, convert the measurement times into timestamp format.

[0030] Step 22: Use measurement data with a time difference greater than a preset time interval threshold as group boundaries.

[0031] Step 23: Divide the oceanographic measurement time series into multiple data groups based on grouping boundaries. Each data group contains multiple consecutive measurement data within a certain period.

[0032] Step 203: Calculate the representative spectrum of each data group and perform a quality assessment on each data group to obtain the quality label of each data group.

[0033] In a specific application example, for each data set, the quality of the remote sensing reflectance data, total radiance data, sky radiance data, and solar irradiance data is assessed, and their representative spectra are calculated. Step 203 includes steps 31 to 34.

[0034] Step 31: Calculate the coefficient of variation of the measured data in each data set within the specified band range: ;in, The coefficient of variation is 1. The standard deviation of the measurement data in the data set. This represents the average or median of the measured data in the data set.

[0035] Specifically, the coefficients of variation include the coefficient of variation for remotely sensed reflectance, the coefficient of variation for total radiation, the coefficient of variation for sky radiation, and the coefficient of variation for solar irradiance. That is, this application calculates the coefficients of variation for remotely sensed reflectance, total radiation, sky radiation, and solar irradiance for each data set separately. The coefficient of variation formula... These correspond to the standard deviations of remote sensing reflectance, total radiance, sky radiance, and solar irradiance in the data set, respectively. The coefficient of variation formula... These correspond to the average or median remote sensing reflectance, the average or median total radiation, the average or median sky radiation, and the average or median solar irradiance in the data set, respectively.

[0036] Step 32: For any data set, if the coefficient of variation corresponding to the data set is less than a set variation threshold, then calculate the representative spectrum of the data set based on the measurement data in the data set.

[0037] Specifically, the representative spectrum includes representative remote sensing reflectance data, representative total radiance data, representative sky radiance data, and representative solar irradiance data. The representative remote sensing reflectance data is the average or median of the remote sensing reflectance data in the data set; the representative total radiance data is the average or median of the total radiance data in the data set; the representative sky radiance data is the average or median of the sky radiance data in the data set; and the representative solar irradiance data is the average or median of the solar irradiance data in the data set.

[0038] Step 33: If the coefficient of variation corresponding to the data set is greater than or equal to a set variation threshold, then the measurement data with the largest deviation in the data set is removed until the data set meets the set conditions. Then, the representative spectrum of the data set is calculated based on the measurement data in the data set when the set conditions are met. The set conditions are that the coefficient of variation is less than the set variation threshold or the number of data points in the data set is 2.

[0039] In this context, the measurement data with the largest deviation in the data set is the measurement data that is furthest from the average value of the data set. For example, if a data set represents measurement data within 10 minutes, and 100 remote sensing reflectance values ​​were collected within 10 minutes, then the coefficient of variation of the 100 remote sensing reflectance values ​​collected within 10 minutes is calculated, and the data with the largest distance from the average value among the 100 remote sensing reflectance values ​​is removed.

[0040] Step 34: Determine the number of measurements for each data group based on the measurement time in each data group.

[0041] Step 35: For any data set, if the coefficient of variation corresponding to the data set is less than a set variation threshold, then the quality identifier of the data set is 0. If the number of measurements for the data set is less than a set number threshold, then the quality identifier of the data set is 1. If the coefficient of variation corresponding to the data set is less than the set variation threshold, then the quality identifier of the data set is 2.

[0042] Step 204: Output the representative spectra and quality labels for each data set to a specified file. The satellite can then be further calibrated or verified based on the representative spectra and quality labels in the file.

[0043] Specifically, the representative spectra output to the specified file include representative remote sensing reflectance data for the entire band, representative remote sensing reflectance data for the specified band, representative total radiance data for the specified band, representative sky radiance data for the specified band, and representative solar irradiance data for the specified band.

[0044] This application automatically completes the entire process of data reading, grouping, outlier removal, representative spectrum calculation, quality assessment, and result output. It achieves automated and rapid processing of massive spectroscopic optical measurement data. Furthermore, through coefficient of variation calculation and outlier removal, it ensures the reliability of representative spectra and improves the accuracy of data processing. It allows for customization of parameters such as time interval threshold, coefficient of variation threshold, and band range to obtain representative spectra at specific times, meeting the needs of different scenarios.

[0045] This application also provides an application scenario in which the above-described method for integrating oceanographic survey data is applied. Specifically, the oceanographic survey data integration method provided in this embodiment can be applied in the calibration scenario of a dedicated ocean color satellite. In the calibration scenario of a dedicated ocean color satellite, by grouping, calculating representative spectra, assessing quality, and outputting results from massive amounts of long-term data from the dedicated ocean color satellite's verification field, the dedicated ocean color satellite is ultimately calibrated or verified based on the output results, thereby improving the correction accuracy and application efficiency of the dedicated ocean color satellite remote sensing data.

[0046] Based on the same inventive concept, this application also provides a marine ophthalmometry data integration device for implementing the above-described method for integrating marine ophthalmometry data. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of one or more embodiments of the marine ophthalmometry data integration device provided below can be found in the limitations of the marine ophthalmometry data integration method described above, and will not be repeated here.

[0047] In one exemplary embodiment, such as Figure 3As shown, a marine surface astronomical measurement data integration device is provided, comprising: a data acquisition module 301, a data division module 302, a quality assessment module 303, and a result output module 304.

[0048] The data acquisition module 301 is used to acquire the oceanographic measurement time series. The oceanographic measurement time series includes multiple measurement times and the corresponding measurement data for each measurement time.

[0049] The data partitioning module 302 is used to divide the oceanographic measurement time series into multiple data groups according to the measurement time.

[0050] The quality assessment module 303 is used to calculate the representative spectrum of each data group and perform a quality assessment on each data group to obtain a quality label for each data group.

[0051] The results output module 304 is used to output the representative spectra and quality labels of each data set to a specified file.

[0052] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 4 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores time series data from oceanographic surveys. The I / O interfaces facilitate information exchange between the processor and external devices. The communication interface allows communication with external terminals via a network connection. When executed by the processor, the computer program implements a method for integrating oceanographic survey data.

[0053] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0054] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0055] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0056] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0057] In this application, all actions to acquire signals, information, or data are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with the authorization granted by the owner of the relevant device.

[0058] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0059] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0060] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0061] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for integrating marine surface observatory measurement data, characterized in that, The method includes: Obtain a time series of oceanographic measurements; the oceanographic measurement time series includes multiple measurement times and the measurement data corresponding to each measurement time. Based on the measurement time, the marine surface astronomical measurement time series is divided into multiple data groups; Calculate the representative spectrum for each data set, and perform a quality assessment for each data set to obtain a quality label for each data set. Output the representative spectra and quality labels for each data set to the specified file.

2. The method for integrating marine visual geophysical measurement data according to claim 1, characterized in that, The measurement data includes geographical location, environmental parameters, remote sensing reflectance data for each band, total radiation data for each band, sky radiation data for each band, and solar irradiance data for each band.

3. The method for integrating marine visual geophysical measurement data according to claim 1, characterized in that, Based on the measurement time, the marine surface geophysical measurement time series is divided into multiple data groups, specifically including: Calculate the time difference between adjacent measurement times in the aforementioned marine surface observatory measurement time series; Measurement data with a time difference greater than a preset time interval threshold are used as group boundaries; The oceanographic measurement time series is divided into multiple data groups based on the grouping boundaries.

4. The method for integrating marine visual geophysical measurement data according to claim 1, characterized in that, Calculate the representative spectrum for each data set separately, specifically including: Calculate the coefficient of variation of the measured data in each data set within the specified band range; For any data set, if the coefficient of variation corresponding to the data set is less than a set variation threshold, then the representative spectrum of the data set is calculated based on the measurement data in the data set. If the coefficient of variation corresponding to the data set is greater than or equal to the set variation threshold, the measurement data with the largest deviation in the data set is removed until the data set meets the set conditions. Then, the representative spectrum of the data set is calculated based on the measurement data in the data set when the set conditions are met. The set conditions are that the coefficient of variation is less than the set variation threshold or the amount of data in the data set is 2.

5. The method for integrating marine visual geophysical measurement data according to claim 4, characterized in that, The coefficients of variation include the coefficient of variation of remote sensing reflectance, the coefficient of variation of total radiation, the coefficient of variation of sky light radiation, and the coefficient of variation of solar irradiance. The representative spectra include representative remote sensing reflectance data, representative total radiation data, representative sky radiation data, and representative solar irradiance data.

6. The method for integrating marine visual geophysical measurement data according to claim 4, characterized in that, The measurement data with the largest deviation in the data set is the measurement data that is furthest from the average value of the data set.

7. The method for integrating marine visual survey data according to claim 4, characterized in that, A quality assessment was performed on each data set to obtain a quality identifier for each data set, specifically including: Determine the number of measurements for each data group based on the measurement time in each data group; For any data set, if the coefficient of variation corresponding to the data set is less than a set variation threshold, the quality identifier of the data set is 0; if the number of measurements of the data set is less than a set number threshold, the quality identifier of the data set is 1; if the coefficient of variation corresponding to the data set is less than a set variation threshold, the quality identifier of the data set is 2.

8. A device for integrating marine surface observatory measurement data, characterized in that, The apparatus performs the marine spectrophotometric data integration method according to any one of claims 1-7, and the apparatus comprises: The data acquisition module is used to acquire the oceanographic measurement time series; the oceanographic measurement time series includes multiple measurement times and the measurement data corresponding to each measurement time; The data partitioning module is used to divide the oceanographic measurement time series into multiple data groups according to the measurement time. The quality assessment module is used to calculate the representative spectrum of each data group and perform a quality assessment on each data group to obtain a quality label for each data group. The results output module is used to output the representative spectra and quality labels of each data set to a specified file.

9. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the marine surface observatory data integration method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the method for integrating marine surface observatory data as described in any one of claims 1-7.

Images