Spectral data inspection method and device

Abstract

The invention provides a spectral data inspection method and device, which can be applied to the field of data processing. The method comprises the following steps: in response to a spectral data inspection request, acquiring a universal field connected with each standard quality inspection database in a preset standard quality inspection database set; the preset standard quality inspection database set is established based on a preset quality inspection database set and a spectral data file stored in a directory, the preset quality inspection database set comprises a plurality of quality inspection databases which are independent and associated with one another, and each quality inspection database corresponds to one spectral data processing link; and according to the universal field, respectively carrying out cross query on each standard quality inspection database, and reversely positioning any link in spectral data processing links. In this way, the traceability mechanism is perfected, the labor cost is greatly reduced, and the detection efficiency, accuracy and coverage rate are improved.

Description

Technical Field

[0001] The present application relates to the field of data processing, specifically to the field of astronomical data processing, and more specifically to a spectral data inspection method, device, equipment, medium and program product. Background Art

[0002] In recent years, with the rapid development of astronomical observation technology and equipment, large-scale spectroscopic surveys have become an important means of astronomical research. Among them, the Large Sky Area Multi-Object Fiber Spectroscopic Telescope (LAMOST, also known as the Guo Shoujing Telescope), with its large field of view, high efficiency, and multi-target observation advantages, generates tens of millions of massive spectral data every year. These data cover different types of celestial bodies such as stars, galaxies, and quasars, and have been widely used in many fields such as Milky Way structure research, galaxy classification, and cosmology.

[0003] However, since LAMOST spectral data needs to go through multiple complex processing links from the initial observation and collection to the subsequent spectrum extraction, calibration, skylight reduction, data splicing, parameter measurement, product production and release, each link may lead to data quality problems due to factors such as instrument failure, environmental impact, and human error. In addition, due to the large scale of data, traditional manual inspection alone is difficult to complete quality control tasks efficiently and accurately. At present, the existing spectral data inspection methods mostly focus on simple file format verification, basic field verification and other basic content. There is a lack of comprehensive and consistency inspection methods for the entire data production chain, and no complete data traceability mechanism has been formed. As a result, the discovery of quality problems is delayed and tracing is difficult, seriously affecting the reliability and application value of scientific research data. Summary of the Invention

[0004] In view of the above problems, the present application provides a spectral data inspection method, apparatus, device, medium and program product that improve the efficiency and accuracy of spectral data inspection.

[0005] According to a first aspect of the present application, a spectral data checking method is provided, comprising:

[0006] In response to a spectral data inspection request, obtaining a common field connecting each standard quality inspection database in a preset standard quality inspection database set; wherein the preset standard quality inspection database set is established based on the preset quality inspection database set and spectral data files stored in a directory, the preset quality inspection database set includes a plurality of independent but interrelated quality inspection databases, each quality inspection database corresponding to a spectral data processing link;

[0007] According to the common fields, cross-query is performed on each standard quality inspection database to reversely locate any link in the spectral data processing process.

[0008] According to an embodiment of the present application, a preset standard quality inspection database set is established based on the preset quality inspection database set and the spectral data files stored in the directory, including:

[0009] Analyze each data table of the quality inspection database and the spectral data files stored in the directory according to the type of celestial body; wherein the celestial body type is obtained from the preset quality inspection database set, and the data table includes various data generated by the corresponding spectral data processing link;

[0010] Based on the analysis results, the preset quality inspection database set is modified to obtain the preset standard quality inspection database set.

[0011] According to an embodiment of the present application, each data table of the quality inspection database and the spectral data files stored in the directory are analyzed separately, including:

[0012] Statistical analysis is performed on each data table of the quality inspection database, and a first spectral data file stored in the directory is compared with the first data table of the analyzed corresponding quality inspection database; wherein the first spectral data file is associated with the celestial body type, and the first data table is associated with the celestial body type.

[0013] According to an embodiment of the present application, the method further includes:

[0014] Based on a preset standard quality inspection database set, a data quality inspection is performed on the spectral data file; wherein, one spectral data file corresponds to a data table and a directory in a standard quality inspection database, and one directory corresponds to one spectral data processing link.

[0015] According to an embodiment of the present application, a data quality check is performed on a spectral data file based on a preset standard quality check database set, including:

[0016] Counting the number of spectral data files in the corresponding directory, and comparing the number of files in each directory obtained by counting with the number of data tables in the standard quality inspection database corresponding to the directory;

[0017] The spectral data file is compared with a data table in a standard quality check database corresponding to the field names of the spectral data file, and the content of the spectral data file is verified according to the type of the field content corresponding to the field name.

[0018] According to an embodiment of the present application, verifying the content of the spectral data file according to the type of field content corresponding to the field name includes:

[0019] If the type of the field content corresponding to the field name is numeric, the field content of the spectral data file corresponding to the field name and the field content of the data table corresponding to the standard quality inspection database corresponding to the field name are converted to floating point types respectively, and the values within the preset floating point precision range are compared;

[0020] If the type of the field content corresponding to the field name is a string type, remove the spaces at both ends of the field content of the spectral data file corresponding to the field name and the spaces at both ends of the field content of the data table in the standard quality inspection database corresponding to the field name, and compare the field contents after the spaces are removed.

[0021] According to an embodiment of the present application, the method further includes:

[0022] Obtaining a standard quality inspection database corresponding to a preset field from a preset standard quality inspection database set, and generating a visual chart based on the characteristics of each data in the standard quality inspection database corresponding to the preset field;

[0023] The spectral data measurement results are checked using a visual chart; wherein the visual chart includes at least one of a scatter plot, a histogram, a density plot, and an error distribution plot.

[0024] According to an embodiment of the present application, the spectral data processing step includes:

[0025] Spectral data acquisition link, spectral data product generation link, spectral data product analysis and measurement link, drawing link and data release link.

[0026] A second aspect of the present application provides a spectral data inspection device, comprising:

[0027] a spectral data inspection response module, configured to, in response to a spectral data inspection request, obtain a common field connecting each standard quality inspection database in a preset standard quality inspection database set; wherein the preset standard quality inspection database set is established based on the preset quality inspection database set and spectral data files stored in a directory, and the preset quality inspection database set includes a plurality of independent yet interrelated quality inspection databases, each quality inspection database corresponding to a spectral data processing link;

[0028] The traceability module of the spectral data processing link is used to cross-query each standard quality inspection database based on the common fields and reversely locate any link in the spectral data processing link.

[0029] The third aspect of the present application provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0030] The fourth aspect of the present application further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the computer program or instructions are executed by a processor.

[0031] The fifth aspect of the present application further provides a computer program product, comprising a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0032] One or more of the above embodiments have the following beneficial effects: (1) a complete traceability mechanism is provided, which facilitates the rapid and accurate positioning of any link in the spectral data processing process, especially when tracing back problematic spectral data, the specific link and source of the problem that generated the spectral data can be quickly found; (2) a full-process consistency check of spectral data from acquisition to final product release is realized, which greatly reduces the workload and time cost of manual inspection, improves inspection efficiency and inspection accuracy, and ensures the reliability of published data; (3) the standard quality inspection database covers data from the entire process from data acquisition to final product release, thereby improving the coverage of spectral data inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The above contents and other objects, features and advantages of the present application will become more apparent through the following description of the embodiments of the present application with reference to the accompanying drawings, in which:

[0034] Figure 1 Schematically illustrates an application scenario diagram of the spectral data inspection method, apparatus, device, medium, and program product according to an embodiment of the present application;

[0035] Figure 2 The flowchart of the spectral data inspection method according to an embodiment of the present application is schematically shown;

[0036] Figure 3 A schematic diagram of a method flow for establishing a preset standard quality inspection database set according to an embodiment of the present application is shown;

[0037] Figure 4 The following schematically shows a data quality checking method according to an embodiment of the present application;

[0038] Figure 5 A density diagram showing the signal-to-noise ratio and redshift velocity error of an A-type star in each wavelength band according to an embodiment of the present application is schematically shown;

[0039] Figure 6 A density diagram showing the relationship between the signal-to-noise ratio and redshift velocity in each band of an F-type star according to an embodiment of the present application is schematically shown;

[0040] Figure 7Schematically shows a redshift distribution histogram of a quasar according to an embodiment of the present application;

[0041] Figure 8 The following schematically shows a structural block diagram of a spectral data inspection device according to an embodiment of the present application;

[0042] Figure 9 The block diagram schematically shows an electronic device suitable for implementing the spectral data inspection method according to an embodiment of the present application. DETAILED DESCRIPTION

[0043] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present application. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present application. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present application.

[0044] The terms used herein are only for describing specific embodiments and are not intended to limit this application. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0045] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0046] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0047] The embodiments of the present application provide a spectral data inspection method, which cross-queries various standard quality inspection databases and provides a complete traceability mechanism. It can quickly and accurately locate any link in the spectral data processing link. When tracing back spectral data with problems, it can quickly find the specific link and source of the problem that generated the spectral data, thereby realizing the consistency inspection of the entire process from spectral data acquisition to final product release, greatly reducing the workload and time cost of manual inspection, improving inspection efficiency and inspection accuracy, and ensuring the reliability of released data. The standard quality inspection database covers data from the entire process from data acquisition to final product release, thereby improving the coverage rate of spectral data inspection.

[0048] Figure 1 The following schematically illustrates an application scenario of the spectral data inspection method according to an embodiment of the present application.

[0049] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or optical fiber cables.

[0050] A user may use a first terminal device 101, a second terminal device 102, or a third terminal device 103 to interact with a server 105 via a network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, or the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (for example only).

[0051] The first terminal device 101 , the second terminal device 102 , and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0052] The server 105 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process received data such as user requests, and feed back processing results (e.g., web pages, information, or data obtained or generated based on user requests) to the terminal devices.

[0053] It should be noted that the spectral data inspection method provided in the embodiment of the present application can generally be executed by the server 105. Accordingly, the spectral data inspection device provided in the embodiment of the present application can generally be set in the server 105. The spectral data inspection method provided in the embodiment of the present application can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the spectral data inspection device provided in the embodiment of the present application can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0054] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0055] The following will be based on Figure 1 The scene described by Figures 2 to 7 The spectral data inspection method according to an embodiment of the present application is described in detail.

[0056] Figure 2 The flowchart of the spectral data inspection method according to an embodiment of the present application is schematically shown.

[0057] like Figure 2 As shown, the spectral data checking method 200 of this embodiment includes operations S210 to S220.

[0058] In operation S210, in response to a spectral data inspection request, a common field connecting each standard quality inspection database in a preset standard quality inspection database set is obtained; wherein the preset standard quality inspection database set is established based on the preset quality inspection database set and the spectral data files stored in the directory, and the preset quality inspection database set includes multiple independent and interrelated quality inspection databases, and each quality inspection database corresponds to a spectral data processing link.

[0059] In operation S220 , a cross-query is performed on each standard quality inspection database according to the common field to reversely locate any link in the spectral data processing link.

[0060] In some embodiments, in operation S210, each quality inspection database includes at least one data table. Each data table includes various data generated by the corresponding spectral data processing step. These data have corresponding fields corresponding to them. The corresponding data in the corresponding data table can be searched by searching the field name. For example, during the spectral data collection stage, data such as the shooting start or end time, the type of the shot celestial body, the data storage path, the moon phase, the shooting duration, the shooting environmental conditions, the central magnitude of the shooting area, and the central coordinates may be generated. These data are stored in a data table named rawdata. Each data also has its own corresponding field name. For example, the field name corresponding to the shooting start time is expsta, the field name corresponding to the shooting end time is expend, the field name corresponding to the type of the shot celestial body is exptype, the field name corresponding to the data storage path is exppath, the field name corresponding to the moon is moonphase, the field name corresponding to the shooting environmental conditions may be windspeed, the field name corresponding to the shooting duration may be exptime, the field name corresponding to the central magnitude of the shooting area is cmag, and the field names corresponding to the central coordinates of the shooting area may be cra and cdec. cra and cdec, one represents the horizontal coordinate and the other represents the vertical coordinate; in the spectral data product generation stage, the data that may be generated include the input coordinates of the photographed celestial body, the apparent magnitude of the celestial body, the selected source type of the celestial body, the sorted selected source type, the selected source catalog, the existence information of the bias, the bias arc length, the number of exposures of the blue-end data, the number of exposures of the red-end data, and the stitching information. These data are stored in the data table named obj_info. Each data in the table has its own corresponding field name. For example, the field names corresponding to the input coordinates of the photographed celestial body are tra and tdec, tra and tdec, one represents the horizontal coordinate and the other represents the vertical coordinate. The field name corresponding to the apparent magnitude of the celestial body is mag, the field name corresponding to the selected source type of the celestial body is objtype, the field name corresponding to the sorted selected source type can be fibertype, the field name corresponding to the selected source catalog can be tfrom, the field name corresponding to the existence information of the bias can be offset, the field name corresponding to the bias arc length can be offset_v, the field name corresponding to the number of exposures of the blue-end data can be num_b, the field name corresponding to the number of exposures of the red-end data can be num_r, and the field name corresponding to the stitching information can be combine;During the analysis and measurement stage of spectral data, the data generated may include the Julian calendar of the shooting date, the signal-to-noise ratio, the classification of spectral data, the redshift of the celestial body spectrum, the best matching template of the spectrum, the measured atmospheric parameters, and the observed celestial body coordinates. These data are stored in a data table named spec_info. Each data in the table has its own corresponding field name. For example, the field name corresponding to the Julian calendar of the shooting date can be mjd or lmjd, the field name corresponding to the signal-to-noise ratio can be snr, the field name corresponding to the classification of spectral data can be class or subclass, the field name corresponding to the redshift of the celestial body spectrum can be z or zerr, the field name corresponding to the best matching template of the spectrum can be tcolumn, the field name corresponding to the measured atmospheric parameters can be teff or logg or feh, etc., and the field name corresponding to the observed celestial body coordinates can be obsra or obsdec. In order to associate the data tables, the common fields are used to connect the data tables, where the common fields can be the fields used to represent the unique ID of the spectral data. For example, the common fields can be The targetid field contains the unique identifier for the spectral data, the specid field contains the unique identifier for the spectral data and its data version number, the obsdate field contains the acquisition date, the planid field contains the acquisition area number, the spid field contains the spectrometer number used, and the fiberid field contains the fiber number. The sum of the obsdate, planid, spid, and fiberid fields equals the targetid field. Connecting various data tables using common fields facilitates querying data generated by different processing stages for the same spectral data in each data table based on field names. This allows for determining which stages of the spectral data have issues, facilitates rapid source tracing, and improves inspection efficiency.

[0061] In some embodiments, in operation S210, a corresponding quality inspection database is established according to the spectral data processing link, wherein the spectral data processing link includes a spectral data acquisition link, a spectral data product generation link, a spectral data product analysis and measurement link, a mapping link, and a data publishing link.

[0062] In some embodiments, in operation S210, data generated by different spectral data processing links are stored in different directories. The spectral data files in the directory correspond to the data tables in the corresponding processing links. The spectral data files include files in fits format, png format and csv format.

[0063] In some embodiments, in operation S210 , the common fields connecting the various standard quality inspection databases may be the same as the common fields connecting the various data tables mentioned above.

[0064] Figure 3 The following schematically shows a flow chart of a method for establishing a preset standard quality inspection data set according to an embodiment of the present application.

[0065] like Figure 3 As shown, the preset standard quality check database set is established based on the preset quality check database set and the spectral data files stored in the directory, including:

[0066] Operation S310 , analyzing the data tables of the quality inspection database and the spectral data files stored in the directory respectively according to the celestial body type; wherein the celestial body type is obtained from the preset quality inspection database set, and the data tables include various data generated by the corresponding spectral data processing link.

[0067] In operation S320 , based on the analysis result, the preset quality inspection database set is modified to obtain a preset standard quality inspection database set.

[0068] For example, first, the type of celestial body represented by each spectral data is obtained from the preset quality inspection database set, and then the data tables related to the classification process of each celestial body type are analyzed respectively. These data tables are data tables in the preset quality inspection database set to determine whether the spectral data representing the same type of celestial body are correct in different spectral data processing links. In addition, the spectral data files generated by each spectral data processing link stored in different directories are analyzed respectively. For the same type of celestial body, the differences between the data tables in the preset quality inspection database set related to the classification process of the type of celestial body and the spectral data files related to the classification process of the type of celestial body are compared to determine whether the classification of the celestial body is correct. If the spectral data representing the same type of celestial body are obtained in different spectral data processing links, If there is inconsistency between the data obtained or there is a large difference between the data table in the preset quality check database set and its corresponding spectral data file (that is, the celestial body type given in the data table does not match the celestial body type given in the corresponding spectral data file), the analysis results will be fed back to the user, and the user will manually check and correct the errors to obtain an accurate preset quality check database set, that is, the preset standard quality check database set. In the subsequent spectral data inspection process, the preset standard quality check database set is used as a benchmark to determine which processing link has a problem. Furthermore, the following command can be used to check the data in the data table named obj_info and the data table named spec_info with a shooting date of January 1, 2020. The command statement is as follows:

[0069] select planid,spid,count(*) from obj_info where obsdate=20200101group by planid,spid;

[0070] select planid,spid,count(*) from spec_info where obsdate=20200101group by planid,spid;

[0071] Among them, count represents the aggregate function, planid represents the first grouping field, and spid represents the second grouping field. The data with a shooting date of January 1, 2020 in the data table named obj_info are grouped according to the first and second grouping fields. The data with a shooting date of January 1, 2020 in the data table named spec_info are also grouped according to the first and second grouping fields. By comparing the data with a shooting date of January 1, 2020 in these two data tables, we can determine whether the data is correct, and thus know which spectral data processing link may have an error.

[0072] In some embodiments, in operation S310, each data table of the quality inspection database and the spectral data files stored in the directory are analyzed separately, including: performing statistical analysis on each data table of the quality inspection database, and comparing the first spectral data file stored in the directory with the first data table in the analyzed corresponding quality inspection database; wherein the first spectral data file is associated with the celestial body type, and the first data table is associated with the celestial body type.

[0073] For example, for a certain type of celestial body, statistical analysis is performed on the data tables related to the spectral data corresponding to the celestial body in the quality inspection database, mainly analyzing the relationship between the data in these data tables to determine whether there is a problem in the spectral data processing link corresponding to the celestial body and which specific link has the problem. In addition, the spectral data files stored in the directory and the data tables in the quality inspection database corresponding to the spectral data processing link corresponding to the directory are compared and analyzed, especially the first spectral data file in the directory and the first data table in the quality inspection database corresponding to the directory are compared and analyzed to determine whether the celestial body classification in the first spectral data file is correct, whether the celestial body classification in the first data table is correct, and whether the two celestial body classifications are consistent. The first spectral data file includes various data related to the celestial body classification, and the first data table includes various data related to the celestial body classification.

[0074] According to the embodiments of the present application, by establishing a preset standard quality inspection database set, the accuracy and reliability of the revised quality inspection database can be improved, thereby improving the traceability accuracy.

[0075] In some embodiments, method 200 further includes: performing data quality inspection on the spectral data file based on a preset standard quality inspection database set; wherein one spectral data file corresponds to one data table and one directory in one standard quality inspection database, and one directory corresponds to one spectral data processing link.

[0076] For example, the spectral data acquisition link generates a large amount of data, which is stored in the first directory. The formats of the spectral data files contained in the first directory may include fits format, png format and csv format, among which files in fits format are data files that store various types of data, files in png format are image files that store various types of images, and files in csv format are star catalog files that store star catalogs. Assume that the data such as the shooting time, shooting duration and the center coordinates of the shooting area stored in the first directory are stored in a spectral data file in fits format, and the standard quality inspection database corresponding to the spectral data acquisition link also stores these data. They may be stored in a data table corresponding to the first table name. If there is no error in the spectral data acquisition link, the data in the spectral data file storing data such as the shooting time, shooting duration and the center coordinates of the shooting area should be consistent with the data in the data table storing these data; further, for each spectral data processing link, the data generated by different spectral data processing links are stored in different directories, each directory includes at least one spectral data file, and the formats of the spectral data files include fits format, png format and csv format. The data generated by each spectral data processing link is also stored in the data table in the corresponding standard quality inspection database. Therefore, for the same spectral data processing link, the corresponding standard quality inspection database Each data table in the spectral data processing link corresponds to the spectral data file in the directory corresponding to the spectral data processing link. For example, in the spectral data product generation link, the data such as the apparent magnitude of the celestial body, the bias arc length, the number of exposures of the blue end and the red end data generated by the link are all stored in the same spectral data file. These data are also stored in a data table in the standard quality inspection database corresponding to the link. The spectral data file and the data table are in a one-to-one correspondence. The source star catalog generated by the link is stored in a csv file, and the source star catalog is also stored in another data table in the standard quality inspection database corresponding to the link. Then this csv file corresponds to the data table storing the source star catalog one-to-one. By comparing the spectral data files and the corresponding data tables, it can be determined whether there is a problem in the corresponding spectral data processing link. For example, when the data generated by the same spectral data processing link are stored in the corresponding directory and the standard quality inspection database respectively, if the data is not updated in the corresponding directory or the corresponding standard quality inspection database in a timely manner, or the data format conversion is incorrect, or the integrity of the data stored in the directory and the corresponding standard quality inspection database is different, etc., it may cause the data stored in the directory to be inconsistent with the data stored in the corresponding standard quality inspection database. The data format may be inconsistent, or the data content may be inconsistent. In this case, the cross-query method given in method 200 can be used to quickly locate the link where the problem occurs, thereby achieving traceability.

[0077] According to an embodiment of the present application, performing a quality check on the spectral data files in the directory can improve the comprehensiveness and accuracy of the check.

[0078] Figure 4 The following schematically shows a data quality checking method according to an embodiment of the present application.

[0079] like Figure 4 As shown, based on the preset standard quality check database set, the spectral data file is checked for data quality, including:

[0080] Operation S410 : Count the number of spectral data files in the corresponding directory, and compare the number of files in each directory obtained by counting with the number of data tables in the standard quality inspection database corresponding to the directory.

[0081] In operation S420 , the spectral data file is compared with a data table in a standard quality check database corresponding to the field names of the spectral data file, and the content of the spectral data file is verified according to the type of the field content corresponding to the field name.

[0082] For example, in a certain spectral data processing link (for example, the spectral data acquisition link or the spectral data product analysis and measurement link or the drawing link, etc.), the number of spectral data files in the directory corresponding to the link is counted, and the number of data tables in the standard quality inspection database corresponding to the link is counted, and the two numbers are compared. If they are consistent, it means that the data in the directory is not missing or redundant. If they are inconsistent, it means that the data in the directory is missing or redundant. By comparing the spectral data files in the directory of the link with their corresponding data tables, it can be known which spectral data files are missing and which are redundant, and then the missing files are supplemented and the redundant files are deleted. Furthermore, the format of the spectral data files in the directory corresponding to the spectral data acquisition link is generally FITs format, the format of the spectral data files corresponding to the intermediate processing link between the spectral data acquisition link and the spectral data product generation link is generally FITs format, and the format of the spectral data files corresponding to the spectral data product generation link is generally FITs format, PNG format and CSV format.

[0083] According to the embodiments of the present application, by performing a quantity check on the spectral data files, it is possible to determine whether the data stored in the directory is missing or redundant; by performing a content check on the spectral data files based on the field content type, the accuracy and efficiency of the spectral data check can be improved.

[0084] In some embodiments, the spectral data file content is verified according to the type of the field content corresponding to the field name, including: if the type of the field content corresponding to the field name is a numeric type, the field content of the spectral data file corresponding to the field name and the field content of the data table corresponding to the standard quality inspection database corresponding to the field name are converted into floating point types, and the values within a preset floating point precision range are compared; if the type of the field content corresponding to the field name is a string type, the spaces at both ends of the field content of the spectral data file corresponding to the field name and the spaces at both ends of the field content of the data table corresponding to the field name are removed, and the field contents after the spaces are removed are compared.

[0085] In some embodiments, the spectral data of each link has multiple fields to store the content corresponding to the corresponding spectral data, that is, the field content. These contents are not only stored in the corresponding spectral data file in the corresponding directory, but also stored in the corresponding data table of the corresponding standard quality inspection database. In order to determine whether the content in the spectral data file is consistent with the content in the corresponding data table, the spectral data file and the corresponding data table are determined according to the field name corresponding to the spectral data file, and then the content in the determined spectral data file is compared with the content in the determined data table according to the type of field content.

[0086] For example, for a spectral data file in the fits format, the field name of the file is stored in the fits header. The field name of the spectral data file corresponds to the table name of the data table in the standard quality inspection database. For field names of spectral data files with a field name length of more than 8 characters, it is necessary to preset the correspondence between the field name in the fits header and the table name of the data table in the standard quality inspection database. Based on the preset correspondence, the corresponding data table can be determined according to the field name of the spectral data file. If there is no problem in the spectral data processing link, the spectral data file that meets the preset correspondence should be consistent with the data stored in the data table, so the corresponding spectral data file content can be verified based on the field content to see if it is correct. Taking the spectral data file in the fits format as an example, if the field content corresponding to the fits file and the content in the corresponding data table are both numeric, the numeric data will be converted to floating-point data according to the preset precision, and the data in the data table corresponding to the fits file will also be converted to floating-point data with the same precision. For example, respectively The data in the fits file and the data in the corresponding data table are taken to 2 significant digits after the decimal point, and the remaining digits are discarded. The retained 2 significant digits after the decimal point are compared. If they are inconsistent, it means that the content of the fits file is incorrect; if the field content corresponding to the fits file and the content in the corresponding data table are both string data, the spaces at both ends of the string in the fits file and the spaces at both ends of the string in the corresponding data table are removed respectively, and then the remaining strings in the fits file and the corresponding data table are compared, and they are not case sensitive. If the two are inconsistent, it means that there is an error in the content of the fits file. Furthermore, this embodiment also presets some equivalent fields, for example, f-std is equivalent to obj, sky, unused, poserr and Non are equivalent, and some other equivalent fields can also be preset as needed; further, for the case of inconsistency in the above comparison, that is, the case where there is an error in the spectral data file, all inconsistent data are output to the log file for user inspection.

[0087] In some embodiments, for the field contents corresponding to commonly used fields such as the Julian day, coordinates in the equatorial coordinate system, target celestial body type, and full moon degree, the corresponding relationship between the fields and the inspection methods shown in Table 1 is used to perform content inspection on the spectral data files of the corresponding fields:

[0088] Table 1

[0089]

[0090] For example, the fields mjd and lmjd both represent the Julian day field. Check whether the difference between the field content corresponding to lmjd in the spectral data file and the field content corresponding to mjd is equal to 1 (recorded in Table 1 as "check whether lmjd-mjd=1 is true"), and at the same time check whether the difference between the field content corresponding to lmjd in the data table corresponding to the spectral data file and the field content corresponding to mjd is equal to 1. If the difference between the two fields in the spectral data file is 1 and the difference between the two fields in the corresponding data table is also 1, it means that the field content corresponding to the Julian day field of the spectral data file is the same as the field content corresponding to the Julian day field in the corresponding data table. Among them, the mjd field represents Greenwich Mean Time, and lmjd The d field indicates the local time; the fields cra, cdec, tra, and tdec respectively indicate the coordinates of the equatorial coordinate system. Specifically, the fields starting with c indicate the center coordinates of the entire sky area, and the fields starting with t indicate the coordinates of the target celestial body in this sky area. That is, the fields cra and cdec, one indicates the central abscissa of the entire sky area, and the other indicates the central ordinate of the entire sky area; the fields tra and tdec, one indicates the abscissa of the target celestial body in this sky area, and the other indicates the ordinate of the target celestial body in this sky area. Assuming that the maximum radius of the sky area is r, check whether the distance between the center coordinate position of the sky area and the coordinate position of the target celestial body in the sky area in the spectral data file is less than or equal to r. Table 1 records it as "check (tra-cra) 2 +(tdec-cdec) 2 <=r 2At the same time, check whether the distance between the center coordinate position of the sky area in the data table corresponding to the spectral data file and the coordinate position of the target celestial body in the sky area is less than or equal to r. If the coordinates in the equatorial coordinate system of the spectral data file and the corresponding data table satisfy the above relationship, it means that the spectral data file and its corresponding data table are consistent in the field contents corresponding to these fields; use the fields tra and tdec to represent the theoretical coordinates of the target celestial body in the sky area respectively, and the fields obsra and obsdec to represent the coordinates of the optical fiber pointing during actual observation respectively. For the fields offset and offset_v, through the field contents corresponding to the fields tra, tdec, obsra, obsdec, offset and offset_v in the spectral data file, judge whether the theoretical coordinates of the target celestial body in the spectral data file and the coordinates of the optical fiber pointing during actual observation are exactly the same, and through the field contents corresponding to the fields tra, tdec, obsra, obsdec, offset and offset_v in the data table corresponding to the spectral data file, judge whether the theoretical coordinates of the target celestial body in the data table and the coordinates of the optical fiber pointing during actual observation are exactly the same, so as to judge whether the spectral data Whether the coordinates of the optical fiber pointing during actual observation in the spectral data file and the corresponding data table are the same, further, whether the theoretical coordinates of the target celestial body in the spectral data file or the corresponding data table are exactly the same as the coordinates of the optical fiber pointing during actual observation is judged in the following way: if the field content corresponding to offset is 1, then the field content corresponding to tra should be different from the field content corresponding to obsra, or the field content corresponding to tdec should be different from the field content corresponding to obsdec, and the distance between the theoretical coordinates of the target celestial body in the same spectral data file or the same data table and the coordinates of the optical fiber pointing during actual observation should be equal to the square of the field content corresponding to the field offset_v in the corresponding spectral data file or data table. Only in this way can the theoretical coordinates of the target celestial body in the same spectral data file or the same data table be the same as the coordinates of the optical fiber pointing during actual observation. If the field content corresponding to offset is 0, then the theoretical coordinates of the target celestial body in the spectral data file or the corresponding data table are exactly the same as the coordinates of the optical fiber pointing during actual observation. The above content is recorded in Table 1 as "If offset=1, then tra≠obsra or tdec≠obsdec, and offset_v 2 =(tra-obsra) 2 +(tdec-obsdec) 2, if offset=0, then the theoretical coordinates of the target celestial body should be exactly the same as the coordinates pointed to by the optical fiber during actual observation"; the fields exptype, objtype, and fibertype respectively represent the types of target celestial bodies. If the field content corresponding to the field objtype in the spectral data file is consistent with the field content corresponding to obj in the corresponding data table, then the field content corresponding to the fibertype in the spectral data file should be consistent with the field content corresponding to the obj or f-std field in the corresponding data table. If the field content corresponding to the objtype in the spectral data file is consistent with the field content of other fields in the corresponding data table or is an empty value, then the field content corresponding to the fibertype in the spectral data file should not be consistent with the field content corresponding to obj in the corresponding data table and should not be consistent with the field content corresponding to f-std. Similarly, if the field content corresponding to the field objtype in the data table is consistent with the field content corresponding to obj in the corresponding spectral data file, then the field content corresponding to the fibertype in the data table should be consistent with the field content corresponding to the obj or f-std field in the corresponding spectral data file. If the field content corresponding to the objtype in the data table is consistent with the field content of other fields in the corresponding spectral data file or is an empty value, then The field content corresponding to fibertype should not be consistent with the field content corresponding to obj in the corresponding spectral data file and should not be consistent with the field content corresponding to f-std; remove the data whose field content corresponding to the combine field is 0 (abbreviated as combine=0) from the data table obj_info, and check whether the number of data in the data table obj_info and the data table spec_info is consistent when the field content of combine is 1. If they are consistent, it means that there is no problem with the data in the two data tables. If they are inconsistent, it means that there is a problem with the data in the two data tables. You can compare the two data tables with The corresponding spectral data file can be used to find the problematic data; the fields num_b and num_r can both represent the number of exposures. Obtain the number of original data (such as shooting time, shooting duration, etc.) in the data table obj_info. Calculate twice the product of the total number of spectral data contained in the data table in each subsequent processing link corresponding to the data table obj_info and the content of the field corresponding to num_b or num_r. Compare the two obtained values to see if they are equal. If not, it means that there is a problem in the data table obj_info or some links in the subsequent processing links. The problem can be found by comparing the data table with the spectral data file corresponding to the data table.The field moonphase indicates the degree of the full moon. It determines whether the field content of this field in the spectral data file matches the shooting date. At the same time, it determines whether the field content of this field in the data table corresponding to the spectral data file matches the shooting date. By comparing the field content of this field in the spectral data file and the field content of this field in the corresponding data table under the same shooting date, we can know whether the relationship between the shooting date and the magnitude corresponds. For example, if the shooting date is close to the full moon, the magnitude (mag) of the photographed celestial body should be brighter than 14. If it is another shooting date, darker stars can be photographed. The field tcolumn can be used to locate the template that best matches the spectral data. If the classification measurement result of a spectral data is found to have a problem after comparing the spectral data file with the corresponding data table, this field can be used to quickly locate the template that best matches the problematic spectral data, so as to correct the error based on the matched template. The field class indicates the celestial body type, and the field z indicates the redshift value. By comparing these two in the spectral data file The spectral data file contains the class field, which can be used to determine whether the object type given in the spectral data file is correct. Comparing these two fields in the data table corresponding to the spectral data file can determine whether the object type given in the data table is correct. Comparing the class field in the spectral data file with the z field in the corresponding data table, or comparing the z field in the spectral data file with the class field in the corresponding data table, can determine whether the object type given in the spectral data file and the corresponding data table are consistent. Furthermore, the relationship between the object type and the redshift value is as follows: if the class field indicates a star, the value of the z field is between ±0.04; if the class field indicates a galaxy, the value of the z field is less than 1; if the class field indicates a quasar (QSO), the value of the z field is less than 7, and the redshift value of non-stellar objects must not be less than 0. Furthermore, if there is data that exceeds the above limits, it should be output to a log file for manual inspection to correct errors or obtain new discoveries.

[0091] In some embodiments, cross-querying each standard quality inspection database is performed based on the common fields to reversely locate any link in the spectral data processing process, including:

[0092] Assume that the first standard quality inspection database stores data generated in the spectral data acquisition link, the second standard quality inspection database stores data generated in the spectral data product generation link, and the third standard quality inspection database stores data generated in the spectral data product analysis and measurement link. The first standard quality inspection database, the second standard quality inspection database and the third standard quality inspection database are connected through common fields. Assume that the common fields include targetid field, obsdate field, planid field, spid field, fiberid field and specid field. According to any common field or a combination of multiple common fields, any one or more standard quality inspection databases can be queried. Check the data in the database in quantity. For example, when problems are found in the analysis and measurement link of the spectral data product, the above general fields can be used to reversely locate the source of the problem. For example, through the general field search, it is found that there is a problem with the data generated in the spectral data acquisition link, so it is determined that the source of the problem occurs in the spectral data acquisition link; when there are no problems in each link, but a new type of celestial body is discovered, the source can also be traced based on the general field, and the characteristics of the data generated by each spectral data processing link corresponding to the new celestial body can be reversely located. For example, based on the combination of the obsdate field and the planid field, it is found that the data generated in the spectral data product generation link has special characteristics, and these special characteristics are marked so that subsequent researchers can explore them.

[0093] In some embodiments, method 200 also includes: obtaining a standard quality inspection database corresponding to a preset field from a preset standard quality inspection database set, generating a visualization chart based on the characteristics of each data in the standard quality inspection database corresponding to the preset field; using the visualization chart to check the spectral data measurement results; wherein the visualization chart includes at least one of a scatter plot, a histogram, a density plot and an error distribution plot.

[0094] For example, in order to draw a density map between the signal-to-noise ratio of each band of a celestial body and the redshift velocity error, a standard quality inspection database of fields corresponding to the spectral data product analysis and measurement link can be obtained from a preset standard quality inspection database set. For example, specifically, the content corresponding to the field named snr, the content corresponding to the field named z or zerr, etc. are obtained, and based on these obtained contents, a density map between the signal-to-noise ratio of each band of a celestial body and the redshift velocity error is drawn. Furthermore, Figure 5 The density diagram between the signal-to-noise ratio and redshift velocity error of A-type stars in each band according to the embodiment of the present application is schematically shown. Figure 5As shown in the figure, the horizontal axis represents the signal-to-noise ratio of each band of A-type stars, where the signal-to-noise ratio is represented by SNR, and the vertical axis represents the redshift velocity error (RV err) of A-type stars. Generally speaking, the higher the signal-to-noise ratio SNR, the smaller the RV err should be. The relationship between RV err and SNR should generally be similar to an inverse proportional function. Figure 5 It can be seen that the redshift velocity error distribution of A-type stars with a signal-to-noise ratio greater than 20 in each band is relatively normal. However, when the SNR is small, the RV err is smaller, which is inconsistent with the distribution law given by the theory, indicating that there may be problems with the spectral data of A-type stars. In this case, it is necessary to output the density map between the problematic signal-to-noise ratio and the redshift velocity error to the log file so that users can check it and find the problem.

[0095] For example, in order to draw a density map between the signal-to-noise ratio and redshift velocity of each band of a celestial body, a standard quality inspection database of fields corresponding to the spectral data product analysis and measurement link can be obtained from a preset standard quality inspection database set. For example, specifically, the content corresponding to the field named snr, the content corresponding to the field named z or zerr, etc. are obtained, and based on these obtained contents, a density map between the signal-to-noise ratio and redshift velocity of each band of the corresponding celestial body is drawn. Furthermore, Figure 6 The density diagram between the signal-to-noise ratio and redshift velocity of an F-type star in each band according to an embodiment of the present application is schematically shown. Figure 6 As shown, the horizontal axis represents the signal-to-noise ratio of each band of F-type stars, where the signal-to-noise ratio is represented by SNR, and the vertical axis represents the redshift velocity (RV) of F-type stars. Generally speaking, the larger the signal-to-noise ratio, the closer the star is, and the smaller the absolute value of the redshift velocity is. The smaller the signal-to-noise ratio, the dimmer the apparent brightness of the star is, and the farther the distance is, the greater the dispersion of the redshift velocity will be. Figure 6 It can be seen that when the signal-to-noise ratio is large, the absolute value of the redshift velocity is small, and when the signal-to-noise ratio is small, the dispersion of the redshift velocity is large, which is in line with the general law. It can be considered that the distribution of the redshift velocity of F-type stars with the signal-to-noise ratio conforms to the general law.

[0096] For example, in order to draw the redshift distribution histogram of celestial bodies, the standard quality inspection database of the fields corresponding to the spectral data product analysis and measurement link, the standard quality inspection database of the fields corresponding to the spectral data product generation link, and the standard quality inspection database of the fields corresponding to the spectral data acquisition link can be obtained from the preset standard quality inspection database set. For example, specifically, the content corresponding to the field named z or zerr is obtained, the content corresponding to the field named objtype is obtained, and the content corresponding to the fields named expsta and expend is obtained, etc., and the corresponding celestial body redshift distribution histogram is drawn based on these obtained contents. Further, Figure 7The redshift distribution histogram of quasars according to an embodiment of the present application is schematically shown. Figure 7 As shown, the horizontal axis represents the redshift value, and the vertical axis represents the number of quasars. Figure 7 In the figure, a trough appears where the redshift value is 1. There are fewer quasars here, and a "redshift desert" phenomenon appears. This may be because the characteristic position of the spectrum of the quasar with a redshift value of 1 is exactly at the junction of the red and blue end data, which makes it impossible to accurately identify the spectral characteristics, and then leads to missing a lot of data. Therefore, the cross-query method can be used to query the relevant standard quality inspection database, trace the relevant information of the red end data and the blue end data, to obtain the missed data, accurately identify the spectral characteristics, and further, Figure 7 The mean of the redshift distribution shown , mean square error , which means that the celestial body is approaching the observer, the absolute value of the average redshift value is 1, and a large mean square deviation means that the distribution of redshift values is more dispersed, and the redshift values fluctuate greatly around the mean.

[0097] In some embodiments, the visualization chart can also be a histogram of the difference in apparent velocity between the current version of release data corresponding to a celestial body and the previous version of release data corresponding to the same celestial body, or a histogram of changes in visible spectrum lines between different versions of release data, etc. Furthermore, the version mentioned here refers to the version of the data processing software. For example, the changes between different versions of data can be verified by the histogram of the radial velocity difference between the current version of the data corresponding to the celestial body and the previous version of the data corresponding to the same celestial body. Generally speaking, when the data is stable and the data processing software version is stable, the difference between the data released in the current version and the data released in the previous version of the same celestial body is very small. Within the allowable error range, it is normal for the difference to be almost 0. For example, assuming that spectral data with the same spectral ID are obtained by processing two different versions of data processing software, the two visible spectral lines (Hα and Hβ) in the spectral data of the corresponding version are determined according to the redshift values of each version, and the spectral intensities of Hα and Hβ in the same version are Gaussian fitted to obtain the corresponding Gaussian fitting curve. The means of the Gaussian fitting curves of different versions are compared, and a histogram of the mean is plotted and Gaussian fitted is performed on it. If the mean square error obtained by fitting is within the preset error range, it is considered normal. Otherwise, it is necessary to start from the initial link of the spectral data processing link and gradually find the cause of the error.

[0098] In some embodiments, method 200 further includes: for a system having multiple servers, method 200 can be used to perform distributed inspection of spectral data according to fields that uniquely identify spectral data, such as spectrometer number, shooting area number, or shooting date, so as to improve inspection efficiency. Specifically, each server uses method 200 to inspect the spectral data corresponding to the corresponding spectrometer number (or shooting area number, shooting date, etc.). Each server executes method 200 simultaneously, which can efficiently complete the data inspection task in real time. By using distributed parallel computing, computing resources are fully utilized, the response speed and processing capability of large-scale data inspection tasks are improved, and the growing data processing needs are met.

[0099] According to the embodiments of the present application, the distribution patterns and rationality of spectral data can be deeply evaluated using visual charts, which is conducive to timely discovery of potential abnormal data or new discoveries. For existing problem data, the problem data can be output to a log file to facilitate user inspection, which is conducive to manual verification, thereby improving inspection accuracy and reliability and timely discovering problems.

[0100] Based on the above spectral data inspection method, the present application also provides a spectral data inspection device. Figure 8 The device is described in detail.

[0101] Figure 8 The structural block diagram of the spectral data inspection device according to an embodiment of the present application is schematically shown.

[0102] like Figure 8 As shown, the spectral data inspection device 800 of this embodiment includes a spectral data inspection response module 810 and a spectral data processing link tracing module 820 .

[0103] Spectral data inspection response module 810 is configured to, in response to a spectral data inspection request, obtain common fields that connect to each standard quality inspection database in a preset standard quality inspection database set. The preset standard quality inspection database set is established based on the preset quality inspection database set and spectral data files stored in a directory. The preset quality inspection database set includes multiple independent yet interconnected quality inspection databases, each corresponding to a spectral data processing step. In one embodiment, spectral data inspection response module 810 can be configured to perform operation S210 described above, which will not be further described here. Furthermore, the spectral data processing step includes a spectral data acquisition step, a spectral data product generation step, a spectral data product analysis and measurement step, a plotting step, and a data publishing step.

[0104] The spectral data processing link traceability module 820 is used to perform cross-queries on each standard quality inspection database based on the common fields to reversely locate any link in the spectral data processing link. In one embodiment, the spectral data processing link traceability module 820 can be used to perform the operation S220 described above, and will not be repeated here.

[0105] In some embodiments, the spectral data inspection response module 810 is specifically configured to:

[0106] According to the type of celestial body, each data table of the quality inspection database and the spectral data files stored in the directory are analyzed respectively; wherein, the celestial body type is obtained from the preset quality inspection database set, and the data table includes various data generated by the corresponding spectral data processing link; based on the analysis results, the preset quality inspection database set is corrected to obtain the preset standard quality inspection database set.

[0107] In some embodiments, the spectral data inspection response module 810 is further configured to:

[0108] Statistical analysis is performed on each data table of the quality inspection database, and a first spectral data file stored in the directory is compared with the first data table of the analyzed corresponding quality inspection database; wherein the first spectral data file is associated with the celestial body type, and the first data table is associated with the celestial body type.

[0109] In some embodiments, the apparatus 800 is further configured to:

[0110] Based on a preset standard quality inspection database set, a data quality inspection is performed on the spectral data file; wherein, one spectral data file corresponds to a data table and a directory in a standard quality inspection database, and one directory corresponds to one spectral data processing link.

[0111] In some embodiments, the apparatus 800 is further configured to:

[0112] Count the number of spectral data files in the corresponding directory, and compare the number of files in each directory obtained by counting with the number of data tables in the standard quality inspection database corresponding to the directory; compare the spectral data files with the data tables in the standard quality inspection database corresponding to the field names of the spectral data files, and verify the content of the spectral data files according to the type of the field content corresponding to the field name.

[0113] In some embodiments, the apparatus 800 is further configured to:

[0114] If the type of the field content corresponding to the field name is numeric, the field content of the spectral data file corresponding to the field name and the field content of the data table corresponding to the standard quality inspection database corresponding to the field name are converted to floating-point types respectively, and the values within the preset floating-point precision range are compared; if the type of the field content corresponding to the field name is string type, the spaces at both ends of the field content of the spectral data file corresponding to the field name and the spaces at both ends of the field content of the data table corresponding to the field name are removed respectively, and the field contents after the spaces are removed are compared.

[0115] In some embodiments, the apparatus 800 is further configured to:

[0116] A standard quality inspection database corresponding to a preset field is obtained from a preset standard quality inspection database set, and a visualization chart is generated based on the characteristics of each data in the standard quality inspection database corresponding to the preset field; and the spectral data measurement results are checked using the visualization chart; wherein the visualization chart includes at least one of a scatter plot, a histogram, a density plot, and an error distribution plot.

[0117] According to the embodiments of the present application, the use of device 800 can greatly reduce the workload of manual inspection, improve data inspection efficiency and coverage, and improve the data traceability mechanism by using the cross-query method, which can quickly and accurately locate the link where the problem data is generated, greatly reducing the time and cost of data problem processing.

[0118] According to embodiments of the present application, any multiple modules in the spectral data inspection response module 810 and the spectral data processing phase traceability module 820 can be combined into a single module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in a single module. According to embodiments of the present application, at least one of the spectral data inspection response module 810 and the spectral data processing phase traceability module 820 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented in hardware or firmware through any other reasonable means of circuit integration or packaging, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or any appropriate combination of any of these. Alternatively, at least one of the spectral data inspection response module 810 and the spectral data processing phase traceability module 820 can be at least partially implemented as a computer program module that, when executed, performs the corresponding functions.

[0119] Figure 9The block diagram schematically shows an electronic device suitable for implementing the spectral data inspection method according to an embodiment of the present application.

[0120] like Figure 9 As shown, an electronic device 900 according to an embodiment of the present application includes a processor 901, which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 902 or programs loaded from a storage unit 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or related chipsets and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present application.

[0121] Various programs and data required for the operation of the electronic device 900 are stored in the RAM 903. The processor 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. The processor 901 performs various operations of the method flow according to the embodiment of the present application by executing the programs in the ROM 902 and / or the RAM 903. It should be noted that the programs may also be stored in one or more memories other than the ROM 902 and the RAM 903. The processor 901 may also perform various operations of the method flow according to the embodiment of the present application by executing the programs stored in the one or more memories.

[0122] According to an embodiment of the present application, electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to bus 904. Electronic device 900 may also include one or more of the following components connected to I / O interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 908 including a hard disk; and a communication section 909 including a network interface card such as a LAN card or modem. Communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to I / O interface 905 as needed. Removable media 911, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 910 as needed, so that computer programs read from the removable media can be installed into storage section 908 as needed.

[0123] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when the one or more programs are executed, the method according to the embodiments of this application is implemented.

[0124] According to an embodiment of the present application, a computer-readable storage medium may be a non-volatile computer-readable storage medium, and may include, for example, but not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present application, a computer-readable storage medium may include the ROM 902 and / or RAM 903 described above and / or one or more memories other than ROM 902 and RAM 903.

[0125] Embodiments of the present application also include a computer program product comprising a computer program containing program code for executing the method shown in the flowchart. When the computer program product is executed in a computer system, the program code is used to cause the computer system to implement the spectral data inspection method provided in the embodiments of the present application.

[0126] The computer program executes the above functions defined in the system / device of the embodiment of the present application when the processor 901 executes the computer program. According to the embodiment of the present application, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0127] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0128] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 909, and / or installed from a removable medium 911. When the computer program is executed by the processor 901, the above-mentioned functions defined in the system of the embodiment of the present application are performed. According to the embodiment of the present application, the systems, devices, means, modules, units, etc. described above can be implemented by computer program modules.

[0129] According to an embodiment of the present application, the program code for executing the computer program provided by the embodiment of the present application can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, languages such as Java, C++, Python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0130] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0131] Those skilled in the art will appreciate that the features described in the various embodiments of this application may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in this application. In particular, the features described in the various embodiments of this application may be combined and / or coupled in various ways without departing from the spirit and teachings of this application. All such combinations and / or couplings fall within the scope of this application.

Claims

1. A spectral data inspection method, characterized in that: The method comprises: In response to a spectral data inspection request, obtaining a common field connecting each standard quality inspection database in a preset standard quality inspection database set; wherein the preset standard quality inspection database set is established based on the preset quality inspection database set and spectral data files stored in a directory, the preset quality inspection database set includes a plurality of independent and interrelated quality inspection databases, each quality inspection database corresponding to a spectral data processing link; According to the common fields, cross-queries are performed on the standard quality inspection databases respectively to reversely locate any link in the spectral data processing link.

2. The method according to claim 1, characterized in that The preset standard quality inspection database set is established based on the preset quality inspection database set and the spectral data files stored in the directory, including: Analyze each data table of the quality inspection database and the spectral data files stored in the directory according to the type of celestial body; wherein the celestial body type is obtained from the preset quality inspection database set, and the data table includes various data generated by the corresponding spectral data processing link; Based on the analysis result, the preset quality inspection database set is modified to obtain a preset standard quality inspection database set.

3. The method according to claim 2, characterized in that The step of respectively analyzing each data table of the quality inspection database and the spectral data files stored in the directory comprises: Statistical analysis is performed on each data table of the quality inspection database, and a first spectral data file stored in a directory is compared with the first data table of the analyzed corresponding quality inspection database; wherein the first spectral data file is associated with the celestial body type, and the first data table is associated with the celestial body type.

4. The method according to claim 2, characterized in that The method further comprises: Based on the preset standard quality inspection database set, the spectral data file is subjected to data quality inspection; wherein, one spectral data file corresponds to one data table and one directory in one standard quality inspection database, and one directory corresponds to one spectral data processing link.

5. The method according to claim 4, characterized in that The step of performing a data quality check on the spectral data file based on the preset standard quality check database set includes: Counting the number of spectral data files in the corresponding directory, and comparing the number of files in each directory obtained by counting with the number of data tables in the standard quality inspection database corresponding to the directory; The spectral data file is compared with a data table in a standard quality inspection database corresponding to the field names of the spectral data file, and the content of the spectral data file is verified according to the type of the field content corresponding to the field name.

6. The method according to claim 5, characterized in that The verifying the content of the spectral data file according to the type of the field content corresponding to the field name includes: If the type of the field content corresponding to the field name is a numeric type, the field content of the spectral data file corresponding to the field name and the field content of the data table corresponding to the standard quality inspection database corresponding to the field name are converted to floating point types respectively, and the values within the preset floating point precision range are compared; If the type of the field content corresponding to the field name is a string type, remove the spaces at both ends of the field content of the spectral data file corresponding to the field name and the spaces at both ends of the field content of the data table in the standard quality inspection database corresponding to the field name, and compare the field contents after the spaces are removed.

7. The method according to claim 1, characterized in that The method further comprises: Obtaining a standard quality inspection database corresponding to a preset field from the preset standard quality inspection database set, and generating a visual chart according to the characteristics of each data in the standard quality inspection database corresponding to the preset field; The spectral data measurement results are checked using the visualization chart; wherein the visualization chart includes at least one of a scatter plot, a histogram, a density plot, and an error distribution plot.

8. The method according to claim 1, characterized in that The spectral data processing step includes: Spectral data acquisition link, spectral data product generation link, spectral data product analysis and measurement link, drawing link and data release link.

9. A spectral data inspection device, characterized in that: The device comprises: a spectral data inspection response module, configured to, in response to a spectral data inspection request, obtain a common field connecting each standard quality inspection database in a preset standard quality inspection database set; wherein the preset standard quality inspection database set is established based on the preset quality inspection database set and spectral data files stored in a directory, the preset quality inspection database set including a plurality of independent yet interrelated quality inspection databases, each quality inspection database corresponding to a spectral data processing link; The spectral data processing link traceability module is used to cross-query the standard quality inspection databases according to the common fields and reversely locate any link in the spectral data processing link.

10. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images