An efficient method for collecting and storing multi-source heterogeneous experimental identification data based on JSON

Abstract

This invention relates to a method for efficient acquisition and storage of multi-source heterogeneous experimental identification data based on JSON, comprising: designing a real-time telemetry data transmission interface based on a JSON-based telemetry parameter transmission protocol and a JSON-based electronic form template data transmission interface on a data acquisition and storage server; a real-time telemetry data sending terminal converting the real-time telemetry data to be transmitted into JSON format and sending it to the data acquisition and storage server; the data acquisition and storage server receiving the JSON-formatted real-time telemetry data, decoding it, and classifying and saving the decoded real-time telemetry data into a database; the data acquisition and storage server pushing the designed electronic form template in JSON format to each data acquisition terminal; the data acquisition terminal importing non-real-time telemetry data into the electronic form template and pushing it to the data acquisition and storage server in JSON format; and the data acquisition and storage server automatically identifying the received JSON-formatted electronic form results, extracting valid data, classifying the data, and saving them separately into the database.

Description

Technical Field

[0001] This invention relates to a method for efficient acquisition and storage of multi-source heterogeneous test and evaluation data based on JSON, belonging to the field of spacecraft test and evaluation technology. Background Technology

[0002] Before launch, spacecraft must complete the procedures and requirements stipulated in the status qualification test outline to assess whether key indicators and boundary performance meet the overall development requirements. The data generated during qualification tests comes from diverse sources, including on-board and ground data, and various data generated during launch site tests. Due to differences in the development units, the formats, types, and protocols of the on-board and ground data generated by different spacecraft vary and are not standardized. The test qualification report obtained after the qualification tests is the main basis for judging the qualification results. The generation of the report mainly comes from the analysis and interpretation of the qualification data. Efficiently collecting and storing this test qualification data from different units is a crucial foundation for evaluating spacecraft launch site test qualification missions.

[0003] Currently, the test and evaluation data generated by spacecraft at the launch site are mainly collected, summarized, organized, analyzed, and judged through relevant software.

[0004] The main shortcomings are: (1) The test identification data format and protocol are not uniform, and the data granularity and content provided by various units are different. Manual processing is required, which is a lot of work; (2) Data acquisition depends on manual labor, which is inefficient and prone to human error, resulting in data distortion; (3) The automatic identification capability of the collected data is insufficient, and it is difficult to process large amounts of data. Summary of the Invention

[0005] The purpose of this invention is to provide an efficient method for collecting and storing multi-source heterogeneous test identification data based on JSON, so as to solve the problems of collection, storage, processing and extraction caused by inconsistent test specimen identification data formats from different units, and at the same time improve the management efficiency of multi-source heterogeneous data.

[0006] To address the aforementioned technical problems, this invention provides a method for efficient acquisition and storage of multi-source heterogeneous experimental identification data based on JSON, comprising:

[0007] Step S1: Design a real-time telemetry data transmission interface based on the JSON telemetry parameter transmission protocol and a JSON electronic form template data transmission interface on the data acquisition and storage server; wherein, the real-time telemetry data transmission interface based on the JSON telemetry parameter transmission protocol is responsible for receiving and processing real-time telemetry data transmitted from different real-time telemetry data sending terminals; the JSON electronic form template data transmission interface is responsible for receiving and processing non-real-time telemetry data transmitted from different data acquisition terminals.

[0008] Step S2: The real-time telemetry data sending terminal converts the real-time telemetry data to be transmitted into JSON format according to the JSON-based telemetry parameter transmission protocol and sends it to the data acquisition and storage server.

[0009] Step S3: The data acquisition and storage server receives real-time telemetry data in JSON format through the real-time telemetry data transmission interface, decodes it according to the JSON-based telemetry parameter transmission protocol, and categorizes and saves the decoded real-time telemetry data into the telemetry engineering value and source code table in the database.

[0010] Step S4: The data collection and storage server pushes the designed electronic form template to each data collection terminal in JSON format;

[0011] Step S5: The data acquisition terminal imports the non-real-time telemetry data into the electronic form template and pushes it to the data acquisition and storage server in JSON format;

[0012] Step S6: The data acquisition and storage server automatically identifies the received JSON-formatted electronic form results through the JSON-based electronic form template data transmission interface, extracts valid data, classifies the data, and saves it to the electronic form filling data table and file table in the database, and establishes electronic form information index and data association.

[0013] Optionally, the JSON-based telemetry parameter transmission protocol includes: model code, data type, time, subsystem code, telemetry parameter code, parameter engineering value, parameter source code value, and whether the limit is exceeded;

[0014] Optional basic elements of an electronic form template include: single-line text, multi-line text, radio button group box, checkbox group box, time picker, date picker, drop-down selection box, text, file, image, subform, editor, cascading selector, grid layout, table layout, tabs, divider, custom area, and custom components;

[0015] Optionally, step S4 may also include designing an electronic form template based on the data type of the non-real-time telemetry test and identification to be collected, generating electronic form template JSON data, and saving it to the electronic form template table in the database.

[0016] Optionally, step S7 may be included after step S6:

[0017] After the data collection and storage for the test and evaluation task are completed, the test and evaluation report will be automatically generated.

[0018] Optionally, step S7 includes:

[0019] Based on the data items specified in the report template, the system automatically retrieves valid data from the telemetry engineering values ​​and source code table, electronic form template table, electronic form data entry table, and file table, inserts it into the corresponding position in the report, and finally generates the test evaluation report.

[0020] The advantages of this invention compared to the prior art are:

[0021] (1) This invention standardizes the protocol interface for multi-source heterogeneous experimental data acquisition and storage. Based on the protocol, it can automatically extract and structure the original experimental data for identification, obtaining data that meets the requirements. This avoids omissions and also improves the efficiency of data integration and extraction.

[0022] (2) Based on JSON syntax, data formatting and parsing are faster, the syntax is standardized, and it has good compatibility, adapting to text, tables, images, data, audio, files, etc. It can be compatible with various types of data from different units, realizing the collection, storage, and management of all experimental and evaluation data. Attached Figure Description

[0023] Figure 1 This is a flowchart of the real-time telemetry data acquisition and storage process provided by the present invention;

[0024] Figure 2 This is a flowchart of the non-real-time telemetry data acquisition and storage (electronic form) provided by the present invention;

[0025] Figure 3 This is a flowchart of the automatic data extraction and test evaluation report generation process provided by the present invention. Detailed Implementation

[0026] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a more detailed explanation of the efficient data acquisition and storage method for multi-source heterogeneous experimental identification based on JSON proposed in this invention. The advantages and features of this invention will become clearer from the following description and claims. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of this invention.

[0027] This invention provides an efficient method for acquiring and storing multi-source heterogeneous experimental identification data based on JSON, the process of which is as follows: Figure 1 , Figure 2 As shown, it includes the following steps:

[0028] Step S1: On the data acquisition and storage server, design a real-time telemetry data transmission interface based on the JSON-based telemetry parameter transmission protocol and a JSON-based electronic form template data transmission interface according to the characteristics of the data source. The real-time telemetry data transmission interface based on the JSON-based telemetry parameter transmission protocol is responsible for receiving and processing real-time telemetry data transmitted from different real-time telemetry data sending terminals; the JSON-based electronic form template data transmission interface is responsible for receiving and processing non-real-time telemetry data transmitted from different data acquisition terminals, including text, tabular data, images, and files. The JSON-based telemetry parameter transmission protocol includes: model code, data type, time, subsystem code, telemetry parameter code, parameter engineering value, parameter source code value, and whether limits are exceeded. The basic elements of the electronic form template include: single-line text, multi-line text, radio button group, checkbox group, time picker, date picker, dropdown selection box, text, file, image, subform, editor, cascading selector, grid layout, table layout, tabs, dividers, custom areas, and custom components. A well-designed electronic form contains information about the data to be collected, such as: project name, test location, test date, subsystem, person in charge, test results (images, text, tables, attachments, etc.), test conclusions, and person in charge.

[0029] (1) Example of design for real-time telemetry data transmission interface based on JSON telemetry parameter transmission protocol:

[0030]

[0031] Attribute description:

[0032] Table 1. Attribute Table Based on JSON Telemetry Parameter Transmission Protocol

[0033] name logo Data types Remark Model code satid String Data types type String time date String Subsystem subsys String Telemetry parameters pars array See Table 2 for details.

[0034] Table 2 Telemetry Parameter Attribute Table

[0035] name logo Data types Example description Telemetry parameter code parid String TMV001 Engineering value v String 89.66 Source code value y String 0X01FF Is it exceeding the limit? n String N: No limit exceeded; Y: Limit exceeded

[0036] (2) Design of JSON-based electronic form template data transmission interface

[0037] Data example:

[0038]

[0039]

[0040]

[0041] Attribute description:

[0042] Table 3. Attribute Table of JSON-based Electronic Form Template Data Transmission Interface

[0043]

[0044] Table 4. Description of the options attribute

[0045] name illustrate Detailed information width Label width percentage defaultValue default value required Is this field required? requiredMessage Error message when required fields dataType Data type to be validated dataTypeCheck Should data type be validated? dataTypeMessage Error messages for data type validation pattern Regular expressions patternCheck Should regular expression validation be used? patternMessage Error messages from regular expression validation placeholder placeholder content customClass Custom class disabled Disable labelWidth Tag width isLabelWidth Is the label width valid? hidden Hide dataBind Is it data-bound? showPassword Should I show password? remoteFunc function id remoteOption option id

[0046] Table 5. Description of config attributes

[0047]

[0048] (3) Database Design

[0049] The database structure includes tables of telemetry engineering values ​​and source code, electronic form templates, electronic form data, and file tables.

[0050] Table 6 Telemetry Engineering Values ​​and Source Code Table

[0051] Fields type name Remark _id ObjectId Primary key t Datetime time p String Parameter code d List Engineering Value List y List Source code list max Float Maximum value min Float Minimum value count Int quantity last Float Last value

[0052] Table 7 Electronic Form Templates

[0053] Fields type name Remark _id ObjectId Primary key name String Form Name data Json Form structure JSON data publish Int Yes 1 No 0 Release type Int Form types 0 General, 1 Special Subform create_time Datetime Creation time create_user ObjectId Create User ID update_time Datetime Last updated update_user ObjectId Last updated operator

[0054] Table 8. Data Table for Electronic Form Submission

[0055]

[0056]

[0057] Table 10 Document Table

[0058]

[0059] Step S2: The real-time telemetry data sending terminals of each unit convert the real-time telemetry data to be transmitted into JSON format according to the JSON-based telemetry parameter transmission protocol and send it to the data acquisition and storage server.

[0060] Step S3: The data acquisition and storage server receives real-time telemetry data in JSON format through the real-time telemetry data transmission interface, decodes it according to the JSON-based telemetry parameter transmission protocol, and categorizes and saves the decoded real-time telemetry data into the telemetry engineering value and source code table in the database.

[0061] Step S4: Design an electronic form template based on the data type (text, table, image, file, etc.) of the non-real-time telemetry test to be collected, generate electronic form template JSON data, and save it to the electronic form template table in the database; the data acquisition and storage server pushes the designed electronic form template to each data acquisition terminal in JSON format;

[0062] Step S5: Each unit fills in its own test and evaluation data at the data acquisition terminal, and after approval, pushes the result electronic form JSON data back to the data acquisition and storage server.

[0063] Step S6: The data collection and storage server will automatically identify the received JSON format electronic form results, extract valid data, classify and store the data, and save it to the electronic form data table and file table in the database respectively, and establish electronic form information index and data association.

[0064] Step S7: After the test evaluation data collection and storage are completed, the automatic test evaluation report generation process is executed. Based on the data items specified in the report template, the system automatically retrieves valid data from telemetry engineering values ​​and source code tables, electronic form template tables, electronic form data entry tables, and file tables, inserts it into the corresponding positions in the report, and finally generates the test evaluation report.

[0065] like Figure 3 As shown, the process of generating the test evaluation report is as follows:

[0066] A. Select the electronic template used for the test and evaluation task, and load the template elements;

[0067] B. Based on the definition, traverse each element of the template, call the data extraction interface, and obtain the report results. The first element type is text and image information stored in the database, which is directly read from the database or file and inserted into the report; the second element type is electronic forms, from which valid data is extracted and filled into the corresponding positions in the report; the third element type is manually filled in; the fourth element type is externally imported files, whose contents are directly inserted into the report; the fifth element type is the stage envelope comparison results based on real-time telemetry data, which needs to be automatically placed into the spacecraft health status test and evaluation record in the report according to the subsystem classification.

[0068] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.

Claims

1. An efficient acquisition and storage method for multi-source heterogeneous test identification data based on JSON, characterized in that, The application relates to a JSON-based telemetry parameter transmission protocol and a JSON-based electronic form template data transmission interface. Step S1: designing a JSON-based telemetry parameter transmission protocol real-time telemetry data transmission interface and a JSON-based electronic form template data transmission interface on a data collection server; wherein the JSON-based telemetry parameter transmission protocol real-time telemetry data transmission interface is responsible for receiving and processing real-time telemetry data transmitted from different real-time telemetry data sending terminals; and the JSON-based electronic form template data transmission interface is responsible for receiving and processing non-real-time telemetry data transmitted from different data collection terminals; Step S2: the real-time telemetry data sending terminal converts the real-time telemetry data to be transmitted into a JSON format according to the JSON-based telemetry parameter transmission protocol, and sends the real-time telemetry data to the data collection server; Step S3: the data collection server receives the JSON format real-time telemetry data through the real-time telemetry data transmission interface, decodes the real-time telemetry data according to the JSON-based telemetry parameter transmission protocol, and classifies and saves the decoded real-time telemetry data into a telemetry engineering value and source code table in a database; Step S4: the data collection server pushes the designed electronic form template in a JSON format to each data collection terminal; Step S5: the data collection terminal imports the non-real-time telemetry data into the electronic form template, and pushes the non-real-time telemetry data to the data collection server in a JSON format; Step S6: the data collection server automatically identifies the received JSON format electronic form result through the JSON-based electronic form template data transmission interface, extracts valid data, classifies the data, and respectively saves the data into an electronic form filling data table and a file table in the database, and establishes an electronic form information index and data correlation; The JSON-based telemetry parameter transmission protocol comprises a model code, a data type, a time, a subsystem code, a telemetry parameter code, a parameter engineering value, a parameter source code value and whether the parameter is out of limit.

2. The method of claim 1, wherein, The electronic form template basic elements comprise a single-line text, a multi-line text, a single selection group box, a multi-selection group box, a time selector, a date selector, a drop-down selection box, a character, a file, a picture, a sub-form, an editor, a cascading selector, a grid layout, a table layout, a tab page, a split line, a self-defined area and a self-defined component.

3. The method of claim 2, wherein, Step S4 further comprises designing an electronic form template according to the type of non-real-time telemetry test identification data to be collected, generating electronic form template JSON data, and saving the electronic form template JSON data into an electronic form template table in a database.

4. The method of claim 3, wherein, After step S6, step S7 is further included: after the data collection of the test identification task is completed, a test identification report automatic generation process is performed.

5. The method of claim 4, wherein, The step S7 comprises: according to the data items defined in the report template, automatically obtaining valid data from the telemetry engineering value and source code table, the electronic form template table, the electronic form filling data table and the file table, inserting the valid data into corresponding positions of the report, and finally generating a test identification report.

Images