Task-oriented data tagging method and system and method for constructing test task

Abstract

The invention discloses a task-oriented data tagging method and system and a method for constructing a test task, and the data tagging method utilizes the internal relation and symbiosis characteristics between a data collection system and a data tag, and carries out modularization data collection and data processing processes. An endogenous automatic data labeling mechanism oriented to a data acquisition-processing process is constructed, and full-chain data labeling management of data from a source to an application is realized; on the basis, an existing mature BI tool is utilized to quickly construct a visual monitoring chart in a data flow type processing pipeline of a test system, and refined monitoring of key data indexes is achieved. According to the method and the system, an endogenous automatic data labeling mechanism oriented to a data acquisition-processing process is constructed through a modularized data acquisition and data processing process by utilizing internal relation and symbiosis characteristics between a data acquisition system and a data label, and full-chain data labeling management from a source to an application is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of data acquisition and processing, and specifically to a task-oriented data labeling method and system, and a method for constructing a test task. Background Art

[0002] As the characteristics of equipment informatization, systematization, intelligence, and precision become more and more obvious, the equipment test objects are expanding from single equipment to equipment systems, and the test environment is changing from a single simple environment to a complex environment, thus bringing about the transformation of the test system from closed isolation to heterogeneous integration. The increasing diversity and complexity of test tasks poses a dual challenge to the test system: on the one hand, it is urgent to enhance the "plug-and-play" component characteristics and fast and agile modular integration capabilities of the test system to ensure that various test systems can be flexibly spliced ​​to quickly respond to task changes and form customized solutions for different task requirements; on the other hand, it also puts forward higher requirements for the effective management and in-depth analysis and utilization of test data, requiring it to not only properly cope with the massive growth and diversity of data, but also to realize the rapid mining of data value and intelligent decision-making support under complex and changing task forms.

[0003] At present, various test systems lack componentization and data labeling mechanisms in the data collection and processing processes. The classification and storage of data rely on manual management and lack a unified labeling management system. In the process of data classification and storage management, there are often obvious differences in data structure and format due to different tasks and differences in business personnel's understanding of data. Furthermore, these significant differences hinder the cross-organizational or cross-disciplinary flow of data, the intelligent use of data, and the release of value. Summary of the invention

[0004] The purpose of the present invention is to overcome the above-mentioned shortcomings and provide a task-oriented data labeling method, system and method for constructing test tasks. The basic concept is to utilize the inherent connection and symbiotic characteristics between the data acquisition system and the data labels, and through componentized data acquisition and data processing processes, construct an endogenous automatic data labeling mechanism for the data acquisition-processing process, so as to realize the full-chain data labeling management from the source to the application; on this basis, using the existing mature BI tools, quickly construct visual monitoring charts in the data streaming processing pipeline of the test system, so as to realize the refined monitoring of key data indicators.

[0005] The technical solution of the present invention is:

[0006] A task-oriented data labeling method includes the following steps:

[0007] Step 1: Determine the required test system based on the test mission scenario (mission scenario refers to a series of preset scenarios, test conditions, test systems and related configurations designed to simulate the real combat environment and meet the purpose and requirements of equipment test missions) and identify the test systems for system components that have not yet been built in the system component design module;

[0008] Step 2: Analyze and componentize the data collection and data processing flow of the test system that has not yet built the collection and monitoring components in step 1, use the basic components provided in the system component management module to build its data processing components and save them to the data storage module for continued use in subsequent test tasks;

[0009] Step 3: According to the characteristics of the test system itself and the data management needs in step 2, a system data tag that is independent of the task is constructed, and the data tag is bound to the test system through the system component ID; the corresponding data tagging trigger is registered and generated in the data tag management module, and the data tagging is automatically triggered when the data "flows through" the system component, and the outflowing data is marked with the corresponding system data tag;

[0010] Step 4: Build task-independent component data tags based on the characteristics and data management needs of each basic component in the data processing process in step 2, and bind the data tags to the corresponding basic component configuration data; register and generate the corresponding data tagging trigger in the data tag management module, and automatically trigger data tagging when the data "flows through" the basic component and is directly "persistently" stored or directly passed to other system components. The component data that meets the tagging conditions are marked with the corresponding component data tags;

[0011] Step 5: Analyze the key indicators that need to be monitored in real time during the actual operation of the test system in step 2, use the visual BI design module to build visual real-time monitoring instruments for these key indicators, and integrate them into the corresponding test system visual BI monitoring components), and bind the constructed visual BI monitoring components to the system components through the system component ID;

[0012] Step 6: According to the experimental task scenario, select the test system components required for the experimental task from the system component library, integrate these system components into a test data collection-processing task instance (hereinafter referred to as task instance) for the experimental task in the task design module according to the scenario, and save it to the storage module;

[0013] Step 7: According to the actual needs of real-time monitoring of the test task, select the monitoring charts corresponding to the indicators that need to be focused on from the system components, and use the visual BI building module combination to build a real-time data monitoring screen for the corresponding test task instance.

[0014] Furthermore, in the step one, the system components of the test system are composed of a data acquisition and processing component and an indicator monitoring component, wherein the data acquisition and processing component is generated and managed by the system component management module, and is responsible for accessing and parsing the data generated by the test system and performing preliminary data processing; the indicator monitoring component is generated and managed by the visual BI design module, and is responsible for drawing charts of various key indicators that need to be monitored in the test system.

[0015] Furthermore, in the step 2, the basic components provided in the system component design module should include: data source component, network connection component, data protocol parsing component, data conversion component, data filtering component, time synchronization component, data encapsulation component, and custom programming data processing component.

[0016] Furthermore, in the step 2, all basic components follow strictly defined interface specifications, have clearly defined inputs and outputs, and through the series and parallel integration and configuration of the basic components, together constitute the "streaming" architecture data acquisition and processing components of the corresponding test system.

[0017] Furthermore, in step 2, the implementation steps corresponding to the test system data acquisition and processing component storage are:

[0018] (1) The system component management module generates a unique identifier (i.e., system component ID);

[0019] (2) Save the configuration metadata of the basic components involved in its data acquisition and processing components (i.e., basic component configuration data), and establish an association relationship between each configuration metadata and its corresponding system component instance through the system component ID;

[0020] (3) According to the order in which data flows in each basic component, metadata describing the series and parallel relationship of the basic components (i.e., basic component relationship data) is generated, and an association relationship is established between the system component ID and its corresponding system component instance. The basic component configuration data and basic component relationship data together complete the functional description of the data acquisition and processing components of the test system.

[0021] Furthermore, in step three, the characteristics of the test system itself include: equipment information, geographical location, test physical quantity characteristics, maintenance users, equipment status, etc.

[0022] A computer-readable storage medium stores a computer program, which can be executed by a processor to implement the steps of a task-oriented data labeling method described in the present invention.

[0023] A system for implementing the task-oriented data labeling method of the present invention includes the following functional modules:

[0024] Task design module: responsible for integrating the system components of the test system of the participating experiments, building and managing the test data collection-processing task instances for specific test tasks;

[0025] System component design module: responsible for managing various basic components and user-defined processing components, building and managing system components of the test system;

[0026] Visual BI design module: responsible for the design of real-time monitoring charts for key data indicators of the test system and the design of real-time data monitoring large screens for test tasks;

[0027] Data tag building module: responsible for managing system data tags and component data tags and their tagging trigger instances;

[0028] Data storage module: provides persistent storage services for the normal operation of other modules.

[0029] A method for constructing a test task includes the following steps:

[0030] Step 1: Determine the required test system according to the test task scenario, and select the required test system components that have been configured in the system component library of the system described in this technical solution.

[0031] Step 2: Connect the system components that need to transfer data (the components selected in step 1) according to the experimental task plan, and configure the input and output of the system components involved, so that the components are integrated into an organic data flow processing complex.

[0032] Step 3: According to the actual needs of real-time task monitoring, select the monitoring charts that need to be focused on from the data key indicator visualization chart set corresponding to the system component selected in step 1, and use the visual BI building module to build a real-time data monitoring screen for the corresponding test task to assist task management and scheduling personnel in task management and decision-making.

[0033] This method and system utilize the inherent connection and symbiotic characteristics between the data collection system and data tags, and construct an endogenous automated data annotation mechanism for the data collection-processing process through componentized data collection and data processing, thereby realizing full-chain data tagging management from source to application, which has the following beneficial effects:

[0034] (1) Enhance the ability to organize and retrieve data efficiently. By labeling data units in an orderly and semantic way, users can quickly locate relevant data groups as needed, thereby improving the agility and accuracy of data utilization.

[0035] (2) Improve the readability and cognitive accessibility of data. By attaching descriptive tags or metadata, users can quickly understand the content and background of the data, effectively reduce errors caused by interpretation bias, and accelerate the effective transmission of data;

[0036] (3) It lays the data foundation for advanced data analysis, intelligence and automation. Labeled data can enable machines to understand and bridge the connections between data features, building a stable fulcrum for promoting the intelligent use of data and releasing its value.

[0037] In summary, this method and system utilize existing mature BI tools to quickly build visual monitoring charts in the test system data streaming pipeline, convert complex test system data into intuitive and easy-to-understand dashboard views, and reflect key data indicators that are strongly related to task execution decisions in real time, thereby achieving refined monitoring of key data indicators, strengthening the data interpretation capabilities of the test system during the test task, enabling task management and scheduling personnel to make data-driven decisions, and promoting the intelligent and efficient transformation of test tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a schematic diagram of the data acquisition and processing flow of the long baseline measurement system.

[0039] Figure 2 Schematic diagram of the long baseline measurement system components.

[0040] Figure 3 A schematic diagram for designing an underwater target positioning test task. DETAILED DESCRIPTION

[0041] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, further detailed description is given below in conjunction with the accompanying drawings and embodiments.

[0042] A task-oriented data labeling method includes the following steps:

[0043] Step 1: Determine the required test system based on the experimental task scenario, and identify the test system that has not yet built the system components in the system component design module. The system components of the test system are composed of data acquisition and processing components and indicator monitoring components. The data acquisition and processing components are generated and managed by the system component management module, which is responsible for accessing and parsing the data generated by the test system and performing preliminary data processing; the indicator monitoring component is generated and managed by the visual BI design module, which is responsible for drawing charts of the key indicators that need to be monitored in the test system.

[0044] Step 2: Analyze and componentize the data collection and data processing flow of the test system for which the collection and monitoring components have not yet been built in step 1. Use the basic components provided in the system component management module (including: data source components, network connection components, data protocol parsing components, data conversion components, data filtering components, time synchronization components, data encapsulation components, custom programming data processing components, etc.) to build its data processing components and save them to the data storage module for continued use in subsequent test tasks.

[0045] In the step 2, all basic components follow strictly defined interface specifications, have clearly defined inputs and outputs, and through series and parallel integration and configuration of basic components, together constitute the "streaming" architecture data acquisition and processing components of the corresponding test system.

[0046] In step 2, the implementation steps corresponding to the test system data acquisition and processing component storage are:

[0047] (1) The system component management module generates a unique identifier (i.e., system component ID);

[0048] (2) Save the configuration metadata of the basic components involved in its data acquisition and processing components (i.e., basic component configuration data), and establish an association relationship between each configuration metadata and its corresponding system component instance through the system component ID;

[0049] (3) According to the order in which data flows in each basic component, metadata describing the series and parallel relationship of the basic components (i.e., basic component relationship data) is generated, and an association relationship is established between the system component ID and its corresponding system component instance. The basic component configuration data and basic component relationship data together complete the functional description of the data acquisition and processing components of the test system.

[0050] In step 2, taking the long baseline underwater positioning system as an example, the data collection and processing process can be performed as follows: Figure 1 The process shown is represented by the following. First, the raw data of the positioning system is accessed through the network, and the structured target positioning data is obtained through data protocol analysis. The target positioning data of the relative positioning system is uniformly mapped to the preset coordinate system through coordinate transformation. The data filtering algorithm is used to systematically remove abnormal position noise in the trajectory data to obtain more complete trajectory data. According to the above data acquisition-processing process, the data protocol analysis component, data conversion component, data filtering component, and network connection component in the basic component library can be used in a serial manner to construct the following Figure 2The long baseline system components shown in the figure. At the same time, according to the needs of data storage, the data source component is connected after the data parsing protocol component to store the system's raw data persistently; the data element component and the network connection component are connected after the data filtering component to store the processed trajectory data persistently and forward it to other business systems.

[0051] Step 3: Build a task-independent system data tag based on the characteristics of the test system itself and the data management needs in step 2, and bind the data tag to the test system through the system component ID; register and generate the corresponding data labeling trigger in the data tag management module, and automatically trigger data labeling when the data "flows through" the system component, and the outgoing data is marked with the corresponding system data tag.

[0052] In step 3, the characteristics of the test system itself include: equipment information, geographic location, test physical quantity characteristics, maintenance and user, equipment status, etc. Taking the long baseline underwater positioning system as an example, its task-independent data tags may include: target positioning, long baseline system, external measurement, etc.

[0053] Step 4: Build task-independent component data tags based on the characteristics and data management needs of each basic component in the data processing process in step 2, and bind the data tags to the corresponding basic component configuration data; register and generate the corresponding data labeling trigger in the data label management module, and automatically trigger data labeling when the data "flows through" the basic component and is directly "persistently" stored or directly passed to other system components. Component data that meets the labeling conditions are marked with the corresponding component data tags.

[0054] Step 5: Analyze the key indicators that need to be monitored in real time during the actual operation of the test system in step 2, use the visual BI design module to build visual real-time monitoring instruments for these key indicators, and integrate them into the corresponding test system visual BI monitoring components), and bind the constructed visual BI monitoring components to the system components through the system component ID. Taking the long baseline underwater positioning system as an example, during the actual task execution process, business personnel pay more attention to the real-time motion trajectory of the target. Therefore, a mature BI tool can be used to build a real-time monitoring diagram of the target motion trajectory of the long baseline positioning system, and bind it to the corresponding long baseline positioning system through the component ID.

[0055] Step 6: According to the experimental task scenario, select the test system components required for the test task in the system component library, integrate these system components into a test data acquisition-processing task instance (hereinafter referred to as task instance) for the test task in the task design module, and save it to the storage module. Taking an underwater target fusion positioning task as an example, an optimal task design process is as follows: Figure 3As shown: In the task design module, the already constructed short baseline positioning system component and long baseline positioning system component are connected through the custom data fusion component, and a data source component is added afterwards to persistently store the fused target trajectory data.

[0056] Step 7: According to the actual needs of real-time monitoring of the test task, select the monitoring charts corresponding to the indicators that need to be focused on from the system components, and use the combination of visual BI building modules to build a real-time data monitoring screen for the corresponding test task instance. The fusion positioning task described in the above step 6 combines the key indicator monitoring dashboards of the long baseline positioning system and the short baseline positioning system into a visual monitoring screen in the visual BI module. The dispatch management personnel can monitor the operating status of each test system in real time through the constructed screen, and grasp the process of task implementation intuitively, accurately and timely.

[0057] In one embodiment, a task process-oriented data labeling system is provided, the system comprising:

[0058] Task design module: responsible for integrating the system components of the test system of the participating experiments, building and managing the test data collection-processing task instances for specific test tasks.

[0059] System component design module: responsible for managing various basic components and user-defined processing components, building and managing system components of the test system.

[0060] Visual BI design module: responsible for the design of real-time monitoring charts for key data indicators of the test system and the design of real-time data monitoring large screens for test tasks.

[0061] Data tag building module: responsible for managing system data tags and component data tags and their tagging trigger instances.

[0062] Once the architecture of various test systems is determined, the data protocol, the processing of each data field, the data label (labels not related to the task) and the key data indicators that need to be monitored in real time are fixed. On this basis, for future test tasks, there is no need to repeatedly design the data components of the required test system, but a more efficient method is adopted: according to the preset assumptions of specific tasks, the existing multiple test system components are integrated and quickly assembled into a highly targeted solution.

[0063] Therefore, a preferred process for quickly constructing a test task includes the following steps:

[0064] Step 1: Determine the required test system according to the test task scenario, and select the required test system components that have been configured in the system component library of the system described in this technical solution.

[0065] Step 2: Connect the system components that need to transfer data (the components selected in step 1) according to the experimental task plan, and configure the input and output of the system components involved, so that the components are integrated into an organic data flow processing complex.

[0066] Step 3: According to the actual needs of real-time task monitoring, select the monitoring charts that need to be focused on from the data key indicator visualization chart set corresponding to the system component selected in step 1, and use the visual BI building module to build a real-time data monitoring screen for the corresponding test task to assist task management and scheduling personnel in task management and decision-making.

Claims

1. A task-oriented data labeling method, characterized by: The method includes a task design module, a system component design module, a visual BI design module, a data label construction module and a data storage module; The following steps are involved: Step 1: Determine the required test system based on the test task scenario, and identify the test system for system components that have not yet been built in the system component design module; Step 2: Analyze and componentize the data collection and data processing flow of the test system that has not yet built the collection and monitoring components in step 1, use the basic components provided in the system component management module to build its data processing components and save them to the data storage module for continued use in subsequent test tasks; Step 3: According to the characteristics of the test system itself and the data management needs in step 2, a system data tag that is independent of the task is constructed, and the data tag is bound to the test system through the system component ID; the corresponding data tagging trigger is registered and generated in the data tag management module, and the data tagging is automatically triggered when the data "flows through" the system component, and the outflowing data is marked with the corresponding system data tag; Step 4: Build task-independent component data tags based on the characteristics and data management needs of each basic component in the data processing process in step 2, and bind the data tags to the corresponding basic component configuration data; register and generate corresponding data tagging triggers in the data tag management module, and automatically trigger data tagging when data flows through the basic component and is directly stored or directly transferred to other system components. Component data that meets the tagging conditions are marked with corresponding component data tags; Step 5: Analyze the key indicators that need to be monitored in real time during the actual operation of the test system in step 2, use the visual BI design module to build visual real-time monitoring instruments for these key indicators, and integrate them into the corresponding test system visual BI monitoring components), and bind the constructed visual BI monitoring components to the system components through the system component ID; Step 6: According to the experimental task scenario, select the test system components required for the experimental task from the system component library, integrate these system components into a test data collection-processing task instance for the experimental task in the task design module according to the scenario, and save them in the storage module; Step 7: According to the actual needs of real-time monitoring of the test task, select the monitoring charts corresponding to the indicators that need to be focused on from the system components, and use the visual BI building module combination to build a real-time data monitoring screen for the test data collection and processing task instance corresponding to the test task.

2. The task-oriented data labeling method according to claim 1, characterized in that: In the step one, the system components of the test system are composed of a data acquisition and processing component and an indicator monitoring component, wherein the data acquisition and processing component is generated and managed by the system component management module, and is used to access and analyze the data generated by the test system and perform preliminary data processing; the indicator monitoring component is generated and managed by the visual BI design module, and is used to draw charts for each key indicator that needs to be monitored in the test system.

3. The task-oriented data labeling method according to claim 1, characterized in that: In the step 2, the basic components provided in the system component design module include: data source component, network connection component, data protocol parsing component, data conversion component, data filtering component, time synchronization component, data encapsulation component and custom programming data processing component.

4. The task-oriented data labeling method according to claim 3, characterized in that: In the step 2, all basic components follow strictly defined interface specifications, have clearly defined inputs and outputs, and through serial and parallel integration and configuration of basic components, together constitute the streaming architecture data acquisition and processing components of the corresponding test system.

5. The task-oriented data labeling method according to claim 1, characterized in that: In step 2, the implementation steps of saving the corresponding test system data acquisition and processing component include: (1) The system component management module generates a system component ID with a unique identifier; (2) Save the configuration metadata of the basic components involved in its data acquisition and processing components, and establish an association relationship between each configuration metadata and its corresponding system component instance through the system component ID; (3) According to the order in which data flows in each basic component, metadata describing the series and parallel relationship of the basic components is generated, and an association relationship is established between the system component ID and its corresponding system component instance; the basic component configuration data and basic component relationship data together complete the functional description of the data acquisition and processing component of the test system.

6. The task-oriented data labeling method according to claim 1, characterized in that: In step three, the characteristics of the test system itself include: equipment information, geographical location, test physical quantity characteristics, maintenance users and equipment status.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program can be executed by a processor to implement the steps of a task-oriented data labeling method as described in any one of claims 1-6.

8. A task-oriented data labeling system for implementing a task-oriented data labeling method according to any one of claims 1 to 6, characterized in that: Includes the following modules: Task design module: responsible for integrating the system components of the test system of the participating experiments, building and managing the test data collection-processing task instances for specific test tasks; System component design module: responsible for managing various basic components and user-defined processing components, building and managing system components of the test system; Visual BI design module: responsible for the design of real-time monitoring charts for key data indicators of the test system and the design of real-time data monitoring large screens for test tasks; Data tag building module: responsible for managing system data tags and component data tags and their tagging trigger instances; Data storage module: provides persistent storage services.

9. A method for constructing a test task, characterized in that: The following steps are involved: S1, determining the required test system according to the test task scenario, and selecting the required test system components that have been configured in the system component design module of the task process-oriented data labeling system as claimed in claim 8; S2, configuring the input and output of the required test system components that have been configured, so that each component is integrated into an organic data stream processing combination; S3, according to the actual needs of real-time task monitoring, select the monitoring charts that need to be focused on from the data key indicator visualization chart set corresponding to the system component selected in step S1, and use the visualization BI building module to build a real-time data monitoring large screen for the corresponding test task to assist task management and scheduling personnel in task management and decision-making.

Images