Data processing methods for digital quality management in the aviation industry
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA AERO POLYTECH ESTAB
- Filing Date
- 2026-02-03
- Publication Date
- 2026-05-26
Smart Images
Figure CN122086873A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of aviation digital analysis technology, and in particular to a data processing method for digital quality management in the aviation industry. Background Technology
[0002] Quality management in the aviation industry is crucial for ensuring the safety, reliability, and efficiency of aircraft. With the digitalization and intelligentization of quality management in the aviation industry, airlines have accumulated a massive amount of quality management documents and records. The digitalization of the aviation industry primarily involves the real-time collection, integration, analysis, and intelligent mining of data from all processes and elements to form data-driven decision-making capabilities, which are ultimately fed back to all aspects of production operations, customer service, and safety management.
[0003] Current digital analysis systems in the aviation industry suffer from problems such as low processing efficiency and serious redundancy due to the increasing amount of data and complex data processing procedures. Summary of the Invention
[0004] At least one aspect and advantage of this application will be set forth in part in the description which follows, or may be apparent from the description, or may be obtained by practicing the subject matter of this disclosure.
[0005] According to a first aspect of this application, a data processing method for intelligent digital quality management in the aviation industry is provided, the method comprising:
[0006] Obtain a first set of data processing units, wherein the metadata of each data processing unit in the first set of data processing units includes its associated data source information;
[0007] A dependency graph is constructed based on the data source information. The dependency graph is a directed graph and contains the dependency relationships between each data processing unit.
[0008] Based on the data extracted from the instantiated data processing units, the correlation between each data processing unit is determined and the dependency graph is updated to obtain the updated dependency graph.
[0009] Grouping of data sources based on the network connectivity within the updated dependency graph;
[0010] Based on the grouping results, redundant items in the original data source are removed to obtain the second set of data processing units.
[0011] The input data is processed based on the dependency graph and the second data processing unit to obtain the data processing results;
[0012] The data processing unit is used to receive user input, obtain corresponding data, and obtain structured output data.
[0013] According to one embodiment of this application, the data source information includes data interface information, data table information associated with the data source, or data processing unit information on which it depends, and the data interface or data table information has a unique global identifier.
[0014] According to one embodiment of this application, the method for constructing a dependency graph includes:
[0015] The vertices of the directed graph are determined based on the data processing unit and its corresponding data source information;
[0016] The association between a data processing unit and its corresponding data source is determined based on the data source information corresponding to the data processing unit.
[0017] According to one embodiment of this application, the process of obtaining the updated dependency graph includes:
[0018] The data processing unit is instantiated based on preset parameters;
[0019] Data is acquired based on instantiated data processing units, and the data is processed to obtain structured output data;
[0020] Based on the correlation between the structured output data obtained by the first data processing unit within the first data processing unit set and the first threshold, a directed edge is added to the dependency graph, and the weight of the directed edge is the correlation.
[0021] According to one embodiment of this application, in response to the difference in the structure of the structured output data of two instantiated data processing units, it is determined that there is no correlation between the corresponding data processing units.
[0022] According to one embodiment of this application, in response to the existence of two edges between two data processing units in the dependency graph, a first edge and a second edge are determined based on the weight of the edges, wherein the weight of the first edge is greater than that of the second edge, and the second edge is removed from the dependency graph.
[0023] According to one embodiment of this application, grouping combinations of data sources based on the updated network connectivity within the dependency graph includes:
[0024] Based on the connectivity of the updated dependency graph, several communities are obtained. Within a community, vertices corresponding to the first data processing unit are directly connected by edges or indirectly connected by vertices corresponding to data source information.
[0025] In response to the inconsistency between vertices corresponding to data processing units that are directly connected within the community and vertices corresponding to data sources, add new edges within the dependency graph to ensure consistent connections between vertices corresponding to data sources.
[0026] According to one embodiment of this application, the process of acquiring the second data processing unit set includes:
[0027] The data processing units are sorted from highest to lowest according to the number of data sources corresponding to the data processing units contained in the grouping results, to obtain the first list;
[0028] The first list is traversed sequentially to obtain the second set of data processing units that are the data sources required to satisfy the results of the data processing units within the grouping results.
[0029] According to one embodiment of this application, the data processing result is obtained in the following manner:
[0030] Instantiate the data processing units within the second data processing unit set, and obtain the data to be processed based on the data source information associated with them;
[0031] Based on the dependency graph, the data source used by each data processing unit is determined, and the data to be processed corresponding to that data source is used as the input of the corresponding data processing unit to obtain the corresponding structured output data.
[0032] According to one embodiment of this application, the process of obtaining the data processing result further includes:
[0033] Identify data processing units that have direct relationships within the dependency graph;
[0034] The difference coefficient is obtained based on the output results of each data processing unit;
[0035] The weights of directly related data processing units are obtained based on the difference coefficient, and the weighted data processing result is obtained based on the weights and the output results of the directly related data processing units.
[0036] A second aspect of this application provides a data processing apparatus for intelligent quality management in the aviation industry, the apparatus comprising:
[0037] The data acquisition module is used to acquire a first set of data processing units, the metadata of which includes the associated data source information; the data processing unit is used to receive user input and acquire the corresponding data, and obtain structured output data.
[0038] The directed graph construction module is used to build a dependency graph based on data source information. The dependency graph is a directed graph that contains the dependency relationships between various data processing units.
[0039] The graph update processing module is used to determine the correlation between each data processing unit and update the dependency graph based on the data extracted from the instantiated data processing units, and obtain the updated dependency graph.
[0040] The data source grouping module is used to group combinations of data sources based on the network connectivity within the updated dependency graph;
[0041] The redundant data processing module removes redundant items from the original data source based on the grouping results, resulting in a set of second data processing units.
[0042] The data result determination module is used to process the input data based on the dependency graph and the second data processing unit to obtain the data processing result.
[0043] A third aspect of this application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.
[0044] In a fourth aspect of this application, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described method.
[0045] This application embodiment obtains a first data processing unit set. The metadata of the data processing units in the first data processing unit set includes their associated data source information. The first data processing unit is used to receive user input and obtain corresponding data, and obtain structured output data to construct a dependency graph based on the data source information. Based on the data extracted by the instantiated data processing units, the correlation between each data processing unit is determined to obtain an updated dependency graph. Based on the network of interconnections within the updated dependency graph, the combinations between data sources are grouped and redundant items in the original data sources are removed to obtain a second data processing unit set. Then, based on the dependency graph and the second data processing unit, the input data is processed to obtain the data processing result. This method manages the first data processing unit set in the form of a dependency graph through metadata and correlation analysis, thereby partitioning and removing redundancy. Access to the original data source is implemented by the second data processing unit set, thereby reducing duplicate data acquisition, optimizing the data access mode, and solving the data redundancy problem in the digital quality management of the aviation industry. Attached Figure Description
[0046] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0047] Figure 1 A flowchart illustrating a data processing method for digital quality management in the aviation industry, provided as an embodiment of this application;
[0048] Figure 2 A block diagram of a data processing device for digital quality management in the aviation industry, provided in another embodiment of this application. Detailed Implementation
[0049] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.
[0050] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used herein are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.
[0051] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."
[0052] According to one embodiment of this application, a data processing method for intelligent digital quality management in the aviation industry is provided, such as... Figure 1 As shown, the method includes steps S101 to S106.
[0053] Step S101: Obtain a first set of data processing units. The metadata of the data processing units in the first set of data processing units includes their associated data source information. The first data processing unit is used to receive user input and obtain corresponding data, and obtain structured output data.
[0054] It should be noted that the data processing unit refers to the functional component in the aviation industry's digital and intelligent quality management system that encapsulates business algorithms. It declares the external data that it depends on during operation through metadata, that is, the associated data source information.
[0055] This embodiment makes the data sources that the data processing unit depends on and the dependencies between the data sources explicit through meta-information declaration, providing data for subsequent construction of the dependency graph and eliminating data redundancy.
[0056] In practical applications, the associated data source information typically includes data interface information, data table information, and the data processing units it depends on. For example, data processing units may include flight delay prediction units, fuel efficiency analysis units, and passenger satisfaction assessment units. Among these, the data sources associated with the metadata of the flight delay prediction unit may include real-time flight APIs, weather APIs, etc.
[0057] Step S102: Construct a dependency graph based on the data source information. The dependency graph is a directed graph and contains the dependency relationships of each data processing unit.
[0058] It should be noted that the data source information of a data processing unit generally includes the current data processing unit's dependency on data sources (such as sensors or database tables), and may also include the current data processing unit's dependency on other data processing units. In practical applications, data source information can be determined through a pre-configured dependency table, which can include two types of identifiers: identifiers representing data sources and identifiers representing the dependencies between data processing units.
[0059] In this embodiment, the dependency graph includes vertices representing data processing units and vertices representing data sources, with the direction of the edges between vertices indicating the dependency direction. For example, suppose the dependency graph includes three data source vertices and two data processing unit vertices in an intelligent quality management system for the aviation industry, where the data source vertices include vertex 1, vertex 2, and vertex 3, and the data processing unit vertices include vertex 4 and vertex 5; the data flow direction is from vertex 1 to vertex 4, and from vertex 4 to vertex 5; this indicates that vertex 4 depends on vertex 1, and vertex 5 depends on vertex 4.
[0060] In practical applications, the first set of data processing units and the set of data sources can be traversed first, and then the vertices of the directed graph can be created based on the traversal results. The vertex information of the data processing units generally includes vertex type, vertex ID, metadata, etc.; the vertex information of the data sources generally includes vertex type, vertex ID, and configuration information used to obtain the data source. The configuration information is determined based on the type of data source and the access protocol, such as the protocol used, the IP address of the data source, and the port number.
[0061] Step S103: Based on the data extracted from the instantiated data processing units, determine the correlation update dependency graph of each data processing unit and obtain the updated dependency graph.
[0062] It should be noted that the data extracted by the instantiated data processing unit refers to the structured output data obtained after receiving user input and acquiring the corresponding data.
[0063] It should be noted that the correlation between data processing units refers to the statistical and business logic correlations in the content, patterns, or trends of the output data after instantiation of different data processing units. Correlation analysis visualizes the implicit data relationships between different data processing units.
[0064] In practice, correlation calculations can be performed on the output data of different data processing units within a predetermined time window, and the dependency graph can be updated based on the data processing units whose correlation values are greater than a threshold. For example, assuming the aviation management system includes a flight delay prediction unit and a fuel consumption prediction unit, instantiate the flight delay prediction unit and the fuel consumption prediction unit, obtain the output data for the predetermined time window, which are sequence A and sequence B, respectively. Analyzing these two sequences reveals that their outputs are highly correlated (because weather affects delays and fuel consumption). Then, add an edge from the delay prediction unit to the fuel consumption prediction unit in the dependency graph with a weight of 0.8. At this time, update the dependency graph constructed in step S102, such as adding an association edge between the vertices corresponding to these two units in the dependency graph.
[0065] Step S104: Group the combinations of data sources based on the network of connectivity within the updated dependency graph.
[0066] It should be noted that the purpose of grouping is to group interdependent data processing units and data sources to reduce duplicate data access and improve processing efficiency. In practice, graph connectivity algorithms can be used to filter out connected subgraphs (i.e., grouping) in the updated dependency graph.
[0067] It should be noted that grouping refers to the partitioning of the vertex set included in the updated dependency graph. That is, each group is a subset of the vertex set. Each group includes two types of vertices (data processing unit vertices and data source vertices), which are directly or indirectly connected by dependency edges or association edges.
[0068] Step S105: Based on the grouping results, remove redundant items from the original data source to obtain the second data processing unit set.
[0069] It should be noted that the second set of data processing units refers to the designated data processing units selected from the group that can cover the set of data sources that all data processing units in the group depend on. During the instantiation process, these designated data processing units connect to the data source to obtain data, and then share the obtained data with other data processing units in the group. In this way, other data processing units in the group do not need to connect to the data source. Compared with the method of each data processing unit directly accessing the original data source, this can reduce duplicate data acquisition, optimize the data access mode, and solve the problem of data redundancy.
[0070] For example, in an aviation scenario, if both the delay prediction unit and the fuel consumption unit use weather data sources, then only the delay prediction unit (if it covers more data sources) may be retained, thereby eliminating redundant data access for the fuel consumption unit.
[0071] Step S106: Process the input data based on the dependency graph and the second data processing unit to obtain the data processing result.
[0072] Specifically, units in the second set of data processing units can be executed based on the topological order of the dependency graph.
[0073] This application embodiment obtains a first data processing unit set. The metadata of the data processing units in the first data processing unit set includes their associated data source information. The first data processing unit is used to receive user input and obtain corresponding data, and obtain structured output data to construct a dependency graph based on the data source information. Based on the data extracted by the instantiated data processing units, the correlation between each data processing unit is determined to obtain an updated dependency graph. Based on the network of interconnections within the updated dependency graph, the combinations between data sources are grouped and redundant items in the original data sources are removed to obtain a second data processing unit set. Then, based on the dependency graph and the second data processing unit, the input data is processed to obtain the data processing result. This method manages the first data processing unit set in the form of a dependency graph through metadata and correlation analysis, thereby partitioning and removing redundancy. Access to the original data source is implemented by the second data processing unit set, thereby reducing duplicate data acquisition, optimizing the data access mode, and solving the data redundancy problem in the digital quality management of the aviation industry.
[0074] According to one embodiment of this application, the data source information includes data interface information, data table information associated with the data source, or data processing unit information on which it depends, and the data interface or data table information has a unique global identifier.
[0075] Since a single data processing unit may rely on multiple heterogeneous data sources, it may encounter problems such as high coupling, maintenance difficulties, and low reliability. To address these issues, this embodiment establishes a global identifier to link data sources with the data processing unit. This allows the data processing unit to query and retrieve data based on this link.
[0076] The data processing unit extracts data from raw data sources (such as databases, APIs, and IoT devices). The definition of data source information ensures its integrity and standardization. For example, when providing services using a RESTful interface, necessary authentication information is provided in the header to retrieve information according to a preset schema, such as / api / data-source / {source_id} / filterArgs, to obtain information from the corresponding API's data source.
[0077] Each data processing unit can be configured individually or globally through a configuration file. The goal of this step is to ensure that each piece of data depends on a single structure, avoiding the situation where a single piece of data depends on multiple data sources. For example, if a data processor needs a and b to obtain c, but a and b are distributed in different databases, then there is strong coupling between the two, which is undesirable.
[0078] You can also use a direct database connection here, but you should pay attention to security.
[0079] According to one embodiment of this application, the method for constructing a dependency graph includes:
[0080] The vertices of the directed graph are determined based on the data processing unit and its corresponding data source information;
[0081] The association between a data processing unit and its corresponding data source is determined based on the data source information corresponding to the data processing unit.
[0082] Specifically, by traversing the first set of data processing units, a corresponding vertex is created for each data processing unit, and the data source declared by each data processing unit is also traversed and a corresponding vertex is created. Then, directed dependency edges of the remaining data source vertices are created based on the data source information of each data processing unit.
[0083] In practice, the adjacency list or adjacency matrix of the graph can be used to represent the dependency graph.
[0084] According to one embodiment of this application, the process of obtaining the updated dependency graph includes:
[0085] The data processing unit is instantiated based on preset parameters;
[0086] Data is acquired based on instantiated data processing units, and the data is processed to obtain structured output data;
[0087] Based on the correlation between the structured output data obtained by the first data processing unit within the first data processing unit set and the first threshold, a directed edge is added to the dependency graph, and the weight of the directed edge is the correlation.
[0088] In practice, preset correlation analysis algorithms can be used, such as Pearson correlation coefficient algorithm, Spearman rank correlation algorithm, Kendall rank correlation algorithm, distance correlation algorithm, etc.
[0089] In one embodiment of this application, in response to the difference in the structure of the structured output data of two instantiated data processing units, it is determined that there is no correlation between the corresponding data processing units.
[0090] It should be noted that differences in structured output data include at least one of the following: field names, data types, number of fields, nesting structure, field semantics, etc. In practice, the structured output data of the two instantiated data processing units can first be converted into a schema representation, and then schema matching can be performed. The schema representation conversion process can include: extracting the names of each field in the output data and their corresponding data types, generating hash values for the extracted fields and their corresponding data types, and comparing the hash values to determine if differences exist.
[0091] For example, the output of data processing unit A1 includes three fields and their respective values: userId, userName, and scorc. The output fields of data processing unit B1 are the same as those of data processing unit A1. After combining the fields and data types of these two data processing units into a string representation, a hash value is calculated. Based on the hash value calculation results, it is determined that the two are the same and there is no difference. Subsequently, correlation calculation is performed.
[0092] For example, the output of data processing unit A2 includes two fields: userId and userName, with values of 12345 and Zhang San, respectively. The output of data processing unit B2 includes three fields: id, name, and email, with values of 12345, Zhang San, and zhangsan@example.com, respectively. Converting the fields and their data types in each of these two data processing units into hash values and comparing them reveals that the hash values are different, confirming the existence of a discrepancy.
[0093] According to one embodiment of this application, in response to the existence of two edges between two data processing units in the dependency graph, a first edge and a second edge are determined based on the weight of the edges, wherein the weight of the first edge is greater than that of the second edge, and the second edge is removed from the dependency graph.
[0094] It should be noted that an edge refers to a directed line segment between two vertices in a dependency graph, and its weight represents the strength or importance of the dependency relationship. The larger the weight, the stronger the relationship.
[0095] It should be noted that two edges generally refer to edges pointing from one data processing unit to another; that is, these two edges are in the same direction but have different weights. These two edges may include edges established during the construction of the dependency graph (i.e., edges created during the construction of the dependency graph based on data source information) and edges newly created during the updating of the dependency graph; or edges added during the updating of the dependency graph using different association algorithms. The existence of multiple edges increases the complexity of the dependency graph and may lead to redundant calculations in subsequent grouping and analysis.
[0096] In practice, it is possible to determine whether there are two edges between the vertices of two data processing units by traversing the dependency graph, and retain the edge with the largest weight if there are two edges and the edges are in the same direction.
[0097] According to one embodiment of this application, grouping combinations of data sources based on the updated network connectivity within the dependency graph includes:
[0098] Based on the connectivity of the updated dependency graph, several communities are obtained. Within a community, vertices corresponding to the first data processing unit are directly connected by edges or indirectly connected by vertices corresponding to data source information.
[0099] In response to the inconsistency between vertices corresponding to data processing units that are directly connected within the community and vertices corresponding to data sources, add new edges within the dependency graph to ensure consistent connections between vertices corresponding to data sources.
[0100] It should be noted that a community refers to a tightly connected subgraph in the dependency graph, where vertices (data processing units and data sources) within a community have strong correlations.
[0101] It's important to note that direct connectivity refers to a direct connection between two vertices in the dependency graph, while indirect connectivity refers to a direct connection between two data processing unit vertices and a vertex of the same data source; that is, the two data sources are connected through this common data source vertex. In practical applications, community detection algorithms, label propagation algorithms, and connectivity component analysis can be used for community detection. The purpose of community detection is to limit the scope of data consistency processing.
[0102] For example, suppose a community built on a dependency graph includes vertex S (data source vertex) and vertices A (data processing unit vertex) and B (data processing unit vertex) that depend on vertex S, where vertex S and vertex A are directly connected, and vertex A and vertex B are indirectly connected.
[0103] In practice, the process of determining the new edge can be as follows:
[0104] The data processing units and data sources included in the community are determined by traversal;
[0105] Identify any two data processing units that are directly connected within the community, and the set of data sources that are directly connected to each of these two data processing units;
[0106] If the intersection of the data source sets connected to any two data processing units is empty, then based on the data source sets connected to each of the two data processing units, the target data source is determined, and an edge is constructed between the two data processing units that are not connected to the target data source.
[0107] In practical applications, we can build usage statistics for data sources within the community and use the data source with the most usage (i.e., the most dependence) as the target data source.
[0108] In practical applications, the quality score of each data source within the community can be calculated, and the data source with the highest quality score can be selected as the target data source. The quality score can be calculated by summing the number of data processing units connected to the data source and the weights of the edges that the data source has.
[0109] It should be noted that the above method for determining the target data source is only an example. Other methods can be used in actual applications, which will not be listed here.
[0110] According to one embodiment of this application, the process of acquiring the second data processing unit set includes:
[0111] The data processing units are sorted from highest to lowest according to the number of data sources corresponding to the data processing units contained in the grouping results, to obtain the first list;
[0112] The first list is traversed sequentially to obtain the second set of data processing units that are the data sources required to satisfy the results of the data processing units within the grouping results.
[0113] It should be noted that the second set of data processing units generally includes a subset of the first set of data processing units, and the data sources on which the elements (i.e., data processing units) depend cover all the data sources on which the elements in the first set of data processing units depend.
[0114] In practical applications, assume that the data processing units in the first data processing unit set include unit A, unit B, unit C and unit D. The data sources that unit A depends on include b1, b2, b3 and b4, the data sources that unit B depends on include b1, b3 and b5, the data sources that unit C depends on include b2 and b4, the data source that unit D depends on includes b5, and the corresponding data source set b includes b1, b2, b3, b4 and b5.
[0115] The elements of the first list obtained by sorting are: Unit A, Unit B, Unit C, and Unit D in sequence; the added data source C is empty, and the second data processing unit set S is empty.
[0116] Traversing the first list, the process is as follows:
[0117] For unit A, the data source set of unit A includes b1, b2, b3 and b4, and its intersection with C is empty. Unit A is taken as an element of the second data processing unit set, and the data source set of unit A is added to set C. At this time, C includes b1, b2, b3 and b4. Unit A is added to the second processing unit set.
[0118] For unit B, its data source set intersects with set C. Add b5 to set C. At this time, C includes b1, b2, b3, b4 and b5. Add unit B to the second processing unit set.
[0119] For unit C, whose data source is a subset of set C, unit C is not added to the second processing unit set;
[0120] For unit D, whose data source is a subset of set C, unit D is not added to the second processing unit set.
[0121] At this point, the traversal stops when the difference between the data source set b and set C is empty, resulting in the second data processing unit set S, which includes unit A and unit B.
[0122] Compared to the approach where each data processing unit acquires data independently, the second set of data processing units has fewer units. It can connect to all data sources using fewer units to obtain data, thereby sharing the acquired data with other data processing units. This decouples other data processing units from the direct dependencies on data sources, achieving a layered architecture design for data processing. This reduces data source connections, avoids duplicate data acquisition, and lowers I / O load and complexity. Furthermore, reduced system coupling lowers data maintenance difficulty.
[0123] According to one embodiment of this application, the data processing result is obtained in the following manner:
[0124] Instantiate the data processing units within the second data processing unit set, and obtain the data to be processed based on the data source information associated with them;
[0125] Based on the dependency graph, the data source used by each data processing unit is determined, and the data to be processed corresponding to that data source is used as the input of the corresponding data processing unit to obtain the corresponding structured output data.
[0126] It should be noted that the instantiated objects are elements of the second data processing unit set. The instantiation of these elements can share the data obtained from the data source with other data processing units. These other data processing units do not need to connect to the data source, but obtain the required data from the second data processing unit, thereby reducing the coupling of these other data processing units.
[0127] It should be noted that this embodiment pre-stores the metadata of each data processing unit and the structure of the dependency graph (including vertices and edges), with the purpose of using this pre-stored information for instantiation. The vertices in the dependency graph generally include labeled groups and labels for the second data processing unit, so that the second data processing unit can be quickly identified from the dependency graph using these labels.
[0128] According to one embodiment of this application, the process of obtaining the data processing result further includes:
[0129] Identify data processing units that have direct relationships within the dependency graph;
[0130] The difference coefficient is obtained based on the output results of each data processing unit;
[0131] The weights of directly related data processing units are obtained based on the difference coefficient, and the weighted data processing result is obtained based on the weights and the output results of the directly related data processing units.
[0132] It should be noted that the difference coefficient refers to the discreteness of the output result; the larger the difference coefficient, the greater its corresponding weight. In practice, the difference coefficient is calculated using the entropy weight method, and its algorithm formula is as follows: ,in, Let be the difference coefficient of the j-th indicator. Let be the information entropy of the j-th indicator. The larger the value, the greater the importance of indicator j; the weight of indicator j is determined by the proportion of its difference coefficient to the sum of the difference coefficients of all indicators.
[0133] It should be noted that, since data processing units with direct relationships in a dependency graph generally process the same or related data, the output results of data processing units with direct relationships may be redundant.
[0134] According to another embodiment of this application, a data processing device for intelligent digital quality management in the aviation industry is provided, such as... Figure 2 As shown, the device includes: a data acquisition module 201, a directed graph construction module 202, a graph update processing module 203, a data source grouping module 204, a redundant data processing module 205, and a data result determination module 206.
[0135] The data acquisition module is used to acquire a first set of data processing units, the metadata of which includes the associated data source information; the data processing unit is used to receive user input and acquire the corresponding data, and obtain structured output data.
[0136] The directed graph construction module 202 is used to construct a dependency graph based on data source information. The dependency graph is a directed graph and contains the dependency relationships of each data processing unit.
[0137] The graph update processing module 203 is used to determine the correlation between each data processing unit and update the dependency graph based on the data extracted from the instantiated data processing units, and obtain the updated dependency graph.
[0138] Data source grouping module 204 is used to group combinations of data sources based on the network of connectivity within the updated dependency graph.
[0139] The redundant data processing module 205 removes redundant items from the original data source based on the grouping results to obtain the second data processing unit set.
[0140] The data result determination module 206 is used to process the input data based on the dependency graph and the second data processing unit to obtain the data processing result.
[0141] This application embodiment obtains a first data processing unit set. The metadata of the data processing units in the first data processing unit set includes their associated data source information. The first data processing unit is used to receive user input and obtain corresponding data, and obtain structured output data to construct a dependency graph based on the data source information. Based on the data extracted by the instantiated data processing units, the correlation between each data processing unit is determined to obtain an updated dependency graph. Based on the network of interconnections within the updated dependency graph, the combinations between data sources are grouped and redundant items in the original data sources are removed to obtain a second data processing unit set. Then, based on the dependency graph and the second data processing unit, the input data is processed to obtain the data processing result. This method manages the first data processing unit set in the form of a dependency graph through metadata and correlation analysis, thereby partitioning and removing redundancy. Access to the original data source is implemented by the second data processing unit set, thereby reducing duplicate data acquisition, optimizing the data access mode, and solving the data redundancy problem in the digital quality management of the aviation industry.
[0142] Furthermore, the data source information includes data interface information, data table information associated with the data source, or information on the data processing unit on which it depends. The data interface or data table information has a unique global identifier.
[0143] Furthermore, methods for constructing dependency graphs include:
[0144] The vertices of the directed graph are determined based on the data processing unit and its corresponding data source information;
[0145] The association between a data processing unit and its corresponding data source is determined based on the data source information corresponding to the data processing unit.
[0146] Furthermore, the process of obtaining the updated dependency graph includes:
[0147] The data processing unit is instantiated based on preset parameters;
[0148] Data is acquired based on instantiated data processing units, and the data is processed to obtain structured output data;
[0149] Based on the correlation between the structured output data obtained by the first data processing unit within the first data processing unit set and the first threshold, a directed edge is added to the dependency graph, and the weight of the directed edge is the correlation.
[0150] Furthermore, in response to the difference in the structure of the structured output data of the two instantiated data processing units, it is determined that there is no correlation between the corresponding data processing units.
[0151] Furthermore, in response to the dependency graph, when there are two edges between two data processing units, the first edge and the second edge are determined based on the weight of the edges. The weight of the first edge is greater than that of the second edge, and the second edge is removed from the dependency graph.
[0152] Furthermore, grouping combinations of data sources based on the network of connectivity within the updated dependency graph includes:
[0153] Based on the connectivity of the updated dependency graph, several communities are obtained. Within a community, vertices corresponding to the first data processing unit are directly connected by edges or indirectly connected by vertices corresponding to data source information.
[0154] In response to the inconsistency between vertices corresponding to data processing units that are directly connected within the community and vertices corresponding to data sources, add new edges within the dependency graph to ensure consistent connections between vertices corresponding to data sources.
[0155] Furthermore, the process of acquiring the second data processing unit set includes:
[0156] The data processing units are sorted from highest to lowest according to the number of data sources corresponding to the data processing units contained in the grouping results, to obtain the first list;
[0157] The first list is traversed sequentially to obtain the second set of data processing units that are the data sources required to satisfy the results of the data processing units within the grouping results.
[0158] Furthermore, the data processing results are obtained in the following manner:
[0159] Instantiate the data processing units within the second data processing unit set, and obtain the data to be processed based on the data source information associated with them;
[0160] Based on the dependency graph, the data source used by each data processing unit is determined, and the data to be processed corresponding to that data source is used as the input of the corresponding data processing unit to obtain the corresponding structured output data.
[0161] Furthermore, the process of obtaining data processing results also includes:
[0162] Identify data processing units that have direct relationships within the dependency graph;
[0163] The difference coefficient is obtained based on the output results of each data processing unit;
[0164] The weights of directly related data processing units are obtained based on the difference coefficient, and the weighted data processing result is obtained based on the weights and the output results of the directly related data processing units.
[0165] The data processing device for digital quality management in the aviation industry in this embodiment can execute the data processing method for digital quality management in the aviation industry shown in the embodiment of this application. The implementation principle is similar and will not be described again here.
[0166] Another embodiment of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described method.
[0167] Specifically, the processor can be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
[0168] Specifically, the processor connects to the memory via a bus, which may include a path for transmitting information. The bus can be a PCI bus or an EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc.
[0169] The memory may be ROM or other types of static storage devices that can store static information and instructions, RAM or other types of dynamic storage devices that can store information and instructions, or EEPROM, CD-ROM or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto.
[0170] Optionally, the memory stores the code of a computer program that executes the scheme of this application, and the execution is controlled by a processor. The processor executes the application code stored in the memory to implement the operation of the apparatus provided in the above embodiments.
[0171] Another embodiment of this application provides a computer-readable storage medium storing computer-executable instructions for performing the methods provided in the above embodiments.
[0172] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0173] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.
[0174] The above is a detailed description of the preferred embodiments of this application. However, this application is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.
Claims
1. A data processing method for intelligent digital quality management in the aviation industry, characterized in that, include: Obtain a first set of data processing units, wherein the metadata of each data processing unit in the first set of data processing units includes its associated data source information; A dependency graph is constructed based on data source information. The dependency graph is a directed graph and contains the dependency relationships between each data processing unit. Based on the data extracted from the instantiated data processing units, the correlation between each data processing unit is determined and the dependency graph is updated to obtain the updated dependency graph. Grouping of data sources based on the network connectivity within the updated dependency graph; Based on the grouping results, redundant items in the original data source are removed to obtain the second set of data processing units. The input data is processed based on the dependency graph and the second data processing unit to obtain the data processing results; The data processing unit is used to receive user input, obtain corresponding data, and obtain structured output data.
2. The data processing method for intelligent digital quality management in the aviation industry as described in claim 1, characterized in that, The data source information includes data interface information, data table information associated with the data source, or information on the data processing unit on which it depends. The data interface or data table information has a unique global identifier.
3. The data processing method for intelligent digital quality management in the aviation industry as described in claim 1, characterized in that, The method for constructing the dependency graph includes: The vertices of the directed graph are determined based on the data processing unit and its corresponding data source information; The association between a data processing unit and its corresponding data source is determined based on the data source information corresponding to the data processing unit.
4. The data processing method for intelligent digital quality management in the aviation industry as described in claim 1, characterized in that, The process of obtaining the updated dependency graph includes: The data processing unit is instantiated based on preset parameters; Data is acquired based on instantiated data processing units, and the data is processed to obtain structured output data; Based on the correlation between the structured output data obtained by the first data processing unit within the first data processing unit set and the first threshold, a directed edge is added to the dependency graph, and the weight of the directed edge is the correlation.
5. The data processing method for intelligent digital quality management in the aviation industry as described in claim 4, characterized in that, In response to the difference in the structure of the structured output data of the two instantiated data processing units, it is determined that there is no correlation between the corresponding data processing units.
6. The data processing method for intelligent digital quality management in the aviation industry as described in claim 1, characterized in that, In response to the dependency graph, when there are two edges between two data processing units, the first edge and the second edge are determined based on the weight of the edges. The weight of the first edge is greater than that of the second edge, and the second edge is removed from the dependency graph.
7. The data processing method for intelligent digital quality management in the aviation industry as described in claim 1, characterized in that, The grouping of combinations of data sources based on the network connectivity of the updated dependency graph includes: Several communities are obtained based on the connectivity of the updated dependency graph, wherein the vertices corresponding to the first data processing unit in the community are directly connected by edges or indirectly connected by vertices corresponding to data source information. In response to the inconsistency between vertices corresponding to data processing units that are directly connected within the community and vertices corresponding to data sources, add new edges within the dependency graph to ensure consistent connections between vertices corresponding to data sources.
8. The data processing method for intelligent digital quality management in the aviation industry as described in claim 1, characterized in that, The process of acquiring the second data processing unit set includes: The data processing units are sorted from highest to lowest according to the number of data sources corresponding to the data processing units contained in the grouping results, to obtain the first list; The first list is traversed sequentially to obtain the second set of data processing units that are the data sources required to satisfy the results of the data processing units within the grouping results.
9. The data processing method for intelligent digital quality management in the aviation industry as described in claim 1, characterized in that, The data processing results are obtained in the following manner: Instantiate the data processing units within the second data processing unit set, and obtain the data to be processed based on the data source information associated with them; Based on the dependency graph, the data source used by each data processing unit is determined, and the data to be processed corresponding to that data source is used as the input of the corresponding data processing unit to obtain the corresponding structured output data.
10. The data processing method for intelligent digital quality management in the aviation industry as described in claim 9, characterized in that, The process of obtaining the data processing results also includes: Identify data processing units that have direct relationships within the dependency graph; The difference coefficient is obtained based on the output results of each data processing unit; The weights of directly related data processing units are obtained based on the difference coefficient, and the weighted data processing result is obtained based on the weights and the output results of the directly related data processing units.