A data operation and maintenance method and system
By connecting to various operation and maintenance platforms through the operation and maintenance data middle platform, data can be acquired, verified, and processed, solving the problems of low efficiency and high threshold in banking business data processing. This enables an efficient and flexible data operation and maintenance approach, suitable for data use in various scenarios and simplifying operations.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-08
- Publication Date
- 2026-03-10
AI Technical Summary
In existing technologies, banking business data processing suffers from low efficiency, poor data integrity, and high operational barriers. In particular, when using scripts to temporarily process data and general data processing platforms, problems such as redundant development, incomplete data, and excessively high professional background requirements for operation and maintenance personnel are encountered.
By using an operations and maintenance data platform, various operations and maintenance platforms can be connected. Message queues are used to obtain reported data, perform data entry verification and operations and maintenance, obtain metadata, and process it for data entry into the warehouse. This breaks down the data gap between operations and maintenance platforms, improves data reusability, and lowers the operational threshold.
It enables a professional and flexible data operation and maintenance approach, improves the efficiency and reusability of data, lowers the operational threshold for operation and maintenance personnel, simplifies the data processing chain, and is suitable for operation and maintenance personnel without a professional background.
Smart Images

Figure CN117033734B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and more specifically, to a data operation and maintenance method and system. Background Technology
[0002] Many business operations in the financial sector have already achieved automation and intelligence through automated and intelligent implementation systems. To maintain the normal operation of these systems, operations and maintenance (O&M) teams have emerged. However, when faced with data processing needs from banking operations, O&M teams typically assign personnel to write scripts on the spot for flexible processing, or use general data processing platforms for specialized business processing.
[0003] In practice, it has been found that the two methods currently in use still have the following problems:
[0004] (1) Scripts process data temporarily, which is inefficient and cannot be reused. Users will develop a lot of scenario-based applications, resulting in a lot of repetitive development work. Poor code engineering makes it impossible to form a platform and the functions cannot be reused.
[0005] (2) Scripts process data temporarily, resulting in poor data integrity. Users need to manually query data from various platforms and perform correlations themselves. This easily leads to incomplete data and thus data distortion.
[0006] (3) General-purpose data processing platforms are too specialized and not user-friendly for operations and maintenance personnel. General-purpose data processing platforms are typically used to process business data and are powerful in function. However, they require users to have a professional big data background. This background requirement is a high barrier for operations and maintenance personnel. Summary of the Invention
[0007] The purpose of this application is to provide a data operation and maintenance method and system that can enable access to various operation and maintenance platforms through an operation and maintenance data platform, thereby achieving a professional and flexible data operation and maintenance approach, breaking down the data gap between operation and maintenance platforms, making data more efficient to be used, improving reusability, and helping to lower the operational threshold.
[0008] The first aspect of this application provides a data operation and maintenance method, which is applied to an operation and maintenance data platform, and the method includes:
[0009] Retrieve reported data submitted via the message queue;
[0010] The reported data is validated for entry into the database based on the validation model in the operation and maintenance data platform.
[0011] When the reported data passes the database entry verification, the reported data is maintained based on the database in the operation and maintenance data platform to obtain metadata.
[0012] The metadata is then processed for inbound storage.
[0013] In the above implementation process, this method can obtain reported data through the message queue via the operation and maintenance data platform; then, it performs database verification on the reported data based on the verification model in the operation and maintenance data platform; when the reported data passes the database verification, it performs operation and maintenance on the reported data based on the database in the operation and maintenance data platform to obtain metadata; finally, it performs warehouse entry processing on the metadata. It is evident that this method can achieve access to various operation and maintenance platforms through the operation and maintenance data platform, thereby realizing a professional and flexible data operation and maintenance approach, bridging the data gap between operation and maintenance platforms, enabling data to be used more efficiently, improving reusability, and lowering the operational threshold.
[0014] Furthermore, the method also includes:
[0015] When the reported data fails the warehousing inspection, the inspection failure information of the reported data is recorded.
[0016] Furthermore, the step of performing warehousing processing on the metadata includes:
[0017] Obtain the inbound task configuration parameters, and store the metadata in the database based on the inbound task configuration parameters.
[0018] Furthermore, the step of performing maintenance on the reported data based on the database in the maintenance data platform to obtain metadata includes:
[0019] Identify the field definition information of the reported data;
[0020] Based on the database in the operation and maintenance data platform, determine the field association information associated with the field definition information;
[0021] Based on the field definition information and the field association information, the lineage of the reported data is maintained to obtain metadata.
[0022] Furthermore, the method also includes:
[0023] When the outbound task is detected, data is automatically retrieved from the user API to obtain the outbound task configuration parameters.
[0024] Determine whether the configuration parameters of the outbound task conform to the preset service configuration;
[0025] When the outbound task configuration parameters match the preset service configuration, the metadata is output.
[0026] A second aspect of this application provides a data operation and maintenance system, which is applied to an operation and maintenance data platform, and the data operation and maintenance system includes:
[0027] The acquisition unit is used to acquire the reported data submitted through the message queue;
[0028] The verification unit is used to perform database verification on the reported data based on the verification model in the operation and maintenance data platform.
[0029] The operation and maintenance unit is used to perform operation and maintenance on the reported data based on the database in the operation and maintenance data platform when the reported data passes the database entry verification, and obtain metadata.
[0030] The inbound unit is used to process the metadata for inbound storage.
[0031] In the above implementation process, this data operation and maintenance system is applied to the operation and maintenance data platform. Specifically, it can acquire reported data through the message queue via the acquisition unit; perform database entry verification on the reported data based on the verification model in the operation and maintenance data platform via the verification unit; when the reported data passes the database entry verification, the operation and maintenance unit performs operation and maintenance on the reported data based on the database in the operation and maintenance data platform to obtain metadata; and then perform warehouse entry processing on the metadata via the warehouse entry unit. It is evident that this system can achieve access to various operation and maintenance platforms through the operation and maintenance data platform, thereby realizing a professional and flexible data operation and maintenance method, bridging the data gap between operation and maintenance platforms, enabling data to be used more efficiently, improving reusability, and lowering the operational threshold.
[0032] Furthermore, the data operation and maintenance system also includes:
[0033] The recording unit is used to record the inspection failure information of the reported data when the reported data fails the warehousing inspection.
[0034] Furthermore, the warehousing unit is specifically used to obtain warehousing task configuration parameters and store the metadata in the database based on the warehousing task configuration parameters.
[0035] Furthermore, the operation and maintenance unit includes:
[0036] An identification subunit is used to identify the field definition information of the reported data;
[0037] The sub-unit is determined based on the database in the operation and maintenance data platform to determine the field association information associated with the field definition information;
[0038] The operation and maintenance subunit is used to maintain the lineage of the reported data based on the field definition information and the field association information to obtain metadata.
[0039] Furthermore, the data operation and maintenance system also includes:
[0040] The data retrieval unit is used to automatically retrieve data from the user API when the outbound task is detected, and obtain the outbound task configuration parameters.
[0041] The judgment unit is used to determine whether the configuration parameters of the outbound task conform to the preset service configuration;
[0042] The outbound unit is used to output the metadata when the outbound task configuration parameters meet the preset service configuration.
[0043] A third aspect of this application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor runs the computer program to cause the electronic device to perform the data maintenance method described in any one of the first aspects of this application.
[0044] The fourth aspect of this application provides a computer-readable storage medium storing computer program instructions, which, when read and executed by a processor, perform the data maintenance method described in any one of the first aspects of this application.
[0045] The beneficial effects of this application are as follows: the method and device can realize the access of various operation and maintenance platforms through the operation and maintenance data platform, thereby realizing a professional and flexible data operation and maintenance method, breaking down the data gap between operation and maintenance platforms, making data more efficient to be used, improving reusability, and helping to reduce the operation threshold. Attached Figure Description
[0046] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0047] Figure 1 A flowchart illustrating a data operation and maintenance method provided in an embodiment of this application;
[0048] Figure 2 A flowchart illustrating another data operation and maintenance method provided in this application embodiment;
[0049] Figure 3 This is a schematic diagram of the structure of a data operation and maintenance system provided in an embodiment of this application;
[0050] Figure 4 This is a schematic diagram of another data operation and maintenance system provided in an embodiment of this application;
[0051] Figure 5 This is a schematic diagram of the architecture of an operation and maintenance data platform provided in an embodiment of this application. Detailed Implementation
[0052] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.
[0053] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0054] Example 1
[0055] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a data operation and maintenance method provided in this embodiment. The data operation and maintenance method is applied to an operation and maintenance data platform and includes:
[0056] S101. Obtain the reported data submitted through the message queue.
[0057] In this embodiment, the operation and maintenance data platform is webPortal.
[0058] S102. Verify the reported data for entry into the database based on the verification model in the operation and maintenance data platform.
[0059] S103. When the reported data passes the database entry verification, the reported data is operated and maintained based on the database in the operation and maintenance data platform to obtain metadata.
[0060] S104. Perform data entry processing on the data repository.
[0061] In the financial sector, various systems within banks typically require maintenance teams for operation and maintenance (O&M). Among these, the maintenance of data and data processing is particularly crucial. Currently, two common approaches are used for O&M: first, O&M personnel write specialized scripts for specific banking operations and perform ad-hoc processing based on these scripts; second, they utilize general data processing platforms corresponding to banking operations for complex and specialized processing. However, in daily work, both of these approaches present certain challenges due to their high level of specialization.
[0062] To address this technical issue, this application proposes a data operation and maintenance method for an operation and maintenance data platform. This method enables diversified data access and integration, and then diversified output based on user needs. By using intermediate means, it reduces the professional difficulty, thereby lowering the entry threshold for operation and maintenance personnel and improving their operation and maintenance efficiency.
[0063] In this embodiment, the subject executing the method can be a computing system such as a computer or server, and no limitation is made in this embodiment.
[0064] In this embodiment, the subject executing the method can also be a smart device such as a smartphone or tablet, and no limitation is made in this embodiment.
[0065] As can be seen, implementing the data operation and maintenance method described in this embodiment enables the acquisition of diverse reported data through the operation and maintenance data platform, thereby achieving data integration with various operation and maintenance platforms. Therefore, this method can bridge the data gap between operation and maintenance platforms, allowing data to be used and integrated more efficiently. Simultaneously, this method can abstract and layer data for common operation and maintenance scenarios through the operation and maintenance data platform, thereby solidifying some common scenario data for direct use by users within the platform. This simplifies user processing logic, improves efficiency, and enhances reusability, eliminating the need for user intervention. Furthermore, this method simplifies the data processing chain, addressing the issue of excessive technical involvement, allowing operation and maintenance personnel to quickly get started and focus on their data processing logic without needing to worry about upstream and downstream processes. This makes it more suitable for operation and maintenance personnel without specialized backgrounds.
[0066] Example 2
[0067] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating a data operation and maintenance method provided in this embodiment. The data operation and maintenance method is applied to an operation and maintenance data platform and includes:
[0068] S201. Obtain the reported data submitted through the message queue.
[0069] In this embodiment, the operation and maintenance data platform is webPortal.
[0070] S202. Verify the reported data for entry into the database based on the verification model in the operation and maintenance data platform.
[0071] As an optional implementation, the method further includes:
[0072] When the reported data fails the warehousing inspection, record the inspection failure information of the reported data.
[0073] In this embodiment, the operation and maintenance data platform predefines many models. Specifically, the operation and maintenance data platform defines the purpose and grouping of the models.
[0074] S203. When the reported data passes the database entry verification, identify the field definition information of the reported data.
[0075] S204. Based on the database in the operation and maintenance data platform, determine the field association information associated with the field definition information.
[0076] S205. Based on field definition information and field association information, perform lineage maintenance on the reported data to obtain metadata.
[0077] In this embodiment, the operations and maintenance data platform also sets field definitions in the database. These field definitions include the field's Chinese name, English name, basic data type, and various constraints and tags.
[0078] In this embodiment, the field definition information and field association information mentioned above can include the Chinese name, English name, basic type, and various constraints and tags of the field.
[0079] In this embodiment, lineage maintenance refers to maintaining the relationship between fields.
[0080] S206. Obtain the inbound task configuration parameters and store the metadata in the database based on the inbound task configuration parameters.
[0081] S207. When an outbound task is detected, the system automatically retrieves data from the user API to obtain the outbound task configuration parameters.
[0082] In this embodiment, the method can be configured for users to retrieve data using APIs, including API verification information, API type, rate limiting control, and permission management.
[0083] In this embodiment, the method can achieve automatic data retrieval through self-service data retrieval management, thereby enabling more precise control over user API call behavior.
[0084] S208. Determine whether the outbound task configuration parameters meet the preset service configuration. If yes, proceed to step S209; otherwise, end this process.
[0085] S209. Output the metadata.
[0086] Please see Figure 5 , Figure 5 This diagram illustrates the architecture of an operations and maintenance data platform.
[0087] The operations and maintenance data platform is webPortal. WebPortal is a unified entry point service that manages the configurations of downstream services. WebPortal primarily implements the configuration and management of the following functions:
[0088] (1) Metadata management: Metadata management mainly involves model definition, field definition, and lineage maintenance.
[0089] Model definition is the core function of metadata management, defining the purpose and grouping of models.
[0090] Field definition refers to the Chinese name, English name, basic type, and various constraints and labels of the field.
[0091] Lineage maintenance maintains the relationships between fields.
[0092] (2) Data integration management: mainly configures the basic information of the data source of the data integration service, the data acquisition method (mainly the domain reports through the message queue), and the data to be reported needs to be validated using the model in the metadata.
[0093] This method can consume user push data based on the webPortal configuration and perform validation using the model in the metadata. Successfully validated data is stored in the database to prepare data for subsequent data entry tasks. Data that fails validation is recorded with failure information for easy backtracking.
[0094] (3) Task Management: This mainly consists of inbound task configuration, calculation task configuration, and outbound task configuration. Inbound tasks, calculation tasks, and outbound tasks constitute a complete data collection, calculation, and output process. The data from the outbound tasks will be used for self-service data retrieval.
[0095] This method can call the inbound job executor, calculate the job executor, and execute the corresponding task based on the webPortal configuration.
[0096] (4) Self-service data retrieval management: This mainly involves configuring user data retrieval via API, including API verification information, API type, rate limiting control, and access control. Self-service data retrieval management allows for more precise control over user API calls.
[0097] This method can serve user API calls based on the webPortal configuration, and perform API request validation, rate limiting, and access control.
[0098] In this embodiment, the subject executing the method can be a computing system such as a computer or server, and no limitation is made in this embodiment.
[0099] In this embodiment, the subject executing the method can also be a smart device such as a smartphone or tablet, and no limitation is made in this embodiment.
[0100] As can be seen, implementing the data operation and maintenance method described in this embodiment can obtain diverse reported data through the operation and maintenance data platform, thereby achieving data integration with various operation and maintenance platforms. Simultaneously, it can provide a unified data outlet with auditing, review, and authorization, enabling diversified data output. Therefore, this method can bridge the data gap between operation and maintenance platforms, allowing data to be used and integrated more efficiently. Furthermore, this method can abstract and layer data for common operation and maintenance scenarios through the operation and maintenance data platform, thereby solidifying some common scenario data for direct use by users within the platform. This simplifies user processing logic, improves efficiency, and enhances reusability, eliminating the need for user intervention. Additionally, this method simplifies the data processing chain, addressing the issue of excessive technical involvement, allowing operation and maintenance personnel to quickly get started and focus on their data processing logic without needing to worry about upstream and downstream processes, making it more suitable for operation and maintenance personnel without specialized backgrounds.
[0101] Example 3
[0102] Please refer to Figure 3 , Figure 3 This is a schematic diagram of the structure of a data operation and maintenance system provided in this embodiment. Figure 3 As shown, this data operation and maintenance system is applied to the operation and maintenance data platform, and the data operation and maintenance system includes:
[0103] The acquisition unit 310 is used to acquire the reported data reported through the message queue;
[0104] Verification unit 320 is used to perform data entry verification on the reported data based on the verification model in the operation and maintenance data platform.
[0105] The operation and maintenance unit 330 is used to perform operation and maintenance on the reported data based on the database in the operation and maintenance data platform when the reported data passes the database entry verification, and obtain metadata.
[0106] Inbound unit 340 is used for inbound processing of metadata.
[0107] In this embodiment, the explanation of the data operation and maintenance system can be referred to the description in Embodiment 1 or Embodiment 2, and will not be repeated here.
[0108] As can be seen, implementing the data operation and maintenance system described in this embodiment can obtain diverse reported data through the operation and maintenance data platform, thereby achieving data integration with various operation and maintenance platforms. Simultaneously, it can provide a unified data outlet with auditing, review, and authorization mechanisms, enabling diversified data output. Therefore, this method can bridge the data gap between operation and maintenance platforms, allowing data to be used and integrated more efficiently. Furthermore, this method can abstract and layer data for common operation and maintenance scenarios through the operation and maintenance data platform, thereby solidifying some common scenario data for direct use by users within the platform. This simplifies user processing logic, improves efficiency, and enhances reusability, eliminating the need for user intervention. Additionally, this method simplifies the data processing chain, addressing the issue of excessive technical involvement, allowing operation and maintenance personnel to quickly get started and focus on their data processing logic without needing to worry about upstream and downstream processes, making it more suitable for operation and maintenance personnel without specialized backgrounds.
[0109] Example 4
[0110] Please refer to Figure 4 , Figure 4 This is a schematic diagram of the structure of a data operation and maintenance system provided in this embodiment. Figure 4 As shown, this data operation and maintenance system is applied to the operation and maintenance data platform, and the data operation and maintenance system includes:
[0111] The acquisition unit 310 is used to acquire the reported data reported through the message queue;
[0112] Verification unit 320 is used to perform data entry verification on the reported data based on the verification model in the operation and maintenance data platform.
[0113] The operation and maintenance unit 330 is used to perform operation and maintenance on the reported data based on the database in the operation and maintenance data platform when the reported data passes the database entry verification, and obtain metadata.
[0114] Inbound unit 340 is used for inbound processing of metadata.
[0115] As an optional implementation, the data operation and maintenance system also includes:
[0116] Recording unit 350 is used to record the inspection failure information of the reported data when the reported data fails the warehousing inspection.
[0117] As an optional implementation, the inbound unit 340 is specifically used to obtain inbound task configuration parameters and store metadata in the database based on the inbound task configuration parameters.
[0118] As an optional implementation, the operation and maintenance unit 330 includes:
[0119] Identification subunit 331 is used to identify the field definition information of the reported data;
[0120] Subunit 332 is defined to determine the field association information associated with the field definition information based on the database in the operation and maintenance data platform.
[0121] Operation and maintenance subunit 333 is used to maintain the lineage of reported data based on field definition information and field association information to obtain metadata.
[0122] As an optional implementation, the data operation and maintenance system also includes:
[0123] The data retrieval unit 360 is used to automatically retrieve data from the user API when an outbound task is detected, and obtain the outbound task configuration parameters.
[0124] The judgment unit 370 is used to determine whether the configuration parameters of the outbound task conform to the preset service configuration.
[0125] Outbound unit 380 is used to output metadata when the outbound task configuration parameters meet the preset service configuration.
[0126] In this embodiment, the explanation of the data operation and maintenance system can be referred to the description in Embodiment 1 or Embodiment 2, and will not be repeated here.
[0127] As can be seen, implementing the data operation and maintenance system described in this embodiment can obtain diverse reported data through the operation and maintenance data platform, thereby achieving data integration with various operation and maintenance platforms. Simultaneously, it can provide a unified data outlet with auditing, review, and authorization mechanisms, enabling diversified data output. Therefore, this method can bridge the data gap between operation and maintenance platforms, allowing data to be used and integrated more efficiently. Furthermore, this method can abstract and layer data for common operation and maintenance scenarios through the operation and maintenance data platform, thereby solidifying some common scenario data for direct use by users within the platform. This simplifies user processing logic, improves efficiency, and enhances reusability, eliminating the need for user intervention. Additionally, this method simplifies the data processing chain, addressing the issue of excessive technical involvement, allowing operation and maintenance personnel to quickly get started and focus on their data processing logic without needing to worry about upstream and downstream processes, making it more suitable for operation and maintenance personnel without specialized backgrounds.
[0128] This application provides an electronic device, including a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the data operation and maintenance method in embodiment 1 or embodiment 2 of this application.
[0129] This application provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, the data operation and maintenance method in embodiment 1 or embodiment 2 of this application is performed.
[0130] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can also be implemented in other ways. The system embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0131] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.
[0132] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0133] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.
[0134] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
[0135] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
Claims
1. A data operation and maintenance method, characterized in that, The method is applied to an operation and maintenance data center, and the method comprises: obtaining reported data reported through a message queue; performing warehouse storage verification on the reported data based on a verification model in usage metadata in the operation and maintenance data center; when the reported data passes the warehouse storage verification, performing warehouse storage processing on the reported data; performing operation and maintenance on the reported data stored in the warehouse based on a database in the operation and maintenance data center to obtain metadata; performing warehouse storage processing on the metadata; wherein the step of performing operation and maintenance on the reported data based on the database in the operation and maintenance data center to obtain metadata comprises: identifying field definition information of the reported data; determining field association information associated with the field definition information based on the database in the operation and maintenance data center; performing blood relationship maintenance on the reported data based on the field definition information and the field association information to obtain metadata.
2. The data operation and maintenance method of claim 1, wherein, The method further comprises: when the reported data fails the warehouse storage verification, recording verification failure information of the reported data.
3. The data operation and maintenance method of claim 1, wherein, The step of performing warehouse storage processing on the metadata comprises: obtaining warehouse storage task configuration parameters, and storing the metadata into the database based on the warehouse storage task configuration parameters.
4. The data operation and maintenance method of claim 1, wherein, The method further comprises: when a warehouse output task is detected, automatically taking data for a user API to obtain warehouse output task configuration parameters; judging whether the warehouse output task configuration parameters meet a preset service configuration; when the warehouse output task configuration parameters meet the preset service configuration, outputting the metadata.
5. A data operation and maintenance system, characterized in that, The data operation and maintenance system is applied to an operation and maintenance data center, and the data operation and maintenance system comprises: an obtaining unit configured to obtain reported data reported through a message queue; a verification unit configured to perform warehouse storage verification on the reported data based on a verification model in usage metadata in the operation and maintenance data center; an operation and maintenance unit configured to, when the reported data passes the warehouse storage verification, perform warehouse storage processing on the reported data; and perform operation and maintenance on the reported data stored in the warehouse based on a database in the operation and maintenance data center to obtain metadata; a warehouse storage unit configured to perform warehouse storage processing on the metadata; wherein the operation and maintenance unit comprises: an identification subunit configured to identify field definition information of the reported data; a determination subunit configured to determine field association information associated with the field definition information based on the database in the operation and maintenance data center; an operation and maintenance subunit configured to perform blood relationship maintenance on the reported data based on the field definition information and the field association information to obtain metadata. 6.The data operation and maintenance system of claim 5, characterized in that, The data operation and maintenance system further comprises: a recording unit configured to, when the reported data fails the warehouse storage verification, record verification failure information of the reported data. 7.The data operation and maintenance system of claim 5, characterized in that, The warehouse storage unit is specifically configured to obtain warehouse storage task configuration parameters, and store the metadata into the database based on the warehouse storage task configuration parameters.
8. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory is configured to store a computer program, and the processor is configured to run the computer program to enable the electronic device to perform the data operation and maintenance method in any one of claims 1 to 4.
9. A readable storage medium, characterized by, The readable storage medium stores computer program instructions, and the computer program instructions are read and run by a processor to execute the data operation and maintenance method in any one of claims 1 to 4.
Citation Information
Patent Citations
Engineering field knowledge graph construction method based on metadata model
CN113010696A
Data processing method, device and equipment for data-in-data station and readable storage medium
CN114546998A