A power supply detection method, device, storage medium and computer device
By using streaming computing and online analytical processing to detect power characteristic data in real time, the problem of untimely power detection in existing technologies is solved, and the efficiency of power anomaly detection is improved.
Patent Information
- Application Number
- CN202110003819.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-01-04
- Publication Date
- 2026-02-13
- Estimated Expiration
- 2041-01-04
AI Technical Summary
In existing technologies, server power supply detection solutions cannot detect power health status in a timely manner, resulting in low efficiency in handling anomalies and potential risks.
A streaming computing component is used to process power characteristic data in real time, store the data in a columnar storage database, and remove duplicates using a materialized view component. Real-time detection is achieved by combining online analytical processing.
It enables real-time detection of power supply anomalies, improving detection efficiency and reducing potential risks.
Smart Images

Figure CN114721495B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of data processing, in particular to a power supply detection method and device, a storage medium and a computer device. BACKGROUND
[0002] In recent years, with the advent of the era of big data, cloud computing and artificial intelligence, the Internet traffic has experienced a sharp increase, and the amount of calculation and the frequency of calculation have also increased. In the server system, the increase in business calculation has increased the carrying pressure of the server, and the overall power consumption of the server has also increased. Therefore, the stability requirement of the server is getting higher and higher, and the server cannot work without power supply, which puts forward higher requirements for the detection of the power supply state of the server.
[0003] In the prior art, the power supply detection scheme for the server mostly adopts offline calculation based on an offline calculation platform to perform offline calculation every day and then display the power supply data results. This offline calculation power supply detection scheme cannot timely detect the health status of the server power supply, and therefore the abnormal processing efficiency of the power supply equipment is relatively low, which may cause the occurrence of risk hidden dangers. SUMMARY
[0004] The embodiments of the present application provide a power supply detection method, device, storage medium and computer device. The power supply abnormality can be detected in real time, and the detection efficiency of the power supply abnormality can be improved.
[0005] A power supply detection method comprises the following steps:
[0006] According to the power supply characteristic data, first data is obtained, and the first data is stored in a first message queue to obtain second data;
[0007] The second data is processed by a stream computing component to obtain third data, and the third data is stored in a second message queue to obtain fourth data;
[0008] The fourth data is connected to a columnar storage database, and a materialized view component is created;
[0009] The fourth data is stored in a first table engine by the materialized view component to remove duplicates, and fifth data is obtained;
[0010] According to a query request, the fifth data is analyzed in real time by online analytical processing, an analysis result is displayed, and the abnormal condition of the power supply is detected based on the analysis result.
[0011] Correspondingly, the embodiments of the present application provide a power supply detection device, which comprises:
[0012] An acquisition unit is configured to obtain first data according to power supply characteristic data, and store the first data into a first message queue to obtain second data;
[0013] A processing unit is configured to perform stream computing processing on the second data by a stream computing component to obtain third data, and store the third data into a second message queue to obtain fourth data;
[0014] A creating unit is configured to access the fourth data into a columnar storage database, and create a materialized view component;
[0015] A storage unit is configured to store the fourth data into a first table engine by the materialized view component and perform deduplication to obtain fifth data;
[0016] A detection unit is configured to trigger real-time analysis on the fifth data based on an online analytical processing according to a query request, display an analysis result, and detect an abnormal condition of a power supply based on the analysis result.
[0017] In an embodiment, the creating unit comprises:
[0018] A first creating sub-unit is configured to create a second table engine by a columnar storage database, and the second table engine is configured to capture the fourth data;
[0019] An accessing sub-unit is configured to access the fourth data into the columnar storage database by the second table engine;
[0020] A second creating sub-unit is configured to create a materialized view component in the columnar storage database.
[0021] In an embodiment, the storage unit comprises:
[0022] A third creating sub-unit is configured to create a first table engine in the columnar storage database;
[0023] A writing sub-unit is configured to convert a format of the fourth data by the materialized view component and write the fourth data into the first table engine;
[0024] A clearing sub-unit is configured to compare contents between the fourth data, clear duplicated target fourth data, and store the fifth data in the first table engine.
[0025] In an embodiment, the acquisition unit is configured to:
[0026] acquire collected power supply characteristic data, and perform analysis processing on the collected power supply characteristic data;
[0027] merge the power supply characteristic data obtained by the analysis processing into one row of data according to attributes to obtain the first data.
[0028] In an embodiment, the detection unit comprises:
[0029] The acquisition subunit is configured to acquire offline data, wherein the offline data is used to assist in detecting abnormal conditions of the power supply.
[0030] The detection subunit is configured to trigger real-time analysis of the fifth data and the offline data together by using online analytical processing according to a query request, and display an analysis result.
[0031] In an embodiment, the acquisition subunit is configured to:
[0032] acquire sixth data, store the sixth data in a local storage database by offline calculation, and obtain offline data.
[0033] create a third table engine, and access the offline data to a columnar storage database by using the third table engine.
[0034] In an embodiment, the detection subunit is configured to:
[0035] acquire a query request;
[0036] associate the fifth data and an offline data source by using a machine serial number according to the query request, wherein the machine serial number is used to identify each server.
[0037] perform real-time analysis of the fifth data and the offline data of each server by using online analytical processing, and display an analysis result of a power supply of each server.
[0038] In addition, an embodiment of the present application further provides a storage medium, wherein the storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute steps in any power supply detection method provided by the embodiment of the present application.
[0039] In addition, an embodiment of the present application further provides a computer device, comprising a processor and a memory, wherein the memory stores an application program, and the processor is used to run the application program in the memory to implement a power supply detection method provided by the embodiment of the present application.
[0040] The embodiment of the application obtains first data according to power supply characteristic data, stores the first data into a first message queue to obtain second data, performs stream computing processing on the second data through a stream computing component to obtain third data, stores the third data into a second message queue to obtain fourth data, accesses the fourth data to a columnar storage database, and creates a materialized view component; stores the fourth data into a first table engine through the materialized view component and performs deduplication to obtain fifth data; triggers online analytical processing on the fifth data for real-time analysis according to a query request, displays an analysis result, and detects an abnormal condition of the power supply based on the analysis result. In this way, the embodiment of the application performs real-time processing on the obtained second data through stream computing processing, stores the data obtained through real-time processing into a columnar storage database, triggers online analytical processing on the fifth data stored in the columnar storage database for real-time processing according to an obtained query request, and then realizes fast query analysis to obtain an analysis result, and realizes real-time detection of the power supply abnormality based on the analysis result, thereby improving the detection efficiency of the power supply abnormality. BRIEF DESCRIPTION OF DRAWINGS
[0041] In order to more clearly illustrate the technical solutions in the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.
[0042] Figure 1 is a power supply detection method implementation scene schematic diagram provided by the embodiment of the application;
[0043] Figure 2 is a block chain block architecture flow schematic diagram provided by the embodiment of the application;
[0044] Figure 3 is a power supply detection method flow schematic diagram provided by the embodiment of the application;
[0045] Figure 4 is a server power supply abnormality detection architecture schematic diagram based on offline computing provided by the embodiment of the application;
[0046] Figure 5 is another power supply detection method flow schematic diagram provided by the embodiment of the application;
[0047] Figure 6 is a power supply abnormality real-time detection architecture schematic diagram based on offline computing provided by the embodiment of the application;
[0048] Figure 7 is a power supply detection device structure schematic diagram provided by the embodiment of the application;
[0049] Figure 8 FIG. 1 is a structural schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0050] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.
[0051] The present application provides a power detection method and device, a storage medium and a computer device. The power detection device can be integrated in the computer device, which can be a server or a terminal device.
[0052] To better illustrate the embodiments of the present application, the following terms are referred to:
[0053] A database (Database) can be briefly regarded as an electronic file cabinet, i.e., a place for storing electronic files. Users can add, query, update, delete, and perform other operations on the data in the files. A "database" is a collection of data stored together in a certain way, shared by multiple users, with as little redundancy as possible, and independent of application programs.
[0054] A database management system (Database Management System, DBMS for short) is a computer software system designed to manage databases, generally having basic functions such as storage, interception, security assurance, backup, etc. Database management systems can be classified according to the database model they support, such as relational, XML (Extensible Markup Language, XML for short); or according to the type of computer they support, such as server clusters, mobile phones; or according to the query language they use, such as SQL (Structured Query Language, SQL for short), XQuery; or according to the performance focus, such as maximum size, highest running speed; or other classification methods. Regardless of the classification method used, some DBMSs can cross categories, such as supporting multiple query languages at the same time.
[0055] Offline computing refers to computing that is performed under the premise that all input data is known before the computing starts, the input data does not change, and the result is obtained immediately after a problem is solved. In big data, the computing part of data corresponds to real-time computing in which offline computing is performed.
[0056] Real-time computing is a kind of low-latency data processing computing, which is almost real-time.
[0057] Message queue refers to a kind of container for saving messages in the process of message transmission.
[0058] Table engine (i.e., the type of table) is a kind of storage engine that can determine the storage mode and location of data, where data is written and read from, which queries are supported and how to support them, concurrent data access, the use of indexes (if any), whether multi-threaded requests can be executed, and data replication parameters.
[0059] ClickHouse is a columnar database management system (DBMS) for online analytical processing (OLAP).
[0060] In the traditional data processing flow, data is always collected first and then put into a database. When people need it, the data is queried from the database to get the answer or to perform related processing. This seems very reasonable, but the result is very tight, especially in some specific problems in real-time search application environment. Offline processing in the manner of MapReduce cannot solve the problem well. This leads to a new data computing structure, stream computing. It can well analyze large-scale streaming data in the process of continuous change and motion, capture possible useful information, and send the result to the next computing node.
[0061] Online Analytical Processing (OLAP) is a software technology that enables analysts to quickly, consistently, and interactively view information from various perspectives to gain insight into the data. It has the characteristics of FASMI (Fast Analysis of Shared Multidimensional Information). F is Fast, which means that the system can respond to most of the user's analysis requirements within seconds; A is Analysis, which means that the user can define new specialized calculations as part of the analysis without programming and give reports in the way the user wants; M is Multi-dimensional, which means that it provides a multi-dimensional view and analysis of data analysis; I is Information, which means that information can be obtained in time and large amounts of information can be managed.
[0062] The power characteristic data refers to data attributes that can represent the health status of the server power supply, such as power supply redundancy status, power supply input power, power supply output power, power supply input voltage, and power supply output voltage.
[0063] The TOC out-of-band data is power supply data that can be used to assist in determining the health status of the server power supply. The TOC is a data acquisition system, and the out-of-band is a data acquisition channel.
[0064] Please refer to Figure 1 , Figure 1 The implementation environment scenario of the power detection method provided by the embodiments of the present application includes a data sharing system 10, a node 11, and a server 12. The server 12 can also be a node in the data sharing system 10, and the server 12 obtains data in the node 11, such as power characteristic data, through the data sharing system 10. The data sharing system 10 and the server 12 or the nodes 11 can be directly or indirectly connected through wired or wireless communication, which is not limited in the present application.
[0065] The data sharing system 10 refers to a system for data sharing between nodes, and can include a plurality of nodes 11, which can be clients or servers in the data sharing system. Each node 11 can receive input information during normal operation and maintain shared data in the data sharing system based on the received input information. To ensure information exchange in the data sharing system, each node in the data sharing system can have an information connection, and the nodes can transmit information through the information connection. For example, when any node in the data sharing system receives input information, other nodes in the data sharing system can obtain the input information according to a consensus algorithm, and store the input information as data in the shared data, so that the data stored on all nodes in the data sharing system is consistent.
[0066] Each node in the data sharing system has a corresponding node identifier, and each node in the data sharing system can store node identifiers of other nodes in the data sharing system, so that the generated block can be broadcast to other nodes in the data sharing system according to the node identifiers of the other nodes. Each node can maintain a node identifier list as shown in the following table, and store the node name and node identifier in the node identifier list. The node identifier can be an IP (Internet Protocol) address or any other information that can be used to identify the node. Please refer to Table 1, which only uses IP addresses as an example for illustration.
[0067] Node name Node identification Node 1 117.114.151.174 Node 2 117.116.189.145 … … Node N 119.123.789.258
[0068] Table 1
[0069] Each node in the data sharing system stores a same block chain. The block chain is composed of a plurality of blocks, as shown in Figure 2 The block chain is composed of a plurality of blocks, and the genesis block includes a block header and a block body. The block header stores an input information feature value, a version number, a timestamp, and a difficulty value. The block body stores input information, which can include power supply characteristic data of power supply equipment, such as power supply characteristic data of each server. The next block of the genesis block takes the genesis block as a parent block, and also includes a block header and a block body. The block header stores an input information feature value of the current block, a block header feature value of the parent block, a version number, a timestamp, and a difficulty value. This is true for each block in the block chain, so that the block data stored in each block is associated with the block data stored in the parent block, ensuring the security of the input information in the block.
[0070] When generating each block in the blockchain, the node where the blockchain is located checks the input information when receiving the input information, stores the input information into the memory pool after completing the check, and updates the hash tree used to record the input information; then, the update timestamp is updated to the time when the input information is received, and different random numbers are tried to perform feature value calculation multiple times, so that the calculated feature value can satisfy the following formula:
[0071] SHA256(SHA256(version+prev_hash+merkle_root+ntime+nbits+x))<TARGET
[0072] Wherein, SHA256 is a feature value algorithm used for calculating the feature value; version (version number) is version information of the relevant block protocol in the blockchain; prev_hash is the block header feature value of the parent block of the current block; merkle_root is the feature value of the input information; ntime is the update time of the update timestamp; nbits is the current difficulty, which is a constant value within a period of time, and is determined again after a fixed period of time; x is a random number; TARGET is a feature value threshold, which can be determined according to nbits.
[0073] In this way, when the random number satisfying the above formula is calculated, the information can be stored correspondingly, the block header and the block body are generated, and the current block is obtained. Subsequently, the node where the blockchain is located sends the newly generated block to other nodes in the data sharing system according to the node identifier of the other nodes in the data sharing system, and the other nodes check the newly generated block, and add the newly generated block to the stored blockchain after completing the check
[0074] The server 12 can be a stand-alone physical server, a server cluster or a distributed system composed of multiple physical servers, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, network acceleration services (CDN), and basic cloud computing services such as big data and artificial intelligence platforms. The server 12 can obtain first data according to power characteristic data, store the first data into a first message queue to obtain second data, perform stream computing processing on the second data through a stream computing component to obtain third data, store the third data into a second message queue to obtain fourth data, access the fourth data to a columnar storage database, and create a materialized view component. The fourth data is stored into a first table engine through the materialized view component and is de-duplicated to obtain fifth data. Real-time analysis is performed on the fifth data through online analytical processing according to a query request, and an analysis result is displayed. The abnormal condition of the power supply is detected based on the analysis result.
[0075] It should be noted that Figure 1 The implementation environment scene diagram of the power supply detection method shown is only an example, and the implementation environment scene of the power supply detection method described in the embodiments of the present application is used to more clearly illustrate the technical solutions of the embodiments of the present application, and does not constitute a limitation on the technical solutions provided by the embodiments of the present application.
[0076] The following will be described in detail. It should be noted that the description order of the following embodiments does not limit the preferred order of the embodiments.
[0077] In this embodiment, the power supply detection device will be described from the perspective of the power supply detection device, which can be integrated in a server. The server can be a single server or a server cluster composed of multiple servers.
[0078] Please refer to Figure 3 , Figure 3 The flowchart of the power supply detection method provided by the embodiments of the present application is shown. The power supply detection method comprises the following steps:
[0079] In step 101, first data is obtained according to power characteristic data, and the first data is stored into a first message queue to obtain second data.
[0080] In an embodiment, the power characteristic data of each server can be collected and parsed to obtain first data in a unified data format that can be recognized and processed by a stream computing framework, wherein the first data is a real-time data stream. In order to decouple and cache, the first data is written into a first message queue, that is, the first data is stored in the first message queue to obtain second data. The data stream represented by the second data is transmitted through the first message queue for data consumption by a downstream consumer, which can be a stream computing component or the like. The first message queue is Kafka, which is a cluster developed by the Apache Software Foundation and is an open source stream processing platform written in Scala and Java. Kafka is a high-throughput distributed message queue based on the publish / subscribe mode, which mainly provides three core capabilities: publishing function for data producers and subscribing function for data consumers, that is, the ability of traditional message queues; buffering data streams in a cache area to provide fault tolerance for data and have certain data storage capacity; and providing some lightweight stream processing capabilities. The purpose of Kafka is to unify online and offline message processing through the parallel loading mechanism of Hadoop Distributed FileSystem (Hadoop for short), and to provide real-time messages through clusters, with stream data storage capacity to queue the written data in a stream data manner for subsequent stream computing processing.
[0081] In the prior art, the power characteristic data is stored and displayed as one power characteristic per row. However, the characteristics that actually represent the health status of a server generally have multiple characteristics, such as power redundancy status, power input power, power output power, power input voltage, and power output voltage. Therefore, in order to determine the health status of a server, multiple characteristic data need to be manually combined and analyzed, which brings inconvenience to the analysis of power characteristic data. In order to solve this problem, in an embodiment, the power characteristic data can be combined according to the attributes during the parsing process to store one row of data. In an embodiment, the parsed power characteristic data can also be combined according to the attributes to store one row of data.
[0082] The collected data is stored in the form of files in the server directory, and the content conforms to certain format specifications. The parsing layer reads the content of these files. For example, it is assumed that there are ten characteristics representing the health status of the power supply, i.e., there are ten power supply characteristic fields. These ten fields can be divided into three acquisition tasks and collected in parallel and distributed in multiple files. In order to merge the power supply characteristic data according to the attributes into a row of data for storage, the characteristics collected by the three tasks can be combined into a data record of ten characteristics according to the field attributes, and then stored, thereby achieving the purpose of storing the data as a row of data. Each row in the table is called a "record", and each record contains all the information in this row, just like the complete information of a person in a contact database. Fields are smaller units than records, and fields form a record. Each field describes a certain feature of the literature, i.e., a data item, and has a unique field identifier for computer recognition. In a database, most of the time, the "columns" of the table are called "fields", and each field contains information on a certain topic, just like the "contact database" database, "name", "contact number" and other attributes shared by all rows in the table. Therefore, these columns are called "name" field and "contact number" field. Therefore, by merging the power supply characteristic data according to the attributes into a row of data for storage, the following power supply characteristic table can be obtained, please refer to Table 2.
[0083]
[0084]
[0085] Table 2
[0086] In the existing server power supply anomaly detection scheme, the power supply characteristic data is calculated based on an offline computing platform on a daily basis, and then exported to the local storage on a daily basis. Please refer to Figure 4 , Figure 4The application discloses a server power abnormality detection architecture based on offline calculation. In the architecture, power characteristic data of a server is collected by a machine and is analyzed and processed. Power data, including the analyzed power characteristic data, server whole machine data and TOC out-of-band data, is accessed to a data warehouse (DW) through a data access module. The power data is subjected to a data cleaning task and is written back to the data warehouse. The power data is exported from the data warehouse to a local storage database through a data export scheduling task. The local storage database can be a relational database (MySQL). The power data is subjected to background processing of a server data platform (Sdata Svr) and is displayed by a front end (Sdata Web). A user can input a query condition through the server data platform to query power abnormality data of a target server. Other services or systems can query the power abnormality data through an application programming interface (Sdata API) service of the server data platform, and perform alarm, email or short message sending and other related operation work. The server whole machine data can refer to Table 3, and the TOC out-of-band data can refer to Table 4.
[0087]
[0088]
[0089] Table 3
[0090] Field Table field Type Server sn Server sn String Whether out-of-band whitelist Whether out-of-band whitelist String Whether out-of-band theoretically available Whether out-of-band theoretically available String Whether out-of-band actually available Whether out-of-band actually available String
[0091] Table 4
[0092] In the server power abnormality detection architecture based on offline calculation, the power abnormality detection scheme in the prior art cannot detect the health state of the server power in time, and thus the abnormality processing efficiency of the power equipment is relatively low, and risk hidden dangers can be caused. In order to solve the problem that the health state of the server power cannot be detected in time, the application embodiment introduces a streaming processing component (Flink) and a columnar storage database (Clickhouse) based on Hadoop offline calculation. The real-time collected power characteristic data is subjected to real-time streaming processing and is stored in the columnar storage database. The relatively stable server whole machine data and TOC out-of-band data can be subjected to offline calculation through the server power abnormality detection architecture based on offline calculation in the prior art, and are stored in the local storage database. The power characteristic data subjected to real-time calculation is also accessed to the columnar storage database, and real-time query analysis is realized through online analytical processing. The health state of the server power is comprehensively judged through online analytical processing results of the three data sources, so that the power abnormality can be detected in real time, and the detection efficiency of the power abnormality is improved. The specific implementation process is described below.
[0093] In step 102, the second data is processed by the stream computing component to obtain third data, and the third data is stored in the second message queue to obtain fourth data.
[0094] In order to be able to detect the abnormal condition of the power supply in real time, it is necessary to obtain real-time power supply characteristic data for real-time query analysis, and therefore, the stream computing component can be used to process the obtained power supply characteristic data including historical data and real-time data to obtain real-time data for real-time analysis to obtain the real-time power supply state.
[0095] The stream computing processing is processing of unbound data, that is, data stream, and after the upstream operator of the stream computing processing completes processing of a piece of data, the data is immediately sent to the downstream operator, for example, the message queue Kafka, and therefore, the stream computing can process the data stream in real time to obtain the calculation result of the real-time data. In the commonly used stream computing framework, the Flink stream computing engine not only provides lower latency than other stream computing engines, but also provides better support for windows and event time, and these advantages of the Flink stream computing engine make real-time detection and query of the server power supply abnormality more convenient. Since the data stream has the requirement of real-time, the most recently arrived data in the data stream is more meaningful to the calculation than the data arrived a long time ago, and therefore, the stream computing component uses the window mechanism to describe the “recent” data in the data stream, that is, real-time data, and the event time refers to the time when the data is actually generated. The stream computing component pulls the data from the Kafka in real time for aggregation calculation, and at the same time, divides the data into time windows for calculation, groups the data in each calculation window according to the preset index, and performs aggregation calculation according to the event time of the data to process out-of-order data and delayed arrival data, thereby realizing stream computing processing of the data. The aggregation refers to selection, analysis, and classification of the content of related data, and finally analysis to obtain the result that people want, mainly refers to any data conversion process that can generate a scalar value from an array. The Flink stream computing engine is a low-latency, high-throughput, and unified big data computing engine, and the core is a stream data stream execution engine, which provides data distribution, data communication, and fault tolerance mechanism for distributed computing of data stream, and is the best performance and most widely used distributed stream computing framework, and has greater advantages compared with other stream computing engines.
[0096] In an embodiment, the Kafka cluster serves as input and output of the upstream and downstream of the stream computing component, the server obtains second data in the first message queue, and performs stream computing on the data through the stream computing component (Flink), that is, performs data cleaning and computing on the real-time data stream, wherein the data cleaning refers to filtering out "dirty" data that does not meet the requirements, and the "dirty" data can be data that is out of the normal range, logically unreasonable or contradictory, at the same time, the window is used to describe the real-time data, and the window is divided according to the event time to perform ordered processing on the data, the second data is aggregated and calculated based on the window and the event time to obtain real-time computing results, that is, third data, and the third data is real-time stored (that is, Sink) in the second message queue (Kafka), so that the third data is stored in the second message queue to obtain fourth data. Wherein, Sink refers to the data after the completion of distributed processing, which is similar to a receiver, and can also be understood as a place where the stream computing component sends the data after conversion and calculation, in the embodiment of the application, the third data after stream computing is Sinked into the second message queue, and a custom Sink is used. Wherein, the first message queue and the second message queue are both a message queue in the Kafka cluster, and the first message queue receives data in a unified format and sends it to the stream computing component, which is mainly used for caching massive data, similar to a data storage pool, to avoid excessive data received from affecting the subsequent stream computing component.
[0097] In an embodiment, in order to facilitate the data after stream computing to be accessed into the columnar storage database to realize the query and analysis of the data in the columnar storage database by online analytical processing, the third data after stream computing can be stored in the second message queue, because the columnar storage database has a message queue engine (KafkaEngine) therein, the message queue engine can be connected with the second message queue, and further, the data in the message queue engine can be grabbed through the creation of a materialized view component (Materialized View), so that the connection between the columnar storage database and the second message queue is easily realized, and therefore, the message queue engine makes the connection between the columnar storage database and the second message queue easier.
[0098] In step 103, the fourth data is accessed into the columnar storage database, and the materialized view component is created.
[0099] In an embodiment, in order to enable online analysis and processing of real-time power supply characteristic data, the data obtained through the stream computing processing can be stored in the columnar storage database, at this time, the fourth data in the second message queue is obtained, and the fourth data is connected to the columnar storage database, for example, by creating a message queue table engine in the columnar storage database as an interface of the second message queue data source, which can also be understood as a consumer of the data in the second message queue, and the fourth data in the second message queue is connected to the columnar storage database through the interface of the message queue table engine. In order to store the fourth data in the columnar storage database, a materialized view component is created in the columnar storage database as a data channel for storing the fourth data from the message queue table engine to the columnar storage database. The materialized view is a database object including a query result, which is a local copy of remote data or is used to generate summary tables based on data table summation. The materialized view stores data based on remote tables, which can also be called snapshots, and the data in a table or view can be grabbed through the materialized view.
[0100] In step 104, the fourth data is stored in the first table engine through the materialized view component and de-duplication is performed to obtain fifth data.
[0101] In an embodiment, in order to store the fourth data in the second message queue in the columnar storage database, a first table engine can be created in the columnar storage database, which is used to store the fourth data in the second message queue, the fourth data in the second message queue is grabbed through the materialized view component, and the data is written into the first table engine, the first table engine is used to remove duplicate fourth data and store the data obtained after the removal processing, so as to realize the storage of the fourth data in the columnar storage database cluster after de-duplication, and obtain the fifth data. The materialized view component plays a role as an intermediate pipeline, which can detect the data in the second message queue in the background, continuously collect the fourth data from the second message queue through the message queue table engine, and convert the collected data into the required format through the Select function.
[0102] In an embodiment, in order to remove duplicate data to save space, the fourth data grabbed by the materialized view component is written into the first table engine, and the first table engine (Replacing MergeTree) belongs to the merge tree series engine in the columnar storage database, and the difference from the merge tree (MergeTree) engine is that it deletes the duplicate items with the same sorting key value, that is, the de-duplication function can be realized. The de-duplication of data is only performed during data merging, and the merging is performed in the background at an uncertain time, so it cannot be planned in advance, therefore, the first table engine is suitable for removing duplicate data in the background to save space and reduce the complexity of data.
[0103] In step 105, real-time analysis of the fifth data is triggered by online analytical processing according to the query request, the analysis result is displayed, and the abnormal condition of the power supply is detected based on the analysis result.
[0104] In an embodiment, the user inputs the query condition on the query page displayed on the front end of the server data platform, the server obtains the query request, and the background of the server data platform is triggered to access the columnar storage database to obtain the fifth data stored in the columnar storage database according to the query request. The fifth data is analyzed by online analytical processing, and the analysis result is displayed on the front page of the server data platform. The dimension refers to the analysis angle. Online analytical processing displays a multi-dimensional view in front of the user. These multi-dimensional views enable the user to quickly, consistently, and interactively observe information from various aspects to achieve the purpose of in-depth understanding of data. Based on real-time query and display of the analysis result, the power supply characteristic data is quickly detected. The columnar storage database (ClickHouse) is a columnar database management system (DBMS) for online analytical processing (OLAP) open sourced by Yandex in 2016, abbreviated as CK, mainly used for online analytical processing query (OLAP), and capable of generating analysis data reports in real time using SQL queries. The main feature of online analytical processing is to directly imitate the multi-angle thinking mode of the user and pre-build a multi-dimensional data model for the user. Therefore, by analyzing the fifth data stored in the columnar storage database through online analytical processing, the power supply abnormality can be detected in real time and efficiently.
[0105] As can be seen from the above, the embodiment of the present application obtains the first data according to the power supply characteristic data, stores the first data into the first message queue to obtain the second data, performs stream computing processing on the second data through the stream computing component to obtain the third data, stores the third data into the second message queue to obtain the fourth data, accesses the fourth data to the columnar storage database, and creates a materialized view component. The fourth data is stored into the first table engine through the materialized view component and de-duplicated to obtain the fifth data. Real-time analysis of the fifth data is triggered by online analytical processing according to the query request, the analysis result is displayed, and the abnormal condition of the power supply is detected based on the analysis result. Therefore, the embodiment of the present application introduces the stream computing component to perform real-time processing on the obtained second data, stores the data obtained by real-time processing into the columnar storage database, triggers online analytical processing according to the query request to perform real-time processing on the fifth data stored in the columnar storage database, and realizes fast query analysis, so that the power supply abnormality can be detected in real time through the analysis result, and the detection efficiency of the power supply abnormality is improved.
[0106] According to the method described in the above embodiment, the following will be further illustrated in detail.
[0107] In this embodiment, the power detection device will be specifically integrated in a computer device. The power detection method takes a server as the execution subject, and simultaneously utilizes an offline computing framework and real-time computing processing to cooperate with real-time detection of power abnormalities.
[0108] As shown in Figure 5 , another flowchart of the power detection method provided by the embodiment of the present application is shown. Figure 5 The specific process is as follows:
[0109] For better illustration of the embodiment of the present application, please refer to Figure 6 for reference, Figure 6 a flowchart of a power abnormality real-time detection architecture based on offline computing provided by the embodiment of the present application.
[0110] In step 201, the server obtains first data according to power characteristic data, and stores the first data into a first message queue to obtain second data.
[0111] In an embodiment, the collected power characteristic data is acquired, and the collected power characteristic data is processed for analysis. The power characteristic data is analyzed into a unified data format, which can be recognized and processed by a message queue and a stream computing component. The power characteristic data is composed of multiple power characteristics, but in the prior art, the data of the multiple power characteristics is stored and displayed separately. Therefore, if the health status of a server is to be determined, multiple data needs to be manually combined and analyzed before the determination. In order to solve this problem and facilitate the analysis of the power health status, the data fields of the multiple power characteristics can be combined into one row of data records for storage in the analysis process, so as to realize the combination of the data of the multiple power characteristics. Based on the above data processing, the first data is obtained from the collected power characteristic data.
[0112] In an embodiment, the server acquires power characteristic data and processes the power characteristic data for analysis to obtain first data. The first data is stored into a first message queue. Before being sent into a stream computing framework, the main role of receiving the data stored in the first message queue in a unified format is to cache massive data, which is similar to a data reservoir, so as to avoid the subsequent distributed stream computing framework from being unable to process due to the excessively large and excessive data received.
[0113] In step 202, the server processes the second data through a stream computing component to obtain third data, and stores the third data into a second message queue to obtain fourth data.
[0114] In an embodiment, the server performs real-time stream processing on the second data in the first message queue through a stream computing component to obtain third data, the first message queue is equivalent to a large cache queue, and the real-time stream computing component processes by subscribing to messages from the first message queue. In an embodiment, in order to facilitate storage of the third data processed by stream computing into the columnar storage database, the third data processed by stream computing is stored into the second message queue, because the columnar storage database has a message queue table engine by itself, so that the connection between the columnar storage database and the second message queue becomes simple, and therefore the third data in the second message queue can be more easily written into the columnar storage database to obtain fourth data. Wherein, the stream computing component processes the real-time data stream represented by the second data in the first message queue in real time, and generates a data stream result in real time, as long as the data source continuously comes, the stream computing component can continue to calculate.
[0115] In step 203, the server creates a second table engine in the columnar storage database, and accesses the fourth data into the columnar storage database through the second table engine.
[0116] In an embodiment, the server creates a second table engine (Kafka Engine) in the columnar storage database, which serves as an interface of the second message queue to the columnar storage database, and is used to grab the fourth data from the second message queue, so that the fourth data in the second message queue can be accessed into the columnar storage database through the second table engine.
[0117] In step 204, the server creates a materialized view component in the columnar storage database, and creates a first table engine in the columnar storage database.
[0118] In an embodiment, in order to store the data processed by stream computing in the columnar storage database, the server creates a materialized view component and a first table engine through the columnar storage database, the materialized view component collects the fourth data in the second message queue through the second table engine in the background, and then stores the collected fourth data into the first table engine.
[0119] In step 205, the server converts the fourth data through the materialized view component and writes it into the first table engine.
[0120] In an embodiment, in order to be able to eliminate duplicate data and store data into the columnar storage database, the server can continuously collect data from the second message queue through the materialized view component and read the fourth data through the select function (Select) and convert the data into the required format, the materialized view component can grab data from a table or view, and write the processed data into the first table engine used to store the data written by the materialized view component.
[0121] In step 206, the server compares the contents between the fourth data, eliminates the target fourth data that is duplicated, and stores it in the first table engine to obtain the fifth data.
[0122] In an embodiment, the server can obtain the fourth data through the first table engine, the contents of the fourth data in the first table engine are sorted according to the sorting key, the server compares the adjacent data between the sorted fourth data two by two, merges the target fourth data that is duplicated to achieve the purpose of deduplication, and stores the data after deduplication into the first table engine to obtain the fifth data. In an embodiment, in order to obtain the unique sorting key and the primary key, an MD5 value can be generated from the combination field of multiple power characteristics of the power characteristic data, and the MD5 value is used as the sorting key and the primary key. Specifically, it can be as follows:
[0123] LOWER(hex(MD5(
[0124] CONCAT(sn,power_index,power_redundancy_status,power_inputpower,power_outptpower,power_input_voltage,power_output_voltage,power_status)
[0125] )))AS md5value
[0126] In step 207, the server obtains the sixth data, stores the sixth data into the local storage database through offline calculation, and obtains offline data.
[0127] Please continue to refer to Figure 6The data sources representing the health status of the server power supply include power supply characteristic data, server whole machine data, and TOC out-of-band data. In actual cases, the power supply characteristic data changes in real time at irregular intervals, so the power supply characteristic data needs to be calculated and queried in real time. The server whole machine data and the TOC out-of-band data are relatively stable data, and the real-time detection of the power supply health status can be met by offline calculation and processing. Therefore, the embodiment of the present application realizes real-time detection of power supply abnormalities through the offline calculation-based power supply abnormality real-time detection architecture diagram as shown in Figure 6
[0128] In an embodiment, the server obtains sixth data including server whole machine data and TOC out-of-band data. The sixth data can be further referred to Tables 3 and 4, and the sixth data is accessed to the data warehouse through the data access module. The data warehouse can be a Tencent data warehouse. After completing the offline calculation task, the export task is started, and the sixth data is stored in the local storage database, which can be a relational database (MySQL) and the like, so as to obtain offline data.
[0129] In step 208, the server creates a third table engine, and accesses the offline data to the columnar storage database through the third table engine.
[0130] In order to access the offline data stored in the local storage database to the columnar storage database, in an embodiment, the server creates a third table engine, which can be a MergeTree table engine and the like. When the task of exporting to the local storage database is completed, the offline data in the local storage database is accessed to the columnar storage database once, so that the offline data is accessed to the columnar storage database through the third table engine.
[0131] In step 209, the server obtains a query request, associates the fifth data and the offline data source according to the query request through the machine serial number, uses online analytical processing to analyze the fifth data and the offline data of each server in real time, and displays the analysis result of the power supply of each server.
[0132] In an embodiment, the three data sources of power supply characteristic data, server whole machine data and TOC out-of-band data are associated by machine serial number (Serial Number), which is used to uniquely identify a server. Through the association of the machine serial number, the fifth data and offline data of each server can be combined and analyzed by online analytical processing. Please continue to refer to the power supply characteristic data table, the server whole machine data table and the TOC out-of-band data table in Table 1, Table 2 and Table 3. From the contents in the tables, it can be seen that each data table has a "server sn" field, which is the machine serial number. The data in the three tables are associated by the machine serial number. Among them, the user can perform query operation through the front-end page of the server data platform. The server obtains the query request, thereby triggering the business background of the server data platform to access the columnar storage database, and then performing real-time analysis on the fifth data and offline data of each server by online analytical processing. The query result is displayed to the front-end of the server data platform for display. The specific fields of the display can be referred to Table 5.
[0133]
[0134]
[0135] Table 5
[0136] The user realizes real-time detection of power supply abnormalities and judges the health status of the power supply through the query result. For example, the query conditions supported by the server data platform can be as follows:
[0137] (1) Server SN: manually input, support multiple selection, no need for drop-down box;
[0138] (2) Computer room, park, computer room management unit, module, rack: input key dynamic drop-down association display, support multiple selection, support cascading;
[0139] (3) Machine type: input key dynamic drop-down association display, support multiple selection;
[0140] (4) Device type: input key dynamic drop-down association display, support multiple selection;
[0141] (5) Major version number: drop-down box, full initialization;
[0142] (6) Cascading relationship: park--> computer room--> computer room management unit--> MODULE--> rack.
[0143] As described above, in this embodiment, the server obtains first data based on power characteristic data, stores the first data in a first message queue to obtain second data; the server performs streaming processing on the second data using a streaming computing component to obtain third data, stores the third data in a second message queue to obtain fourth data; the server creates a second table engine using a columnar storage database, and uses the second table engine to access the fourth data into the columnar storage database; the server creates a materialized view component in the columnar storage database, and creates a first table engine in the columnar storage database; the server uses the materialized view component to convert the format of the fourth data and writes it into the first table engine; the server compares the contents of the fourth data, removes duplicate target fourth data, and stores it in the first table engine to obtain fifth data; the server obtains sixth data, stores the sixth data in a local storage database through offline computing to obtain offline data; the server creates a third table engine, and accesses the offline data into the columnar storage database through the third table engine; the server obtains a query request, associates the fifth data with the offline data source through the machine serial number in the query request, and uses online analytical processing to perform real-time analysis on the fifth data and offline data of each server, displaying the power analysis results of each server. Therefore, this embodiment of the application introduces a streaming computing component to process the acquired second data in real time and stores the data obtained from the real-time processing in a columnar storage database. The sixth data is then processed offline and stored in a local storage database. The sixth data is then accessed through a third engine in the columnar storage database. Based on the combination of offline computing framework and real-time computing, online analytical processing is used to process the fifth data stored in the columnar storage database and the offline data in real time, thereby achieving rapid query analysis. Based on the analysis results, real-time detection of power supply anomalies can be achieved, improving the efficiency of power supply anomaly detection.
[0144] To better implement the above methods, this application also provides a power detection device, which can be integrated into network devices, such as servers or terminals. The terminal may include tablet computers, laptop computers, and / or personal computers.
[0145] For example, such as Figure 7 As shown, Figure 7 This is a schematic diagram of the power detection device provided in an embodiment of this application. The power detection device may include an acquisition unit 301, a processing unit 302, a creation unit 303, a storage unit 304, and a detection unit 305, as follows:
[0146] The acquisition unit 301 is used to obtain first data based on power characteristic data, and store the first data in a first message queue to obtain second data;
[0147] The processing unit 302 is configured to perform stream computing processing on the second data by a stream computing component to obtain third data, and store the third data into a second message queue to obtain fourth data.
[0148] The creating unit 303 is configured to access the fourth data into a columnar storage database, and create a materialized view component.
[0149] The storage unit 304 is configured to store the fourth data into a first table engine by the materialized view component and perform deduplication to obtain fifth data.
[0150] The detection unit 305 is configured to trigger real-time analysis on the fifth data based on a query request, display an analysis result, and detect an abnormal condition of the power supply based on the analysis result.
[0151] In an embodiment, the creating unit 303 comprises:
[0152] The first creating sub-unit is configured to create a second table engine by the columnar storage database, and the second table engine is configured to capture the fourth data.
[0153] The accessing sub-unit is configured to access the fourth data into the columnar storage database by the second table engine.
[0154] The second creating sub-unit is configured to create a materialized view component in the columnar storage database.
[0155] In an embodiment, the storage unit 304 comprises:
[0156] The third creating sub-unit is configured to create a first table engine in the columnar storage database.
[0157] The writing sub-unit is configured to convert the fourth data into a format and write the fourth data into the first table engine by the materialized view component.
[0158] The clearing sub-unit is configured to compare contents between the fourth data, clear duplicated target fourth data, and store the target fourth data in the first table engine to obtain the fifth data.
[0159] In an embodiment, the obtaining sub-unit is configured to:
[0160] Obtain the collected power supply characteristic data, and perform analysis processing on the collected power supply characteristic data.
[0161] Merge the power supply characteristic data obtained by the analysis processing into one row of data according to attributes to obtain first data.
[0162] In an embodiment, the detection unit 305 comprises:
[0163] An obtaining subunit is configured to obtain offline data, which is used to assist in detecting abnormal conditions of the power supply;
[0164] A detecting subunit is configured to trigger real-time analysis of the fifth data and the offline data together by online analytical processing according to a query request, and display an analysis result.
[0165] In an embodiment, the obtaining subunit is configured to:
[0166] Obtain sixth data, store the sixth data into a local storage database by offline calculation, and obtain offline data;
[0167] Create a third table engine, and access the offline data into a columnar storage database through the third table engine.
[0168] In an embodiment, the detecting subunit is configured to:
[0169] Obtain a query request;
[0170] Associate the fifth data and the offline data source according to a machine serial number, which is used to identify each server, according to the query request;
[0171] Use online analytical processing to perform real-time analysis on the fifth data and the offline data of each server, and display an analysis result of the power supply of each server.
[0172] In a specific implementation, each of the above subunits can be implemented as an independent entity, or can be combined as the same or several entities. The specific implementation of each of the above subunits can be referred to the method embodiments above, and will not be described herein.
[0173] From the above, the embodiment of the application obtains first data according to the power supply characteristic data by the acquisition unit 301, stores the first data into the first message queue to obtain second data; the processing unit 302 performs stream computing processing on the second data by a stream computing component to obtain third data, stores the third data into the second message queue to obtain fourth data; the creating unit 303 accesses the fourth data to the columnar storage database and creates a materialized view component; the storage unit 304 stores the fourth data to the first table engine by the materialized view component to perform deduplication to obtain fifth data; the detection unit 305 triggers online analytical processing on the fifth data for real-time analysis according to a query request, displays an analysis result, and detects an abnormal condition of the power supply based on the analysis result. In this way, the embodiment of the application introduces a stream computing component to perform real-time processing on the obtained second data, stores the data obtained by the real-time processing into the columnar storage database, performs real-time processing on the fifth data stored in the columnar storage database by online analytical processing, and thus realizes fast query analysis, so that the real-time detection of the power supply abnormality can be realized according to the analysis result, and the detection efficiency of the power supply abnormality is improved.
[0174] The embodiment of the application further provides a computer device, as shown in the figure, which shows a structural schematic diagram of the computer device related to the embodiment of the application. The computer device can be a server, and specifically: Figure 8
[0175] The computer device can include a processor 401 with one or more processing cores, a memory 402 with one or more computer readable storage media, a power supply 403, and an input unit 404, and the like. Those skilled in the art can understand that the computer device structure shown in the figure does not constitute a limitation on the computer device, and can include more or fewer components than the figure, or combine certain components, or different component arrangements. Among them: Figure 8 The processor 401 is the control center of the computer device, connects various parts of the computer device through various interfaces and lines, executes the software programs and / or modules stored in the memory 402 and the data stored in the memory 402, and processes various functions and data of the computer device, thereby detecting the computer device as a whole. Optionally, the processor 401 can include one or more processing cores; preferably, the processor 401 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application program, and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 401.
[0176]
[0177] The memory 402 can be used to store software programs and modules, and the processor 401 executes various function applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; and the data storage area can store data created according to the use of the computer device, etc. In addition, the memory 402 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory 402 can also include a memory controller to provide the processor 401 with access to the memory 402.
[0178] The computer device further includes a power supply 403 for supplying power to various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 403 can also include one or more than one direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power state indicator, and the like.
[0179] The computer device can further include an input unit 404, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.
[0180] Although not shown, the computer device can also include a display unit and the like, which will not be described here. Specifically, in the present embodiment, the processor 401 in the computer device loads the executable file corresponding to the process of one or more than one application program into the memory 402 according to the following instructions, and runs the application program stored in the memory 402 by the processor 401, so as to realize various functions, as follows:
[0181] The first data is obtained through the power supply characteristic data, the second data is obtained by storing the first data into the first message queue, the third data is obtained by performing stream computing processing on the second data through the stream computing component, the fourth data is obtained by storing the third data into the second message queue, the fourth data is accessed into the columnar storage database, and the creation of the materialized view component is performed, the fifth data is obtained by storing the fourth data into the first table engine and performing deduplication through the materialized view component, the real-time analysis on the fifth data is performed according to the query request triggering the online analytical processing, the analysis result is displayed, and the abnormal situation of the power supply is detected based on the analysis result.
[0182] The specific implementation of the above operations can refer to the foregoing embodiments, which will not be described herein again. It should be noted that the computer device provided in the embodiments of the present application and the power supply detection method in the foregoing embodiments belong to the same concept, and the specific implementation process is described in the foregoing method embodiments, which will not be described herein again.
[0183] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions or by controlling relevant hardware by the instructions, which can be stored in a computer readable storage medium and loaded and executed by a processor.
[0184] To this end, the embodiments of the present application provide a storage medium, which stores a plurality of instructions. The instructions can be loaded by a processor to execute the steps in any power supply detection method provided by the embodiments of the present application. For example, the instructions can execute the following steps:
[0185] The first data is obtained according to the power supply characteristic data, the first data is stored in the first message queue to obtain the second data, the second data is processed by the stream computing component to obtain the third data, the third data is stored in the second message queue to obtain the fourth data, the fourth data is accessed to the columnar storage database, and the creation of the materialized view component is performed, the fourth data is stored in the first table engine by the materialized view component to perform deduplication to obtain the fifth data, the real-time analysis of the fifth data is performed according to the query request triggering online analytical processing, the analysis result is displayed, and the abnormal situation of the power supply is detected based on the analysis result.
[0186] The specific implementation of the above operations can refer to the foregoing embodiments, which will not be described herein again.
[0187] The storage medium can include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0188] Since the instructions stored in the storage medium can execute the steps in any power supply detection method provided by the embodiments of the present application, the beneficial effects of any power supply detection method provided by the embodiments of the present application can be achieved, which are described in detail in the foregoing embodiments, which will not be described herein again.
[0189] According to an aspect of the present application, a computer program product or computer program is provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in various optional implementations provided by the above embodiments.
[0190] The power supply detection method, device, storage medium and computer device provided by the embodiments of the present application are described in detail above, and the principles and implementation manners of the present application are described by applying specific examples. The above embodiment descriptions are only used to help understand the method of the present application and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application ranges will be changed, and the above description of the present application should not be understood as a limitation.
Claims
1. A power supply detection method, characterized by, The method comprises the following steps: obtaining first data according to power characteristic data, storing the first data into a first message queue to obtain second data; performing stream computing processing on the second data through a stream computing component to obtain third data, storing the third data into a second message queue to obtain fourth data; accessing the fourth data into a columnar storage database and creating a materialized view component; storing the fourth data into a first table engine through the materialized view component and performing deduplication to obtain fifth data; triggering online analytical processing to perform real-time analysis on the fifth data according to a query request, wherein sixth data including server whole machine data and TOC out-of-band data is obtained, the sixth data is stored into a local storage database through offline computing to obtain offline data, the offline data is used for assisting in detecting abnormal conditions of the power supply, the fifth data and the offline data are analyzed together in real time according to the query request, and analysis results are displayed, and the abnormal conditions of the power supply are detected based on the analysis results.
2. The power supply detection method of claim 1, wherein, The step of accessing the fourth data into the columnar storage database and creating the materialized view component comprises the following steps: creating a second table engine through the columnar storage database, wherein the second table engine is used for grabbing the fourth data; accessing the fourth data into the columnar storage database through the second table engine; creating the materialized view component in the columnar storage database.
3. The power supply detection method of claim 1, wherein, The step of storing the fourth data into the first table engine through the materialized view component and performing deduplication to obtain the fifth data comprises the following steps: creating a first table engine in the columnar storage database; converting the fourth data into a format and writing the fourth data into the first table engine through the materialized view component; comparing the contents of the fourth data, removing duplicate target fourth data, and storing the target fourth data in the first table engine to obtain the fifth data.
4. The power supply detection method of claim 1, wherein, The step of obtaining first data according to power characteristic data comprises the following steps: obtaining collected power characteristic data, and performing analysis processing on the collected power characteristic data; merging the power characteristic data obtained through the analysis processing into one row of data according to attributes to obtain the first data.
5. The power detection method of claim 1, wherein, After the step of storing the sixth data into the local storage database through offline computing to obtain offline data, the method further comprises the following steps: creating a third table engine, and accessing the offline data into the columnar storage database through the third table engine.
6. The power detection method of claim 1, wherein, The step of triggering online analytical processing to perform real-time analysis on the fifth data and the offline data according to a query request and displaying analysis results comprises the following steps: obtaining a query request; associating the fifth data and the offline data source through a machine serial number according to the query request, wherein the machine serial number is used for identifying each server; performing real-time analysis on the fifth data and the offline data of each server using online analytical processing, and displaying analysis results of the power supply of each server.
7. A power supply detecting apparatus characterized by comprising: The method comprises the following steps: an obtaining unit is configured to obtain first data according to power characteristic data, and store the first data into a first message queue to obtain second data; The processing unit is configured to perform stream computing processing on the second data by a stream computing component to obtain third data, and store the third data into a second message queue to obtain fourth data; The creating unit is configured to access the fourth data into a columnar storage database, and create a materialized view component; The storage unit is configured to store the fourth data into a first table engine by the materialized view component to perform deduplication to obtain fifth data; The detecting unit is configured to trigger real-time analysis on the fifth data according to a query request, wherein sixth data is obtained, the sixth data includes server whole machine data and TOC out-of-band data, the sixth data is stored into a local storage database by offline computing to obtain offline data, the offline data is used to assist in detecting abnormal conditions of the power supply, real-time analysis is performed on the fifth data and the offline data according to the query request, analysis results are displayed, and the abnormal conditions of the power supply are detected based on the analysis results.
8. A storage medium, characterized by The storage medium stores a plurality of instructions, and the instructions are adapted to be loaded by the processor to perform the steps in the power supply detection method of any one of claims 1 to 6.
9. A computer device, comprising: The computer device comprises a memory and a processor; the memory stores an application program, and the processor is used to run the application program in the memory to perform the steps in the power supply detection method of any one of claims 1 to 6.
10. A computer program product, characterised in that, The computer program product comprises computer instructions stored in a computer readable storage medium, a processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the power supply detection method of any one of claims 1 to 7.
Citation Information
Patent Citations
Real-time flow calculation monitoring system and method for processing monitoring big data
CN109800129A