Data early warning method, device, electronic equipment and storage medium

Abstract

The embodiment of the present invention discloses a data early warning method, device, electronic device and storage medium. The data early warning method comprises: obtaining the API call data to be monitored of the data source management platform; determining the call monitoring data of the API call data to be monitored; and performing an alarm process on the API call data when it is determined that the call monitoring data meets the data early warning condition. The technical solution of the embodiment of the present invention can realize real-time monitoring and early warning of API call data.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of data processing technology, and in particular to a data early warning method, device, electronic device and storage medium. Background Art

[0002] With the development of Internet technology, high concurrency of big data is becoming more and more common. Monitoring and early warning of external interface call data in high concurrency scenarios of big data has become a technical point in the field of Internet technology.

[0003] In the prior art, database locking is mainly used to deal with the problem of high concurrency of big data. However, the database locking processing method will cause frequent locking processing, which will lead to the data processing time not meeting the processing time limit requirements under the condition of high concurrency of big data, and cannot provide real-time warning for the high concurrency of big data. Summary of the invention

[0004] The embodiments of the present invention provide a data early warning method, device, electronic device and storage medium, which achieve the effect of real-time monitoring and early warning of API call data.

[0005] In a first aspect, an embodiment of the present invention provides a data early warning method, comprising:

[0006] Obtain the API (Application Program Interface) call data to be monitored on the data source management platform;

[0007] Determine the call monitoring data of the API call data to be monitored;

[0008] When it is determined that the call monitoring data meets the data warning conditions, an alarm is processed for the API call data.

[0009] In a second aspect, an embodiment of the present invention further provides a data early warning device, including:

[0010] The data acquisition module is used to obtain the API call data to be monitored from the data source management platform;

[0011] A call monitoring data determination module is used to determine the call monitoring data of the API call data to be monitored;

[0012] The data warning module is used to perform alarm processing on the API call data when it is determined that the call monitoring data meets the data warning conditions.

[0013] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0014] one or more processors;

[0015] A storage device for storing one or more programs;

[0016] When one or more programs are executed by one or more processors, the one or more processors implement the data early warning method provided by any embodiment of the present invention.

[0017] In a fourth aspect, an embodiment of the present invention further provides a computer storage medium on which a computer program is stored, and when the program is executed by a processor, the data early warning method provided by any embodiment of the present invention is implemented.

[0018] The technical solution of this embodiment parses the API call data to be monitored from the data source management platform, further processes the API call data to be monitored to obtain call monitoring data, and finally performs alarm processing on the API call data corresponding to the call monitoring data that meets the data warning conditions. This solves the problem of time delay in the prior art for warning API calls under high concurrency of big data, realizes real-time warning of API call data, and enhances the availability of external data resources and the controllability of each external API. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a flow chart of a data early warning method provided by Embodiment 1 of the present invention;

[0020] Figure 2 is a flow chart of a data early warning method provided by Embodiment 2 of the present invention;

[0021] Figure 3 is a schematic diagram of a data early warning device provided in Embodiment 3 of the present invention;

[0022] Figure 4 A schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0023] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention.

[0024] It should also be noted that, for ease of description, only the part relevant to the present invention but not all content is shown in the accompanying drawings. It should be mentioned before discussing exemplary embodiments in more detail that some exemplary embodiments are described as processing or methods depicted as flow charts. Although the flow chart describes each operation (or step) as sequential processing, many operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. When its operation is completed, the processing can be terminated, but it can also have additional steps not included in the accompanying drawings. The processing can correspond to methods, functions, procedures, subroutines, subprograms, etc.

[0025] Embodiment 1

[0026] Figure 1 is a flow chart of a data warning method provided by the first embodiment of the present invention. This embodiment is applicable to the case of real-time warning of API calls in a scenario of high concurrency of big data. The method can be executed by a data warning device, which can be implemented by software and / or hardware and can generally be integrated in an electronic device. Figure 1 As shown, the method includes the following operations:

[0027] S110: Obtain the API call data to be monitored from the data source management platform.

[0028] The data source management platform may be a platform for managing data sources. The data source may be multiple databases that meet business needs. The API call data to be monitored may be all or part of the API call data that the data source management platform needs to monitor. The API call data may be data called from a data source by an external application through an external API. The embodiments of the present invention do not limit the specific data content of the API call data and the API call data to be monitored.

[0029] At present, every Internet application or internal Internet program needs to access a large number of APIs. However, the development departments or teams of Internet companies are independent of each other. Different project teams may access different external APIs, or different departments may access the same external APIs. This leads to confusion and resource waste in the external API management within a company, increasing the company's operating costs. Therefore, a unified data source management platform is established to aggregate and manage a large number of calls. However, when a large number of API calls are aggregated together, a high-concurrency scenario of big data will be formed. Therefore, how to issue real-time warnings for abnormal API calls in the data source management platform has become an urgent problem to be solved.

[0030] In an embodiment of the present invention, before issuing a data warning for the API of the data source management platform, the various API call data of the data source management platform can be parsed first, and all or part of the parsed API call data can be used as the API call data to be monitored, so as to use the API call data to be monitored to issue a real-time warning for the API call situation.

[0031] Exemplarily, when an external application or user needs to use the data source management platform to call external data resources, the external application can send a data call request to the unified interface of the data source management platform. The data source management platform calls the API with the request data according to the data call request. The data source management platform monitors the unified interface and the called API, and can use the parsed call data of all or part of the interface of the data source management platform as the API call data to be monitored. Among them, the external application can be an application that can call data from the data source management platform. The external data resources can be different databases connected to the data source management platform through various APIs. The unified interface can be a unified interface provided by the data source management platform for external applications or users. The data call request can be a request sent by an external application or user to the data source management platform. According to the data call request, the data source management platform can feed back the data to be called to the sender of the data call request.

[0032] In an optional embodiment of the present invention, obtaining the API call data to be monitored of the data source management platform may include: obtaining mirror copy data of the original API call data from each external API of the data source management platform through a data mirror copy method as the API call data to be monitored.

[0033] The data mirroring method may be a method for retaining two or more online data. The original API call data may be data generated during the call of the external API. The external API may be an interface for connecting the data source management platform with external data resources. The mirrored data may be data obtained by mirroring the API call data of each external API of the data source management platform using the data mirroring method.

[0034] Specifically, before obtaining the API call data to be monitored from the data source management platform, the original API call data is parsed from each external API, and the original API call data is copied using a data mirror copy method to obtain mirror copy data, and the mirror copy data of the original API call data is further used as the API call data to be monitored. It should be noted that the acquisition of the mirror copy data will not affect the main process of data call, so the use of the data mirror copy method will not increase the time consumption of the main process of data call. Among them, the main process of data call can be a process for users or applications to call data.

[0035] In an optional embodiment of the present invention, after obtaining the mirror copy data of the original API call data from each external API of the data source management platform through the data mirror copy method, it can also include: sending the API call data to be monitored to a message queue.

[0036] Among them, the message queue can be a container for storing data.

[0037] In an embodiment of the present invention, after obtaining the mirror copy data of the original API call data from each external API of the data source management platform, the mirror copy data of the original API call data is sent to the message queue, and the data in the message queue is further processed to complete the alarm processing of the API call data.

[0038] S120: Determine call monitoring data of the API call data to be monitored.

[0039] The call monitoring data may be data after data analysis or monitoring processing is performed on the API call data to be monitored.

[0040] Specifically, after obtaining the API call data to be monitored, data processing is performed on the API call data to be monitored, and the processed API call data to be monitored is further used as call monitoring data, and finally alarm processing is performed according to the call monitoring data.

[0041] S130: When it is determined that the call monitoring data meets the data warning condition, an alarm process is performed on the API call data.

[0042] Among them, the data warning condition may be a condition for triggering alarm processing. Alarm processing may be an operation of calling an API corresponding to monitoring data that meets the data warning condition. For example, alarm processing may include interrupting external data resource services, switching routes, and sending relevant information of the faulty API to a data source management platform. The embodiments of the present invention do not limit the specific operations of alarm processing.

[0043] In the embodiment of the present invention, once it is found that the call monitoring data meets the data warning condition, the real-time alarm processing is immediately performed on each API call data that meets the data warning condition. If the call monitoring data does not meet the data warning condition, the real-time monitoring of each API is performed normally, and the API call data is not alarmed.

[0044] In an embodiment of the present invention, the data source management platform can configure early warning conditions or API call permissions for each external API. Through the early warning conditions and call permission settings, each link of the interface call of the data source management platform can be controlled. In addition, if the call monitoring data meets the early warning conditions, the data source management platform can perceive the abnormality of the external data resources and the abnormal operation of the user. The data source management platform can issue an operation and maintenance alarm notification so that the operation and maintenance personnel can handle the fault in time or limit the access rights of the abnormal operation user. If there is a problem with the external data resource, the data source management platform can switch to a similar data source, and the operation and maintenance personnel can perform data cleaning, release user permissions or increase bandwidth for the problematic external data resources to enhance the availability of external data resources.

[0045] The technical solution of this embodiment first parses the API call data to be monitored from the data source management platform, further processes the API call data to be monitored to obtain call monitoring data, and finally performs alarm processing on the API call data corresponding to the call monitoring data that meets the data warning conditions, thereby solving the problem of time delay in the prior art for warning API calls under high concurrency of big data, realizing real-time warning of API call data, and enhancing the availability of external data resources and the controllability of each external API.

[0046] Embodiment 2

[0047] Figure 2 is a flow chart of a data early warning method provided in Example 2 of the present invention. This embodiment is specific based on the above embodiment. In this embodiment, a specific optional implementation scheme for determining the call monitoring data of the API call data to be monitored according to the data monitoring dimension is provided. Accordingly, Figure 2 As shown, the method includes the following operations:

[0048] S210: Obtain the API call data to be monitored from the data source management platform.

[0049] S220: Determine data monitoring dimensions.

[0050] The data monitoring dimension may be a dimension for monitoring the API call data to be monitored. Different types of API call data to be monitored may correspond to the same or different data monitoring dimensions.

[0051] Specifically, when the data source management platform monitors the API call status in real time, it can determine the data monitoring dimension according to the early warning requirements, and further analyze the API call data to be monitored according to the data monitoring dimension.

[0052] S230. Determine call monitoring data of the API call data to be monitored according to the data monitoring dimension.

[0053] Specifically, after the data monitoring dimension is determined according to the warning demand, the API call data dimension to be monitored is determined according to the data monitoring dimension, and the call monitoring data is further determined according to the API call data dimension to be monitored.

[0054] Exemplarily, assuming that the data monitoring dimension is the call volume of a certain API, the call data dimension of the API to be monitored is the number of times the API is called by each user or external application. The number of times the API is called by each user or external application is summed up, and the sum is used as the call monitoring data. Alternatively, only user or external application can be used as a data monitoring dimension to obtain the call monitoring data of a certain user or external application calling a certain API to be monitored. Alternatively, the monitoring cycle can be determined to determine the call monitoring data of a certain API call data to be monitored within a certain cycle range.

[0055] In an optional embodiment of the present invention, the data monitoring dimension may include an interface monitoring dimension and / or a user monitoring dimension; determining the call monitoring data of the API call data to be monitored according to the data monitoring dimension may include: determining the user call monitoring data of the target calling user according to the user monitoring dimension; and / or determining the interface call monitoring data of the target calling interface according to the interface monitoring dimension.

[0056] Among them, the interface monitoring dimension can be the monitoring of some or all interface call dimensions. The user monitoring dimension can be the monitoring of some or all user dimensions. The target calling user can be the user who needs to be monitored. The user call monitoring data can be the associated data of the interface called by the target calling user. For example, the user call monitoring data can include but is not limited to the monitoring data of the API type called by a certain user, the call volume of a certain API by a certain user, and the total number of API calls by a certain user. The target calling interface can be the interface that needs to be monitored. The interface call monitoring data can be the associated data of the target calling interface being called. For example, the interface call monitoring data can include but is not limited to the monitoring data of the API call volume, API call type, average API call time and API call failure rate. The embodiment of the present invention does not limit the specific data types of the user call monitoring data and the interface call monitoring data.

[0057] Specifically, before performing alarm processing on the API call data, first determine the data monitoring dimension. If it is necessary to monitor the interface call situation, then determine the data monitoring dimension as the interface monitoring dimension, determine the target calling interface according to the interface monitoring dimension, and further perform data analysis based on the target calling interface to obtain interface call monitoring data. If it is necessary to monitor the user call situation, then determine the data monitoring dimension as the user monitoring dimension, determine the target calling user according to the user monitoring dimension, and further perform data analysis on the API call data of the target calling user to obtain user call monitoring data. Therefore, the embodiments of the present invention can realize the user call monitoring data of the target calling user determined according to the user monitoring dimension, and / or determine the interface call monitoring data of the target calling interface according to the interface monitoring dimension.

[0058] In an optional embodiment of the present invention, determining call monitoring data of API call data to be monitored based on data monitoring dimensions may include: subscribing to a message queue through a distributed data processing server; obtaining the API call data to be monitored based on the data monitoring dimensions and the message queue through the distributed data processing server to calculate the call monitoring data; wherein the distributed data processing server determines the call monitoring data using a distributed atomic counting method.

[0059] The distributed data processing server may be a server capable of performing parallel data processing. The distributed atomic counting method may be a method for performing batch processing task counting.

[0060] Correspondingly, after the API call data to be monitored is sent to the message queue, the distributed data processing server subscribed to the message queue can obtain the API call data to be monitored stored in the message queue. Specifically, the distributed data processing server can obtain the API call data to be monitored in the message queue according to the data monitoring dimension. Further, the distributed data processing server uses a distributed atomic counting method according to the data monitoring dimension and the API call data to be monitored to calculate the call monitoring data.

[0061] Exemplarily, assuming that the data monitoring dimension is the user monitoring dimension, the distributed data processing server can parse the user call monitoring data based on the API call data to be monitored in the message queue, and the distributed data processing server processes the user call monitoring data in a distributed atomic counting manner to obtain the call monitoring data of the target calling user. Specifically, the number of times a certain user calls a certain API, the total number of API calls by a certain user, or the number of API calls by all users calculated by the distributed data processing server can all be used as the call monitoring data of the target calling user. In addition, the distributed data processing server can give play to the advantages of distributed computing, making it possible to linearly increase data processing performance by increasing data processing servers, greatly speeding up data processing efficiency, achieving the effect of rapid consumption of API call data to be monitored, and improving the speed of early warning feedback.

[0062] S240: Determine a threshold for calling monitoring data.

[0063] The call monitoring data threshold may be a fixed value or a data range.

[0064] Specifically, the call monitoring data threshold is determined according to the interface monitoring dimension and / or the user monitoring dimension. The call monitoring data threshold can be set according to actual warning requirements.

[0065] For example, the number of times a user calls a certain interface within a certain time range can be determined based on the user monitoring dimension. If the number of times a user calls a certain interface falls within a certain data interval, it indicates that the user's calling of the interface is a normal access behavior, and the data interval can be further used as a monitoring data threshold. Similarly, the data interval can also be a fixed value.

[0066] S250: When the data statistics of the call monitoring data are greater than or equal to the call monitoring data threshold, determine that the call monitoring data meets the data warning condition, and perform alarm processing on the API call data.

[0067] The data statistics may be the result of calling monitoring data for data processing.

[0068] Specifically, before determining the data warning conditions, the call monitoring data can be determined first, and then the call monitoring data can be further processed to obtain data statistics. If the data statistics are greater than or equal to the call monitoring data threshold, the call monitoring data meets the data warning conditions, and the API call data corresponding to the call monitoring data that meets the warning conditions is parsed, and the API call data is alarmed. Otherwise, it indicates that the call monitoring data does not meet the data warning conditions, and no alarm processing is required at this time.

[0069] In the embodiment of the present invention, since the monitoring calculation of the API call data to be monitored is separated from the call of the external data resources of the data source management platform, that is, the main process of data call, it will not affect the operation of the main process of data call, and thus will not cause external data resource call delay.

[0070] The embodiment of the present invention, after obtaining the API call data to be monitored from the data source management platform, determines the call monitoring data of the API call data to be monitored according to the data monitoring dimension, and further performs alarm processing on the API call data when it is determined that the call monitoring data meets the data warning condition, thereby realizing real-time alarm processing of the API call data from various data dimensions, improving the data dimension of the data warning, and enhancing the scalability of the data warning.

[0071] It should be noted that any arrangement and combination of the technical features in the above embodiments also falls within the protection scope of the present invention.

[0072] Embodiment 3

[0073] Figure 3 is a schematic diagram of a data early warning device provided by Embodiment 3 of the present invention, such as Figure 3 As shown, the device includes: a data acquisition module 310, a call monitoring data determination module 320 and a data warning module 330, wherein:

[0074] The data acquisition module 310 is used to acquire the API call data to be monitored from the data source management platform;

[0075] A call monitoring data determination module 320, used to determine the call monitoring data of the API call data to be monitored;

[0076] The data warning module 330 is used to perform warning processing on the API call data when it is determined that the call monitoring data meets the data warning condition.

[0077] Optionally, the data acquisition module 310 is specifically used to: obtain mirror copy data of original API call data from each external API interface of the data source management platform through a data mirror copy method as the API call data to be monitored.

[0078] Optionally, the data warning device may further include: a module for sending API call data to be monitored, configured to send the API call data to be monitored to a message queue.

[0079] Optionally, the call monitoring data determination module 320 is specifically used to: determine a data monitoring dimension; and determine the call monitoring data of the API call data to be monitored according to the data monitoring dimension.

[0080] Optionally, the data monitoring dimension includes an interface monitoring dimension and / or a user monitoring dimension; the call monitoring data determination module 320 is specifically used to: determine the user call monitoring data of the target calling user according to the user monitoring dimension; and / or determine the interface call monitoring data of the target calling interface according to the interface monitoring dimension.

[0081] Optionally, the call monitoring data determination module 320 is specifically used to: subscribe to a message queue through a distributed data processing server; obtain the API call data to be monitored according to the data monitoring dimension and the message queue through the distributed data processing server to calculate the call monitoring data; wherein, the distributed data processing server uses a distributed atomic counting method to determine the call monitoring data.

[0082] Optionally, the data warning module 330 is specifically used to: determine a call monitoring data threshold; and determine that the call monitoring data meets a data warning condition when a data statistic of the call monitoring data is greater than or equal to the call monitoring data threshold.

[0083] The technical solution of this embodiment first parses the API call data to be monitored from the data source management platform, further processes the API call data to be monitored to obtain call monitoring data, and finally performs alarm processing on the API call data corresponding to the call monitoring data that meets the data warning conditions, thereby solving the problem of time delay in the prior art for warning API calls under high concurrency of big data, realizing real-time warning of API call data, and enhancing the availability of external data resources and the controllability of each external API.

[0084] The above data warning device can execute the data warning method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not described in detail in this embodiment, please refer to the data warning method provided by any embodiment of the present invention.

[0085] Since the data early warning device introduced above is a device that can execute the data early warning method in the embodiment of the present invention, based on the data early warning method introduced in the embodiment of the present invention, the technical personnel of the field can understand the specific implementation of the data early warning device of the present embodiment and its various variations, so how the data early warning device implements the data early warning method in the embodiment of the present invention is not introduced in detail here. As long as the technical personnel of the field implement the device used in the data early warning method in the embodiment of the present invention, it belongs to the scope of protection of this application.

[0086] Embodiment 4

[0087] Figure 4A schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Figure 4 A block diagram of an electronic device 412 suitable for implementing embodiments of the present invention is shown. Figure 4 The electronic device 412 shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention. The electronic device 412 may be, for example, a computer device or a server device.

[0088] like Figure 4 As shown, the electronic device 412 is in the form of a general purpose computing device. The components of the electronic device 412 may include, but are not limited to: one or more processors 416, a storage device 428, and a bus 418 connecting different system components (including the storage device 428 and the processor 416).

[0089] Bus 418 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0090] The electronic device 412 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 412, including volatile and non-volatile media, removable and non-removable media.

[0091] The storage device 428 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 430 and / or cache memory 432. The electronic device 412 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 434 may be used to read and write non-removable, non-volatile magnetic media ( Figure 4 not shown, usually called a "hard drive"). Although Figure 4Not shown in the figure, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a read-only optical disk (Compact Disc-Read Only Memory, CD-ROM), a digital video disk (Digital Video Disc-Read Only Memory, DVD-ROM) or other optical media) may be provided. In these cases, each drive may be connected to the bus 418 via one or more data medium interfaces. The storage device 428 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0092] A program 436 having a set (at least one) of program modules 426 may be stored, for example, in a storage device 428, such program modules 426 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination thereof may include an implementation of a network environment. Program modules 426 generally perform the functions and / or methods of the embodiments described herein.

[0093] The electronic device 412 may also communicate with one or more external devices 414 (e.g., keyboard, pointing device, camera, display 424, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 412, and / or communicate with any device that enables the electronic device 412 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 422. In addition, the electronic device 412 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN) and / or a public network, such as the Internet) through a network adapter 420. As shown, the network adapter 420 communicates with other modules of the electronic device 412 through a bus 418. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 412, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, disk arrays (Redundant Arrays of Independent Disks, RAID) systems, tape drives, and data backup storage systems.

[0094] The processor 416 executes various functional applications and data processing by running the programs stored in the storage device 428, for example, implementing the data warning method provided in the above-mentioned embodiment of the present invention: obtaining the API call data to be monitored of the data source management platform; determining the call monitoring data of the API call data to be monitored; and when it is determined that the call monitoring data meets the data warning conditions, performing alarm processing on the API call data.

[0095] The technical solution of this embodiment first parses the API call data to be monitored from the data source management platform, further processes the API call data to be monitored to obtain call monitoring data, and finally performs alarm processing on the API call data corresponding to the call monitoring data that meets the data warning conditions, thereby solving the problem of time delay in the prior art for warning API calls under high concurrency of big data, realizing real-time warning of API call data, and enhancing the availability of external data resources and the controllability of each external API.

[0096] Embodiment 5

[0097] Embodiment 5 of the present invention further provides a computer storage medium storing a computer program, wherein the computer program, when executed by a computer processor, is used to execute the data warning method described in any one of the above embodiments of the present invention: obtaining the API call data to be monitored of the data source management platform; determining the call monitoring data of the API call data to be monitored; and performing alarm processing on the API call data when it is determined that the call monitoring data meets the data warning conditions.

[0098] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0099] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0100] The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the foregoing.

[0101] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0102] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A data early warning method, characterized in that: include: Obtain the API call data to be monitored from the data source management platform; Determine call monitoring data of the API call data to be monitored, wherein the call monitoring data is obtained by performing data analysis or monitoring processing on the API call data to be monitored; When it is determined that the call monitoring data meets the data warning condition, an alarm process is performed on the API call data, wherein the alarm process includes at least one of interrupting the external data resource service, switching the route, and sending the relevant information of the faulty API to the data source management platform; The step of obtaining the API call data to be monitored of the data source management platform includes: Obtain mirrored copy data of original API call data from each external API of the data source management platform through a data mirroring replication method as the API call data to be monitored, wherein the data mirroring replication method is a replication method that retains two or more online data, the original API call data is data generated during the call process of the external API, the external API is an interface for connecting the data source management platform with external data resources, and the mirrored copy data is data obtained by mirroring the API call data of each external API of the data source management platform using the data mirroring replication method; The API call data to be monitored is sent to a message queue, wherein the message queue is a container for storing data.

2. The method according to claim 1, characterized in that The step of determining the call monitoring data of the API call data to be monitored includes: Determine the data monitoring dimensions; Determine the call monitoring data of the API call data to be monitored according to the data monitoring dimension.

3. The method according to claim 2, characterized in that The data monitoring dimension includes an interface monitoring dimension and / or a user monitoring dimension; The determining the call monitoring data of the API call data to be monitored according to the data monitoring dimension includes: Determining user call monitoring data of a target calling user according to the user monitoring dimension; and / or The interface call monitoring data of the target calling interface is determined according to the interface monitoring dimension.

4. The method according to claim 2 or 3, characterized in that: The determining the call monitoring data of the API call data to be monitored according to the data monitoring dimension includes: Subscribe to the message queue through the distributed data processing server; Obtaining the API call data to be monitored according to the data monitoring dimension and the message queue through the distributed data processing server to calculate the call monitoring data; The distributed data processing server determines the call monitoring data by using a distributed atomic counting method.

5. The method according to claim 1, characterized in that The determining that the call monitoring data meets the data warning condition includes: Determine the call monitoring data threshold; When the data statistic of the call monitoring data is greater than or equal to the call monitoring data threshold, it is determined that the call monitoring data meets the data warning condition.

6. A data early warning device, characterized in that: include: The data acquisition module is used to obtain the API call data to be monitored from the data source management platform; A call monitoring data determination module is used to determine the call monitoring data of the API call data to be monitored, wherein the call monitoring data is obtained by performing data analysis or monitoring processing on the API call data to be monitored; A data warning module, configured to perform warning processing on the API call data when it is determined that the call monitoring data meets the data warning conditions, wherein the warning processing includes at least one of interrupting the external data resource service, switching the route, and sending the relevant information of the faulty API to the data source management platform; The data acquisition module is specifically used to: obtain mirror copy data of original API call data from each external API interface of the data source management platform through a data mirror copy method as the API call data to be monitored, wherein the data mirror copy method is a copy method that retains two or more online data, the original API call data is the data generated during the call process of the external API, the external API is the interface for connecting the data source management platform with external data resources, and the mirror copy data is the data obtained after mirror copying the API call data of each external API of the data source management platform using the data mirror copy method; The data warning module includes: a module for sending API call data to be monitored, which is used to send the API call data to be monitored to a message queue, wherein the message queue is a container for storing data.

7. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the data early warning method as described in any one of claims 1-5.

8. A computer storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the data early warning method as described in any one of claims 1 to 5 is implemented.

Images