Data processing method, device and equipment and computer storage medium

Abstract

The embodiment of the invention discloses a data processing method and device, equipment and a computer storage medium. The method comprises the steps of obtaining alarm data; generating alarm information under the condition that the alarm data meets a preset alarm condition; acquiring unit fault data according to the alarm information; and processing the fault data according to a preset processing rule under the condition that the fault data meets the preset event processing condition. According to the data processing method provided by the embodiment of the invention, the corresponding fault data can be determined according to the obtained alarm data, and when the fault data meets the preset condition, the fault data is processed according to the corresponding processing rule, so that automatic processing of the fault data is realized, and the operation and maintenance efficiency is improved.

Description

Technical Field

[0001] This application belongs to the field of Internet technologies, and in particular, relates to a method, apparatus, device, and computer storage medium for data processing. Background Art

[0002] Currently, due to the rapid development of IT technologies, IT operation and maintenance also play an increasingly important role. During IT operation and maintenance, manual operation and maintenance and semi-automated operation and maintenance are mainly adopted. After an IT failure occurs, operation and maintenance personnel take corresponding remedial measures for processing to ensure the normal operation of IT. However, when performing IT operation and maintenance using the above methods, a large amount of time is wasted on some simple and repetitive problems, resulting in low efficiency. Summary of the Invention

[0003] Embodiments of this application provide a method, apparatus, device, computer storage medium, and computer program product for data processing, which can implement automated processing of fault data and improve the efficiency of fault handling.

[0004] In a first aspect, embodiments of this application provide a method for data processing, the method including:

[0005] Obtain alarm data;

[0006] When the alarm data meets a preset alarm condition, generate alarm information according to the alarm data, where the alarm information includes keywords, fault unit information, and fault occurrence time;

[0007] According to the alarm information, obtain the fault data of the unit within the occurrence time;

[0008] When the fault data meets a preset event processing condition, process the fault data according to a preset processing rule.

[0009] In a second aspect, embodiments of this application provide an apparatus for data processing, the apparatus including:

[0010] An obtaining module, configured to obtain alarm data;

[0011] A generating module, configured to generate alarm information according to the alarm data when the alarm data meets a preset alarm condition, where the alarm information includes keywords, fault unit information, and fault occurrence time;

[0012] The obtaining module is further configured to obtain the fault data of the unit within the occurrence time according to the alarm information;

[0013] A processing module, configured to process the fault data according to a preset processing rule when the fault data meets a preset event processing condition.

[0014] In a third aspect, an embodiment of the present application provides a data processing device, which includes: a processor and a memory storing computer program instructions;

[0015] When the processor executes the computer program instructions, it implements the data processing method as in the first aspect of the present application.

[0016] In a fourth aspect, an embodiment of the present application provides a computer storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, they implement the data processing method as in the first aspect of the present application.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device is caused to execute the data processing method as in the first aspect of the present application

[0018] The data processing method, device, equipment, and computer storage medium according to the embodiments of the present application can determine corresponding fault data based on the obtained alarm data, and when the fault data meets the preset conditions, process the fault data according to the corresponding processing rules, realizing the automated processing of the fault data and improving the operation and maintenance efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required to be used in the embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0020] Figure 1 is a flowchart of the data processing method provided by an embodiment of the present application;

[0021] Figure 2 is a structural schematic diagram of the data processing device provided by an embodiment of the present application;

[0022] Figure 3 is a structural schematic diagram of the data processing equipment provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] The features and exemplary embodiments of each aspect of the present application will be described in detail below. In order to make the purpose, technical solutions, and advantages of the present application clearer, the following further describes the present application in detail with reference to the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than limiting the present application. For those skilled in the art, the present application can be implemented without some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by showing examples of the present application.

[0024] It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the said element.

[0025] Currently, when performing IT operation and maintenance, it is mainly manual operation and maintenance and semi-automated operation and maintenance. When an IT failure occurs, operation and maintenance personnel need to take corresponding remedial measures, resulting in low efficiency.

[0026] To solve the problems of the prior art, the embodiments of the present application provide a method, device, equipment, computer storage medium and computer program product for data processing. First, the data processing method provided by the embodiments of the present application will be introduced below.

[0027] Figure 1 The flowchart of the data processing method provided by an embodiment of the present application is shown. As Figure 1 shown, the method may include the following steps:

[0028] S110. Obtain alarm data.

[0029] By collecting and monitoring data, alarm data is obtained. The alarm data includes configured parameter values.

[0030] S120. When the alarm data meets a preset alarm condition, generate alarm information according to the alarm data. The alarm information includes keywords, fault unit information and fault occurrence time.

[0031] When the alarm data meets a preset alarm threshold, corresponding alarm information is generated according to the obtained alarm data. The alarm information includes keywords, fault unit information and fault occurrence time, where the keyword is the fault return code corresponding to the alarm data.

[0032] In some embodiments, the alarm information further includes fault data fields such as fault occurrence type and occurrence reason.

[0033] In some embodiments, the alarm information is the alarm information generated by platforms such as Kibana and Oms, and the alarm information is transmitted through the Kafka platform. Specifically, the information transmission can select a point-to-point transmission mode or a publish-subscribe transmission mode. Among them, in the publish-subscribe mode, the consumer subscribes to and consumes the information to handle operation and maintenance matters.

[0034] S130. Obtain the fault data of the unit within the occurrence time according to the alarm information.

[0035] According to the alarm information obtained by consuming Kafka information, obtain the key fault domain, and obtain the fault data of the faulty unit within the current time by means of capacity query.

[0036] S140. When the fault data meets the preset event processing conditions, process the fault data according to the preset processing rules.

[0037] Judge whether the obtained fault data meets the event standard according to the preset event processing conditions. When the fault data meets the event standard, process the fault data according to the preset processing rules corresponding to the preset event processing conditions.

[0038] In some embodiments, when the fault data meets the event standard, generate a corresponding event work order, and process the fault data according to the preset processing rules corresponding to the preset event processing conditions according to the event work order.

[0039] The data processing method provided by the embodiments of the present application can determine the corresponding fault data according to the obtained alarm data, and when the fault data meets the preset conditions, process the fault data according to the corresponding processing rules, realizing the automatic processing of the fault data and improving the operation and maintenance efficiency.

[0040] In some embodiments, after processing the fault data according to the preset processing rules, the method further includes: sending the first fault information to the preset data domain device, where the first fault information is the fault information obtained by processing the fault data according to the preset processing rules. Obtain the abnormal fault data after the fault data is processed by the preset processing rules as the first fault information, and send the first fault information to the set information receiving group, that is, the preset data domain device. Among them, when sending the first fault information to the set information receiving group, the first fault information is sent to different groups according to the data domains with different requirements of the fault data.

[0041] In some embodiments, during the work order processing, corresponding query mechanisms are classified and formulated according to the actual operation characteristics of the member units.

[0042] In some embodiments, when it is detected that the event is in an unrecovered state, the fault data is automatically updated, and the fault data with synchronization exceptions is sent to the corresponding group according to different required data domains. Among them, the recovery of the event is judged by the unit transaction request volume and success rate. When the unit transaction request volume and success rate meet the preset conditions, the event is determined to be in a recovered state.

[0043] In some embodiments, the method further includes: detecting the status of the unit; when the status of the unit is in a recovered state and the number of recoveries meets the preset threshold, updating the fault data of the unit from the first time to the second time. Detect the status of the faulty unit. When the system repeatedly queries and detects that the unit is in a recovered state, the system automatically updates the fault data from the first time to the second time, where the first time is the start time of unit recovery and the second time is the end time of unit recovery. At the same time, the synchronized fault data is sent to the corresponding group. Among them, the preset threshold can be set and is not limited herein.

[0044] In some embodiments, after updating the fault data of the unit from the first time to the second time, the method further includes: generating a second fault message; sending the second fault message to a preset data domain device. After updating the fault data of the unit from the first time to the second time, according to the updated data, the data with synchronization exceptions in the updated data is sent to the preset data domain device, that is, the corresponding group.

[0045] In some embodiments, the alarm message further includes information about the priority level; when obtaining the fault data of multiple units at the same occurrence time and the fault data of multiple units all meet the preset event processing conditions, the method further includes: processing the fault data of multiple units according to the preset processing rules according to the priority level. When the fault data obtained at the same time is the fault data of multiple faulty units and the fault data of multiple units all meet the event processing conditions, the multiple fault data is processed according to the preset priority order. Specifically, according to the priority processing principle, the fault data is divided into different levels, and the multiple fault data is processed from high to low according to the levels.

[0046] In one example, the fault events are divided into minor events and serious events, where the minor events are processed routinely, specifically, they can be processed in the order of occurrence time; the serious events are processed according to the preset priority order, and the events with higher levels are processed first.

[0047] The data processing method provided by the embodiments of the present application can determine the corresponding fault data according to the obtained alarm data, process the fault data according to the corresponding processing rules when the fault data meets the preset conditions, and at the same time detect based on the recovery status of the faulty unit, realizing the automated processing of the fault data and improving the operation and maintenance efficiency.

[0048] Figure 2 This is a schematic structural diagram of a data processing device provided by an embodiment of the present application. As Figure 2 shown, the device 200 may include an acquisition module 210, a generation module 220, and a processing module 230.

[0049] The acquisition module 210 is configured to acquire alarm data;

[0050] The generation module 220 is configured to generate alarm information according to the alarm data when the above-mentioned alarm data meets a preset alarm condition. The alarm information includes keywords, fault unit information, and fault occurrence time;

[0051] The acquisition module 210 is further configured to acquire the fault data of the unit within the occurrence time according to the alarm information;

[0052] The processing module 230 is configured to process the fault data according to a preset processing rule when the fault data meets a preset event processing condition.

[0053] The data processing device provided by the embodiment of the present application can determine corresponding fault data according to the acquired alarm data, and when the fault data meets the preset conditions, process the fault data according to the corresponding processing rules, realizing the automatic processing of the fault data and improving the operation and maintenance efficiency.

[0054] In some embodiments, the device further includes: a first sending module, configured to send first fault information to a preset data domain device, where the first fault information is the fault information obtained by processing the fault data according to a preset processing rule.

[0055] In some embodiments, the device further includes: a detection module, configured to detect the status of the unit; an update module, configured to update the fault data of the unit from the first time to the second time when the status of the unit is a recovery status and the number of recoveries meets a preset threshold.

[0056] In some embodiments, the device further includes: a second generation module, configured to generate second fault information; a second sending module, configured to send the second fault information to a preset data domain device.

[0057] In some embodiments, the alarm information further includes priority information; the processing module 230 is specifically configured to: when acquiring the fault data of multiple units at the same occurrence time and the fault data of the multiple units all meet the preset event processing conditions, process the fault data of the multiple units according to the preset processing rule according to the priority.

[0058] The data processing device provided by the embodiment of the present application can determine corresponding fault data according to the acquired alarm data. When the fault data meets the preset conditions, the fault data is processed according to the corresponding processing rules, and at the same time, the recovery status of the faulty unit is detected to realize the automated processing of the fault data and improve the operation and maintenance efficiency.

[0059] Figure 2 Each module in the shown device has the function of implementing Figure 1 each step in and can achieve its corresponding technical effects. For the sake of brief description, it will not be elaborated here.

[0060] Figure 3 The figure shows a schematic hardware structure diagram of the data processing device provided by the embodiment of the present application.

[0061] The data processing device may include a processor 301 and a memory 302 storing computer program instructions.

[0062] Specifically, the above-mentioned processor 301 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0063] The memory 302 may include a mass storage for data or instructions. By way of example and not limitation, the memory 302 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disc, a magneto-optical disc, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In a suitable case, the memory 302 may include a removable or non-removable (or fixed) medium. In a suitable case, the memory 302 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 302 is a non-volatile solid state memory.

[0064] The memory may include a read only memory (ROM), a random access memory (RAM), a magnetic disk storage media device, an optical storage media device, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described in reference to the method according to one aspect of the present disclosure.

[0065] The processor 301 reads and executes the computer program instructions stored in the memory 302 to implement any one of the data processing methods in the above embodiments.

[0066] As an example, the data processing device may further include a communication interface 303 and a bus 310. Among them, as Figure 3 shown, the processor 301, the memory 302, and the communication interface 303 are connected through the bus 310 and complete communication with each other.

[0067] The communication interface 303 is mainly used to implement communication between each module, device, unit, and / or device in the embodiments of the present application.

[0068] The bus 310 includes hardware, software, or both, and couples the components of the data processing device to each other. By way of example and not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses or a combination of two or more of these. In a suitable case, the bus 310 may include one or more buses. Although the embodiments of the present application describe and illustrate specific buses, the present application contemplates any suitable bus or interconnect.

[0069] The data processing device can execute the data processing method in the embodiments of the present application, thereby implementing the data processing method and apparatus combined with Figure 1 and Figure 2 described data processing methods and apparatuses.

[0070] In addition, in combination with the data processing method in the above embodiments, the embodiments of the present application can be implemented by providing a computer storage medium. Computer program instructions are stored on the computer storage medium; when the computer program instructions are executed by a processor, any one of the data processing methods in the above embodiments is implemented.

[0071] The embodiments of the present application provide a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes any one of the data processing methods in the above embodiments.

[0072] It should be clear that the present application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of the present application.

[0073] The functional blocks shown in the above-described block diagrams can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, and so on. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted via a data signal carried in a carrier wave on a transmission medium or a communication link. A "machine-readable medium" can include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, and so on. The code segment can be downloaded via a computer network such as the Internet, an intranet, and so on.

[0074] It should also be noted that the exemplary embodiments mentioned in the present application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above steps, that is, the steps can be executed in the order mentioned in the embodiments, or different from the order in the embodiments, or several steps can be executed simultaneously.

[0075] Aspects of the present disclosure have been described above with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block in the flowcharts and / or block diagrams, and the combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing devices to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing devices enable the implementation of the functions / actions specified in one or more blocks of the flowcharts and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and the combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0076] As described above, this is only the specific implementation manner of the present application. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein. It should be understood that the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present application.

Claims

1. A method for data processing, characterized in that, Including: Obtain alarm data; When the alarm data meets the preset alarm condition, generate alarm information according to the alarm data, where the alarm information includes keywords, fault unit information, and fault occurrence time; According to the alarm information, obtain the fault data of the unit during the occurrence time; When the fault data meets the preset event processing condition, process the fault data according to the preset processing rules.

2. The method according to claim 1, characterized in that, After processing the fault data according to the preset processing rules, the method further includes: Send the first fault information to a preset data domain device, where the first fault information is the fault information obtained by processing the fault data according to the preset processing rules.

3. The method according to claim 1 or 2, characterized in that, The method further includes: Detect the status of the unit; When the status of the unit is the recovery status and the number of recoveries meets the preset threshold, update the fault data of the unit from the first time to the second time.

4. The method according to claim 3, characterized in that, After updating the fault data of the unit from the first time to the second time, the method further includes: Generate the second fault information; Send the second fault information to the preset data domain device.

5. The method according to claim 1, characterized in that, The alarm information further includes priority information; when obtaining the fault data of multiple units at the same occurrence time and the fault data of the multiple units all meet the preset event processing condition, the method further includes: According to the preset processing rules, process the fault data of the multiple units according to the priority.

6. A device for data processing, characterized in that, The device includes: An acquisition module for obtaining alarm data; A generation module for generating alarm information according to the alarm data when the alarm data meets the preset alarm condition, where the alarm information includes keywords, fault unit information, and fault occurrence time; The acquisition module is further configured to obtain the fault data of the unit during the occurrence time according to the alarm information; A processing module for processing the fault data according to the preset processing rules when the fault data meets the preset event processing condition.

7. The device according to claim 6, characterized in that, The device further includes: A first sending module for sending the first fault information to a preset data domain device, where the first fault information is the fault information obtained by processing the fault data according to the preset processing rules.

8. The device according to claim 6 or 7, characterized in that, The device further includes: A detection module for detecting the status of the unit; An update module for updating the fault data of the unit from the first time to the second time when the status of the unit is the recovery status and the number of recoveries meets the preset threshold.

9. The device according to claim 8, characterized in that, The device further includes: A second generation module for generating the second fault information; A second sending module for sending the second fault information to the preset data domain device.

10. The device according to claim 6, characterized in that, The alarm information further includes priority information; the processing module is specifically configured to: When obtaining the fault data of multiple units at the same occurrence time and the fault data of the multiple units all meet the preset event processing condition, process the fault data of the multiple units according to the preset processing rules according to the priority.

11. A data processing device, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the data processing method described in any one of claims 1-5 is implemented.

12. A computer storage medium, characterized in that, Computer program instructions are stored on the computer storage medium, and when the computer program instructions are executed by a processor, the data processing method described in any one of claims 1-5 is implemented.

13. A computer program product, characterized in that, When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device is caused to execute the data processing method described in any one of claims 1-5.

Images