Data platform monitoring method and device and storage medium

Abstract

The invention discloses a data platform monitoring method and device and a storage medium. The method comprises the steps that a detection instruction of a target object is received, a service instance corresponding to a data platform is determined, and the data platform comprises a control platform and a code development platform which are used for industrial production; target running parameters of a target application program are determined from the service instance, and the target application program is any application program installed in the data platform; and under the condition that the target operation parameter meets a preset condition, determining that the target application program in the data platform is abnormal. According to the method and the device, the technical problems that the running health state of the application program on the platform can only be detected manually, the manual detection means is lagged, the efficiency is low, and the problems occurring in the running process of the application program cannot be perceived in advance in the prior art are solved.

Description

Technical Field

[0001] The present application relates to the field of industrial automation, and more specifically, to a monitoring method, device and storage medium for a data platform. Background Art

[0002] In the current context of industrial automation and informatization, platforms such as supOS-ADP, supOS-Lite and i-CDS play a vital role in the field of industrial production. These platforms support low-code development, allowing engineers to quickly develop entity configuration applications suitable for specific scenarios. However, in related technologies, various problems that arise during the operation of entity configuration applications on the platform cannot be perceived in advance, and users are often required to query the operating status of each application one by one to determine whether there is a problem. In addition, users receive feedback after the problem occurs and then process it, which increases the user's loss.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a data platform monitoring method, device and storage medium to at least solve the technical problem that the related technology can only rely on manual detection of the health status of the application running on the platform, the manual detection means are delayed and inefficient, and the problems that arise during the operation of the application cannot be perceived in advance.

[0005] According to one aspect of an embodiment of the present application, a method for monitoring a data platform is provided, including: receiving a detection instruction of a target object, determining a service instance corresponding to the data platform, wherein the data platform includes a control platform and a code development platform used in industrial production; determining target operating parameters of a target application from the service instance, wherein the target application is any application installed in the data platform; and determining that an abnormality occurs in the target application in the data platform when the target operating parameters meet preset conditions.

[0006] In some embodiments of the present application, before receiving a detection instruction of a target object and determining a service instance corresponding to a data platform, the method further includes: determining whether an inspection tool for detecting an application is installed in the data platform; if the inspection tool is not installed on the data platform, generating a first prompt message, wherein the first prompt message is used to prompt the target object to install the inspection tool on the data platform; if the inspection tool is already installed on the data platform, sending the detection instruction to the inspection tool.

[0007] In some embodiments of the present application, a detection instruction of a target object is received, and a service instance corresponding to a data platform is determined, including: obtaining all running applications in the data platform; determining a collection program from a database based on the detection instruction, wherein the collection program is used to collect running parameters of the applications on the data platform; determining the running parameters of the running applications based on the collection program; and determining the service instance corresponding to the data platform based on the running parameters.

[0008] In some embodiments of the present application, the method also includes: determining the middleware and database used by the data platform; executing a first detection program according to the detection instruction to determine the process information of the middleware, and executing a second detection program according to the detection instruction to determine the connection information of the database; determining a first operating state of the middleware based on the process information, and determining a second operating state of the database based on the connection information.

[0009] In some embodiments of the present application, when the target operating parameters meet preset conditions, determining that an abnormality occurs in a target application in the data platform includes: determining a first parameter set from the target operating parameters, wherein the first parameter set includes at least one of the following: the number of full garbage collections (Full Garbage Collection, referred to as FullGC for short), and the total time spent on garbage collection; when the number of FullGCs exceeds a preset number of times, or the total time spent on garbage collection is greater than a preset time, determining that the target operating parameters meet the preset conditions; determining the target application whose number of FullGCs exceeds a preset number of times or the total time spent on garbage collection is greater than a preset time as an abnormal application; performing a restart operation on the abnormal application and generating a first alarm message, wherein the first alarm message is used to prompt that an abnormal application exists in the target object.

[0010] In some embodiments of the present application, the above method also includes: obtaining a first operating parameter of a server used by the data platform, and obtaining a second operating parameter of a database used by the data platform; when any one of the first operating parameter and / or the second operating parameter is greater than a standard threshold, generating a second alarm information, wherein the second alarm information is used to prompt that there is an abnormality in the target object server or database.

[0011] In some embodiments of the present application, the above method also includes: determining the operating parameters of each application in the data platform according to a preset frequency through a timer; generating an operating curve based on the operating parameters, and determining abnormal applications based on the relationship between the operating curve and the standard curve, wherein the operating curve is used to indicate changes in the operating parameters; and sending alarm information containing abnormal applications to the target object through the middleware.

[0012] According to another aspect of an embodiment of the present application, a monitoring device for a data platform is also provided, including: a receiving module, used to receive a detection instruction of a target object, and determine a service instance corresponding to the data platform, wherein the data platform includes a control platform and a code development platform used in industrial production; a first determination module, used to determine target operating parameters of a target application from the service instance, wherein the application is any application installed in the data platform; and a second determination module, used to determine that an abnormality has occurred in the target application in the data platform when the target operating parameters meet preset conditions.

[0013] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned data platform monitoring method by running the computer program.

[0014] According to another aspect of the embodiments of the present application, a computer program product is provided, including computer instructions, which implement the above-mentioned data platform monitoring method when executed by a processor.

[0015] In an embodiment of the present application, by receiving a detection instruction of the target object, a service instance corresponding to the detection instruction on the data platform is obtained, and the target operating parameters of the target application are determined from the service instance. Finally, the target operating parameters are compared with the preset conditions. When the preset conditions are met, it is determined that an abnormality has occurred in the target application in the data platform, thereby achieving the purpose of monitoring the applications running on the data platform. When an abnormality occurs in an application, it is ensured that the problem can be quickly paid attention to, thereby solving the technical problem that the related technology can only rely on manual detection of the health status of the application operation on the platform, the manual detection means are delayed and inefficient, and problems occurring during the operation of the application cannot be perceived in advance. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 It is a hardware structure block diagram of a computer terminal for implementing a monitoring method of a data platform according to an embodiment of the present application;

[0018] Figure 2 is a flow chart of a method for implementing a monitoring method of a data platform according to an embodiment of the present application;

[0019] Figure 3It is a schematic diagram of overall deployment interaction for implementing a monitoring method for a data platform according to an embodiment of the present application;

[0020] Figure 4 is an interactive timing diagram a for implementing a monitoring method of a data platform according to an embodiment of the present application;

[0021] Figure 5 is an interactive timing diagram b of a method for implementing a monitoring method of a data platform according to an embodiment of the present application;

[0022] Figure 6 It is a device diagram for implementing a monitoring method of a data platform according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained as follows:

[0026] supOS-ADP / supOS-Lite: An industrial intelligent production control system used in the field of industrial production.

[0027] i-CDS: A low-code development platform integrated with supOS-ADP / supOS-Lite.

[0028] Entity configuration application: A Java application that is rapidly developed based on the i-CDS platform.

[0029] In the current context of industrial automation and informatization, platforms such as supOS-ADP, supOS-Lite and i-CDS play a vital role in the field of industrial production. However, when deploying these platforms and applications, engineers face a series of challenges. First, in terms of problem perception and response, when detecting whether there is a problem with the operation of an application, the relevant technology needs to take manual detection means, by logging into the server to query the operation status of the application one by one, or querying the operation information of each application through configuration management tools such as Nacos to determine whether the application is abnormal. Manual detection is not timely, and feedback is often received after the system has problems, and then processed, which increases the user's loss; secondly, in terms of the overall deployment of the platform, the relevant technology lacks effective detection means for the operation status of the middleware in the platform and the database connection status, resulting in the system often unable to respond to faults in time, and the relevant technology requires manual creation and connection to the database. This process lacks automation and is prone to configuration errors and connection problems; finally, in terms of system resource usage, the relevant technology lacks monitoring of the system server resource usage and database resource usage, resulting in system performance degradation. In order to solve the above problems, the present application provides a relevant solution in the embodiment, which is described in detail below.

[0030] The data platform monitoring method embodiment provided in the embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing a monitoring method for a data platform. Figure 1 As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions connected via a wired and / or wireless network. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. Those skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0031] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0032] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the monitoring method of the data platform in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the monitoring method of the data platform mentioned above. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0033] The transmission module 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0034] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0035] It should be noted that, in some optional embodiments, the above Figure 1 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0036] Figure 2 is a flow chart of a data platform monitoring method according to an embodiment of the present application, such as Figure 2 As shown, the method comprises the following steps:

[0037] Step S202, receiving a detection instruction of a target object, and determining a service instance corresponding to a data platform, wherein the data platform includes a control platform and a code development platform used in industrial production.

[0038] In some embodiments, the detection instruction is used to trigger the execution steps of determining the above service instance, and the above detection instruction at least includes: an identification of the object to be monitored, and the object to be monitored includes but is not limited to: an application, middleware, server, database, etc.

[0039] In the above step S202, the detection instructions input by the target object may be application detection instructions, middleware detection instructions, server detection instructions, database detection instructions, etc., wherein the application detection instructions include application monitoring identifiers, the middleware detection instructions include middleware monitoring identifiers, the server detection instructions include server monitoring identifiers, and the database detection instructions include database monitoring identifiers. The system obtains information about the corresponding service instances on the current data platform according to different detection instructions.

[0040] In an optional embodiment, after receiving the above-mentioned detection instruction, the service instance corresponding to the data platform is determined from a predetermined Instance list (i.e., a service instance list). The information of the service instance in the Instance list may be an IP address, port number, version information, etc. If the Instance list is not empty, it can be considered that at least one corresponding service instance is running.

[0041] In an optional embodiment, the control platform used in industrial production may be, for example, a supOS-ADP or supOS-Lite system, and the code development platform may be an i-CDS platform integrated with the supOS-ADP or supOS-Lite system.

[0042] Step S204, determining target operating parameters of a target application from the service instance, wherein the target application is any application installed in the data platform.

[0043] In the above step S204, when the detection instruction is an application detection instruction, the information of all application service instances running on the current platform can be obtained, and the target operating parameters of the target application are obtained from the information of all service instances. The target operating parameters can be, for example, the heap memory usage ratio, the number of new generation garbage collection YGC times, the number of full garbage collection FullGC times, the total time spent on garbage collection, metaspace, number of threads, etc., and the obtained target operating parameters are stored in the database. In an optional embodiment, the target application can be an entity configuration application developed based on the i-CDS platform.

[0044] Step S206: When the target operating parameters meet the preset conditions, it is determined that an abnormality occurs in the target application in the data platform.

[0045] In the above step S206, the acquired target application operating parameters are compared with preset conditions, which may be set thresholds corresponding to the target operating parameters, such as preset FullGC times, preset total time spent on garbage collection, etc. When the target operating parameters meet the preset conditions, including but not limited to: the target operating parameters are greater than the set thresholds, the target operating parameters are equal to the set thresholds, and the target operating parameters are less than the set thresholds, the system finally determines that the target application corresponding to the current target operating parameters on the data platform is abnormal.

[0046] Through the above steps S202 to S206, the system receives the application detection instruction of the target object, obtains the corresponding application service instance on the current data platform according to the instruction, and obtains the target parameters from the service instance for comparison with the preset conditions. When the preset conditions are met, it is determined that the target application corresponding to the target parameters has an exception. Therefore, when the operating parameters of a certain application are abnormal, it is ensured that the exception can be quickly paid attention to, thereby solving the technical problem that the related technology can only rely on manual detection of the health status of the application on the platform, the manual detection means are lagging and inefficient, and the problems that occur during the operation of the application cannot be perceived in advance. The following is an explanation.

[0047] In order to save operating resources, before receiving the detection instruction of the target object and determining the service instance corresponding to the data platform, the following process can also be performed: determine whether an inspection tool for detecting the application is installed in the data platform; if the inspection tool is not installed on the data platform, generate a first prompt message, wherein the first prompt message is used to prompt the target object to install the inspection tool on the data platform; if the inspection tool is already installed on the data platform, send the detection instruction to the inspection tool. Through the above process, since the detection instruction is sent to the inspection tool only when the inspection tool is already installed on the data platform, waste of resources can be avoided.

[0048] In an optional embodiment, the inspection tool used to detect the application can be a dynamic service discovery, configuration management and service management tool (Naming and Configuration Service, referred to as Nacos). In a Windows system, the task manager can be used to check whether the Nacos process is running. In a Linux system, the ps command can be used to check whether the Nacos process is running. When it is confirmed based on the process information that Nacos is not installed on the data platform, the first prompt information used to prompt the target object to install the inspection tool on the data platform is sent to the message center through the Kafka middleware. The message center notifies the target object through a text message. Otherwise, the detection instruction entered by the target user is sent to Nacos.

[0049] In some embodiments of the present application, a detection instruction of a target object is received, and a service instance corresponding to a data platform is determined, including: obtaining all running applications in the data platform; determining a collection program from a database based on the detection instruction, wherein the collection program is used to collect running parameters of the applications on the data platform; determining the running parameters of the running applications based on the collection program; and determining the service instance corresponding to the data platform based on the running parameters.

[0050] In the embodiment of the present application, when the method is deployed in a Windows operating system, all running applications on the current platform can be obtained through the task manager under the Windows system. When the method is deployed in a Linux operating system, all running applications on the current platform can be obtained through K8s commands under the Linux system, such as kubectl get pod and kubectl describe commands. The collection program is an executable program written by engineers for collecting data, and different detection instructions correspond to different collection programs.

[0051] In an optional embodiment, the collection program execution steps corresponding to the application detection instruction include: registering the database, collector, and data visualization tool as Windows services, using the JVM information collector to collect the operating parameters of all running applications, storing data through the time series database Prometheus or a custom database, and determining each application service instance corresponding to the data platform based on the collected operating parameters. In addition, the collected operating parameters can also be connected to the data visualization tool Grafana, and the operating parameters of different applications can be displayed to the user through the icons in Grafana, providing a basis for human intervention, judgment, and troubleshooting.

[0052] In some embodiments of the present application, in the process of monitoring the running status of the application on the data platform, in addition to monitoring the application, the middleware and database can also be monitored, specifically: determine the middleware and database used by the data platform; execute the first detection program according to the detection instruction to determine the process information of the middleware, and execute the second detection program according to the detection instruction to determine the connection information of the database; determine the first running status of the middleware according to the process information, and determine the second running status of the database according to the connection information. Through the above process, the monitoring of the running status of the middleware and the connection status of the database can be realized. The normal operation of the middleware ensures that the system abnormal information can be transmitted in time, and the normal connection of the database can reduce a series of troubleshooting and maintenance costs caused by service interruptions.

[0053] When monitoring the running status of the middleware, first obtain the middleware information used on the data platform, and when the detection instruction input by the target object is a middleware detection instruction, execute the first detection program. In an optional embodiment, the first detection program is a Java program that controls the execution of the Windows Command Prompt (CMD) command, and outputs the process information of the middleware by executing the command. The process information can be the process status, CPU usage, memory usage, etc. The first running state of the middleware is determined according to the process information, and the first running state can be normal or abnormal. For example, when judging the running state of the Apache Kafka (Kafka for short) middleware according to the process status, in the monitoring indicators of Kafka, ActiveControllerCount represents the active controller count. Usually this value cannot be greater than 1, but if this value is equal to 0 and lasts for a short period of time (<1 second), it can be confirmed that the first running state of the middleware is abnormal, and the running state of the middleware is monitored. Among them, the middleware can be a cache middleware Redis (Remote DictionaryServer), a distributed middleware Kafka, etc.

[0054] When monitoring the database connection status, first obtain the database configuration information on the data platform, and when the detection instruction input by the target object is a database detection instruction, execute the second detection program. In an optional embodiment, the second detection program implementation method includes: collecting database-related information input by the target object, and the relevant information can be information such as database type, database address, port number, etc.; creating a database connection according to the database-related information, and determining the second operating state of the database according to the connection result, for example, when the connection result is successful, determine that the second operating state is normal, when the connection result is failed or the application is down, determine that the second operating state is abnormal; when the second operating state is normal, store the configuration information of the current database in Nacos, and subsequently perform centralized management through Nacos, including starting the database, stopping the database, restarting the database, etc. When the second operating state is abnormal, restart the database with the current abnormal connection or the application that is down, to achieve monitoring of the database connection state.

[0055] In some embodiments of the present application, when the target operating parameters meet preset conditions, determining that an abnormality occurs in a target application in the data platform includes: determining a first parameter set from the target operating parameters, wherein the first parameter set includes at least one of the following: the number of full garbage collections FullGC, the total time spent on garbage collection; when the number of FullGCs exceeds a preset number of times, or the total time spent on garbage collection is greater than a preset time, determining that the target operating parameters meet the preset conditions; determining the target application whose number of FullGCs exceeds a preset number of times or the total time spent on garbage collection is greater than a preset time as an abnormal application; performing a restart operation on the abnormal application and generating a first alarm message, wherein the first alarm message is used to prompt that an abnormal application exists in the target object.

[0056] In an optional embodiment, when the first parameter set is the number of full garbage collections FullGC, a JVM monitoring tool or log can be used to track the number of FullGC occurrences. When the number of FullGC occurrences exceeds a preset number, the application corresponding to the number of FullGC occurrences is determined to be an abnormal application. The restart service program script triggers the restart operation, and the alarm script generates a first alarm message. The first alarm message may include information such as the name of the abnormal application and the name of the abnormal operating parameters. The first alarm message is sent to the message center through the Kafka middleware, and the message center notifies the target object via SMS.

[0057] In another optional embodiment, when the first parameter set is the total time spent on garbage collection, the garbage collection time of the application is monitored through the API provided by Java, and each time garbage collection occurs, its duration is recorded and added to the total time spent on garbage collection. When the total time spent on garbage collection is greater than the preset time, the application corresponding to the total time spent on garbage collection is determined to be an abnormal application, and the restart service program script triggers the restart operation. The alarm script generates a second alarm message. The second alarm message may include the name of the abnormal application, the current garbage collection time, a timestamp and other information. The second alarm message is sent to the message center through the Kafka middleware, and the message center notifies the target object via SMS.

[0058] In some embodiments of the present application, the usage of server and database resources can also be monitored, specifically: obtaining the first operating parameter of the server used by the data platform, and obtaining the second operating parameter of the database used by the data platform; generating a second alarm message when any one of the first operating parameter and / or the second operating parameter is greater than the standard threshold, wherein the second alarm message is used to prompt that the target object server or database has an abnormality. By monitoring the usage of server and database resources, the monitoring scope is expanded and the stability of the system is improved.

[0059] When monitoring a server, the server may be a server in the current environment or a server in another environment, and the first operating parameter may be CPU usage, remaining disk capacity, memory usage, etc. In an optional embodiment, a JVM information collector may be used to collect the CPU usage of the current server, and the collected CPU usage may be stored in a time series database Prometheus or a custom database, and the CPU usage may be compared with a standard threshold. If the current CPU usage is greater than the standard threshold, a second alarm message is generated to indicate that the server is abnormal, and the alarm message is sent to a message center through the Kafka middleware, and the message center notifies the target object via SMS.

[0060] When monitoring a database, the database may be a database of the current environment or a database of another environment, and the second operating parameter may be throughput, response time, CPU usage, etc. In an optional embodiment, a JVM information collector may be used to collect the throughput of the current database, and the collected throughput may be stored through a time series database Prometheus or a custom database, and the throughput may be compared with a standard threshold. If the current throughput is greater than the standard threshold, a second alarm message is generated to indicate that an abnormality exists in the database, and the second alarm message is sent to a message center through the Kafka middleware, and the message center notifies the target object through a text message.

[0061] In some embodiments of the present application, a timer can also be used to collect data in real time to achieve automated real-time monitoring of applications running on the data platform. Specifically: the operating parameters of each application in the data platform are determined according to a preset frequency through a timer; an operating curve is generated based on the operating parameters, and abnormal applications are determined based on the relationship between the operating curve and the standard curve, wherein the operating curve is used to indicate changes in the operating parameters; and alarm information containing abnormal applications is sent to the target object through the middleware.

[0062] In an optional embodiment, the application operating parameters that need to be monitored in real time, such as CPU usage, memory usage, etc., are determined, the timer trigger frequency is set, the target operating parameters are collected regularly by the timer, and an operating curve of the corresponding relationship between the target operating parameter value and time is generated. The standard curve is a horizontal line with a preset value that remains unchanged. When the operating curve intersects with the standard curve, it is determined that the operating parameter value reaches the preset value, and it is determined whether the operating parameter value at the next moment corresponding to the operating parameter value is greater than the preset value. If the operating parameter value at the next moment is greater than the preset value, it is determined that the application corresponding to the target operating parameter has an abnormality, and the alarm script generates an alarm message, and sends the alarm message to the message center through the Kafka middleware, and the message center notifies the target object via SMS.

[0063] Figure 3 FIG. 1 is a schematic diagram of an overall deployment interaction of a monitoring method for a data platform according to an embodiment of the present application. Figure 3 As shown, in an optional embodiment, the entire environment is composed of multiple Windows servers and database servers, which are uniformly managed and configured by the monitoring method of the data platform of the present application. The monitoring method of the data platform of the present application is deployed in the Windows server where the current data platform is located, and can monitor the applications, middleware, and databases running on the current server and data platform. At the same time, it can also monitor the usage of server resources in other environments.

[0064] In another optional embodiment, the method can also be deployed to other Linux servers to monitor the running status of the application. Specifically: under the k8s platform, the server where the application is running is accessed through the IP address, and the running parameters of each application are obtained through the k8s command. The running parameters include running time, running node, resource allocation, etc. The obtained application running parameters are saved in the database, and the application running parameters stored in the database are traversed and compared with the corresponding running parameter preset conditions. When a certain running parameter meets the preset condition, it is determined that the application corresponding to the running parameter has an abnormality, and the alarm script generates an alarm message, and the alarm message is sent to the message center through the middleware. The message center notifies the target object through SMS; under the non-k8s platform, by directly requesting the health check interface of the corresponding application, it is determined whether the application has an abnormality through the information returned by the interface. If it is determined that the application has an abnormality, the restart service program script triggers the restart operation.

[0065] Figure 4 FIG. 1 is an interactive timing diagram a of a method for implementing a monitoring method of a data platform according to an embodiment of the present application. Figure 4As shown, in an optional embodiment, the environment management tool Nacos can initiate a request to the application server at a preset frequency to obtain the currently available platform version information, ensuring that the target user can understand the software version that can be used in the current environment, so as to make corresponding choices, and the supOS-AD / supOS-Lite application server (hereinafter referred to as the application server) returns the platform version information to the environment management tool. Obtain the detection instruction of the target user, for example, the target user selects the platform version and clicks the detection button, and initiates a request to the application server through the environment management tool to obtain the Nacos service port information and the application server port usage information, ensuring the correct configuration and availability of each service port in the current environment, and the application server returns the corresponding server port information and port usage information to the environment management tool, and displays it to the target user through a page for further analysis and configuration. In addition, you can also obtain the target user's application detection instructions. For example, the target user clicks the detection button on the application (app) tab, and Nacos initiates a request to the application server to obtain the information and startup status of all application modules in the environment, monitor the applications running in the current environment, and the application server returns all app module information and startup status to Nacos, and displays it to the target user through the page, ensuring that the target user can intuitively understand the overall running status of each application in the current environment. You can also obtain the target user's middleware detection instructions. For example, the target user clicks the detection button on the system middleware tab, and Nacos initiates a request to the application server to obtain the information and startup status of the specified middleware in the current environment, ensuring the normal operation of the middleware. The application server returns the information and startup status of the specified middleware in the current environment to Nacos, and displays it to the target user through the page.You can also obtain the database detection instructions of the target user. For example, the target user clicks the Detect button on the System Database tab, and Nacos initiates a request to the application server to obtain the Nacos configuration database information on the environment to ensure the correctness of the database configuration. After the application server returns the database information to the target user through Nacos, the target user fills in the database-related information and clicks the link. Nacos sends the database information detection instruction to the application server, and the application server returns the detection result to Nacos and returns it to the target user in the form of a page display. Specifically, the environment management tool obtains the database-related information and database connection instructions entered by the target user, creates a database connection based on the database-related information and detects whether the connection is successful, and displays the database connection result on the interactive page; obtain the system resource detection instructions of the target user. For example, the target user clicks the Detect button on the System Resource tab, and Nacos initiates a request to the application server to obtain the system resource usage information on the current environment to assist in system performance optimization and troubleshooting. The application server returns the obtained resource usage information to Nacos and returns it to the target user in the form of a page display to ensure that the target user has a comprehensive understanding of the health status of the system.

[0066] Figure 5 FIG. 1 is an interactive timing diagram b of a method for implementing a monitoring method of a data platform according to an embodiment of the present application. Figure 5 As shown, in an optional embodiment, the method can not only monitor the system resources of the current environment, but also monitor the Windows resource information, database information, application information, etc. of other environments. Specifically: obtain the resource detection instruction of the target user, for example, the target user clicks the detection button on the resource detection tab, and after Nacos jumps to the tab, the target user clicks the detection tab. After Nacos jumps to the tab, it initiates a request to other database servers to obtain other environment Windows resource information, other environment database monitoring information, and current server entity configuration application JVM monitoring information. Other database servers return the obtained information to Nacos and return it to the target user in the form of page display so that the target user can perform comprehensive analysis. Through the above steps, the monitoring scope is expanded, which is conducive to cross-environment resource management and optimization.

[0067] In the above process, the service instance corresponding to the data platform is obtained through the detection instruction, the operating parameters of the application are obtained from the service instance, and the operating parameters are compared with the preset conditions to determine whether the application is abnormal, thereby achieving the purpose of monitoring the application running on the data platform and being able to perceive problems that occur on the data platform in advance. At the same time, the method can also monitor the middleware operating status, server resource usage, database connection status, and database resource usage, expanding the monitoring scope and improving the stability of the system.

[0068] According to an embodiment of the present application, a monitoring device for a data platform is provided. It should be noted that the monitoring device for a data platform in the embodiment of the present application can be used to execute the monitoring method for a data platform provided in the embodiment of the present application. The monitoring device for a data platform provided in the embodiment of the present application is introduced below.

[0069] Figure 6 1 is a structural diagram of a monitoring device for a data platform provided according to an embodiment of the present application. Figure 6 As shown, the device comprises:

[0070] A receiving module 40 is used to receive a detection instruction of a target object and determine a service instance corresponding to a data platform, wherein the data platform includes a control platform and a code development platform used in industrial production;

[0071] A first determination module 42 is used to determine target operating parameters of a target application from a service instance, wherein the application is any application installed in the data platform;

[0072] The second determination module 44 is used to determine whether an abnormality occurs in the target application in the data platform when the target operating parameters meet the preset conditions.

[0073] Through the above-mentioned receiving module 40, the first determination module 42 and the second determination module 44, the purpose of monitoring the applications running on the data platform is achieved, thereby achieving the technical effect of ensuring that when an abnormality occurs in the operating parameters of a certain application, the abnormality can be quickly paid attention to, thereby solving the technical problem that the related technology can only rely on manual detection of the health status of the application on the platform, the manual detection means are lagging and inefficient, and the problems that occur during the operation of the application cannot be perceived in advance.

[0074] In a monitoring device for a data platform provided in an embodiment of the present application, the receiving module is also used to determine whether an inspection tool for detecting an application is installed in the data platform; if the inspection tool is not installed on the data platform, a first prompt message is generated, wherein the first prompt message is used to prompt the target object to install the inspection tool on the data platform; if the inspection tool has been installed on the data platform, a detection instruction is sent to the inspection tool.

[0075] In a monitoring device for a data platform provided in an embodiment of the present application, the receiving module is also used to obtain all running applications in the data platform; determine a collection program from a database based on a detection instruction, wherein the collection program is used to collect running parameters of the applications on the data platform; determine the running parameters of the running applications based on the collection program; and determine a service instance corresponding to the data platform based on the running parameters.

[0076] In a monitoring device for a data platform provided in an embodiment of the present application, the first determination module is also used to determine the middleware and database used by the data platform; execute the first detection program according to the detection instruction to determine the process information of the middleware, and execute the second detection program according to the detection instruction to determine the connection information of the database; determine the first operating state of the middleware according to the process information, and determine the second operating state of the database according to the connection information.

[0077] In a monitoring device for a data platform provided in an embodiment of the present application, the first determination module is also used to obtain a first operating parameter of a server used by the data platform, and to obtain a second operating parameter of a database used by the data platform; when any one of the first operating parameter and / or the second operating parameter is greater than a standard threshold, a second alarm message is generated, wherein the second alarm message is used to prompt that there is an abnormality in the target object server or database.

[0078] In a monitoring device for a data platform provided in an embodiment of the present application, the second determination module is further used to determine a first parameter set from the target operating parameters, wherein the first parameter set includes at least one of the following: the number of full garbage collections FullGC and the total time spent on garbage collection; when the number of FullGCs exceeds a preset number of times, or the total time spent on garbage collection is greater than a preset time, determining that the target operating parameters meet preset conditions; determining a target application whose number of FullGCs exceeds a preset number of times or the total time spent on garbage collection is greater than a preset time as an abnormal application; performing a restart operation on the abnormal application and generating a first alarm message, wherein the first alarm message is used to prompt that an abnormal application exists in the target object.

[0079] In a data platform monitoring device provided in an embodiment of the present application, the second determination module is also used to determine the operating parameters of each application in the data platform according to a preset frequency through a timer; generate an operating curve based on the operating parameters, and determine abnormal applications based on the relationship between the operating curve and the standard curve, wherein the operating curve is used to represent changes in the operating parameters; and send alarm information containing abnormal applications to the target object through the middleware.

[0080] An embodiment of the present application also provides an electronic device, which includes a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory, and is used to execute program instructions to implement the following functions: receiving a detection instruction of a target object, and determining a service instance corresponding to a data platform, wherein the data platform includes a control platform and a code development platform used in industrial production; determining target operating parameters of a target application from the service instance, wherein the target application is any application installed in the data platform; and determining that an abnormality has occurred in the target application in the data platform when the target operating parameters meet preset conditions.

[0081] It should be noted that the above electronic equipment is used to execute Figure 2 The monitoring method of the data platform shown, therefore the relevant explanations in the above-mentioned monitoring method of the data platform are also applicable to the electronic device, and will not be repeated here.

[0082] An embodiment of the present application also provides a non-volatile storage medium, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned data platform monitoring method by running the computer program.

[0083] It should be noted that the above non-volatile storage medium is used to execute Figure 2 The monitoring method of the data platform shown, therefore the relevant explanations in the above-mentioned monitoring method of the data platform are also applicable to the non-volatile storage medium, and will not be repeated here.

[0084] An embodiment of the present application also provides a computer program product, including a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of the data platform monitoring method in each embodiment of the present application are implemented.

[0085] An embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the data platform monitoring method in each embodiment of the present application.

[0086] The embodiments of the present application also provide a computer program, which, when executed by a processor, implements the steps of the data platform monitoring method in each embodiment of the present application.

[0087] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0088] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0089] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0090] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0091] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0092] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly or all or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc., which can store program code.

[0093] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A data platform monitoring method, characterized in that: include: Receiving a detection instruction of a target object, and determining a service instance corresponding to a data platform, wherein the data platform includes a control platform and a code development platform used in industrial production; Determining target operating parameters of a target application from the service instance, wherein the target application is any application installed in the data platform; When the target operating parameters meet preset conditions, it is determined that an abnormality occurs in the target application in the data platform.

2. The method according to claim 1, characterized in that Before receiving a detection instruction of a target object and determining a service instance corresponding to the data platform, the method further includes: Determining whether a checking tool for detecting the application is installed in the data platform; If the inspection tool is not installed on the data platform, generating first prompt information, wherein the first prompt information is used to prompt the target object to install the inspection tool on the data platform; In the case where the inspection tool has been installed on the data platform, the detection instruction is sent to the inspection tool.

3. The method according to claim 1, characterized in that Receive the detection instruction of the target object and determine the service instance corresponding to the data platform, including: Obtaining all running applications in the data platform; According to the detection instruction, a collection program is determined from a database, wherein the collection program is used to collect operating parameters of an application program of the data platform; Determining the operating parameters of the application program in operation according to the acquisition program; A service instance corresponding to the data platform is determined according to the operating parameters.

4. The method according to claim 1, characterized in that The method further comprises: Determine the middleware and database used by the data platform; executing a first detection program according to the detection instruction to determine the process information of the middleware, and executing a second detection program according to the detection instruction to determine the connection information of the database; A first operating state of the middleware is determined according to the process information, and a second operating state of the database is determined according to the connection information.

5. The method according to claim 1, characterized in that When the target operating parameters meet the preset conditions, determining that an abnormality occurs in the target application in the data platform includes: Determine a first parameter set from the target operating parameters, wherein the first parameter set includes at least one of the following: the number of full garbage collections (FullGC) and the total time spent on garbage collection; When the number of FullGCs exceeds a preset number, or the total duration of garbage collection is greater than a preset duration, determining that the target operating parameter satisfies the preset condition; Determine the target application program whose number of FullGC exceeds the preset number or whose total time spent on garbage collection is longer than the preset time as an abnormal application program; A restart operation is performed on the abnormal application and a first alarm message is generated, wherein the first alarm message is used to prompt the target object that the abnormal application exists.

6. The method according to claim 1, characterized in that The method further comprises: Acquire a first operating parameter of a server used by the data platform, and acquire a second operating parameter of a database used by the data platform; When any one of the first operating parameter and / or the second operating parameter is greater than a standard threshold, a second alarm message is generated, wherein the second alarm message is used to prompt the target object, the server or the database, that an abnormality exists.

7. The method according to claim 1, characterized in that The method further comprises: Determining the operating parameters of each application in the data platform according to a preset frequency through a timer; generating an operation curve according to the operation parameters, and determining abnormal applications according to the relationship between the operation curve and a standard curve, wherein the operation curve is used to represent the change of the operation parameters; The warning information containing the abnormal application is sent to the target object through the middleware.

8. A monitoring device for a data platform, characterized in that: include: A receiving module, configured to receive a detection instruction of a target object and determine a service instance corresponding to a data platform, wherein the data platform includes a control platform and a code development platform used in industrial production; A first determination module, configured to determine target operating parameters of a target application from the service instance, wherein the application is any application installed in the data platform; The second determination module is used to determine that an abnormality occurs in the target application in the data platform when the target operating parameters meet preset conditions.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the data platform monitoring method described in any one of claims 1 to 7 by running the computer program.

10. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the data platform monitoring method described in any one of claims 1 to 7 is implemented.

Images