Website inspection monitoring method, system and device and storage medium

Abstract

The invention provides a website inspection monitoring method, system and device and a storage medium. The method comprises the following steps: S100, customizing a monitoring task; s200, executing a monitoring task; s300, processing a monitoring result; s400, the AI analyzes a monitoring result; and S500, automatically processing the exception by the PRA, and returning to execute the S200. According to the method, stable operation of the target website can be guaranteed, and abnormal conditions can be responded in time.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a website inspection and monitoring method, system, device and storage medium. Background Art

[0002] In the modern Internet era, all industries are inseparable from their official websites, which include interfaces that support business capabilities. Therefore, it is particularly important to ensure the stable operation of the official website and respond to abnormal situations in a timely manner. Summary of the invention

[0003] The embodiments of the present application provide a website inspection and monitoring method, system, device and storage medium to solve the problems existing in the related technologies. The technical solutions are as follows:

[0004] In a first aspect, an embodiment of the present application provides a website inspection and monitoring method, comprising:

[0005] S100, customizing monitoring tasks: customizing multiple monitoring targets of the target website, and then customizing and configuring the Yaml configuration file corresponding to each of the monitoring targets, wherein the Yaml configuration file corresponding to each of the monitoring targets records the monitoring mode and monitoring frequency of each of the monitoring targets, and the multiple monitoring targets include port monitoring targets, HTTP monitoring targets, and SSL certificate monitoring targets, and the monitoring mode of each of the monitoring targets is traversal execution;

[0006] S200, executing the monitoring task: parsing the YAML configuration file corresponding to each monitoring target, performing patrol monitoring on each monitoring target based on the monitoring mode and monitoring frequency of each monitoring target, and returning relevant alarm messages;

[0007] S300, processing monitoring results: storing the inspection monitoring data of S200, and calling the corresponding alarm robot to push the relevant alarm message returned by S200 to the corresponding person in charge;

[0008] S400, AI analysis monitoring results: calling the AI ​​big model to analyze and process the relevant alarm messages returned by S200, obtaining analysis results, storing the analysis results locally or in MQ, where MQ refers to a message queue, and the analysis results include alarm events of corresponding categories;

[0009] S500, PRA automated exception processing: the analysis result is read by the PRA robot, and an external API processing service is called to process the alarm event of the corresponding category, and the process returns to execute S200.

[0010] In one embodiment, the port-type monitoring target includes a plurality of TCP ports; in S200, the following inspection and monitoring operations are performed on the port-type monitoring target each time according to the monitoring frequency of the port-type monitoring target:

[0011] S201, calling Python's socket library to create a TCP entity class;

[0012] S202, compile a first request mode, a first timeout time and a first maximum monitoring number corresponding to the TCP entity class;

[0013] S203, calling the TCP entity class to traverse and access each of the TCP ports using the first request method within the first timeout period, and determining whether each of the TCP ports is connected; if all of the TCP ports are connected, executing S204, or if at least one of the TCP ports is not connected, executing S205;

[0014] S204, determining that the TCP port response of the target website is normal, and completing TCP port monitoring;

[0015] S205, triggering cyclic monitoring, returning to S203, and recording the first cyclic monitoring times until all the TCP ports are connected, otherwise executing S206;

[0016] S206, determining whether the first cycle number reaches the first maximum monitoring number; if the first cycle number is less than the first maximum monitoring number, continue to execute S205, or if the first cycle number reaches the first maximum monitoring number, execute S207;

[0017] S207. Return a first alarm message, where the first alarm message is a message in msg format and is used to record an abnormal message of at least one of the TCP ports.

[0018] In one embodiment, the HTTP monitoring target includes a plurality of URL addresses, and the plurality of URL addresses include a URL address corresponding to a web page and a URL address corresponding to an interface; in S200, the following inspection and monitoring operations are performed on the HTTP monitoring target each time according to the monitoring frequency of the HTTP monitoring target:

[0019] S201a, call Python's requests library to create the monitorByHttp entity class;

[0020] S202a, compile the second request method, the second timeout period, the second maximum monitoring number and the third maximum monitoring number corresponding to the monitorByHttp entity class; wherein the second request method self-defined selects the GET request method or the POST request method, and the parameter data structure of the second request method is converted using the json.dumps method;

[0021] S203a, calling the monitorByHttp entity class to traverse and access each of the URL addresses using the second request method within the second timeout period, and determining whether each of the URL addresses is connected; if all of the URL addresses are connected, executing S204a, or if at least one of the URL addresses is not connected, executing S209a;

[0022] S204a, matching the response content of each URL address with a preset verification keyword; if the response content of all URL addresses is consistent with the verification keyword, executing S205a, or if the response content of at least one URL address is inconsistent with the verification keyword, executing S206a;

[0023] S205a, determining that the URL address of the target website responds normally, and completing HTTP monitoring;

[0024] S206a, triggering loop monitoring, returning to execute S203a, and recording the second loop monitoring times, until the response contents of all the URL addresses are consistent with the verification keyword, otherwise execute S207a;

[0025] S207a, determining whether the second cycle number reaches the second maximum monitoring number; if the second cycle number is less than the second maximum monitoring number, continue to execute S206a, or if the second cycle number reaches the second maximum monitoring number, execute S208a;

[0026] S208a, returning a second warning message, wherein the second warning message is a message in msg format, and is used to warn that the URL response message is abnormal;

[0027] S209a, triggering loop monitoring, returning to execute S203a, and recording the third loop monitoring times until all the URL addresses are connected, otherwise execute S210a;

[0028] S210a, determining whether the third cycle number reaches the third maximum monitoring number; if the third cycle number is less than the third maximum monitoring number, continue to execute S209a, or if the third cycle number reaches the third maximum monitoring number, execute S211a;

[0029] S211a. Return a third alarm message, where the third alarm message is used to warn of URL port timeout.

[0030] In one implementation, in S203a, if the URL corresponds to a common interface, the second request method is a custom GET request method;

[0031] If the URL address corresponds to a special interface, the second request method customizes the selection of the GET request method or the POST request method, and configures the interface account password, Headers, and Body fields in the parameter data structure of the second request method according to the corresponding request, and then uses the json.dumps method for conversion. The special interface refers to the interface that requires verification of the correctness of the interface data response;

[0032] If the URL address corresponds to a web page, the second request method customizes the GET request method.

[0033] In one embodiment, the SSL certificate monitoring target includes several domain names used by the target SSL certificate of the target website; in S200, the following inspection and monitoring operations are performed on the HTTP monitoring target each time according to the monitoring frequency of the HTTP monitoring target:

[0034] S201b, call Python's ssl library to create the monitorBySsl entity class;

[0035] S202b, calling the monitorBySsl entity class to read a number of the domain names, and using the wrap_socket function to verify whether the connectivity of each of the domain names is normal; if the connectivity of each of the domain names is normal, executing S203b, or, if the connectivity of each of the domain names is abnormal, executing S207b;

[0036] S203b, parsing each of the domain names, obtaining all information of the target SSL certificate, wherein all the information includes the issuing object, issuing time and expiration time, and the issuing object includes several of the domain names;

[0037] S204b, calculating the remaining validity period of the SSL certificate corresponding to each of the domain names based on all the information, and determining whether the remaining validity period of the SSL certificate reaches the alarm time threshold; if the remaining validity period of the SSL certificate reaches the alarm time threshold, executing S205b, or if the remaining validity period of the SSL certificate does not reach the alarm time threshold, executing S206b;

[0038] S205b, returning a fourth warning message, wherein the fourth warning message is used to warn that the remaining time of the SSL certificate of each domain name is insufficient;

[0039] S206b, determining that each of the domain names does not need to renew the target SSL certificate, and completing SSL certificate monitoring for each of the domain names;

[0040] S207b. Return fifth warning information, where the fifth warning information is used to warn that each domain name cannot be connected.

[0041] In one embodiment, the AI ​​model is a TensorFlow model.

[0042] In one implementation, the PRA robot is Python's Invoke Python Activity or Invoke Code Activity.

[0043] In a second aspect, the embodiment of the present application further provides a website inspection and monitoring system, including:

[0044] A customization unit is used to customize monitoring tasks: customize multiple monitoring targets of a target website, and then customize and configure a Yaml configuration file corresponding to each of the monitoring targets, wherein the Yaml configuration file corresponding to each of the monitoring targets records the monitoring mode and monitoring frequency of each of the monitoring targets, and the multiple monitoring targets include port monitoring targets, HTTP monitoring targets, and SSL certificate monitoring targets, and the monitoring mode of each of the monitoring targets is traversal execution;

[0045] An execution unit is used to execute the monitoring task: parse the YAML configuration file corresponding to each monitoring target, perform patrol monitoring on each monitoring target based on the monitoring mode and monitoring frequency of each monitoring target, and return relevant alarm messages;

[0046] The first processing unit is used to process the monitoring results: store the inspection monitoring data of the execution unit, and call the corresponding alarm robot to push the relevant alarm message returned by the execution unit to the corresponding person in charge;

[0047] The analysis unit is used for AI analysis of monitoring results: calling the AI ​​big model to analyze and process the relevant alarm messages returned by the execution unit to obtain analysis results, and storing the analysis results locally or in MQ, where MQ refers to a message queue. The analysis results include alarm events of corresponding categories;

[0048] The second processing unit is used for PRA automated exception processing: the analysis result is read by the PRA robot, and the external API processing service is called to process the alarm event of the corresponding category, and the monitoring task is returned to the execution unit.

[0049] In a third aspect, an embodiment of the present application further provides a computer device, comprising: a memory and a processor, wherein the memory stores instructions, and the instructions are loaded and executed by the processor to implement a method in any one of the above-mentioned aspects, wherein the memory and the processor communicate with each other through an internal connection path.

[0050] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer, the method in any one of the above-mentioned embodiments is implemented.

[0051] The advantages or beneficial effects of the above technical solution include at least:

[0052] This application can be used from the user's perspective, using Python program to automatically patrol and monitor the target website to perform page status detection, interface response monitoring, TCP port listening, SSL certificate validity detection, etc., so as to timely discover system failures of the target website and provide feedback to the relevant person in charge for processing as soon as possible. For example, the alarm message can be pushed to the alarm notification group through the open interface of DingTalk / WeChat robot, and the relevant person in charge can respond to and handle the exception in time, thereby improving the system stability and response efficiency of the target website. At the same time, the introduction of AI big models and RPA automation in the next stage will reduce manual operations to a certain extent and free up most of the manual workload. In this way, the service can be handled through automation as soon as possible. If it cannot be solved, manual intervention will be carried out.

[0053] In summary, this application can ensure the stable operation of the target website and respond to abnormal situations in a timely manner.

[0054] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present application will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed in the present application and should not be regarded as limiting the scope of the present application.

[0056] Figure 1 A flowchart of a website inspection and monitoring method provided in an embodiment of the present application;

[0057] Figure 2An example diagram of a process for inspecting and monitoring a port type monitoring target provided in an embodiment of the present application;

[0058] Figure 3 An example diagram of a process for patrolling and monitoring HTTP-type monitoring targets provided in an embodiment of the present application;

[0059] Figure 4 An example diagram of a process for patrolling and monitoring SSL certificate monitoring targets provided in an embodiment of the present application;

[0060] Figure 5 A schematic diagram of the structure of a website inspection and monitoring system provided in an embodiment of the present application;

[0061] Figure 6 A structural block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0063] Before introducing the embodiments of the present application, some terms in the present application are first explained to facilitate understanding by those skilled in the art.

[0064] Robotic Process Automation (RPA): It is a software automation technology that usually runs in the form of "software" on the office staff's computer, and is sometimes called "RPA robot". RPA robots can automatically operate various software in the computer according to pre-set operating rules, such as opening websites, clicking the mouse, keyboard input, etc., thereby realizing the technology of automated operation of business processes.

[0065] Artificial Intelligence (AI) is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It mainly includes the principles of computer intelligence and the manufacture of computers similar to human brain intelligence, so that computers can achieve higher-level applications.

[0066] Harbor is an enterprise-level registry service for storing Docker images.

[0067] ConfigMap is an API object used to store non-confidential data (such as configuration files, environment variables, command line parameters, etc.). ConfigMap allows the application configuration to be managed separately from the containerized application, thereby achieving flexible and dynamic management of configuration.

[0068] An Application Programming Interface (API) is a set of predefined functions that aims to provide applications and developers with the ability to access a set of routines based on certain software or hardware without having to access the source code or understand the details of the internal working mechanism.

[0069] Ceph is a distributed storage system that is very flexible in storing data. For example, if you need to expand capacity, you only need to add servers to the Ceph cluster.

[0070] A message queue (MQ) is a container for storing messages. It is essentially a queue that allows applications to send and receive messages asynchronously, ensures reliable transmission of messages, and improves the system's decoupling capabilities and scalability.

[0071] Uipath is a Robotic Process Automation tool for Windows desktop automation that automates repetitive / redundant tasks with the help of drag-and-drop functionality and eliminates human intervention.

[0072] The requests library is a third-party request library for Python, which is rewritten based on the urllib library (Python's built-in standard library for HTTP requests). Therefore, compared with the urllib library, requests is more user-friendly and convenient, and is the most frequently used HTTP request library in crawler logic.

[0073] Various technical solutions provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0074] Figure 1 FIG. 1 is a flow chart showing a website inspection and monitoring method according to an embodiment of the present application. Figure 1 As shown, the method may include the following steps:

[0075] S100, customizing monitoring tasks: customizing multiple monitoring targets of the target website, and then customizing and configuring the Yaml configuration file corresponding to each of the monitoring targets.

[0076] In one implementation, the Yaml configuration file corresponding to each monitoring target records the monitoring mode and monitoring frequency of each monitoring target. It can be understood that one Yaml configuration file corresponds to one monitoring target.

[0077] In specific implementation, the various monitoring targets may include but are not limited to: port type monitoring targets, HTTP type monitoring targets and SSL certificate type monitoring targets.

[0078] In a specific implementation, the target website may be any website that needs to be monitored, and this embodiment of the present application does not limit this.

[0079] During specific implementation, the monitoring mode and monitoring frequency of each monitoring target can be passed in through its corresponding YAML configuration file.

[0080] As an example, in the process of customizing multiple monitoring targets of a target website, the information such as the port, address, and domain name certificate to be monitored can be configured in advance and then passed in through the corresponding YAML configuration file.

[0081] In one embodiment, the monitoring mode of each monitoring target can be set to traversal execution. The monitoring frequency of each monitoring target can be set to perform a patrol monitoring operation at a specified interval, wherein the configuration time unit of the specified interval can be seconds (s), and its specific value can be set according to actual needs, and the embodiment of the present application does not limit this.

[0082] In the embodiment of the present application, by executing step S100 to customize the monitoring task, it is possible to monitor multiple monitoring items of the target website to ensure the stable operation of the target website and respond to abnormal situations in a timely manner.

[0083] S200, executing monitoring tasks: parsing the YAML configuration file corresponding to each monitoring target, performing patrol monitoring on each monitoring target based on the monitoring mode and monitoring frequency of each monitoring target, and returning relevant alarm messages.

[0084] In one embodiment, the port class monitoring target may include several TCP ports. The Yaml configuration file corresponding to the port class monitoring target may record these several TCP ports in a list, and the recorded list is a TCP list. The specific values ​​of these several TCP ports may be set according to actual needs, and the embodiment of the present application does not limit this.

[0085] In one implementation, in step S200, by parsing the Yaml configuration file corresponding to the port-type monitoring target, the monitoring mode and monitoring frequency of the port-type monitoring target and the TCP list can be obtained.

[0086] As an example, taking the port monitoring target including a TCP port as an example, its corresponding YAML configuration file can be expressed as follows:

[0087]

[0088] Monitoring target: TCP port

[0089] Monitoring method: traversal inspection, traversal IP corresponding port: 127.0.0.1 1234

[0090] Monitoring frequency: once a minute, YAML configuration: timer: 60(s)

[0091] In specific implementation, by parsing the Yaml configuration file corresponding to the port class monitoring target, the TCP list in the Yaml configuration file corresponding to the port class monitoring target can be loaded into the monitoring list, so as to facilitate the subsequent detection of the TCP port situation through the TCP entity class.

[0092] In one embodiment, Figure 2 As shown, in step S200, the following patrol monitoring operation can be performed on the port-type monitoring target each time according to the monitoring frequency of the port-type monitoring target. For example, if the monitoring frequency of the port-type monitoring target is set to perform a patrol monitoring operation every 10 minutes, then the following patrol monitoring operation can be performed every 10 minutes:

[0093] S201. Call Python's socket library to create a TCP entity class.

[0094] As an example, the TCP entity class may be a monitorByTcp entity class.

[0095] S202: Write a first request mode, a first timeout period, and a first maximum number of monitoring times corresponding to the TCP entity class.

[0096] In specific implementation, the first request method may select a request method such as GET or POST.

[0097] In a specific implementation, the first timeout period (ie, the timeout period) may be set according to actual needs, for example, it may be set to 60 seconds, and this embodiment of the present application does not limit this.

[0098] In a specific implementation, the first maximum number of monitoring times can be understood as the maximum number of times that a cyclic detection of a TCP port can be triggered, which can be set according to actual needs, and the embodiments of the present application do not limit this.

[0099] S203, calling the TCP entity class to traverse and access each TCP port using the first request method within the first timeout period, and determining whether each TCP port is connected. If all TCP ports are connected, executing S204, or if there is at least one TCP port that is not connected, executing S205.

[0100] S204: Determine that the TCP port response of the target website is normal, and complete TCP port monitoring.

[0101] S205, triggering cyclic monitoring, returning to execute S203, and recording the first cyclic monitoring times until all TCP ports are connected, otherwise execute S206.

[0102] That is, in an embodiment of the present application, during the TCP port inspection process, when an abnormal TCP port situation is detected, there will be a retry interval, namely the first timeout period, and cyclic detection will be performed again on the abnormal TCP port. In this way, the network instantaneous jitter triggering alarm can be avoided, and the operation and maintenance personnel can be prevented from generating alarm fatigue.

[0103] That is, in the embodiment of the present application, through step S203 and step S205, the connectivity detection of the TCP port of the target website can be implemented to timely discover the abnormal situation of the TCP port of the target website.

[0104] S206: Determine whether the first cycle number reaches the first maximum monitoring number. If the first cycle number is less than the first maximum monitoring number, continue to execute S205, or if the first cycle number reaches the first maximum monitoring number, execute S207.

[0105] S207. Return the first warning message.

[0106] In a specific implementation, the first alarm message may be a message in msg format, which is used to record an abnormal message of at least one of the TCP ports.

[0107] As an example, the msg format of the first warning message can be customized in the following manner:

[0108] self.msg = '####**' + self.name + '**Monitoring alarm\n\n' + \'Port service:

[0109] '+self.ipaddres+':'+str(self.port)+'\n\n'\'\n\n'+\'Port exceptions have been triggered 10 times in a row, please handle it urgently! ! ! \n\n'+\'######**'+date+'**Release [Alarm](#)\n\n'+\"

[0110] In an embodiment of the present application, in step S200, Python built-in functions (i.e., Socket library) can be used to implement connectivity monitoring of the TCP port of the target website, so as to promptly detect abnormal conditions of the TCP port of the target website.

[0111] In one embodiment, the HTTP class monitoring target may include several URL addresses, each URL address corresponds to a web page or an interface, that is, the several URL addresses may include URL addresses corresponding to the web page and URL addresses corresponding to the interface. Among them, the Yaml configuration file corresponding to the HTTP class monitoring target can record these several URL addresses in the form of a list, and the recorded list is the HTTP list. Among them, the specific values ​​of these several URL addresses can be set according to actual needs, and the embodiment of the present application does not limit this.

[0112] In one implementation, in step S200, by parsing the Yaml configuration file corresponding to the HTTP-type monitoring target, the monitoring mode and monitoring frequency of the HTTP-type monitoring target and the HTTP list can be obtained.

[0113] As an example, the Yaml configuration file corresponding to the HTTP monitoring target can be expressed as follows:

[0114]

[0115]

[0116] Monitoring target: web page, interface

[0117] Monitoring method: traversal inspection, traversal interface: http: / / 127.0.0.1:1234 / test

[0118] Monitoring frequency: once a minute, YAML configuration: timer: 60(s)

[0119] In specific implementation, by parsing the Yaml configuration file corresponding to the HTTP class monitoring target, the HTTP list in the Yaml configuration file corresponding to the HTTP class monitoring target can be loaded into the monitoring list, so as to facilitate the subsequent detection of HTTP conditions through monitorByHttp.

[0120] In one embodiment, in step S200, the following patrol monitoring operation may be performed on the HTTP monitoring target each time according to the monitoring frequency of the HTTP monitoring target. For example, if the monitoring frequency of the HTTP monitoring target is set to perform a patrol monitoring operation every 10 minutes, then the following patrol monitoring operation may be performed every 10 minutes:

[0121] S201a. Call Python's requests library to create the monitorByHttp entity class.

[0122] S202a. Write the second request mode, the second timeout period, the second maximum monitoring times and the third maximum monitoring times corresponding to the monitorByHttp entity class.

[0123] In specific implementation, the second request method can be customized to select a GET request method or a POST request method, and the parameter data structure of the second request method is converted using the json.dumps method.

[0124] In specific implementation, the second maximum monitoring number and the third maximum monitoring number can be set according to actual needs, and the second maximum monitoring number and the third maximum monitoring number can be set to be the same or different, which is not limited in the embodiment of the present application.

[0125] In specific implementation, the second timeout period (ie, timeout period) may be set according to actual needs, for example, may be set to 60s, and this embodiment of the present application does not limit this.

[0126] S203a, call the monitorByHttp entity class to traverse and access each URL address using the second request method within the second timeout period, and determine whether each URL address is connected. If all URL addresses are connected, execute S204a, or if there is at least one URL address that is not connected, execute S209a.

[0127] In specific implementation, for common interfaces, it can be set to only verify their connectivity. Then in step S203a, if the URL corresponds to a common interface, the second request method can be customized to select a GET request method.

[0128] In the specific implementation, for special interfaces, that is, interfaces that need to verify the correctness of interface data responses, it is necessary to configure interface account password, Headers, Body (Json, Xml) and other fields according to the corresponding request. Then in step S203a, if the URL address corresponds to a special interface, the second request method can customize the selection of GET request method or POST request method, and configure the interface account password, Headers, Body and other fields in the parameter data structure of the second request method according to the corresponding request, and then use the json.dumps method for conversion. For example, the json.dumps method can be used to convert the parameter data structure of the second request method into an XML structure or a JSON structure according to the input Body specification.

[0129] That is, in the trial implementation of this application, interface response monitoring supports detecting interface response status through GET request method, and can match the returned content. At the same time, it also supports detecting interface response status through POST request method, and can input parameters, request headers, etc., and supports Body as XML and JSON format message input.

[0130] In specific implementation, for a web page (which can be an HTTP web page or an HTTPS web page), it can be set to perform a GET request only for a specific address to verify the connectivity of the web page address, and whether the response content is consistent with the preset verification keyword to verify its connectivity. Then in step S203a, if the URL address corresponds to a web page, the second request method can be customized to select the GET request method.

[0131] That is, in an embodiment of the present application, during the HTTP inspection process, when an abnormal URL address is detected, there will be a retry interval, i.e., a second timeout period, and cyclic detection will be performed on the abnormal URL address. In this way, it is possible to avoid the network instantaneous jitter triggering an alarm and prevent the operation and maintenance personnel from generating alarm fatigue. At the same time, it is also possible to avoid the situation where the response time exceeds the default threshold and a false alarm is caused when the response time is slightly longer due to excessive interface data.

[0132] S204a, match the response content of each URL address with a preset verification keyword; if the response content of all URL addresses is consistent with the verification keyword, execute S205a, or if the response content of at least one URL address is inconsistent with the verification keyword, execute S206a.

[0133] That is, in the embodiment of the present application, the webpage response monitoring can detect whether the page can be opened normally and whether it is consistent with the verification keyword under normal circumstances by returning a preset verification keyword through a specified keyword. For example, the returned information can be matched with the response content of a successful custom request, and an alarm is triggered if it is inconsistent.

[0134] S205a, determine that the URL address of the target website responds normally, and complete HTTP monitoring.

[0135] S206a, triggering loop monitoring, returning to execute S203a, and recording the number of second loop monitoring, until the response contents of all URL addresses are consistent with the verification keyword, otherwise execute S207a.

[0136] S207a, determine whether the second cycle number reaches the second maximum monitoring number; if the second cycle number is less than the second maximum monitoring number, continue to execute S206a, or if the second cycle number reaches the second maximum monitoring number, execute S208a.

[0137] S208a. Return a second warning message.

[0138] In a specific implementation, the second alarm message may be a message in msg format, which is used to alarm that the URL response message is abnormal.

[0139] As an example, the msg format of the second warning message can be customized as follows:

[0140] self.tmp='Request response data:'+'Request interface timeout, no data response'+'\n\n'+\'Request response code:'+'Timeout'+'\n\n'self.msg='####**'+self.name+'**Monitoring alarm\n\n'+\'Alarm information:'+self.alterMsg+'\n\n'+\'Interface service address:'+self.url+'\n\n'+\self.tmp+\'\n\n'+\'Interface exceptions have been triggered 10 times in a row, please deal with it urgently! ! ! \n\n'+\'######**'+date+'**Release [Alarm](#)\n\n'+\"

[0141] #The status code is normal, but the POST / GET request returns an abnormal message

[0142] self.tmp = 'Request response code:' + str (self.response.status_code) + '\n\n' + \'Request response data:' + str (self.response.text) + '\n\n'

[0143] self.msg = '####**'+self.name+'Response message abnormal**\n\n'+\'Alarm information:'+self.alterMsg+'\n\n'+\'Interface service address:'+self.url+'\n\n'+\self.tmp+\'\n\n'+\'\n\n'+\'**Interface message abnormality! ! ! **\n\n'+\'\n\n'+\'\n\n'+\'######**'+date+'**Release [Message abnormality](#)\n\n'+\"

[0144] S209a, triggering loop monitoring, returning to execute S203a, and recording the third loop monitoring times until all URL addresses are connected, otherwise execute S210a.

[0145] In the embodiment of the present application, by executing step S206a and step S209a, the HTTP connectivity detection of the target website can be implemented, so as to timely discover the abnormality of the web page and interface of the target website.

[0146] S210a, determine whether the third cycle number reaches the third maximum monitoring number; if the third cycle number is less than the third maximum monitoring number, continue to execute S209a, or if the third cycle number reaches the third maximum monitoring number, execute S211a.

[0147] S211a. Return the third warning message.

[0148] In a specific implementation, the third warning information can be used to warn of URL port timeout.

[0149] In the embodiment of the present application, in step S200, Python built-in functions (i.e., requests library) can be used to implement connectivity monitoring of the target website's web pages and interfaces, and abnormal situations of the target website's web pages and interfaces can be discovered in a timely manner.

[0150] In one embodiment, the SSL certificate monitoring target may include several domain names used by the target SSL certificate of the target website. The Yaml configuration file corresponding to the SSL certificate monitoring target may record these several domain names in a list, and the recorded list is the domain name list. The specific values ​​of these several domain names may be set according to the number of domain names specifically used by the target SSL certificate, and this embodiment of the application does not limit this.

[0151] In specific implementation, each domain name can set an SSL certificate validity threshold, which can be obtained through the issuance time and expiration time of the target SSL certificate.

[0152] In one implementation, in step S200, by parsing the Yaml configuration file corresponding to the SSL certificate monitoring target, the monitoring mode and monitoring frequency and the domain name list of the SSL certificate monitoring target can be obtained.

[0153] As an example, the Yaml configuration file corresponding to the SSL certificate monitoring target can be expressed as follows:

[0154]

[0155]

[0156] Monitoring target: SSL certificate

[0157] Monitoring method: traversal inspection, traversal domain name array: ['abc.xxx.cn', 'def.xxx.cn']

[0158] Monitoring frequency: once a day, YAML configuration: timer:09:30(am)

[0159] That is, in this example, the detection time point that can be used to detect the validity period of the SSL certificate every day can be set to be detected once a day.

[0160] In the specific implementation, by parsing the Yaml configuration file corresponding to the SSL certificate class monitoring target, the domain name list in the Yaml configuration file corresponding to the SSL certificate class monitoring target can be loaded into the monitoring list, so as to facilitate the subsequent detection of the SSL certificate status through the monitorBySsl entity class.

[0161] In one embodiment, in step S200, the following patrol monitoring operation may be performed on the SSL certificate monitoring target each time according to the monitoring frequency of the SSL certificate monitoring target. For example, if the monitoring frequency of the SSL certificate monitoring target is set to perform a patrol monitoring operation every 10 minutes, then the following patrol monitoring operation may be performed every 10 minutes:

[0162] S201b. Call Python's ssl library to create the monitorBySsl entity class.

[0163] S202b, call the monitorBySsl entity class to read the domain names, and use the wrap_socket function to verify whether the connectivity of each domain name is normal; if the connectivity of each domain name is normal, execute S203b, or, if the connectivity of each domain name is abnormal, execute S207b.

[0164] S203b, parse each domain name and obtain all information of the target SSL certificate.

[0165] In specific implementation, all the information may include but is not limited to: the issuing object, issuing time and expiration time, wherein the issuing object includes several domain names.

[0166] In specific implementation, the monitorBySsl entity class may be called to use the getpeercert function to obtain all information in the SSL certificate.

[0167] That is, in the embodiment of the present application, the wrap_socket function in the ssl library is used to first connect to each configured domain name to verify whether the connectivity is normal; then the getpeercert function in the ssl library is used to obtain all the information in the SSL certificate to obtain the issuing object, issuing date, expiration date, etc.

[0168] S204b. Calculate the remaining validity period of the SSL certificate corresponding to each domain name based on all the information, and determine whether the remaining validity period of the SSL certificate reaches the alarm time threshold; if the remaining validity period of the SSL certificate reaches the alarm time threshold, execute S205b; or, if the remaining validity period of the SSL certificate does not reach the alarm time threshold, execute S206b.

[0169] In specific implementation, the alarm time threshold can be set according to actual needs, and the embodiments of the present application do not limit this.

[0170] In specific implementation, the remaining validity period of the SSL certificate corresponding to each domain name can be calculated based on the issuance time and expiration time corresponding to each domain name (i.e., the issuing object) in all the information, and the time format of the remaining validity period of the SSL certificate can be standardized and compared with the set alarm time threshold.

[0171] In specific implementation, in step S204, when a domain name in the temporary change policy whitelist among the several domain names causes the monitoring machine to be inaccessible, the try except exception can be captured to continue to detect the remaining domain names.

[0172] That is, in an embodiment of the present application, SSL certificate monitoring can parse the domain name through the SSL library, obtain the issuing object, issuing date and expiration date of the target certificate, and calculate the remaining validity period of the corresponding SSL certificate.

[0173] S205b. Return the fourth warning information.

[0174] In specific implementation, the fourth warning information can be used to warn that the remaining time of the SSL certificate of each domain name is insufficient.

[0175] In the embodiment of the present application, by executing step S205b, message push can be performed only on the domain name that needs to renew the target SSL certificate.

[0176] S206b, determining that each domain name does not need to renew the target SSL certificate, and completing SSL certificate monitoring for each domain name.

[0177] S207b. Return the fifth warning information.

[0178] In a specific implementation, the fifth warning information may be used to prompt that each domain name cannot be connected.

[0179] In an embodiment of the present application, in step S200, the Python built-in function (i.e., the ssl library) can be used to detect the validity period of the SSL certificate, which can detect the validity period of the SSL certificate and promptly discover the expiration of the SSL certificate of each domain name.

[0180] In an embodiment of the present application, by executing step S200, the user's failure waiting period can be reduced based on the purpose of reducing the impact of information system failures on users. From the user's perspective, the target website can be automatically inspected and monitored by using a Python program to perform page status detection, interface response monitoring, TCP port listening, SSL certificate validity period detection, etc., so as to timely discover abnormal operations and responses of the target website, thereby ensuring the stable operation of the target website and timely response to abnormal situations.

[0181] S300, process monitoring results: store the inspection monitoring data of S200, and call the corresponding alarm robot to push the relevant alarm message returned by S200 to the corresponding person in charge.

[0182] As an example, the inspection and monitoring data of step S200 may be stored locally or in MQ.

[0183] For example, an example of storing the inspection and monitoring data of step S200 locally may be as follows:

[0184]

[0185]

[0186] For example, an example of storing the inspection and monitoring data of step S200 in MQ may be as follows:

[0187]

[0188]

[0189] In one implementation, in step S300, when step S200 returns a first alarm message during the inspection and monitoring of a port-type monitoring target, a corresponding alarm robot may be called to push the first alarm message to a corresponding person in charge.

[0190] For example, if the Yaml configuration file of the port monitoring target configures the alarm information notification method as: @ the corresponding person in charge through the alarm notification group on the Dingding platform, then the corresponding alarm robot can be called to send the first alarm message to the alarm notification group and @ the corresponding person in charge.

[0191] In one embodiment, in step S300, when step S200 returns the second alarm message and / or the third alarm message during the inspection and monitoring of the HTTP type monitoring target, the corresponding alarm robot can be called to push the second alarm message and / or the third alarm message to the corresponding person in charge.

[0192] For example, if the Yaml configuration file of the HTTP monitoring target configures the alarm information notification method as: @ the corresponding person in charge through the alarm notification group on the Dingding platform, then the corresponding alarm robot can be called to send the second alarm message and / or the third alarm message to the alarm notification group and @ the corresponding person in charge.

[0193] In one embodiment, in step S300, when step S200 returns the fourth alarm information and / or the fifth alarm information during the inspection and monitoring of the SSL certificate monitoring target, the corresponding alarm robot can be called to push the fourth alarm information and / or the fifth alarm information to the corresponding person in charge.

[0194] For example, if the alarm information notification method is configured through the Yaml configuration file of the SSL certificate monitoring target as: @ the corresponding person in charge through the alarm notification group on the Dingding platform, then the corresponding alarm robot can be called to send the fourth alarm information and / or the fifth alarm information to the alarm notification group and @ the corresponding person in charge.

[0195] In specific implementation, the above-mentioned alarm robot can be supported by the webhook of the DingTalk / WeChat open platform, which can be encapsulated by a module. For example, based on the platform and the beauty and readability of the text format, markdown can be selected as the alarm information output type of the above-mentioned alarm robot.

[0196] During the specific implementation, one alarm robot can be set to correspond to each system according to specific needs, and each alarm robot can be set to correspond to the person in charge who needs to be @ed. Among them, each alarm robot can use a pop-up window to notify the person in charge, so that the alarm information can be pushed to the persons in charge who use different systems.

[0197] During specific implementation, a unique alarm robot can be set up and used according to the specific message notification platform used, so as to directly respond to relevant alarm messages (such as the corresponding alarm messages mentioned above).

[0198] As an example, each of the above alarm robots can also be configured through a Yaml configuration file. In this process, each alarm robot can be set to match the corresponding response keyword as required, and IP segment access restrictions can be configured if necessary. Among them, the token allocated to each alarm robot can also be configured in the Yaml configuration file.

[0199] For example, the Yaml configuration file corresponding to a certain alarm robot can be as follows:

[0200]

[0201]

[0202] In step S300, the alarm robot has high readability and can easily and quickly identify abnormal systems and their interfaces. On this basis, the corresponding person in charge of the designated system will be @notified through the alarm robot. Since the abnormal message notification is in the enterprise office software, the efficiency of transmitting it to the corresponding person in charge is in seconds, which is an instant response, so that the abnormal message (such as the corresponding alarm message mentioned above) can be pushed to the corresponding person in charge in time.

[0203] In an embodiment of the present application, by executing step S300, the abnormal message can be fed back to the relevant user (such as the corresponding person in charge mentioned above) for maintenance processing as soon as possible, thereby reducing the user's perception time of system failure, and can increase efficiency and improve user satisfaction for the enterprise.

[0204] S400, AI analysis monitoring results: call the AI ​​big model to analyze and process the relevant alarm messages returned by S200, obtain the analysis results, and store the analysis results locally or in MQ.

[0205] In specific implementation, the analysis result may include corresponding types of alarm events, wherein the corresponding types of alarm events may include critical events and abnormal events.

[0206] As an example, the key event may include an SSL certificate validity period reminder event, and the abnormal event may include a URL response message event, a URL port timeout event, a domain name unreachable abnormal event, and a TCP port abnormal event.

[0207] The TensorFlow model is flexible and scalable, supports deployment from mobile devices to large-scale distributed systems, and allows developers to deploy machine learning models on multiple platforms. The TensorFlow model has its advanced optimizers, including Adam, Adagrad, etc. These optimizers help accelerate the model training process and improve the convergence speed. Based on this, in the embodiment of the present application, the AI ​​model can be set as a TensorFlow model.

[0208] In specific implementation, the AI ​​big model can be called by using Python script to analyze and process the inspection and monitoring data of S200.

[0209] For example, an example of using Python script to call the AI ​​big model to analyze and process the inspection and monitoring data of S200 can be as follows:

[0210]

[0211]

[0212] In an embodiment of the present application, by executing step S400, the AI ​​model can be used to analyze and process the relevant alarm messages returned by step S200 to identify key events or abnormal events.

[0213] S500, PRA automated exception processing: the analysis result is read by the PRA robot, and the external API processing service is called to process the corresponding category of alarm events in the analysis result, and then the process returns to execute S200.

[0214] In specific implementation, MQ can be used as a communication medium between the AI ​​model and the RPA robot. The analysis results generated by the AI ​​model can be published to MQ, and the RPA robot can subscribe to the MQ to receive and process messages. For example, MQ can be used to save the corresponding category of alarm events in the analysis results to a specific folder or send them to the input folder monitored by the RPA robot. The RPA robot can periodically check the specific folder or the input folder and read the corresponding category of alarm events stored therein for subsequent operations.

[0215] In one implementation, the PRA robot may be an Invoke Python Activity or an Invoke Code Activity of Python.

[0216] In this implementation, you can use Invoke Python Activity or Invoke Code Activity to call a Python script, pass the AI ​​analysis results as input parameters, and then use HTTP Request or other appropriate activities to call the external API processing service. When necessary, you can pass the necessary data to control service restart, expansion, etc.

[0217] As an example, the external API processing service may be a CCSE api-server processing service. In this example, calling the external API processing service to process the corresponding alarm event may be implemented in the following manner:

[0218] Pay attention to the request header configuration of CCSE API

[0219] Pay attention to CCSE API authentication and authorization

[0220] Introducing HTTP requests from RPA tools or specialized Kubernetes API integration packages

[0221] UiPath Automation Platform simple sample code:

[0222]

[0223] In specific implementation, after the PRA robot calls the external API processing service to process the corresponding category of alarm events in the analysis result, it can return to step S200 to determine whether the TCP port, interface, web page and SSL certificate are restored.

[0224] In an embodiment of the present application, by executing step S500, the UiPath automation platform (which can provide a PRA robot) combines with MQ to call an external Kubernetes API according to classification information to handle exceptions, such as handling alarm events of corresponding categories.

[0225] That is, in the embodiment of the present application, by executing step S400 and step S500, efficient data processing and external API calls can be achieved by integrating artificial intelligence (AI) big models and robotic process automation (RPA) technology, thereby reducing manual input.

[0226] And in the embodiment of the present application, by executing steps S200-S500, it is possible to use the Python program to automatically patrol and monitor the target website from the user's perspective, and perform page status detection, interface response monitoring, TCP port monitoring, SSL certificate validity period detection, etc., and automatically handle some abnormal problems by combining AI big models and RPA automation. The main monitoring alarm program can be very lightweight, occupies little resources, and has high fault identification efficiency. It can cover most of the health manifestations of the system, which is more efficient and rapid than the existing traversal golden process.

[0227] In summary, the website inspection and monitoring method provided in the embodiment of the present application can be used from the user's perspective to perform page status detection, interface response monitoring, TCP port monitoring, SSL certificate validity period detection, etc. on the target website by using Python program automatic inspection and monitoring. It can promptly discover system failures of the target website and can be fed back to the corresponding person in charge for processing at the first time, such as through the open interface of DingTalk / Enterprise Micro Robot, etc., to push the alarm message to the alarm notification group, @The corresponding person in charge responds to the exception in time, thereby improving the system stability and response efficiency of the target website. At the same time, the introduction of AI big models and RPA automation in the next stage will reduce manual operations to a certain extent and liberate most of the manual workload. In this way, the service can be processed by automation at the first time. If it cannot be solved, manual intervention will be carried out. In short, the website inspection and monitoring method provided in the embodiment of the present application can ensure the stable operation of the target website and respond to abnormal situations in a timely manner.

[0228] Figure 5 FIG. 2 shows a structural block diagram of a website inspection and monitoring system according to an embodiment of the present application. Figure 5 As shown, the system may include:

[0229] The customization unit 11 is used to customize the monitoring task: customize multiple monitoring targets of the target website, and then customize and configure the Yaml configuration file corresponding to each monitoring target, wherein the Yaml configuration file corresponding to each monitoring target records the monitoring mode and monitoring frequency of each monitoring target, and the multiple monitoring targets include port monitoring targets, HTTP monitoring targets and SSL certificate monitoring targets, and the monitoring mode of each monitoring target is traversal execution;

[0230] The execution unit 12 is used to execute the monitoring task: parse the YAML configuration file corresponding to each monitoring target, perform patrol monitoring on each monitoring target based on the monitoring mode and monitoring frequency of each monitoring target, and return relevant alarm messages;

[0231] The first processing unit 13 is used to process the monitoring results: store the inspection monitoring data of the execution unit 12, and call the corresponding alarm robot to push the relevant alarm message returned by the execution unit 12 to the corresponding person in charge;

[0232] The analysis unit 14 is used for AI analysis of monitoring results: calling the AI ​​big model to analyze and process the relevant alarm messages returned by the execution unit 12, obtaining the analysis results, and storing the analysis results locally or in MQ, where MQ refers to a message queue. The analysis results include alarm events of corresponding categories;

[0233] The second processing unit 15 is used for PRA automated exception processing: the analysis results are read by the PRA robot, and the external API processing service is called to process the corresponding category of alarm events, and the monitoring task is returned to the execution unit 12.

[0234] The functions of each unit in the website inspection and monitoring system of the embodiment of the present application can be found in the corresponding description in the above method, and will not be repeated here.

[0235] In practical applications, in applicable scenarios, the above website inspection and monitoring system can be deployed in the following ways:

[0236] Environmental Support

[0237] 1. Windows:

[0238] 1. It can be packaged into an executable exe file through compilation tools, and all dependencies can be integrated into the program for easy deployment. There is no need to configure running dependencies separately in a new environment.

[0239] 2. You can install the Python environment by directly running the .py program. This method requires the installation of dependencies.

[0240] 2. Linux:

[0241] 1. It can be packaged into a Linux executable file through a compilation tool, and all dependencies can be integrated into the program. It is convenient for deployment, and there is no need to configure running dependencies separately in a new environment.

[0242] 2. You can install the Python environment by directly running the .py program. This method requires the installation of dependencies.

[0243] Summary: If the environment allows, it is not recommended to package it into an executable file. Because if there is a special need to modify the code logic later, you can directly modify the .py file and rerun it. If it is an executable file, it cannot be modified and needs to be repackaged. Combining the current mainstream direction of cloud native, we can provide a deployment solution for the website inspection and monitoring system.

[0244] Tianyi Cloud CCSE Platform Deployment

[0245] (I) Creating a Docker Image

[0246] Customized Dockerfile file, based on the basic CentOS7 image, installs the required Chinese environment to provide support for normal log display. The required program for installation depends on the Python environment to provide support for the normal operation of the program.

[0247] In making a Docker image, you can create a new Deployment, set the data volume, specify the configMap configuration item, select a distributed high-availability ceph disk, set the image version, and mount the configMap configuration item into the container. You can specify the configuration directory, log directory, and script directory. Set the number of CPU cores and memory size. Because the program is simple and efficient, it can meet the monitoring needs without too many resources.

[0248] (2) Run the Docker image

[0249] Deployment is configured and run through the PAAS platform. The image comes from the Harbor warehouse, which can provide automated operations and achieve elastic scaling based on demand in multiple systems.

[0250] Among them, when running the Docker image, you can choose to monitor the system running node, configure the harbor warehouse pull credentials, and start the corresponding number of sub-containers as needed.

[0251] 3. Support across multiple cloud platforms

[0252] Kubernetes can run on a variety of cloud platforms and on-premises environments, providing consistency across multiple cloud platforms and enabling applications to be easily migrated and deployed in different environments.

[0253] (IV) Log storage

[0254] When the above website inspection and monitoring system is running, the log is written to the file through redirection. The log can be saved locally, but localization is no longer the mainstream nowadays.

[0255] The above website inspection and monitoring system can run in the PAAS environment. The PAAS platform provides the functions of automatically deploying, expanding and managing containerized applications, reducing the need for manual intervention and operation. Ceph distributed storage disks are mounted in Pod containers to provide persistent log storage. The monitoring situation at a certain point in time can be analyzed at any time later.

[0256] That is, in the above applicable scenarios, customized Dockerfile can be used to build image instructions and customized message alarm information terminal templates. In this way, the above website inspection and monitoring system can be customized to build the required environment dependencies into the image, providing a container that can be used across platforms, and through customized alarm templates, it can intuitively display the abnormal system and target address, greatly improving the efficiency of locating abnormalities.

[0257] And in the above applicable scenarios, by adopting the method of combining Tianyi Cloud CCSE with Cpeh distributed storage for deployment, the above website inspection and monitoring system configures the Yaml file through the configMap of the CCSE platform, passes in key monitoring targets, monitoring methods, and monitoring frequencies, and uses Ceph distributed storage to mount it to the container, and persistently saves the log flow records generated by the monitoring system. The accuracy and timeliness have reached 99.5% since it went online, so that anomalies can be detected within 5 seconds of system anomalies.

[0258] Figure 6 FIG. 2 shows a block diagram of a computer device according to an embodiment of the present application. Figure 6 As shown, the computer device includes: a memory 10 and a processor 20. The memory 10 stores instructions, and the instructions are loaded and executed by the processor 20 to implement the website inspection and monitoring method in the above embodiment. The number of the memory 10 and the processor 20 can be one or more.

[0259] The computer device also includes:

[0260] The communication interface 30 is used to communicate with external devices and perform data exchange transmission.

[0261] If the memory 10, the processor 20 and the communication interface 30 are implemented independently, the memory 10, the processor 20 and the communication interface 30 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0262] Optionally, in a specific implementation, if the memory 10, the processor 20 and the communication interface 30 are integrated on a chip, the memory 10, the processor 20 and the communication interface 30 can communicate with each other through an internal interface.

[0263] An embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer, the method provided in the embodiment of the present application is implemented.

[0264] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in a memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present application.

[0265] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the application.

[0266] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0267] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory, and may also include a non-volatile random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct rambus RAM (DR RAM).

[0268] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0269] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0270] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0271] Any process or method description in the flow chart or otherwise described herein can be understood to represent a module, fragment or portion of a code including one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0272] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with these instruction execution systems, apparatuses or devices.

[0273] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0274] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0275] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A website inspection and monitoring method, characterized in that: include: S100, customizing monitoring tasks: customizing multiple monitoring targets of the target website, and then customizing and configuring the Yaml configuration file corresponding to each of the monitoring targets, wherein the Yaml configuration file corresponding to each of the monitoring targets records the monitoring mode and monitoring frequency of each of the monitoring targets, and the multiple monitoring targets include port monitoring targets, HTTP monitoring targets, and SSL certificate monitoring targets, and the monitoring mode of each of the monitoring targets is traversal execution; S200, executing the monitoring task: parsing the YAML configuration file corresponding to each monitoring target, performing patrol monitoring on each monitoring target based on the monitoring mode and monitoring frequency of each monitoring target, and returning relevant alarm messages; S300, processing monitoring results: storing the inspection monitoring data of S200, and calling the corresponding alarm robot to push the relevant alarm message returned by S200 to the corresponding person in charge; S400, AI analysis monitoring results: calling the AI ​​big model to analyze and process the relevant alarm messages returned by S200, obtaining analysis results, storing the analysis results locally or in MQ, where MQ refers to a message queue, and the analysis results include alarm events of corresponding categories; S500, PRA automated exception processing: the analysis result is read by the PRA robot, and an external API processing service is called to process the alarm event of the corresponding category, and the process returns to execute S200.

2. The method according to claim 1, characterized in that The port-type monitoring target includes several TCP ports. In S200, the following inspection and monitoring operations are performed on the port-type monitoring target each time according to the monitoring frequency of the port-type monitoring target: S201, calling Python's socket library to create a TCP entity class; S202, compile a first request mode, a first timeout time and a first maximum monitoring number corresponding to the TCP entity class; S203, calling the TCP entity class to traverse and access each of the TCP ports using the first request method within the first timeout period, and determining whether each of the TCP ports is connected; if all of the TCP ports are connected, executing S204, or if at least one of the TCP ports is not connected, executing S205; S204, determining that the TCP port response of the target website is normal, and completing TCP port monitoring; S205, triggering cyclic monitoring, returning to S203, and recording the first cyclic monitoring times until all the TCP ports are connected, otherwise executing S206; S206, determining whether the first cycle number reaches the first maximum monitoring number; If the first cycle number is less than the first maximum monitoring number, continue to execute S205, or if the first cycle number reaches the first maximum monitoring number, execute S207; S207. Return a first alarm message, where the first alarm message is a message in msg format and is used to record an abnormal message of at least one of the TCP ports.

3. The method according to claim 1, characterized in that The HTTP monitoring target includes a plurality of URL addresses, and the plurality of URL addresses include URL addresses corresponding to web pages and URL addresses corresponding to interfaces. In S200, the following inspection and monitoring operations are performed on the HTTP monitoring target each time according to the monitoring frequency of the HTTP monitoring target: S201a, call Python's requests library to create the monitorByHttp entity class; S202a, compile the second request method, the second timeout period, the second maximum monitoring number and the third maximum monitoring number corresponding to the monitorByHttp entity class; wherein the second request method self-defined selects the GET request method or the POST request method, and the parameter data structure of the second request method is converted using the json.dumps method; S203a, calling the monitorByHttp entity class to traverse and access each of the URL addresses using the second request method within the second timeout period, and determining whether each of the URL addresses is connected; if all of the URL addresses are connected, executing S204a, or if at least one of the URL addresses is not connected, executing S209a; S204a, matching the response content of each URL address with a preset verification keyword; if the response content of all URL addresses is consistent with the verification keyword, executing S205a, or if the response content of at least one URL address is inconsistent with the verification keyword, executing S206a; S205a, determining that the URL address of the target website responds normally, and completing HTTP monitoring; S206a, triggering loop monitoring, returning to execute S203a, and recording the second loop monitoring times, until the response contents of all the URL addresses are consistent with the verification keyword, otherwise execute S207a; S207a, determining whether the second cycle number reaches the second maximum monitoring number; if the second cycle number is less than the second maximum monitoring number, continue to execute S206a, or if the second cycle number reaches the second maximum monitoring number, execute S208a; S208a, returning a second warning message, wherein the second warning message is a message in msg format, and is used to warn that the URL response message is abnormal; S209a, triggering loop monitoring, returning to execute S203a, and recording the third loop monitoring times until all the URL addresses are connected, otherwise execute S210a; S210a, determining whether the third cycle number reaches the third maximum monitoring number; if the third cycle number is less than the third maximum monitoring number, continue to execute S209a, or if the third cycle number reaches the third maximum monitoring number, execute S211a; S211a. Return a third alarm message, where the third alarm message is used to warn of URL port timeout.

4. The method according to claim 3, characterized in that In S203a, if the URL corresponds to a common interface, the second request method is customized to select a GET request method; If the URL address corresponds to a special interface, the second request method customizes the selection of the GET request method or the POST request method, and configures the interface account password, Headers, and Body fields in the parameter data structure of the second request method according to the corresponding request, and then uses the json.dumps method for conversion. The special interface refers to the interface that requires verification of the correctness of the interface data response; If the URL address corresponds to a web page, the second request method customizes the GET request method.

5. The method according to claim 1, characterized in that The SSL certificate monitoring target includes several domain names used by the target SSL certificate of the target website; in S200, the following inspection and monitoring operations are performed on the HTTP monitoring target each time according to the monitoring frequency of the HTTP monitoring target: S201b, call Python's ssl library to create the monitorBySsl entity class; S202b, calling the monitorBySsl entity class to read a number of the domain names, and using the wrap_socket function to verify whether the connectivity of each of the domain names is normal; if the connectivity of each of the domain names is normal, executing S203b, or, if the connectivity of each of the domain names is abnormal, executing S207b; S203b, parsing each of the domain names, obtaining all information of the target SSL certificate, wherein all the information includes the issuing object, issuing time and expiration time, and the issuing object includes several of the domain names; S204b, calculating the remaining validity period of the SSL certificate corresponding to each of the domain names based on all the information, and determining whether the remaining validity period of the SSL certificate reaches the alarm time threshold; if the remaining validity period of the SSL certificate reaches the alarm time threshold, executing S205b, or if the remaining validity period of the SSL certificate does not reach the alarm time threshold, executing S206b; S205b, returning a fourth warning message, wherein the fourth warning message is used to warn that the remaining time of the SSL certificate of each domain name is insufficient; S206b, determining that each of the domain names does not need to renew the target SSL certificate, and completing SSL certificate monitoring for each of the domain names; S207b. Return fifth warning information, where the fifth warning information is used to warn that each domain name cannot be connected.

6. The method according to claim 1, characterized in that The AI ​​model is a TensorFlow model.

7. The method according to any one of claims 1 to 6, characterized in that The PRA robot is Python's Invoke Python Activity or Invoke Code Activity.

8. A website inspection and monitoring system, characterized in that: include: A customization unit is used to customize monitoring tasks: customize multiple monitoring targets of a target website, and then customize and configure a Yaml configuration file corresponding to each of the monitoring targets, wherein the Yaml configuration file corresponding to each of the monitoring targets records the monitoring mode and monitoring frequency of each of the monitoring targets, and the multiple monitoring targets include port monitoring targets, HTTP monitoring targets, and SSL certificate monitoring targets, and the monitoring mode of each of the monitoring targets is traversal execution; An execution unit is used to execute the monitoring task: parse the YAML configuration file corresponding to each monitoring target, perform patrol monitoring on each monitoring target based on the monitoring mode and monitoring frequency of each monitoring target, and return relevant alarm messages; The first processing unit is used to process the monitoring results: store the inspection monitoring data of the execution unit, and call the corresponding alarm robot to push the relevant alarm message returned by the execution unit to the corresponding person in charge; The analysis unit is used for AI analysis of monitoring results: calling the AI ​​big model to analyze and process the relevant alarm messages returned by the execution unit to obtain analysis results, and storing the analysis results locally or in MQ, where MQ refers to a message queue. The analysis results include alarm events of corresponding categories; The second processing unit is used for PRA automated exception processing: the analysis result is read by the PRA robot, and the external API processing service is called to process the alarm event of the corresponding category, and the monitoring task is returned to the execution unit.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory stores instructions, and the instructions are loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the method according to any one of claims 1 to 7 is implemented.

Images