Index data processing method and device

Abstract

The invention discloses an index data processing method and device, and relates to the technical field of big data. A specific embodiment of the method comprises the steps of obtaining basic information of an index capable of being displayed on a page, wherein the basic information comprises a domain name address where the index is located, an index verification address and an index weight; performing domain name connectivity verification according to the domain name address where the index is located, and performing data displayability verification according to the index verification address; deleting the first index which does not pass the domain name connectivity verification and the second index which does not pass the data displayability verification from the indexes which can be displayed in the page to obtain an index to be displayed; performing page drawing according to the to-be-displayed indexes and the index weight of each to-be-displayed index to obtain a display page; and obtaining index data according to the domain name address of the to-be-displayed index, and filling the obtained index data into a display page for display. According to the embodiment, the display effect of the index data is improved, and the availability of the system is improved.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a method and device for processing indicator data. Background Art

[0002] With the popularization of big data technology, large-screen visualization is becoming more and more common. In order to display various indicators on the front-end interface, the existing technology basically makes one or several systems in a targeted manner. If the amount of data is particularly large, it is displayed in a multi-screen or multi-page mode. Of course, there is also a user-configurable interface, and users can choose the configured database source according to their needs. In the existing technical solutions, when there are many data indicators to be displayed, if the page has abnormalities such as indicator offline or data errors, the entire system needs to be offline, resulting in the unavailability of the entire system. Summary of the invention

[0003] In view of this, an embodiment of the present invention provides a method and device for processing indicator data, which can intelligently screen indicator data by performing domain name connectivity verification and data displayability verification to delete abnormal indicators. When an indicator abnormality occurs, there is no need to take the entire system offline, which improves the display effect of the indicator data and improves the availability of the system.

[0004] To achieve the above object, according to one aspect of an embodiment of the present invention, a method for processing indicator data is provided, comprising:

[0005] Obtain basic information of indicators that can be displayed on the page, including the domain name address of the indicator, the indicator verification address and the indicator weight;

[0006] Perform domain name connectivity verification based on the domain name address where the indicator is located, and perform data displayability verification based on the indicator verification address;

[0007] Deleting the first indicator that fails the domain name connectivity verification and the second indicator that fails the data displayability verification from the indicators that can be displayed on the page, to obtain indicators to be displayed;

[0008] Drawing a page according to the indicators to be displayed and the indicator weight of each indicator to be displayed to obtain a display page;

[0009] The indicator data is obtained according to the domain name address where the indicator to be displayed is located, and the obtained indicator data is filled into the display page for display.

[0010] Optionally, after obtaining the basic information of the indicators that can be displayed on the page, it also includes: storing the basic information of the indicators that can be displayed on the page in an array; before deleting the first indicator that fails the domain name connectivity verification and the second indicator that fails the data displayability verification from the indicators that can be displayed on the page, it also includes: deleting the basic information of the first indicator that fails the domain name connectivity verification and the second indicator that fails the data displayability verification from the array.

[0011] Optionally, a page is drawn according to the indicators to be displayed and the indicator weight of each indicator to be displayed to obtain a display page, including: in response to monitoring a change in the array, a page is drawn according to the indicators to be displayed and the indicator weight of each indicator to be displayed in the changed array to obtain a display page.

[0012] Optionally, domain name connectivity verification is performed according to the domain name address where the indicator is located, including: requesting access to the corresponding domain name according to the domain name address where the indicator is located, and in response to the domain name access is unavailable or the access times out, the domain name connectivity verification fails; data displayability verification is performed according to the indicator verification address, including: judging whether the data of a sub-page corresponding to a certain indicator is all 0, empty, or the data query times out according to the indicator verification address, and in response to the data of the sub-page corresponding to the indicator is all 0, empty, or the data query times out, the data displayability verification fails.

[0013] Optionally, there are multiple indicators to be displayed; corresponding indicator data is obtained according to the domain name address where the indicator to be displayed is located, including: for each indicator to be displayed, request authentication data of the current domain name is obtained according to the domain name address where the indicator to be displayed is located, and authority verification is performed according to the request authentication data, and in response to the authority verification being passed, corresponding indicator data is obtained from the domain name address where the indicator to be displayed is located.

[0014] Optionally, obtaining corresponding indicator data according to the domain name address where the indicator to be displayed is located also includes: in response to failing to obtain request authentication data of the current domain name according to the domain name address where the indicator to be displayed is located, determining whether the current domain name is within the domain name range that can be shared with the request authentication data; in response to the current domain name being within the domain name range that can be shared with the request authentication data, reading the request authentication data of the primary domain name to which the current domain name belongs, and writing the request authentication data of the primary domain name into the domain name address where the indicator to be displayed is located, so as to obtain the request authentication data of the current domain name according to the domain name address where the indicator to be displayed is located.

[0015] Optionally, the indicator data is generated in the following manner: regularly collecting internal system data and external system data and generating data processing tasks; classifying and aggregating the collected data based on the data processing tasks to obtain first indicator data; monitoring the real-time status and process processing status of the server through process probes, and analyzing and aggregating the monitoring data to obtain second indicator data; generating the indicator data based on the first indicator data and the second indicator data.

[0016] Optionally, before performing domain name connectivity verification according to the domain name address where the indicator is located and performing data displayability verification according to the indicator verification address, it also includes: drawing a page according to the indicators that can be displayed on the page and the indicator weight of each indicator to obtain a first display page, the first display page including the indicator name of each indicator and a data placeholder corresponding to each indicator, and the data placeholder is embedded with the domain name address where the indicator is located.

[0017] According to another aspect of an embodiment of the present invention, there is provided a device for processing indicator data, including:

[0018] The indicator information acquisition module is used to obtain the basic information of the indicators that can be displayed on the page, and the basic information includes the domain name address of the indicator, the indicator verification address and the indicator weight;

[0019] A data verification module, used to verify domain name connectivity according to the domain name address where the indicator is located, and to verify data displayability according to the indicator verification address;

[0020] An indicator screening module, used to delete the first indicator that fails the domain name connectivity verification and the second indicator that fails the data displayability verification from the indicators that can be displayed on the page, to obtain the indicators to be displayed;

[0021] A page drawing module, used for drawing a page according to the indicators to be displayed and the indicator weight of each indicator to be displayed to obtain a display page;

[0022] The indicator filling and display module is used to obtain indicator data according to the domain name address where the indicator to be displayed is located, and fill the obtained indicator data into the display page for display.

[0023] According to another aspect of an embodiment of the present invention, there is provided an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for processing indicator data provided by an embodiment of the present invention.

[0024] According to another aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for processing indicator data provided by an embodiment of the present invention is implemented.

[0025] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method for processing indicator data provided by an embodiment of the present invention is implemented.

[0026] An embodiment of the above invention has the following advantages or beneficial effects: by obtaining basic information of indicators that can be displayed on the page, the basic information includes the domain name address of the indicator, the indicator verification address and the indicator weight; performing domain name connectivity verification according to the domain name address of the indicator, and performing data displayability verification according to the indicator verification address; deleting the first indicator that fails the domain name connectivity verification and the second indicator that fails the data displayability verification from the indicators that can be displayed on the page to obtain the indicators to be displayed; drawing the page according to the indicators to be displayed and the indicator weight of each indicator to be displayed to obtain a display page; obtaining indicator data according to the domain name address of the indicator to be displayed, and filling the obtained indicator data into the display page for display. The technical solution realizes intelligent screening of indicator data by performing domain name connectivity verification and data displayability verification to delete abnormal indicators. When an indicator abnormality occurs, there is no need to take the entire system offline, thereby improving the display effect of indicator data and improving the availability of the system.

[0027] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific implementation examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0029] Figure 1 is a schematic diagram of main steps of a method for processing indicator data according to an embodiment of the present invention;

[0030] Figure 2 This is a schematic diagram of the implementation principle of cross-domain sharing of authentication request data according to an embodiment of the present invention;

[0031] Figure 3 It is a schematic diagram of the process of generating indicator data according to an embodiment of the present invention;

[0032] Figure 4 It is a schematic diagram of the processing flow of indicator data according to an embodiment of the present invention;

[0033] Figure 5 is a schematic diagram of main modules of an indicator data processing device according to an embodiment of the present invention;

[0034] Figure 6 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0035] Figure 7 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0037] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution disclosed in the present invention are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0038] Currently, when loading and displaying indicator data, if there are too many indicators, the page can easily take a long time to load. Sometimes, even if data aggregation preprocessing is done on the server, there will still be freezes. If it is a multi-page mode, it becomes difficult to see all the indicators in one interface. If it is a user-configurable interface, the page will eventually become very stuck or even crash as the number of tabs increases. Regardless of the solution, if the page has abnormal conditions such as indicator offline or data errors, the entire system needs to be offline.

[0039] In order to solve the above technical problems, the present invention provides a method for processing indicator data, which can realize the decoupling between page monitoring indicators and can load quickly. If a certain indicator data is abnormal or needs to be processed offline, the front-end page automatically blocks it without the need for overall offline processing, thereby improving the display effect of the indicator data and the availability of the system.

[0040] Figure 1 FIG. 1 is a schematic diagram of the main steps of the method for processing indicator data according to an embodiment of the present invention. Figure 1 As shown, the method for processing indicator data according to the embodiment of the present invention mainly includes the following steps S101 to S105.

[0041] Step S101: Obtain basic information of indicators that can be displayed on the page, the basic information includes the domain name address of the indicator, the indicator verification address and the indicator weight. In an embodiment of the present invention, a preset data range interface can be called to obtain basic information of indicators that can be displayed on the page, wherein the basic information of the indicator can include, for example, the indicator name, the domain name address of the indicator, the indicator verification address and the indicator weight, the indicator verification address stores a logic program for verifying the correctness of the indicator, and the indicator weight can be pre-configured to indicate the importance of the indicator. The preset data range interface is, for example, a data range interface under the main domain name of a system, and the indicators that can be displayed on the page obtained are usually multiple.

[0042] According to an embodiment of the present invention, after obtaining the basic information of the indicators that can be displayed on the page, it also includes: storing the basic information of the indicators that can be displayed on the page in an array. For each indicator obtained, the basic information of the indicator can be stored in an array, such as a viewList array or a vueList array, and the basic information of each indicator can be stored in a list.

[0043] Step S102: Verify domain connectivity based on the domain address where the indicator is located, and verify data displayability based on the indicator verification address. After obtaining the indicators that can be displayed on the page, the obtained indicators will be verified to determine whether there are abnormal indicators that cannot obtain data.

[0044] According to an embodiment of the present invention, domain name connectivity verification is performed according to the domain name address where the indicator is located, which may specifically include: requesting access to the corresponding domain name according to the domain name address where the indicator is located, and responding to the domain name access failure or access timeout, the domain name connectivity verification fails. Specifically, an access request can be generated and sent to the domain name where the indicator is located (determined according to the domain name address where the indicator is located). If the domain name is accessible and there is no access delay, it means that the domain name connectivity verification passes, otherwise it means that the domain name connectivity verification fails. The indicator that fails the domain name connectivity verification is used as the first indicator of data anomaly. Among them, when generating an access request and sending it to the domain name where the indicator is located, specifically, the access request is sent to the server corresponding to the domain name, and the indicator data is stored in the server. If the domain name is connectable, after the server receives the access request, it will generate response data and return it; otherwise, if the domain name is not connectable, the server will not receive the access request, and accordingly, it will not return response data.

[0045] Performing data displayability verification according to the indicator verification address may specifically include: judging whether the data of a sub-page corresponding to a certain indicator is all 0, empty, or the data query has timed out according to the indicator verification address, and in response to the data of the sub-page corresponding to the indicator being all 0, empty, or the data query having timed out, the data displayability verification fails. Specifically, a data acquisition request may be generated and sent to the domain name address where the indicator is located to request the data of the sub-page corresponding to the indicator. Then, judging whether the data of the sub-page corresponding to the indicator is all 0, empty, or the data query has timed out according to the indicator verification address. Here, when requesting to obtain the data of the sub-page corresponding to the indicator according to the data acquisition request, the specific implementation method may refer to the specific implementation method of obtaining the indicator data according to the domain name address where the indicator to be displayed is located in step S105.

[0046] According to an embodiment of the present invention, when there are many indicators to be displayed, in order to view all indicators in one page, a main page can be used to display brief indicator data, and the detailed information of each indicator in the main page can be displayed in each sub-page nested in the main page. When verifying the indicator, for the indicator data displayed in the main page, the domain name connectivity verification can be performed on the domain name address where the indicator is located to filter out the first indicator that fails the domain name connectivity verification. For the detailed information of the indicator displayed in each sub-page, a logic program for verifying the correctness of the indicator can be obtained from the indicator verification address, and then based on the obtained logic program, it is verified whether the detailed information data corresponding to the indicator to be displayed on the sub-page is all 0, empty, or the data query timed out. If the detailed information data corresponding to the indicator is all 0, empty, or the data query timed out, it means that the indicator has data anomalies and the data displayability verification fails. The indicator that fails the data displayability verification is used as the second indicator of data anomalies. For the indicator that passes the data displayability verification, the data of the sub-page corresponding to the indicator can be obtained, and the sub-page can be refreshed in real time.

[0047] Step S103: The first indicator that fails the domain name connectivity verification and the second indicator that fails the data presentability verification are deleted from the indicators that can be presented on the page to obtain indicators to be presented.

[0048] According to an embodiment of the present invention, after obtaining the basic information of the indicators that can be displayed on the page, the basic information of the indicators that can be displayed on the page is also stored in an array. Therefore, before deleting the first indicator that failed the domain name connectivity verification and the second indicator that failed the data displayability verification from the indicators that can be displayed on the page, it can also include: deleting the basic information of the first indicator that failed the domain name connectivity verification and the second indicator that failed the data displayability verification from the array.

[0049] Specifically, when deleting the basic information of the first indicator and the second indicator from the array, for example, the viewList array may be copied and backed up into a viewListBase backup array, and then the viewListBase backup array may be copied to generate an array viewListBase2; then the basic information of the first indicator is deleted from the generated array viewListBase2, and the viewList array is updated according to the array viewListBase2 after the basic information is deleted; then the basic information of the second indicator is deleted from the updated viewList array. The basic information of the indicator to be displayed is stored in the final viewList array.

[0050] Step S104: Drawing a page according to the indicators to be displayed and the indicator weight of each indicator to be displayed to obtain a display page. After obtaining the indicators to be displayed according to the aforementioned steps S101 to S103, the display page can be drawn according to the indicators to be displayed.

[0051] According to one embodiment of the present invention, drawing a page according to the indicators to be displayed and the indicator weight of each indicator to be displayed to obtain a display page may specifically include: in response to monitoring that the array has changed, drawing a page according to the indicators to be displayed and the indicator weight of each indicator to be displayed in the changed array to obtain a display page. Specifically, the viewList array may be monitored, and when a change in the viewList array is monitored, the page may be drawn according to the indicators to be displayed and the indicator weight of each indicator to be displayed. In another embodiment, it may also be possible to determine whether the viewList array has changed by comparing the viewList array with the viewListBase backup array.

[0052] According to one of the embodiments of the present invention, the front end of the display page indicators can be developed using the vue (a JavaScript framework for building user interfaces) framework, which has built-in data two-way binding (a synchronous update mechanism between data models and views). If a change in the array vueList is detected, the display page will be automatically redrawn.

[0053] Step S105: Obtain indicator data according to the domain name address where the indicator to be displayed is located, and fill the obtained indicator data into the display page for display.

[0054] From step S101 to step S105, it can be seen that as long as an indicator is abnormal, the display page will not display the indicator. At the same time, the data loading between various indicators is completely synchronous, including the corresponding backend services are also synchronously queried, which greatly improves the page loading speed. As long as the server responds quickly, the page can be loaded in seconds.

[0055] In an embodiment of the present invention, when obtaining indicator data according to the domain name address where the indicator to be displayed is located, specifically, a data acquisition request may be generated, and according to the domain name address where the indicator to be displayed is located, the data acquisition request may be sent to a server corresponding to the domain name address to obtain the indicator data. In other embodiments, when data displayability verification is performed in step S102, that is, sub-page data corresponding to each indicator that has passed the displayability verification is obtained, and then the indicator data of the indicator to be displayed is obtained from the sub-page data corresponding to these indicators.

[0056] According to an embodiment of the present invention, there are multiple indicators to be displayed; obtaining corresponding indicator data according to the domain name address where the indicator to be displayed is located may specifically include: for each indicator to be displayed, obtaining the request authentication data of the current domain name according to the domain name address where the indicator to be displayed is located, and performing authority verification according to the request authentication data, and in response to the authority verification being passed, obtaining the corresponding indicator data from the domain name address where the indicator to be displayed is located. In an embodiment of the present invention, the request authentication data is, for example, a cookie (data stored on the user's local terminal for identifying the user's identity).

[0057] In an embodiment of the present invention, there are multiple indicators to be displayed, and there are also multiple corresponding domain name addresses where the indicators to be displayed are located. When acquiring data of different domain names, for the sake of data security, it is necessary to first perform login authentication or authority authentication of different domain names according to cookies, so as to obtain indicator data after the login authentication or authority authentication is passed. For each indicator to be displayed, when obtaining indicator data according to the domain name address where the indicator to be displayed is located, the request authentication data of the current domain name can be obtained first, and the authority verification can be performed according to the request authentication data, and after the authority verification is passed, the corresponding indicator data can be obtained from the domain name address where the indicator to be displayed is located.

[0058] According to an embodiment of the present invention, when displaying data indicators, in order to avoid the problem of poor user experience caused by login authentication or permission authentication for each system, or repeated page jumps, it is necessary to enable cookies to be shared across domains.

[0059] According to one embodiment of the present invention, obtaining corresponding indicator data according to the domain name address where the indicator to be displayed is located may also include: in response to failing to obtain request authentication data of the current domain name according to the domain name address where the indicator to be displayed is located, determining whether the current domain name is within the domain name range that can be shared with the request authentication data; in response to the current domain name being within the domain name range that can be shared with the request authentication data, reading the request authentication data of the primary domain name to which the current domain name belongs, and writing the request authentication data of the primary domain name into the domain name address where the indicator to be displayed is located, so as to obtain the request authentication data of the current domain name according to the domain name address where the indicator to be displayed is located.

[0060] In an embodiment of the present invention, an interception plug-in is introduced on the server side of each domain name to read out all cookies under the domain name, and then all cookies are stored in the cache redis in the form of key-value pairs to facilitate subsequent permission verification.

[0061] When the request authentication data of the current domain name cannot be obtained according to the domain name address where the indicator to be displayed is located, the request authentication data cookie of the current domain name can be searched and obtained from all cookies stored in the cache redis. If the request authentication data cookie of the current domain name can be obtained, the permission verification is performed according to the cookie; if the request authentication data cookie of the current domain name cannot be obtained, it is determined whether the current domain name is within the domain name range that the request authentication data can be shared, so as to determine whether the cookie can be shared across domains. If the current domain name is within the domain name range that the request authentication data can be shared (which can be pre-set), the request authentication data cookie in the redis under the primary domain name to which the current domain name belongs is read, and the request authentication data cookie of the primary domain name is written into the domain name address where the indicator to be displayed is located, so as to obtain the request authentication data cookie of the current domain name according to the domain name address where the indicator to be displayed is located, thereby realizing cookie cross-domain synchronization. If the current domain name is not within the domain name range that the request authentication data can be shared, the data acquisition request needs to be returned immediately, and the response data is set to empty data, in order to prevent request forgery.

[0062] In addition, according to the needs of the subsystem of the current domain name, when verifying login, authority, security and other information, you can call a third-party unified authentication center for verification. If a subsystem also has special authority verification, based on the account parsed from the cookie, the separate verification logic of the subsystem can be used. If the verification passes, the indicator data can be queried from the domain name address where the indicator to be displayed is located; if the verification fails, the data acquisition request needs to be returned immediately, and the response data is set to empty data, so that when the data displayability verification is performed according to the indicator verification address, it can be prompted that the data displayability verification fails, and the indicator corresponding to the data acquisition request is used as the second indicator.

[0063] Figure 2 FIG. 1 is a schematic diagram showing the implementation principle of cross-domain sharing of authentication request data according to an embodiment of the present invention. Figure 2 As shown, in an embodiment of the present invention, the request authentication data is a cookie. The implementation principle of requesting cross-domain sharing of authentication data mainly includes the following steps: the indicator data display system generates a cookie acquisition request, and sends the cookie acquisition request to the server corresponding to the domain name address of the indicator to be displayed, so as to request to obtain the cookie of the current domain name. Afterwards, it is determined whether the cookie can be obtained. If the cookie can be obtained, the authority verification is performed according to the cookie. If the cookie cannot be obtained, it is determined whether the current domain name is within the domain name range of the sharable cookie. If the current domain name is within the domain name range of the sharable cookie, the cookie of the primary domain name to which the current domain name belongs is read, and the cookie of the primary domain name is written into the domain name address of the indicator to be displayed to complete the cross-domain sharing of the cookie, and the authority verification is performed according to the cookie; otherwise, if the current domain name is not within the domain name range of the sharable cookie, the cross-domain sharing of the cookie is not performed, and the failure to obtain the cookie is returned, and the authority verification fails.

[0064] Among them, when the permission verification is performed based on the cookie, if the permission verification passes, the indicator data is obtained according to the data acquisition request; otherwise, the permission verification fails and the process ends.

[0065] According to one embodiment of the present invention, the indicator data is generated in the following manner: regularly collecting internal system data and external system data and generating data processing tasks; classifying and aggregating the collected data based on the data processing tasks to obtain first indicator data; monitoring the real-time status and process processing status of the server through process probes, and analyzing and aggregating the monitoring data to obtain second indicator data; generating the indicator data based on the first indicator data and the second indicator data. In an embodiment of the present invention, the data sources of the indicator data mainly include three dimensions: internal system data screening, internal system process data collection, and external system data collection.

[0066] Among them, when filtering the internal system data, the data can be obtained through the scheduled inspection task, and then the data processing is performed. Specifically, first, according to the inspection scope and indicator scope configured by the system, the database, log, etc. of the template system are queried, the target data is filtered out, stored in a large database (such as hive), and then a task message is pushed to the message queue MQ. After that, the data is initially processed and multiple processes are started to consume the tasks in MQ. These tasks are all used to calculate or aggregate the target data. Then, according to the purpose of the data, the data is classified, such as order category, payment category, price category, server abnormality category, online accident category, etc. The reason for the small category is to trace the source of abnormal alarms later. Then, according to the similarity of the small categories, the data is aggregated into large categories, such as order category, payment category, resource category, order category, etc., because different large categories face different pages or system displays. Then, deep data processing is performed, and the data of each major category is processed offline in depth according to the requirements of the page. The aggregated data is aggregated, such as the order volume on xxx day, the system failure duration on xxx day, etc., and the processed data is stored in the database (such as MySQL) as required.

[0067] When collecting data from internal system processes, you can use process probes to collect data. Process probes can be divided into two categories: software probes and hardware probes. Hardware probes are used to monitor the real-time status of the server, such as disk or hard disk usage, and push messages to a large database (such as hive) in real time. Software probes are mainly designed for processes. For example, in the order process, if an exception or reverse scheduling occurs in the order process, the probe will be triggered immediately and push messages to the large database; in the approval process, if a process card point occurs and the process cannot be executed, the probe will immediately push the exception message to the large database; in another example, the price fluctuation probe is used to monitor the price fluctuation of online products in real time. If the price changes, the information will be pushed to the large database, etc. Then monitor the data changes in the large database and analyze the data in the large database. If it is a process card point problem, immediately start the abnormal alarm, aggregate the data, and store it in the analysis result library according to the category; if it is not a process card point problem, such as price fluctuation, not only analyze the reasons for the large price fluctuation, and generate alarm information to the relevant people, but also record the fluctuation trajectory and store it in the analysis result library for subsequent page display.

[0068] When collecting data from external systems, the main data collected is public data, such as new products, competitor product quotations, complaints about own products, and other information. First, lock the data collection website, scan the page data regularly, store the scanned page data as a string, and then parse the string according to certain rules. For example, in an HTML fragment, filter out the information: "xx product on xx website is a fake", "xx product on xx website is priced at 88 yuan", etc., and then store this information in the original library. Start the scheduled task, scan the new content of the original library, and then classify and analyze it (new products, quotations, complaints, etc.), and then store the data analysis results in the analysis result library, so that the subsequent front-end display page can make display decisions.

[0069] Figure 3 FIG. 1 is a schematic diagram of the generation process of indicator data according to an embodiment of the present invention. Figure 3 As shown, in one embodiment of the present invention, the indicator data is mainly obtained by processing three parts of data: internal system data, internal system process data and external system data.

[0070] Among them, when processing the internal system data, first, the database and log of the template system are queried through the scheduled inspection task, the target data is filtered out, and the target data is stored in the large database; then the data is preliminarily processed and the data is aggregated; finally, the aggregated internal system data is stored in the database.

[0071] When processing internal system process data, first, the real-time status data of the server and process data such as process card points and price fluctuations that occur during the execution of the internal system process are collected in real time through process probes, and the real-time collected internal system process data is pushed to the big database; then the data changes in the big database are monitored, the data in the big database is analyzed, and the data analysis results are stored in the analysis result library.

[0072] When processing external system data, first collect data from external systems in the configured scanning range through a scheduled scanning task, such as data disclosed by external systems such as new high-quality products, competitor product quotations, and complaints about the company's own products. The data disclosed by the external system is stored as a string, and then the string is parsed according to certain rules, and the string parsing results are stored in the original library; then, start the scheduled task to perform data analysis on the new content of the original library, and store the data analysis results in the analysis result library.

[0073] Finally, the data stored in the database and analysis result library are used as indicator data so that they can be displayed on the front-end display page or abnormal warnings can be issued later.

[0074] According to another embodiment of the present invention, before performing domain name connectivity verification according to the domain name address where the indicator is located and performing data displayability verification according to the indicator verification address, it may also include: drawing a page according to the indicators that can be displayed on the page and the indicator weight of each indicator to obtain a first display page, the first display page includes the indicator name of each indicator and the data placeholder corresponding to each indicator, and the data placeholder is embedded with the domain name address where the indicator is located. In an embodiment of the present invention, after obtaining the basic information of the indicators that can be displayed on the page, before performing domain name connectivity verification and data displayability verification on the indicators, it is also possible to perform an initial page drawing to the first display page according to the basic information of the indicators that can be displayed on the page, and display the first display page to the user, so that before the page loads the indicator data to generate the final display page, the user can browse the first display page in advance, thereby improving the user experience. In addition, a data placeholder corresponding to each indicator is set in the first display page, and the domain name address where the indicator is located is embedded in the data placeholder, so that the indicator data corresponding to the indicator is obtained according to the domain name address where the indicator is located in the subsequent, and the indicator data of the indicator to be displayed is filled into the display page in combination with the domain name connectivity verification and data displayability verification results.

[0075] In an embodiment of the present invention, the data placeholder can be implemented based on an iframe tag. An iframe is an HTML tag that acts as a document in a document, or a floating frame (FRAME). An iframe element creates an inline frame (i.e., an inline frame) containing another document. Then, the domain name address where the indicator is located is embedded in the iframe tag, so that the data loading between the domain names does not affect each other.

[0076] Figure 4 FIG. 1 is a schematic diagram of the processing flow of indicator data according to an embodiment of the present invention. Figure 4 As shown, in one embodiment of the present invention, the processing flow of indicator data is mainly as follows: first, draw the first display page according to the basic information of the indicators that can be displayed on the page, wherein the basic information includes the domain name address of the indicator, the indicator verification address and the indicator weight; then, perform domain name connectivity verification and data displayability verification on each indicator, and extract the indicators to be displayed from the indicators that can be displayed on the page according to the verification results; re-draw the page according to the basic information of the indicators to be displayed to obtain the display page; then, obtain the indicator data according to the domain name address of the indicators to be displayed, and fill the obtained indicator data into the display page for indicator data display. Among them, the indicator data can be collected by proxy service, direct connection service or Socket socket, etc., and the three parts of the collected data are processed and generated.

[0077] Figure 5 Schematic diagram of the main modules of the indicator data processing device according to an embodiment of the present invention. Figure 5 As shown, the indicator data processing device 500 of the embodiment of the present invention mainly includes an indicator information acquisition module 501, a data verification module 502, an indicator screening module 503, a page drawing module 504 and an indicator filling and display module 505.

[0078] The indicator information acquisition module 501 is used to acquire basic information of indicators that can be displayed on the page, and the basic information includes the domain name address where the indicator is located, the indicator verification address and the indicator weight;

[0079] A data verification module 502, configured to verify domain name connectivity according to the domain name address where the indicator is located, and verify data displayability according to the indicator verification address;

[0080] An indicator screening module 503, used to delete the first indicator that fails the domain name connectivity verification and the second indicator that fails the data displayability verification from the indicators that can be displayed on the page, to obtain indicators to be displayed;

[0081] A page drawing module 504, configured to draw a page according to the indicators to be displayed and the indicator weight of each indicator to be displayed to obtain a display page;

[0082] The indicator filling and display module 505 is used to obtain indicator data according to the domain name address where the indicator to be displayed is located, and fill the obtained indicator data into the display page for display.

[0083] According to one embodiment of the present invention, the indicator data processing device 500 may further include a basic information management module (not shown in the figure), which is used to: after obtaining the basic information of the indicators that can be displayed on the page, store the basic information of the indicators that can be displayed on the page in an array; before deleting the first indicator that fails the domain name connectivity verification and the second indicator that fails the data displayability verification from the indicators that can be displayed on the page, delete the basic information of the first indicator that fails the domain name connectivity verification and the second indicator that fails the data displayability verification from the array.

[0084] According to another embodiment of the present invention, the page drawing module 504 may also be used for: in response to monitoring the change of the array, drawing a page according to the to-be-displayed indicator and the indicator weight of each to-be-displayed indicator in the changed array to obtain a display page.

[0085] According to another embodiment of the present invention, when the data verification module 502 performs domain name connectivity verification according to the domain name address where the indicator is located, it can be specifically used to: request access to the corresponding domain name according to the domain name address where the indicator is located, and in response to the domain name access is unavailable or the access times out, the domain name connectivity verification fails; when the data verification module 502 performs data displayability verification according to the indicator verification address, it can be specifically used to: determine whether the data of a sub-page corresponding to a certain indicator is all 0, empty, or the data query times out according to the indicator verification address, and in response to the data of the sub-page corresponding to the indicator being all 0, empty, or the data query times out, the data displayability verification fails.

[0086] According to another embodiment of the present invention, there are multiple indicators to be displayed; the indicator filling and display module 505 can also be used to: for each indicator to be displayed, obtain the request authentication data of the current domain name according to the domain name address where the indicator to be displayed is located, and perform authority verification according to the request authentication data, and in response to the authority verification being passed, obtain the corresponding indicator data from the domain name address where the indicator to be displayed is located.

[0087] According to another embodiment of the present invention, the indicator filling and display module 505 can also be used for: in response to failing to obtain the request authentication data of the current domain name according to the domain name address where the indicator to be displayed is located, determining whether the current domain name is within the domain name range that can be shared with the request authentication data; in response to the current domain name being within the domain name range that can be shared with the request authentication data, reading the request authentication data of the primary domain name to which the current domain name belongs, and writing the request authentication data of the primary domain name into the domain name address where the indicator to be displayed is located, so as to obtain the request authentication data of the current domain name according to the domain name address where the indicator to be displayed is located.

[0088] According to another embodiment of the present invention, the indicator data is generated in the following manner: regularly collecting internal system data and external system data and generating data processing tasks; classifying and aggregating the collected data based on the data processing tasks to obtain first indicator data; monitoring the real-time status and process processing status of the server through process probes, and analyzing and aggregating the monitoring data to obtain second indicator data; generating the indicator data based on the first indicator data and the second indicator data.

[0089] According to another embodiment of the present invention, the indicator data processing device 500 may also include a first page drawing module (not shown in the figure), which is used to: before performing domain name connectivity verification according to the domain name address where the indicator is located and performing data displayability verification according to the indicator verification address, draw a page according to the indicators that can be displayed on the page and the indicator weight of each indicator to obtain a first display page, wherein the first display page includes the indicator name of each indicator and a data placeholder corresponding to each indicator, and the data placeholder is embedded with the domain name address where the indicator is located.

[0090] According to the technical solution of the embodiment of the present invention, by obtaining basic information of indicators that can be displayed on the page, the basic information includes the domain name address of the indicator, the indicator verification address and the indicator weight; performing domain name connectivity verification according to the domain name address of the indicator, and performing data displayability verification according to the indicator verification address; deleting the first indicator that fails the domain name connectivity verification and the second indicator that fails the data displayability verification from the indicators that can be displayed on the page, to obtain the indicators to be displayed; drawing the page according to the indicators to be displayed and the indicator weight of each indicator to be displayed to obtain a display page; obtaining indicator data according to the domain name address of the indicator to be displayed, and filling the obtained indicator data into the display page for display. The technical solution realizes intelligent screening of indicator data by performing domain name connectivity verification and data displayability verification to delete abnormal indicators. When an indicator abnormality occurs, there is no need to take the entire system offline, thereby improving the display effect of indicator data and improving the availability of the system.

[0091] Figure 6An exemplary system architecture 600 is shown to which the method for processing indicator data or the apparatus for processing indicator data according to the embodiment of the present invention can be applied.

[0092] like Figure 6 As shown, system architecture 600 may include terminal devices 601, 602, 603, network 604 and server 605. Network 604 is used to provide a medium for communication links between terminal devices 601, 602, 603 and server 605. Network 604 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0093] Users can use terminal devices 601, 602, and 603 to interact with server 605 through network 604 to receive or send messages, etc. Various client applications can be installed on terminal devices 601, 602, and 603, such as data display applications, indicator data processing applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0094] The terminal devices 601 , 602 , and 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers, etc.

[0095] Server 605 may be a server that provides various services, such as a backend management server (for example only) that supports shopping websites browsed by users using terminal devices 601, 602, and 603. The backend management server may obtain basic information of indicators that can be displayed on the page from the received processing request for indicator data and other data, and the basic information includes the domain name address where the indicator is located, the indicator verification address, and the indicator weight; perform domain name connectivity verification according to the domain name address where the indicator is located, and perform data displayability verification according to the indicator verification address; delete the first indicator that fails the domain name connectivity verification and the second indicator that fails the data displayability verification from the indicators that can be displayed on the page to obtain the indicators to be displayed; draw the page according to the indicators to be displayed and the indicator weight of each indicator to be displayed to obtain the display page; obtain indicator data according to the domain name address where the indicator to be displayed is located, and fill the obtained indicator data into the display page for display and other processing, and feed back the processing result (for example, display page, display result - for example only) to the terminal device.

[0096] It should be noted that the method for processing indicator data provided in the embodiment of the present invention is generally executed by the server 605 , and accordingly, the device for processing indicator data is generally disposed in the server 605 .

[0097] It should be understood that Figure 6The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0098] Reference below Figure 7 , which shows a schematic diagram of the structure of a computer system 700 of a terminal device or a server suitable for implementing an embodiment of the present invention. Figure 7 The terminal device or server shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0099] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0100] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage section 708 as needed.

[0101] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above-mentioned functions defined in the system of the present invention are executed.

[0102] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0103] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0104] The units or modules involved in the embodiments of the present invention may be implemented by software or hardware. The units or modules described may also be provided in a processor. For example, they may be described as follows: a processor includes an indicator information acquisition module, an acquisition unit, a data verification module, an indicator screening module, a page drawing module, and an indicator filling and display module. Among them, the names of these units or modules do not constitute limitations on the units or modules themselves in certain circumstances. For example, the indicator filling and display module may also be described as "a module for acquiring indicator data according to the domain name address of the indicator to be displayed, and filling the acquired indicator data into the display page for display."

[0105] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by a device, the device includes: obtaining basic information of indicators that can be displayed on the page, the basic information includes the domain name address where the indicator is located, the indicator verification address and the indicator weight; performing domain name connectivity verification according to the domain name address where the indicator is located, and performing data displayability verification according to the indicator verification address; deleting the first indicator that fails the domain name connectivity verification and the second indicator that fails the data displayability verification from the indicators that can be displayed on the page, and obtaining the indicators to be displayed; drawing the page according to the indicators to be displayed and the indicator weight of each indicator to be displayed to obtain the display page; obtaining indicator data according to the domain name address where the indicator to be displayed is located, and filling the obtained indicator data into the display page for display.

[0106] According to the technical solution of the embodiment of the present invention, by obtaining basic information of indicators that can be displayed on the page, the basic information includes the domain name address of the indicator, the indicator verification address and the indicator weight; performing domain name connectivity verification according to the domain name address of the indicator, and performing data displayability verification according to the indicator verification address; deleting the first indicator that fails the domain name connectivity verification and the second indicator that fails the data displayability verification from the indicators that can be displayed on the page, to obtain the indicators to be displayed; drawing the page according to the indicators to be displayed and the indicator weight of each indicator to be displayed to obtain a display page; obtaining indicator data according to the domain name address of the indicator to be displayed, and filling the obtained indicator data into the display page for display. The technical solution realizes intelligent screening of indicator data by performing domain name connectivity verification and data displayability verification to delete abnormal indicators. When an indicator abnormality occurs, there is no need to take the entire system offline, thereby improving the display effect of indicator data and improving the availability of the system.

[0107] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for processing indicator data, characterized in that: include: Obtain basic information of indicators that can be displayed on the page, including the domain name address of the indicator, the indicator verification address and the indicator weight; Perform domain name connectivity verification based on the domain name address where the indicator is located, and perform data displayability verification based on the indicator verification address; Deleting the first indicator that fails the domain name connectivity verification and the second indicator that fails the data displayability verification from the indicators that can be displayed on the page to obtain indicators to be displayed; Drawing a page according to the indicators to be displayed and the indicator weight of each indicator to be displayed to obtain a display page; The indicator data is obtained according to the domain name address where the indicator to be displayed is located, and the obtained indicator data is filled into the display page for display.

2. The method according to claim 1, characterized in that After obtaining the basic information of the indicators that can be displayed on the page, the method further includes: storing the basic information of the indicators that can be displayed on the page into an array; Before deleting the first indicator that fails the domain name connectivity verification and the second indicator that fails the data displayability verification from the indicators that can be displayed on the page, it also includes: deleting the basic information of the first indicator that fails the domain name connectivity verification and the second indicator that fails the data displayability verification from the array.

3. The method according to claim 2, characterized in that Drawing a page according to the indicators to be displayed and the indicator weight of each indicator to be displayed to obtain a display page includes: In response to monitoring the change of the array, a page is drawn according to the to-be-displayed indicators and the indicator weight of each to-be-displayed indicator in the changed array to obtain a display page.

4. The method according to claim 1, characterized in that: Performing domain name connectivity verification according to the domain name address where the indicator is located, including: requesting access to the corresponding domain name according to the domain name address where the indicator is located, and responding that the domain name access is unavailable or the access times out, the domain name connectivity verification fails; Performing data displayability verification according to the indicator verification address includes: determining whether the data of a subpage corresponding to a certain indicator is all 0, empty, or the data query has timed out according to the indicator verification address, and in response to the data of the subpage corresponding to the indicator being all 0, empty, or the data query has timed out, the data displayability verification fails.

5. The method according to claim 1, characterized in that: There are multiple indicators to be displayed; Obtaining corresponding indicator data according to the domain name address where the indicator to be displayed is located includes: For each indicator to be displayed, the request authentication data of the current domain name is obtained according to the domain name address where the indicator to be displayed is located, and authority verification is performed according to the request authentication data. In response to the authority verification being passed, the corresponding indicator data is obtained from the domain name address where the indicator to be displayed is located.

6. The method according to claim 5, characterized in that Acquiring corresponding indicator data according to the domain name address where the indicator to be displayed is located also includes: In response to failure to obtain the request authentication data of the current domain name according to the domain name address where the indicator to be displayed is located, determining whether the current domain name is within the domain name range that can be shared with the request authentication data; In response to the current domain name being within the domain name range that can be shared by the request authentication data, the request authentication data of the primary domain name to which the current domain name belongs is read, and the request authentication data of the primary domain name is written into the domain name address where the indicator to be displayed is located, so as to obtain the request authentication data of the current domain name according to the domain name address where the indicator to be displayed is located.

7. The method according to claim 1, characterized in that The indicator data is generated in the following way: Regularly collect internal system data and external system data and generate data processing tasks; Based on the data processing task, the collected data is classified and aggregated to obtain first indicator data; The real-time status of the server and the process processing status are monitored through the process probe, and the monitoring data is analyzed and aggregated to obtain the second indicator data; The indicator data is generated according to the first indicator data and the second indicator data.

8. The method according to claim 1, characterized in that Before performing domain name connectivity verification according to the domain name address where the indicator is located and performing data displayability verification according to the indicator verification address, it also includes: The page is drawn according to the indicators that can be displayed on the page and the indicator weight of each indicator to obtain a first display page, wherein the first display page includes the indicator name of each indicator and a data placeholder corresponding to each indicator, wherein the data placeholder is embedded with the domain name address of the indicator.

9. A device for processing index data, characterized in that: include: The indicator information acquisition module is used to obtain the basic information of the indicators that can be displayed on the page, and the basic information includes the domain name address where the indicator is located, the indicator verification address and the indicator weight; A data verification module, used to verify domain name connectivity according to the domain name address where the indicator is located, and to verify data displayability according to the indicator verification address; An indicator screening module, used to delete the first indicator that fails the domain name connectivity verification and the second indicator that fails the data displayability verification from the indicators that can be displayed on the page, to obtain the indicators to be displayed; A page drawing module, used for drawing a page according to the indicators to be displayed and the indicator weight of each indicator to be displayed to obtain a display page; The indicator filling and display module is used to obtain indicator data according to the domain name address where the indicator to be displayed is located, and fill the obtained indicator data into the display page for display.

10. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

11. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images