Buried point quality detection method, device, equipment, medium and program product

Abstract

The embodiment of the invention discloses a burying point quality detection method and device, equipment, a medium and a program product, and the method comprises the steps: obtaining log data of a target burying point; performing log error detection based on the log data to obtain a log detection error result of the target burying point; performing data reporting time interval detection based on the log data to obtain a data reporting time interval of the target burying point; and determining the burying point quality of the target burying point based on the log error detection result and the data reporting time interval. According to the embodiment of the invention, the buried point data quality can be automatically, quickly and accurately detected, the labor cost is reduced, and the efficiency is high.

Description

Technical Field

[0001] The present disclosure relates to data processing technology and web page technology, and in particular to a method, device, equipment, medium and program product for detecting quality of buried points. Background Art

[0002] Tracking refers to the technology and implementation process for capturing, processing and sending specific user behaviors or business processes.

[0003] The processing flow of tracking points can include tracking point registration, development, testing, online, data quality monitoring and query use. Tracking point data quality refers to the accuracy of tracking point data, which mainly includes the accuracy of the number of tracking point collection reports and the accuracy of tracking point reporting information. Only when the tracking point can accurately capture user behavior data can the user behavior be determined based on the user behavior data and then analyzed.

[0004] Currently, data quality monitoring is done through manual inventory, which has high labor costs and low efficiency. Summary of the invention

[0005] The embodiments of the present disclosure provide a method and device for detecting the quality of buried points, an electronic device and a storage medium to solve the above-mentioned problems.

[0006] A first aspect of an embodiment of the present disclosure provides a method for detecting quality of buried points, including:

[0007] Get the log data of the target tracking point;

[0008] Perform log error detection based on the log data to obtain a log error detection result of the target embedding point;

[0009] Perform data reporting time interval detection based on the log data to obtain the data reporting time interval of the target tracking point;

[0010] Based on the log error detection result and the data reporting time interval, the burying point quality of the target burying point is determined.

[0011] In some embodiments of the present disclosure, log error detection is performed based on the log data to obtain a log error detection result of the target tracking point, including:

[0012] Perform metadata information verification error detection based on the log data to obtain the metadata information verification error detection result of the target embedding point;

[0013] Perform custom parameter verification error detection based on the log data to obtain a custom parameter verification error detection result of the target embedding point;

[0014] The log detection error result is obtained based on the metadata information verification error detection result and the custom parameter verification error detection result.

[0015] In some embodiments of the present disclosure, it further includes:

[0016] Based on the numerical relationship between the data amounts of each buried point in the associated buried points, the buried point quality of the associated buried points is determined.

[0017] In some embodiments of the present disclosure, it further includes:

[0018] Build a mapping relationship between the custom parameters of each tracking point and the fields of the business table;

[0019] Based on the mapping relationship, field reduction is performed on the tracking point log data of each tracking point;

[0020] The tracking point log data of each tracking point after the field reduction is stored.

[0021] In some embodiments of the present disclosure, it further includes:

[0022] Determine the illegal and legal data in the tracking log data of each tracking point;

[0023] Deleting the illegal data, or writing the illegal data into an illegal data table;

[0024] The legal data is written into the legal data table.

[0025] In some embodiments of the present disclosure, deleting the illegal data or writing the illegal data into an illegal data table includes:

[0026] In response to the type of the illegal data being the first type, deleting the illegal data;

[0027] In response to the type of the illegal data being the second type, the illegal data is written into an illegal data table.

[0028] In some embodiments of the present disclosure, after determining the burying point quality of the target burying point based on the log error detection result and the data reporting time interval, the method further includes:

[0029] In response to the burial point quality of the target burial point indicating that the target burial point has quality problems, an alarm is issued.

[0030] A second aspect of the embodiments of the present disclosure provides a device for detecting quality of buried points, including:

[0031] The acquisition module is used to obtain the log data of the target embedding point;

[0032] A log error detection module, used to perform log error detection based on the log data to obtain a log error detection result of the target embedding point;

[0033] A time interval detection module, used to detect the data reporting time interval based on the log data to obtain the data reporting time interval of the target buried point;

[0034] The quality detection module is used to determine the burying point quality of the target burying point based on the log error detection result and the data reporting time interval.

[0035] In some embodiments of the present disclosure, the log error detection module includes:

[0036] A first detection unit is used to perform metadata information verification error detection based on the log data to obtain a metadata information verification error detection result of the target embedding point;

[0037] A second detection unit is used to perform custom parameter verification error detection based on the log data to obtain a custom parameter verification error detection result of the target embedding point;

[0038] The log detection error result determining unit is used to obtain the log detection error result based on the metadata information verification error detection result and the custom parameter verification error detection result.

[0039] In some embodiments of the present disclosure, the buried point quality detection device further includes:

[0040] The associated buried point quality detection module is used to determine the buried point quality of the associated buried points based on the numerical relationship between the data amounts of each buried point in the associated buried points.

[0041] In some embodiments of the present disclosure, the buried point quality detection device further includes:

[0042] The mapping relationship building module is used to build the mapping relationship between the custom parameters of each tracking point and the fields of the business table;

[0043] A field reduction module, used for reducing fields of the tracking point log data of each tracking point based on the mapping relationship;

[0044] The storage module is used to store the tracking point log data of each tracking point after field reduction.

[0045] In some embodiments of the present disclosure, the storage module includes:

[0046] A determination unit, used to determine illegal data and legal data in the tracking point log data of each tracking point;

[0047] An illegal data processing unit, used for deleting the illegal data or writing the illegal data into an illegal data table;

[0048] The legal data processing unit is used to write the legal data into the legal data table.

[0049] In some embodiments of the present disclosure, the illegal data processing unit is used to:

[0050] In response to the type of the illegal data being the first type, deleting the illegal data;

[0051] In response to the type of the illegal data being the second type, the illegal data is written into an illegal data table.

[0052] In some embodiments of the present disclosure, the burial point quality detection further includes:

[0053] The alarm module is used to issue an alarm in response to the comprehensive evaluation result indicating that there is a quality problem with the target burial point.

[0054] According to a third aspect of the present disclosure, an electronic device is provided, including:

[0055] A memory for storing a computer program product;

[0056] The processor is used to execute the computer program product stored in the memory, and when the computer program product is executed, the method described in the first aspect above is implemented.

[0057] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described in the first aspect is implemented.

[0058] According to a fifth aspect of the embodiments of the present disclosure, a computer program product is provided, comprising computer program instructions, which, when executed by a processor, enable the processor to execute the method described in the first aspect.

[0059] The buried point quality detection apparatus, device, equipment, medium and program product of the embodiments of the present invention, since there may be log errors in the log data of the buried point, and usually the time interval for uploading data of the buried point has a certain regularity, therefore log error detection and data upload time interval detection are performed on the log data of the target buried point from the dimensions of log accuracy and data upload time interval, and the log detection error result of the target buried point and the data reporting time interval of the target buried point can be obtained, and then the buried point quality of the target buried point can be obtained, thereby realizing automatic, rapid and accurate detection of the buried point data quality, reducing labor costs and high efficiency.

[0060] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0062] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0063] Figure 1 A schematic diagram of a method for detecting quality of buried points in one embodiment of the present disclosure;

[0064] Figure 2 is a flowchart of step S2 in some embodiments of the present disclosure;

[0065] Figure 3 It is a partial flow chart of a method for detecting quality of buried points in some embodiments of the present disclosure;

[0066] Figure 4 is a flowchart of step S6 in some embodiments of the present disclosure;

[0067] Figure 5 This is a working principle diagram of a point-of-care log data management system in an example of the present disclosure;

[0068] Figure 6 is a flowchart of an alarm process in an example of the present disclosure;

[0069] Figure 7 This is a structural block diagram of a buried point quality detection device in one embodiment of the present disclosure;

[0070] Figure 8 This is a structural block diagram of a log error detection module 200 in one embodiment of the present disclosure;

[0071] Fig. 9 This is a structural block diagram of a buried point quality detection device in another embodiment of the present disclosure;

[0072] Fig.10 This is a structural block diagram of a storage module 800 in one embodiment of the present disclosure;

[0073] Fig.11 The structure block diagram of an electronic device in one embodiment of the present disclosure is shown in FIG. DETAILED DESCRIPTION

[0074] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0075] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0076] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0077] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0078] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.

[0079] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0080] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0081] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0082] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0083] The embodiments of the present disclosure can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0084] Electronic devices such as terminal devices, computer systems, servers, etc. can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0085] Figure 1 FIG. 1 is a flow chart of a method for detecting the quality of a buried point in an embodiment of the present disclosure. Figure 1 As shown, the method for detecting the quality of buried points includes the following steps:

[0086] S1: Get the log data of the target tracking point.

[0087] The management system of the buried point log data (hereinafter referred to as the management system) extracts the buried point log data of the target buried point from the storage device of the buried point data (hereinafter referred to as the storage device). The target buried point may be a buried point for capturing page events, exposure events or click events.

[0088] The management system in the disclosed embodiment may be ClickHouse. ClickHouse is a column-based database management system for online analytical processing (OLAP), supporting flexible massively parallel processing (MPP) architecture and linear expansion. Its advantages include column-based storage, vectorized computing, high compression ratio, and high performance.

[0089] S2: Perform log error detection based on log data to obtain the log error detection result of the target embedding point.

[0090] The management system performs log error detection on the log data of the target embedding point according to the set conditions of the correct log data to detect whether a log error occurs in the embedding point log data of the target embedding point, thereby obtaining the log error detection result of the target embedding point. The log error detection result may include the error type, error location, and error quantity of the log error, etc.

[0091] S3: Perform data reporting time interval detection based on log data to obtain target buried data reporting time interval detection.

[0092] The management system determines the upload time of each data based on the log data of the target tracking point, and then performs data reporting time interval detection to obtain the time interval between adjacent data uploaded by the target tracking point.

[0093] S4: Based on the log error detection results and data reporting time interval, determine the comprehensive evaluation results of the target tracking point quality.

[0094] The management system can evaluate the quality of the target embedding point from the log error dimension according to the preset quality assessment method and the log error detection result, and evaluate the quality of the target embedding point from the data upload time interval dimension according to the data upload time interval of the target embedding point, and finally obtain the embedding point quality of the target embedding point.

[0095] In this embodiment, since there may be log errors in the log data of the burial point, and usually the time interval for uploading data of the burial point has a certain regularity, log error detection and data upload time interval detection are performed on the log data of the target burial point from the dimensions of log accuracy and data upload time interval. The log detection error result of the target burial point and the data reporting time interval of the target burial point can be obtained, and then the burial point quality of the target burial point can be obtained, thereby realizing automatic, fast and accurate detection of the burial point data quality, reducing labor costs and high efficiency.

[0096] Figure 2 is a flow chart of step S2 in some embodiments of the present disclosure. Figure 2 As shown, step S2 includes the following steps:

[0097] S2-1: Perform metadata information verification error detection based on log data to obtain metadata information verification error detection results.

[0098] The tracking point design scheme of the disclosed embodiment mainly includes business line (pid), page (uicode), tracking point identifier (evtId) and custom attributes. pid, uicode, evtId mainly describe the business scenario in which the target tracking point is located and on which page the target tracking point is triggered to be reported through clicks or exposure. Custom parameters mainly describe which custom parameters need to be attached in the above business scenarios to calculate business indicators.

[0099] The management system sets pid, uicode and evtId as one type and custom parameters as another type, and further divides the log error types into: meta information verification error and custom parameter verification error.

[0100] Among them, metadata information verification errors may include metadata value errors and combination errors, that is, single field reporting errors and combination value errors between fields.

[0101] Custom parameter validation errors can include missing custom information and multiple reporting errors, that is, the target tracking point has custom parameters but is not reported, or the target tracking point does not have custom parameters but reports custom parameters. Custom parameter validation errors can also include custom information type errors, custom information range errors, and regular expression errors, that is, the custom information reported by the target tracking point does not meet the standard type, does not meet the standard range of custom information, or the regular expression of custom information does not meet the specifications.

[0102] The management system performs metadata information verification error detection on the target burial point's burial point log data according to the preset metadata verification rules to detect whether metadata information verification errors occur in the burial point's burial point log data and obtain metadata information verification error detection results.

[0103] S2-2: Perform custom parameter verification error detection based on log data to obtain the custom parameter verification error detection result of the target embedding point.

[0104] The management system performs custom parameter verification error detection on the target embedding point's log data according to preset custom parameter verification rules to detect whether custom parameter verification errors occur in the target embedding point's log data and obtain custom parameter verification error detection results.

[0105] S2-3: Verify the error detection result based on the metadata information and the custom parameter to obtain a log detection error result.

[0106] The management system obtains the log detection error result based on the metadata information verification error detection result and the custom parameter verification error detection result. In this embodiment, since the log error type can include metadata information verification error and custom parameter verification error, metadata information verification error detection and custom parameter verification error detection can be performed on the log data of the target buried point respectively, and the metadata information verification error detection result and custom parameter verification error detection result can be obtained accordingly, and then the log detection error result can be obtained, so as to realize the quality detection of the target buried point from the log error dimension.

[0107] In some embodiments of the present disclosure, the method for detecting buried point quality further includes the following steps: determining the buried point quality of associated buried points based on the numerical relationship between the data amounts of each buried point in the associated buried points.

[0108] The management system can extract the data volume of all associated tracking points from the storage device, wherein the associated relationship can include the click tracking point and the exposure tracking point.

[0109] The management system sets the data volume of each associated embedding point to satisfy the funnel relationship, that is, the first layer is the page event, the second layer is the exposure event, and the third layer is the click event. The data volume of the embedding points corresponding to the three layers of events should satisfy the following: the data volume of the first layer embedding point ≥ the data volume of the second layer embedding point ≥ the data volume of the third layer embedding point.

[0110] The management node detects whether the data volume of the associated buried point meets the data volume requirement corresponding to the funnel form. If so, the data volume of the associated node is determined to be normal, otherwise, the data volume of the associated node is determined to be abnormal. The management node obtains the buried point quality of the associated buried point based on the comparison result between the data volume of the associated buried point and the data volume requirement corresponding to the funnel form.

[0111] In this embodiment, since the data amounts of the associated buried points have a set numerical relationship under normal circumstances, the buried point quality of the associated buried points can be quickly and accurately obtained based on the numerical relationship between the data amounts of the associated buried points.

[0112] In some embodiments of the present disclosure, the method for detecting the quality of the buried point further includes the following steps: constructing a mapping relationship between the custom parameters of each buried point and the fields of the business table; based on the mapping relationship, performing field reduction on the buried point log data of each buried point; and writing the buried point log data of each buried point after the field reduction into the business table.

[0113] In the embodiment of the present disclosure, the number of custom parameters used by all tracking points is N, and the number of fields in the business table of the management system is M. Wherein, N and M are both integers greater than 1, and N is much greater than M. For example, N is 2000 and M is 30.

[0114] The management system constructs a mapping between N custom parameters and M fields. When each tracking point reports data, the management system reduces the fields of each tracking point report data according to the mapping relationship and writes it into the business table.

[0115] In an example of the present disclosure, 9 fields (custom parameters) are mapped to 3 fields. Buried point situation: Buried point 1 has field 1, field 2 and field 3; Buried point 2 has field 4, field 5 and field 6; Buried point 3 has field 7, field 8 and field 9. Buried point 1 reports data: field 1 is A, field 2 is B; Buried point 3 reports data: field 7 is C, field 8 is D.

[0116] Before mapping is performed, the management system's database needs to have: field 1, field 2, ..., field 8, field 9.

[0117] The following two pieces of data will be generated in the management system database:

[0118] Tracking point 1: Field 1 is A, Field 2 is B, and the remaining 7 fields are empty;

[0119] Buried point 3: Field 7 is C, field 8 is D, and the remaining 7 fields are empty.

[0120] After mapping, tracking point 1: field 1 is mapped to action1, field 2 is mapped to action2, and field 3 is mapped to action; tracking point 2: field 4 is mapped to action1, field 5 is mapped to action2, and field 6 is mapped to action; tracking point 3: field 7 is mapped to action1, field 8 is mapped to action2, and field 9 is mapped to action. At this point, the business table has 3 fields, namely, tracking point 1, action2, and action.

[0121] The following two pieces of data will be generated in the database of the management system:

[0122] Tracking point 1: action1 is A, action2 is B, and action is empty;

[0123] Tracking point 3: action1 is C, action2 is D, and action is empty.

[0124] Those skilled in the art can understand that the way of mapping N fields into M fields is similar to the way of mapping 9 fields into 3 fields in the above example.

[0125] When you need to use a business table to display a piece of data, you can perform a reverse mapping based on the mapping relationship to obtain the corresponding field.

[0126] In this embodiment, since the total number of custom parameters used by all embedding points is much larger than the number of custom parameters actually used by each embedding point, the embedding point log data of each embedding point is reduced in fields based on the mapping relationship between the custom parameters of each embedding point and the fields of the business table, which can significantly reduce the data storage capacity.

[0127] Figure 3 FIG. 1 is a partial flow chart of a method for detecting the quality of buried points in some embodiments of the present disclosure. Figure 3 As shown, the method for detecting the quality of buried points may further include the following steps:

[0128] S5: Determine the illegal data and legal data in the tracking point log data of each tracking point.

[0129] The management system detects the tracking log data of each tracking point according to the preset data legitimacy detection rules, and obtains the illegal data and legal data in the tracking log data of each tracking point. Among them, illegal data is data that does not comply with the data legitimacy detection rules, and legal data is data that complies with the data legitimacy detection rules.

[0130] S6: Delete the illegal data, or write the illegal data into the illegal data table.

[0131] According to the illegal data processing rules, the management system can choose to delete the illegal data or write the illegal data into the illegal data table, where the illegal data table is a table storing illegal data.

[0132] S7: Write the legal data into the legal data table, where the legal data table is a table storing legal data.

[0133] It should be noted that the present disclosure does not limit the execution order of S1-S4 and S5-S7, that is, S1-S4 may be executed first and then S5-S7, or S5-S7 may be executed first and then S1-S4, or S1-S4 and S5-S7 may be executed simultaneously.

[0134] In this embodiment, by distinguishing between legal data and illegal data in the tracking log data of each tracking point, the storage capacity of the legal data table can be reduced, thereby improving the reading and writing performance of legal data, and ensuring the stability of the task of writing legal data.

[0135] Figure 4 FIG. 5 is a flow chart of step S6 in some embodiments of the present disclosure. Figure 4 As shown, step S5 includes the following steps:

[0136] S6-1: In response to the type of the illegal data being the first type, deleting the illegal data.

[0137] The first type may be the type of sending the tracking point log data when the tracking point is offline or deleted. The first type of illegal data is not helpful for user behavior analysis. When the management system determines that the type of illegal data is the first type, the illegal data is deleted.

[0138] S6-2: In response to the type of the illegal data being the second type, writing the illegal data into the illegal data table.

[0139] The second type may include error in reporting address of buried point log data, error in buried point unique identification and error filtering of pid. The second type of illegal data is written into the illegal data table so as to analyze the user behavior after correcting the error.

[0140] In this embodiment, different types of illegal data are processed in different ways, which can not only utilize data useful for user behavior analysis, but also reduce data storage capacity by deleting useless illegal data.

[0141] Figure 5 This is a working principle diagram of a point-of-care log data management system in an example of the present disclosure. Figure 5As shown, the tracking point log data management system mainly includes: a receiving and uploading module for tracking point log data, a field parsing module and a writing task module.

[0142] The receiving and uploading module of the buried point log data is used to receive the buried point log data of the first enterprise user, the second enterprise user and the client through the service tool. The service tool may include a tool for implementing the dig service, which is a business line service for the enterprise. The first enterprise terminal may be a terminal of a real estate transaction enterprise, the second enterprise user may be a terminal of a real estate rental enterprise, and the client may be a client for logging into the first enterprise website or the second enterprise website.

[0143] The Data Pipeline (DP) module is used to implement data exchange and communication between devices. Among them, the switching chip module is mainly used to perform embedded field analysis, and send the analysis results of embedded field analysis to the writing task module through Kafka. Kafka is a high-throughput distributed publish-subscribe messaging system.

[0144] The write task module is used to perform data writing tasks. Among them, the metadata of the management system (such as pid, uicode, evtId and action data) verifies the buried log errors in real time through the remote dictionary server (Remote Dictionary Server, Redis), writes the legal data into the legal data table through the write and filter task flink, and discards the first type of illegal data. For the second type of illegal data, the write and filter task flink is sent to the write task flink through kafka, and the write task flink writes the second type of illegal data into the illegal data table, so that the second type of illegal data in the illegal data table can be corrected and used for analyzing user behavior. Flink is a distributed system that requires effective allocation and management of computing resources to execute stream applications. In flink, a job is a complete computing process submitted by the user to the flink system for execution. A job consists of multiple operations. Flink converts the job into one or more tasks and executes these tasks in parallel on the cluster.

[0145] In some embodiments of the present disclosure, the following steps are also included after step S1: in response to the burial point quality of the target burial point indicating that the target burial point has a quality problem, an alarm is issued.

[0146] After the management system determines that there is a quality problem with the target burying point based on the burying point quality of the target burying point, it can alert the administrator of the management system in a variety of ways such as text alarm, text and image combined alarm and video alarm.

[0147] In the alarm entry of the management system, the alarm status is divided into pending, accepted, closed, and whitelisted. The repair results are divided into unrepaired and repaired. The monitoring identification of log content monitoring, seven-day no data anomaly monitoring, and funnel anomaly monitoring is abstracted into a reasonable processing logic.

[0148] Figure 6 FIG. 1 is a flowchart of an alarm process in an example of the present disclosure. Figure 6 As shown, for the data to be verified in the alarm information, the information of each buried point and the buried point log data in the data to be verified are extracted to determine whether the data volume of the associated buried points satisfies the funnel relationship.

[0149] If the data volume of the associated tracking points satisfies the funnel relationship, the whitelist is updated, the tracking point log data of each tracking point in the whitelist is traversed, and it is determined whether there are quality problems in the tracking point log data of each tracking point.

[0150] If a certain tracking point is determined to have a quality problem, it is checked whether an alarm has been issued for the quality problem. If it is detected that no alarm has been issued for the tracking point, an alarm is issued; if it is detected that an alarm has been issued for the tracking point, the alarm status is updated according to the alarm processing status (pending, accepted or closed).

[0151] If it is determined that a certain tracking point has no quality problems, then check whether the tracking point has no historical alarms and the alarm status of the tracking point is closed. If it is detected that the tracking point has no historical alarms and the alarm status of the tracking point is closed, the tracking point will not be processed. If it is detected that the tracking point has historical alarms or the alarm status of the tracking point is open, the alarm status of the tracking point is adjusted to repaired or closed.

[0152] Figure 7 FIG. 1 is a structural block diagram of a device for detecting the quality of a buried point in one embodiment of the present disclosure. Figure 7 As shown, the buried point quality detection device includes:

[0153] The acquisition module 100 is used to acquire the log data of the target embedding point;

[0154] The log error detection module 200 is used to perform log error detection based on log data and obtain the log error detection result of the target embedding point;

[0155] The time interval detection module 300 is used to detect the data reporting time interval based on the log data to obtain the data reporting time interval of the target buried point;

[0156] The quality detection module 400 is used to determine the tracking quality of the target tracking point based on the log error detection result and the data reporting time interval.

[0157] Figure 8FIG. 2 is a structural block diagram of a log error detection module 200 in an embodiment of the present disclosure. Figure 8 As shown, the log error detection module 200 includes:

[0158] The first detection unit 210 is used to perform metadata information verification error detection based on the log data to obtain the metadata information verification error detection result of the target buried point;

[0159] The second detection unit 220 is used to perform custom parameter verification error detection based on the log data to obtain a custom parameter verification error detection result of the target embedding point;

[0160] The log detection error result determining unit 230 is used to determine the log detection error result based on the metadata information verification error detection result and the custom parameter verification error detection result.

[0161] Fig. 9 FIG. 1 is a structural block diagram of a device for detecting the quality of a buried point in another embodiment of the present disclosure. Fig. 9 As shown, the buried point quality detection device also includes:

[0162] The associated buried point quality detection module 500 is used to determine the buried point quality of the associated buried points based on the numerical relationship between the data amounts of each buried point in the associated buried points.

[0163] In some embodiments of the present disclosure, the device for detecting the quality of buried points further includes:

[0164] A mapping relationship building module 600 is used to build a mapping relationship between the custom parameters of each tracking point and the fields of the business table;

[0165] The field reduction module 700 is used to reduce the fields of the tracking point log data of each tracking point based on the mapping relationship;

[0166] The storage module 800 is used to store the log data of each embedding point after field reduction.

[0167] Fig.10 FIG. 8 is a structural block diagram of a storage module 800 in one embodiment of the present disclosure. Fig.10 As shown, the storage module 800 includes:

[0168] A determination unit 810, configured to determine illegal data and legal data in the tracking point log data of each tracking point;

[0169] An illegal data processing unit 820, used to delete illegal data or write illegal data into an illegal data table;

[0170] The legal data processing unit 830 is used to write the legal data into the legal data table.

[0171] In some embodiments of the present disclosure, the illegal data processing unit 820 is used to:

[0172] In response to the type of the illegal data being the first type, deleting the illegal data;

[0173] In response to the type of the illegal data being the second type, the illegal data is written into the illegal data table.

[0174] In some embodiments of the present disclosure, the device for detecting the quality of buried points further includes:

[0175] The alarm module 900 is used to generate an alarm in response to the comprehensive evaluation result indicating that there is a quality problem with the target buried point.

[0176] It should be noted that the specific implementation of the burial point quality detection device of the embodiment of the present disclosure is similar to the specific implementation of the burial point quality detection method of the embodiment of the present disclosure, and the technical effect of the burial point quality detection device of the embodiment of the present disclosure is similar to the technical effect of the burial point quality detection method of the embodiment of the present disclosure. Please refer to the description of the burial point quality detection method part for details. In order to reduce redundancy, it will not be repeated.

[0177] In addition, an embodiment of the present disclosure further provides an electronic device, including:

[0178] Memory for storing computer programs;

[0179] The processor is used to execute the computer program stored in the memory, and when the computer program is executed, the method for detecting the quality of the buried points described in any of the above embodiments of the present disclosure is implemented.

[0180] Below, reference Fig.11 The electronic device according to the embodiment of the present disclosure is described. Fig.11 As shown, the electronic device includes one or more processors and memory.

[0181] The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0182] The memory may store one or more computer program products, and the memory may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program products may be stored on the computer-readable storage medium, and the processor may run the computer program product to implement the method for monitoring the quality of the burial point data of the various embodiments of the present disclosure described above and / or other desired functions.

[0183] In one example, the electronic device may further include: an input device and an output device, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0184] In addition, the input device may also include, for example, a keyboard, a mouse, and the like.

[0185] The output device can output various information to the outside, including determined distance information, direction information, etc. The output device can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0186] Of course, to simplify, Fig.11 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device may further include any other appropriate components.

[0187] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method for monitoring the quality of burial point data according to various embodiments of the present disclosure described in the above part of this specification.

[0188] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0189] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the method for monitoring the quality of burial point data according to various embodiments of the present disclosure described in the above part of this specification.

[0190] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0191] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limiting. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0192] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0193] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0194] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0195] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0196] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0197] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for detecting the quality of buried points, characterized in that: include: Get the log data of the target tracking point; Perform log error detection based on the log data to obtain a log error detection result of the target embedding point; Perform data reporting time interval detection based on the log data to obtain the data reporting time interval of the target tracking point; Based on the log error detection result and the data reporting time interval, the burying point quality of the target burying point is determined.

2. The method according to claim 1, characterized in that The performing log error detection based on the log data to obtain the log error detection result of the target embedding point includes: Perform metadata information verification error detection based on the log data to obtain the metadata information verification error detection result of the target embedding point; Perform custom parameter verification error detection based on the log data to obtain a custom parameter verification error detection result of the target embedding point; The log detection error result is obtained based on the metadata information verification error detection result and the custom parameter verification error detection result.

3. The method according to claim 1, characterized in that Also includes: Based on the numerical relationship between the data amounts of each buried point in the associated buried points, the buried point quality of the associated buried points is determined.

4. The method according to any one of claims 1 to 3, characterized in that: Also includes: Build a mapping relationship between the custom parameters of each tracking point and the fields of the business table; Based on the mapping relationship, field reduction is performed on the tracking point log data of each tracking point; The tracking point log data of each tracking point after the field reduction is stored.

5. The method according to any one of claims 1 to 3, characterized in that: Also includes: Determine the illegal and legal data in the tracking log data of each tracking point; Deleting the illegal data, or writing the illegal data into an illegal data table; The legal data is written into the legal data table.

6. The method according to claim 5, characterized in that The deleting the illegal data or writing the illegal data into an illegal data table includes: In response to the type of the illegal data being the first type, deleting the illegal data; In response to the type of the illegal data being the second type, the illegal data is written into an illegal data table.

7. The method according to any one of claims 1 to 3, characterized in that: After determining the burying point quality of the target burying point based on the log error detection result and the data reporting time interval, the method further includes: In response to the burial point quality of the target burial point indicating that the target burial point has quality problems, an alarm is issued.

8. A buried point quality detection device, characterized in that: include: The acquisition module is used to obtain the log data of the target embedding point; A log error detection module, used to perform log error detection based on the log data to obtain a log error detection result of the target embedding point; A time interval detection module, used to detect the data reporting time interval based on the log data to obtain the data reporting time interval of the target buried point; The quality detection module is used to determine the burying point quality of the target burying point based on the log error detection result and the data reporting time interval.

9. The device according to claim 8, characterized in that The log error detection module includes: A first detection unit is used to perform metadata information verification error detection based on the log data to obtain a metadata information verification error detection result of the target embedding point; A second detection unit is used to perform custom parameter verification error detection based on the log data to obtain a custom parameter verification error detection result of the target embedding point; The log detection error result determining unit is used to obtain the log detection error result based on the metadata information verification error detection result and the custom parameter verification error detection result.

10. The device according to claim 8, characterized in that Also includes: The associated buried point quality detection module is used to determine the buried point quality of the associated buried points based on the numerical relationship between the data amounts of each buried point in the associated buried points.

11. The device according to any one of claims 8 to 10, characterized in that: Also includes: The mapping relationship building module is used to build the mapping relationship between the custom parameters of each tracking point and the fields of the business table; A field reduction module, used for reducing fields of the tracking point log data of each tracking point based on the mapping relationship; The storage module is used to store the log data of each tracking point after field reduction.

12. An electronic device, characterized in that: include: a memory for storing a computer program product; A processor is used to execute the computer program product stored in the memory, and when the computer program product is executed, it implements the method described in any one of claims 1 to 7.

13. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method described in any one of claims 1 to 7 is implemented.

14. A computer program product, characterized in that The method comprises computer program instructions, which, when executed by a processor, enable the processor to execute the method according to any one of claims 1 to 7.

Images