Check method and device, electronic equipment and computer readable storage medium

Abstract

The method comprises the steps of: obtaining a data request of an object; analyzing the data request of the object to obtain to-be-verified data; inputting the to-be-verified data into a pre-trained verification model to predict a data request verification result of the object, wherein the verification model is pre-trained based on historical data request logs of the object. The technical solution is simple in operation, can not only save operation time and complexity and save labor input, but also effectively reduce the maintenance cost in the later period.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, specifically to a verification method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] With the development of data technology, remote data requests and transmissions are becoming increasingly frequent, and the amount of data transmitted is also increasing. When a requester requests data for a specific object, it usually needs to carry parameters related to the requested data. The backend server or service personnel need to determine whether the requested data parameters and the corresponding data are correct and valid. In existing technologies, fixed-value validation and regular expression validation methods are commonly used. Fixed-value validation refers to configuring one or more specific validation values ​​to validate the data when the requester requests it, but this method requires establishing all validation rules in advance, which is complex and time-consuming. Regular expression validation refers to using pre-configured regular expressions to validate the data when the requester requests it, but this method has high maintenance costs in the later stages, and the modification operation is complicated when the validation content needs to be changed. Therefore, there is an urgent need for a data validation solution that is simple to operate, saves operation time and complexity, and saves manpower. Summary of the Invention

[0003] This disclosure provides a verification method, apparatus, electronic device, and computer-readable storage medium.

[0004] Firstly, this disclosure provides a verification method.

[0005] Specifically, the verification method includes:

[0006] Data request for retrieving an object;

[0007] Parse the data request of the object to obtain the data to be verified;

[0008] The data to be verified is input into a pre-trained verification model to predict the data request verification result of the object, wherein the verification model is pre-trained based on the object's historical data request logs.

[0009] In conjunction with the first aspect, in a first implementation of the first aspect of this disclosure, parsing the data request of the object to obtain the data to be verified is implemented as follows:

[0010] The data requests for the object are parsed and cached hierarchically to obtain the data to be verified.

[0011] In conjunction with the first aspect and the first implementation of the first aspect, in the second implementation of the first aspect of this disclosure, the step of parsing and hierarchically caching the data request for the object to obtain the data to be verified is implemented as follows:

[0012] The data request for the object is parsed to obtain the valid data of the object's data request;

[0013] The valid data request data of the object is stored in the first-level cache, and the pre-stored first-level comparison data is retrieved from the first-level cache;

[0014] The valid data of the data request of the object is matched with the first comparison data. If there is data in the valid data of the data request of the object that corresponds to the first comparison data, the data corresponding to the first comparison data and its subordinate data are stored as candidate data to be verified in the secondary cache, and the pre-stored secondary comparison data is obtained from the secondary cache.

[0015] Match the candidate data to be verified with the secondary comparison data. If there is data in the candidate data to be verified that corresponds to the secondary comparison data, store the data corresponding to the secondary comparison data and its lower-level data as the updated candidate data to be verified in the lower-level cache.

[0016] Traverse the comparison data in each level of cache, and take the candidate data to be verified obtained after matching with the last level comparison data as the data to be verified.

[0017] In conjunction with the first aspect, the first implementation of the first aspect, and the second implementation of the first aspect, in the third implementation of the first aspect of this disclosure, before inputting the data to be verified into the pre-trained verification model to predict the data request verification result of the object, the method further includes:

[0018] A preset verification model is determined, and the preset verification model is trained offline using the historical data request logs of the object to obtain the verification model.

[0019] In conjunction with the first aspect, the first implementation of the first aspect, the second implementation of the first aspect, and the third implementation of the first aspect, in the fourth implementation of the first aspect of this disclosure, the step of using the historical data request logs of the object to perform offline training on the preset verification model to obtain the verification model is implemented as follows:

[0020] Request logs to retrieve historical data of an object using a message queue;

[0021] Filter out abnormal data from the historical data request logs.

[0022] The historical data request logs obtained after filtering abnormal data are parsed to obtain the data characteristics of different parameters in the historical data request logs.

[0023] The preset verification model is trained offline using the data features to obtain the verification model.

[0024] In conjunction with the first aspect, the first implementation of the first aspect, the second implementation of the first aspect, the third implementation of the first aspect, and the fourth implementation of the first aspect, this disclosure, in the fifth implementation of the first aspect, further includes, before performing abnormal data filtering on the historical data request log:

[0025] The historical data request logs are formatted and normalized based on preset format information.

[0026] In conjunction with the first aspect, the first implementation of the first aspect, the second implementation of the first aspect, the third implementation of the first aspect, the fourth implementation of the first aspect, and the fifth implementation of the first aspect, this disclosure, in the sixth implementation of the first aspect, further includes, before performing abnormal data filtering on the historical data request log:

[0027] Configure identification information for the historical data request logs for differentiated storage.

[0028] In conjunction with the first aspect, the first implementation of the first aspect, the second implementation of the first aspect, the third implementation of the first aspect, the fourth implementation of the first aspect, the fifth implementation of the first aspect, and the sixth implementation of the first aspect, in the seventh implementation of the first aspect of this disclosure, the step of using the data features to perform offline training on the preset verification model to obtain the verification model is implemented as follows:

[0029] The data features are used as input to the preset verification model, and the data corresponding to the data features are used as output to the preset verification model for offline training to obtain the verification model.

[0030] In conjunction with the first aspect, the first implementation of the first aspect, the second implementation of the first aspect, the third implementation of the first aspect, the fourth implementation of the first aspect, the fifth implementation of the first aspect, the sixth implementation of the first aspect, and the seventh implementation of the first aspect, in the eighth implementation of the first aspect, after obtaining the verification model by offline training of the preset verification model using the data features, the method further includes:

[0031] Obtain preset data verification requirement information, and adjust the verification model according to the preset data verification requirement information.

[0032] Secondly, this disclosure provides a verification method.

[0033] Specifically, the verification method includes:

[0034] Request advertising data from the advertising requester;

[0035] Parse the advertising data request from the advertising requester to obtain the data to be verified;

[0036] The data to be verified is input into a pre-trained verification model to predict the verification result of the advertising data request, wherein the verification model is obtained by pre-training based on the advertising historical data request logs.

[0037] Thirdly, this disclosure provides a verification method.

[0038] Specifically, the verification method includes:

[0039] The controller receives a data request from an object and forwards the object's data request to the online server;

[0040] The online server parses the data request of the object, obtains the data to be verified, inputs the data to be verified into a pre-trained verification model, predicts the data request verification result of the object, and feeds back the data request verification result of the object to the controller.

[0041] In conjunction with the third aspect, in the first implementation of the third aspect of this disclosure, the online server parses the data request of the object to obtain the data to be verified, which is implemented as follows:

[0042] The online server parses the data request of the object to obtain the valid data of the object's data request;

[0043] The valid data request data of the object is stored in the first-level cache, and the pre-stored first-level comparison data is retrieved from the first-level cache;

[0044] The valid data of the data request of the object is matched with the first comparison data. If there is data in the valid data of the data request of the object that corresponds to the first comparison data, the data corresponding to the first comparison data and its subordinate data are stored as candidate data to be verified in the secondary cache, and the pre-stored secondary comparison data is obtained from the secondary cache.

[0045] Match the candidate data to be verified with the secondary comparison data. If there is data in the candidate data to be verified that corresponds to the secondary comparison data, store the data corresponding to the secondary comparison data and its lower-level data as the updated candidate data to be verified in the lower-level cache.

[0046] Traverse the comparison data in each level of cache, and take the candidate data to be verified obtained after matching with the last level comparison data as the data to be verified.

[0047] In conjunction with the third aspect and the first implementation of the third aspect, the second implementation of the present disclosure further includes:

[0048] The offline server obtains the historical data request logs of the object, determines the preset verification model, and uses the historical data request logs to perform offline training on the preset verification model to obtain the verification model.

[0049] Combining the third aspect, the first implementation of the third aspect, and the second implementation of the third aspect, in the third implementation of the third aspect of this disclosure, the offline server obtains the historical data request logs of the object, determines a preset verification model, and uses the historical data request logs to perform offline training on the preset verification model to obtain the verification model, which is implemented as follows:

[0050] The offline server obtains the historical data request logs of the object through a message middleware.

[0051] Filter out abnormal data from the historical data request logs.

[0052] The historical data request logs obtained after filtering abnormal data are parsed to obtain the data characteristics of different parameters in the historical data request logs.

[0053] The preset verification model is trained offline using the data features to obtain the verification model.

[0054] In conjunction with the third aspect, the first implementation of the third aspect, the second implementation of the third aspect, and the third implementation of the third aspect, this disclosure, in the fourth implementation of the third aspect, further includes, before performing abnormal data filtering on the historical data request log:

[0055] The historical data request logs are formatted and normalized based on preset format information.

[0056] In conjunction with the third aspect, the first implementation of the third aspect, the second implementation of the third aspect, the third implementation of the third aspect, and the fourth implementation of the third aspect, this disclosure, in the fifth implementation of the third aspect, further includes, before performing abnormal data filtering on the historical data request log:

[0057] Configure identification information for the historical data request logs for differentiated storage.

[0058] In conjunction with the third aspect, the first implementation of the third aspect, the second implementation of the third aspect, the third implementation of the third aspect, the fourth implementation of the third aspect, and the fifth implementation of the third aspect, in the sixth implementation of the third aspect of this disclosure, the step of using the data features to perform offline training on the preset verification model to obtain the verification model is implemented as follows:

[0059] The data features are used as input to the preset verification model, and the data corresponding to the data features are used as output to the preset verification model for offline training to obtain the verification model.

[0060] In conjunction with the third aspect, the first implementation of the third aspect, the second implementation of the third aspect, the third implementation of the third aspect, the fourth implementation of the third aspect, the fifth implementation of the third aspect, and the sixth implementation of the third aspect, this disclosure further includes, in the seventh implementation of the third aspect:

[0061] The controller acquires preset data verification requirement information and sends the preset data verification requirement information to the offline server, so that the offline server adjusts the verification model according to the preset data verification requirement information.

[0062] Fourthly, this disclosure provides a verification device.

[0063] Specifically, the verification device includes:

[0064] The first acquisition module is configured to retrieve data requests from objects;

[0065] The first parsing module is configured to parse the data request of the object to obtain the data to be verified.

[0066] The first verification module is configured to input the data to be verified into a pre-trained verification model to predict the data request verification result of the object, wherein the verification model is pre-trained based on the object's historical data request logs.

[0067] Fifthly, this disclosure provides a verification device.

[0068] Specifically, the verification device includes:

[0069] The second acquisition module is configured to acquire advertising data requests from the advertising requester.

[0070] The second parsing module is configured to parse the advertising data request from the advertising requester to obtain the data to be verified.

[0071] The second verification module is configured to input the data to be verified into a pre-trained verification model for prediction, and obtain the verification result of the advertising data request, wherein the verification model is pre-trained based on the advertising historical data request log.

[0072] Sixthly, this disclosure provides a verification system.

[0073] Specifically, the verification system includes:

[0074] The controller is configured to receive data requests from an object and forward those requests to an online server.

[0075] An online server is configured to parse the data request of the object, obtain the data to be verified, input the data to be verified into a pre-trained verification model, predict the data request verification result of the object, and feed back the data request verification result of the object to the controller.

[0076] In conjunction with the sixth aspect, the first implementation of the present disclosure further includes:

[0077] An offline server is configured to obtain historical data request logs of an object, determine a preset verification model, and use the historical data request logs to perform offline training on the preset verification model to obtain the verification model.

[0078] In a seventh aspect, embodiments of this disclosure provide an electronic device including a memory and a processor. The memory stores one or more computer instructions that support a verification device in executing the aforementioned verification method. The processor is configured to execute the computer instructions stored in the memory. The verification device may further include a communication interface for communicating with other devices or communication networks.

[0079] Eighthly, embodiments of this disclosure provide a computer-readable storage medium for storing computer instructions used by a verification device, which includes computer instructions for performing the verification method described above in connection with the verification device.

[0080] The technical solutions provided in this disclosure may have the following beneficial effects:

[0081] The above technical solution trains a verification model offline and uses this model to perform real-time verification of the data to be verified obtained from parsing online object data requests. This technical solution is simple to operate, saves on operation time and complexity, reduces manpower investment, and effectively reduces subsequent maintenance costs.

[0082] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the embodiments of this disclosure. Attached Figure Description

[0083] Other features, objects, and advantages of embodiments of this disclosure will become more apparent from the following detailed description of non-limiting implementations, taken in conjunction with the accompanying drawings. In the drawings:

[0084] Figure 1 A flowchart illustrating a verification method according to an embodiment of the present disclosure is shown;

[0085] Figure 2 A flowchart illustrating the overall process of a verification method according to an embodiment of the present disclosure is shown.

[0086] Figure 3 A system timing diagram of a verification method according to an embodiment of the present disclosure is shown;

[0087] Figure 4 A flowchart illustrating a verification method according to another embodiment of this disclosure is shown;

[0088] Figure 5 A flowchart illustrating a verification method according to another embodiment of the present disclosure is shown;

[0089] Figure 6 A structural block diagram of a verification apparatus according to an embodiment of the present disclosure is shown;

[0090] Figure 7 A structural block diagram of a verification apparatus according to another embodiment of the present disclosure is shown;

[0091] Figure 8 A structural block diagram of a verification apparatus according to another embodiment of the present disclosure is shown;

[0092] Figure 9 This is a schematic diagram of the structure of a computer system suitable for implementing the verification method according to an embodiment of the present disclosure. Detailed Implementation

[0093] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings to enable those skilled in the art to readily implement them. Furthermore, for clarity, portions unrelated to the description of the exemplary embodiments have been omitted from the drawings.

[0094] In embodiments disclosed herein, it should be understood that terms such as “comprising” or “having” are intended to indicate the presence of features, numbers, steps, behaviors, components, portions or combinations thereof disclosed herein, and are not intended to exclude the possibility that one or more other features, numbers, steps, behaviors, components, portions or combinations thereof are present or added.

[0095] It should also be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings and examples.

[0096] The technical solution provided in this disclosure uses an offline trained verification model to perform real-time verification of the data to be verified obtained from parsing online object data requests. This technical solution is simple to operate, saves operation time and complexity, reduces manpower investment, and effectively reduces subsequent maintenance costs.

[0097] Figure 1 A flowchart illustrating a verification method according to an embodiment of the present disclosure is shown, as follows: Figure 1 As shown, the verification method includes the following steps S101-S103:

[0098] In step S101, the data request for the object is obtained;

[0099] In step S102, the data request of the object is parsed to obtain the data to be verified;

[0100] In step S103, the data to be verified is input into a pre-trained verification model to predict the data request verification result of the object, wherein the verification model is obtained by pre-training based on the object's historical data request logs.

[0101] As mentioned above, with the development of data technology, remote data requests and transmissions are becoming increasingly frequent, and the amount of data transmitted is also increasing. When a requester requests data for a specific object, it usually needs to carry parameters related to the requested data. The backend server or service personnel need to determine whether the requested data parameters and the corresponding data are correct and valid. In existing technologies, fixed-value validation and regular expression validation methods are commonly used. Fixed-value validation refers to configuring one or more specific validation values ​​to validate the data when the requester requests it, but this method requires establishing all validation rules in advance, which is complex and time-consuming. Regular expression validation refers to using pre-configured regular expressions to validate the data when the requester requests it, but this method has high maintenance costs in the later stages, and the modification operation is relatively complex when the validation content needs to be changed. Therefore, there is an urgent need for a data validation solution that is simple to operate, saves operation time and complexity, and saves manpower.

[0102] In view of the above problems, this embodiment proposes a verification method. This method trains a verification model offline and uses the model to perform real-time verification of the data to be verified obtained from parsing online object data requests. This technical solution is simple to operate, saves operation time and complexity, reduces manpower investment, and effectively reduces subsequent maintenance costs.

[0103] In one embodiment of this disclosure, the verification method is applicable to computers, computing devices, electronic devices, servers, service clusters, etc., that can perform verification processing.

[0104] In one embodiment of this disclosure, the object refers to an object that contains certain related data, can be requested, can be transmitted, and can be displayed, such as a prompt or explanation of a certain thing or a certain type of thing, or a promotion or advertisement for a certain item or a certain type of item, etc.

[0105] In one embodiment of this disclosure, the object data request refers to a request issued by an object data requester to obtain data of a certain object. The object data requester can be a user, an application, or a device. The object data request may include one or more of the following parameters: applicable application environment information, object data loading location information in the object data requester, object data requester geographical location information, object data requester domain name information, object data request parameters, and their corresponding data. The application environment information of the object is used to characterize the current application environment of the object, such as the operating system version used or applicable to the object. The object data loading location information in the object data requester refers to the location where the object data requester loads the required object data after requesting it. For example, if the object is an advertisement and the object data requester is an application, then the object data loading location information in the object data requester refers to the location where the application loads or displays the advertisement. The domain name information of the object data requester refers to the domain name information of the object data requester, such as Sohu, Sina, Taobao, etc. The object data request parameters and their corresponding data refer to the names, number, types, and numerical ranges of the parameters included in the object data request.

[0106] In one embodiment of this disclosure, the object data request log refers to a log formed based on the object data requests issued by the object data requester. The object data request log may include one or more of the following information: object data request time, object data requester version information, object data requester operating system version information, object data requester application environment information, object data request parameter information, etc. The object data request parameter information may include the name, quantity, type, and corresponding numerical range of the object data request parameters, etc.

[0107] In one embodiment of this disclosure, the object's data request can be sent by the object data requester to the object server, and the object's data request log can be obtained by a logging component requesting it from the object server. In another embodiment of this disclosure, the object's data request log can also be obtained using a pre-configured proxy component.

[0108] In one embodiment of this disclosure, the data to be verified refers to data existing in the data request of the object that needs to be verified to determine whether the data is correct or valid. The data to be verified may be domain name information in the data request of the object, one or more parameter types carried in the data request of the object, or the numerical value corresponding to one or more parameters carried in the data request of the object.

[0109] In one embodiment of this disclosure, the verification model refers to a model used to determine whether the data to be verified is correct or valid. The verification model is obtained through offline training based on the object's historical data request logs. The verification model may include what content a specific piece of data should contain, what type a parameter should be, or what range the value of a parameter should be within, etc.

[0110] In one embodiment of this disclosure, the data request verification result of the object refers to the result obtained after verifying the data to be verified using the verification model, such as whether a certain data to be verified is correct or incorrect, valid or invalid, or whether its value conforms to or does not conform to a preset numerical range, etc.

[0111] In one embodiment of this disclosure, the verification result will be fed back in real time after it is obtained. For example, it can be fed back to the feedback recipient or the proxy component through a web page, table, document, message, etc. The feedback recipient can be the object data requester, the object data request tester or the object data request test device, or any other party that needs to receive the data request verification result.

[0112] In one embodiment of this disclosure, step S102, namely, parsing the data request of the object to obtain the data to be verified, can be implemented as follows:

[0113] The data requests for the object are parsed and cached hierarchically to obtain the data to be verified.

[0114] Considering that the number of data requests for an object may be large at a certain time or within a certain time period, resulting in a large amount of data processing, in this embodiment, when parsing the data requests for the object and obtaining the data to be verified, a parsing and hierarchical caching method is used to obtain the data to be verified for the object's data requests, so as to reduce the data processing pressure, reduce the data processing latency, and enhance the real-time performance of data processing.

[0115] In one embodiment of this disclosure, the step of parsing and hierarchically caching the data request for the object to obtain the data to be verified can be implemented as follows:

[0116] The data request for the object is parsed to obtain the valid data of the object's data request;

[0117] The valid data request data of the object is stored in the first-level cache, and the pre-stored first-level comparison data is retrieved from the first-level cache;

[0118] The valid data of the data request of the object is matched with the first comparison data. If there is data in the valid data of the data request of the object that corresponds to the first comparison data, the data corresponding to the first comparison data and its subordinate data are stored as candidate data to be verified in the secondary cache, and the pre-stored secondary comparison data is obtained from the secondary cache.

[0119] Match the candidate data to be verified with the secondary comparison data. If there is data in the candidate data to be verified that corresponds to the secondary comparison data, store the data corresponding to the secondary comparison data and its lower-level data as the updated candidate data to be verified in the lower-level cache.

[0120] Traverse the comparison data in each level of cache, and take the candidate data to be verified obtained after matching with the last level comparison data as the data to be verified.

[0121] To alleviate data processing pressure, reduce data processing latency, and enhance the real-time performance of data processing, while considering that not all object data requests will include data requiring verification, this implementation employs parsing and hierarchical caching when acquiring data to be verified. Specifically, firstly, the object's data request is parsed to remove invalid data, obtaining the valid data of the object's data request, such as the requested destination domain name data; then, a multi-level cache space is pre-set to hierarchically store the parsed valid data. This multi-level cache space pre-stores preset multi-level comparison data, which is used to compare with the valid data of the object's data request to determine whether the object's data request contains data that needs subsequent verification; then, the valid data of the object's data request that may include data to be verified is stored in the first-level cache. The pre-stored first-level comparison data is retrieved from the first-level cache and matched against the valid data of the object's data request to determine whether the valid data of the object's data request contains data corresponding to the first-level comparison data. If the valid data in the object's data request contains data corresponding to the primary comparison data, then the valid data in the object's data request is considered to contain data that needs to be verified or further determined to decide whether verification is necessary. The data corresponding to the primary comparison data and its subordinate data are stored as candidate data to be verified in the secondary cache. Pre-stored secondary comparison data is retrieved from the secondary cache and compared with the candidate data to be verified. If the candidate data to be verified contains data corresponding to the secondary comparison data, similarly, the data corresponding to the secondary comparison data and its subordinate data are stored as updated candidate data to be verified in the lower-level cache. This process continues until all comparison data in each level of cache has been traversed. Finally, the latest candidate data to be verified, obtained after matching with the final comparison data, is used as the data to be verified.

[0122] For example, assuming the cache space has two levels, the primary comparison data is preset domain name data, and the valid data of the object's data request contains domain name data corresponding to the preset domain name data, such as domain name data consistent with the preset domain name data: domain name 1 and domain name 2, then the data in the valid data of the object's data request that corresponds to the preset domain name data and its subordinate data domain name 1: parameter 1 and parameter 2, domain name 2: parameter 4 are stored as candidate data to be verified in the secondary cache. The pre-stored secondary comparison data is obtained from the secondary cache. Assuming the secondary comparison data is domain name 1: parameter 1 and parameter 2, domain name 2: parameter 3, it is obvious that the candidate data to be verified includes domain name 1 and its subordinate data parameter 1 and parameter 2. The subordinate data of domain name 2 is inconsistent with the secondary comparison data. At this time, domain name 1: parameter 1 and parameter 2 are used as the final data to be verified.

[0123] In another embodiment of this disclosure, the step of parsing the data request for the object to obtain the valid data of the object's data request can also be performed after the hierarchical caching step. That is, the step of parsing and hierarchically caching the data request for the object to obtain the data to be verified can be implemented as follows:

[0124] The data request for the object is stored in the first-level cache, and the pre-stored first-level comparison data is retrieved from the first-level cache;

[0125] The data request of the object is matched with the first-level comparison data. If the data request of the object contains data corresponding to the first-level comparison data, the data corresponding to the first-level comparison data and its subordinate data are stored as candidate data to be verified in the secondary cache, and the pre-stored secondary comparison data is obtained from the secondary cache.

[0126] Match the candidate data to be verified with the secondary comparison data. If there is data in the candidate data to be verified that corresponds to the secondary comparison data, store the data corresponding to the secondary comparison data and its lower-level data as the updated candidate data to be verified in the lower-level cache.

[0127] Traverse the comparison data in each level of cache and obtain the candidate data to be verified after matching with the last level comparison data;

[0128] The candidate data to be verified is parsed, and the valid data obtained from the candidate data to be verified is used as the data to be verified.

[0129] In one embodiment of this disclosure, before step S103, which involves inputting the data to be verified into a pre-trained verification model to predict the data request verification result of the object, the following steps are further included:

[0130] A preset verification model is determined, and the preset verification model is trained offline using the historical data request logs of the object to obtain the verification model.

[0131] In order to save online data processing resources and improve online data processing speed, in this embodiment, a preset verification model is first determined, and then the preset verification model is trained offline based on the historical data request logs of objects obtained within a preset historical time period to obtain the verification model.

[0132] In one embodiment of this disclosure, the step of offline training of the preset verification model using the historical data request logs of the object to obtain the verification model can be implemented as follows:

[0133] Request logs to retrieve historical data of an object using a message queue;

[0134] Filter out abnormal data from the historical data request logs.

[0135] The historical data request logs obtained after filtering abnormal data are parsed to obtain the data characteristics of different parameters in the historical data request logs.

[0136] The preset verification model is trained offline using the data features to obtain the verification model.

[0137] In this implementation:

[0138] First, historical data request logs of objects within a preset historical time period are obtained through a message middleware. After the historical data request logs are obtained, they can be persisted. The preset historical time period can be set according to the needs of the actual application. The message middleware can be a high-capacity asynchronous streaming log message middleware available on the technology market. The historical data request logs are log data corresponding to the historical data requests of objects. The historical data request refers to a request issued by the historical data requester to obtain data of a certain object. Similar to the data requester, the historical data requester can be a user, an application, or a device.

[0139] Then, abnormal data in the historical data request log is filtered out. Abnormal data refers to non-standard or invalid data, such as noisy data, test data entered by testers that is unrelated to the normal data acquisition request log, etc. Existing abnormal data filtering methods can be used for filtering abnormal data, which will not be elaborated upon in this disclosure.

[0140] Then, the historical data request logs obtained after filtering for abnormal data are parsed to obtain the data characteristics of different parameters in the historical data request logs. These data characteristics may include one or more of the following: parameter type, the data type corresponding to the parameter, the range of data values ​​corresponding to the parameter, the correlation between parameters, and the correlation between the data corresponding to parameters. For example, is a parameter a system-level parameter or an application-level parameter? Is the data type corresponding to a parameter long integer or short integer? What is the range of data values ​​corresponding to a parameter? Does a correlation exist between two or more parameters, and what kind of correlation exists? Does a parameter have a correlation in different application environments? For example, the value of a parameter should be A in operating system 1, while the value should be B in operating system 2, which means that the parameter has a correlation in different application environments. Does a correlation exist between the data corresponding to two or more parameters, and what kind of correlation exists? And so on.

[0141] Finally, the data features obtained from the analysis are used to train the preset verification model offline, thus obtaining the verification model.

[0142] In one embodiment of this disclosure, before the step of filtering abnormal data from the historical data request log, the following steps are further included:

[0143] The historical data request logs are formatted and normalized based on preset format information.

[0144] To improve data processing efficiency, in this embodiment, the historical data request log is also normalized based on preset format information so that the historical data request log has a unified data structure consistent with the preset format information. The preset format information can be preset according to the needs of actual application and the specific data to be processed.

[0145] In one embodiment of this disclosure, before the step of filtering abnormal data from the historical data request log, the following steps are further included:

[0146] Configure identification information for the historical data request logs for differentiated storage.

[0147] The identification information refers to information used to distinguish the historical data request logs, such as ID information. The configuration of the identification information enables the historical data request logs to be stored in a distinguishable and orderly manner, and also facilitates the tracing of the historical data request logs.

[0148] In one embodiment of this disclosure, the step of offline training of the preset verification model using the data features to obtain the verification model can be implemented as follows:

[0149] The data features are used as input to the preset verification model, and the data corresponding to the data features are used as output to the preset verification model for offline training to obtain the verification model.

[0150] After obtaining the data features that can be used as training data, an available preset verification model can be determined, such as a decision tree model. Then, the data features are used as the input of the preset verification model, and the data corresponding to the data features, such as parameters, are used as the output of the preset verification model for offline training to obtain the verification model.

[0151] In one embodiment of this disclosure, after offline training of the preset verification model using the data features to obtain the verification model, the method further includes:

[0152] Obtain preset data verification requirement information, and adjust the verification model according to the preset data verification requirement information.

[0153] Considering that in practical applications, data requesters or other relevant parties may have explicit and specific requirements for the verification of data request logs, which are more effective and targeted for verification, this embodiment needs to simultaneously consider the verification requirements provided by the data requester or other relevant parties, i.e., preset data verification requirement information, to improve the verification model and form an information closed loop of data learning, data verification, requirement feedback, and data relearning. In other words, if the preset data verification requirement information is received, the verification model needs to be adjusted according to it, for example, by modifying or adjusting the training data of the verification model. The preset data verification requirement information may include one or more of the following: preset verification parameters, the type of the preset verification parameters, the corresponding values ​​of the preset verification parameters, preset verification data, the type of the preset verification data, the value of the preset verification data, etc. This allows the training data of the verification model to be modified or adjusted according to the specific content of the preset data verification requirement information.

[0154] In one embodiment of this disclosure, the preset data verification requirement information can be obtained from a preset database. That is, the data requester or other relevant party can first store the preset data verification requirement information in a pre-specified or pre-set database, and then the verification server can obtain it from the preset database.

[0155] Figure 2 A flowchart illustrating the overall process of a verification method according to an embodiment of this disclosure is shown. Figure 2 As shown, in the online process, multiple data requesters issue data requests for objects online. These requests are parsed and cached hierarchically to obtain data to be verified. Then, a verification model trained offline is used to predict the verification result of the data request. In the offline process, historical data request logs from one or more data requesters are obtained. These logs are then format-normalized, and corresponding identification information is configured for each normalized log. Abnormal data in the logs is filtered out, and the filtered logs are parsed to obtain data features of different parameters. These features are used to train a preset verification model offline, resulting in the verification model, which is then delivered to the online service. Furthermore, preset data verification requirements sent by relevant parties are considered during the verification model training to modify or adjust the model.

[0156] Figure 3 A system timing diagram of a verification method according to an embodiment of the present disclosure is shown. Figure 3 As shown, in the offline process, the historical data requester sends a historical data request to the verification server. The log component requests the corresponding historical data request log from the verification server. The verification server performs persistence processing, format normalization processing, abnormal data filtering, and parsing on the historical data request log to obtain data features and trains it offline to obtain a verification model, which is then sent to the online service. In addition, the preset data verification requirements sent by relevant parties, such as the data requester, are considered during the offline training of the verification model. In the online service, the verification server uses a proxy component to obtain the data request log corresponding to the data request sent by the data requester. The verification server parses and caches the data request log in a hierarchical manner to obtain the data to be verified. Then, it uses the verification model trained offline to predict the data to be verified, obtains the data request verification result, and feeds it back to the proxy component or the feedback receiver.

[0157] In one embodiment of this disclosure, the verification method can be used to verify advertising data requests, where the object is an advertisement, such as... Figure 4 As shown, when validating an advertising data request, the validation method includes the following steps S401-S403:

[0158] In step S401, the advertising data request from the advertising requesting end is obtained;

[0159] In step S402, the advertising data request from the advertising requesting end is parsed to obtain the data to be verified;

[0160] In step S403, the data to be verified is input into a pre-trained verification model to predict the verification result of the advertising data request, wherein the verification model is obtained by pre-training based on the advertising historical data request log.

[0161] The verification method employs an offline-trained verification model, which is then used to perform real-time verification of the data to be verified obtained from parsing online advertising data requests. This technical solution is simple to operate, saves on operation time and complexity, reduces manpower investment, and effectively lowers subsequent maintenance costs.

[0162] In one embodiment of this disclosure, the verification method can be applied to computers, computing devices, electronic devices, servers, service clusters, etc., that can perform advertising data request verification processing.

[0163] Figure 4 The technical terms and technical features involved in the illustrated and related embodiments are consistent with Figure 1-3 The technical terms and technical features mentioned in the illustrated and related embodiments are the same or similar, for Figure 4 The explanations and descriptions of the technical terms and features involved in the illustrated and related embodiments can be found in the above-mentioned... Figure 1-3 The explanations of the illustrated and related embodiments are not repeated here.

[0164] Figure 5 A flowchart illustrating a verification method according to another embodiment of the present disclosure is shown, as follows: Figure 5 As shown, the verification method includes the following steps S501-S502:

[0165] In step S501, the controller receives the data request from the object and forwards the data request to the online server;

[0166] In step S502, the online server parses the data request of the object to obtain the data to be verified, inputs the data to be verified into a pre-trained verification model, predicts the data request verification result of the object, and feeds back the data request verification result of the object to the controller.

[0167] As mentioned above, with the development of data technology, remote data requests and transmissions are becoming increasingly frequent, and the amount of data transmitted is also increasing. When a requester requests data for a specific object, it usually needs to carry parameters related to the requested data. The backend server or service personnel need to determine whether the requested data parameters and the corresponding data are correct and valid. In existing technologies, fixed-value validation and regular expression validation methods are commonly used. Fixed-value validation refers to configuring one or more specific validation values ​​to validate the data when the requester requests it, but this method requires establishing all validation rules in advance, which is complex and time-consuming. Regular expression validation refers to using pre-configured regular expressions to validate the data when the requester requests it, but this method has high maintenance costs in the later stages, and the modification operation is relatively complex when the validation content needs to be changed. Therefore, there is an urgent need for a data validation solution that is simple to operate, saves operation time and complexity, and saves manpower.

[0168] In view of the above problems, this embodiment proposes a verification method. This method trains a verification model offline and uses the model to perform real-time verification of the data to be verified obtained from parsing online object data requests. This technical solution is simple to operate, saves operation time and complexity, reduces manpower investment, and effectively reduces subsequent maintenance costs.

[0169] In one embodiment of this disclosure, the verification method is applicable to a verification system that includes a controller and an online server.

[0170] In one embodiment of this disclosure, the step of the online server parsing the data request of the object to obtain the data to be verified can be implemented as follows:

[0171] The online server parses the data request of the object to obtain the valid data of the object's data request;

[0172] The valid data request data of the object is stored in the first-level cache, and the pre-stored first-level comparison data is retrieved from the first-level cache;

[0173] The valid data of the data request of the object is matched with the first comparison data. If there is data in the valid data of the data request of the object that corresponds to the first comparison data, the data corresponding to the first comparison data and its subordinate data are stored as candidate data to be verified in the secondary cache, and the pre-stored secondary comparison data is obtained from the secondary cache.

[0174] Match the candidate data to be verified with the secondary comparison data. If there is data in the candidate data to be verified that corresponds to the secondary comparison data, store the data corresponding to the secondary comparison data and its lower-level data as the updated candidate data to be verified in the lower-level cache.

[0175] Traverse the comparison data in each level of cache, and take the candidate data to be verified obtained after matching with the last level comparison data as the data to be verified.

[0176] In one embodiment of this disclosure, it further includes:

[0177] The offline server obtains the historical data request logs of the object, determines the preset verification model, and uses the historical data request logs to perform offline training on the preset verification model to obtain the verification model.

[0178] In one embodiment of this disclosure, the step of obtaining historical data request logs of an object by an offline server, determining a preset verification model, and using the historical data request logs to perform offline training on the preset verification model to obtain the verification model can be implemented as follows:

[0179] The offline server obtains the historical data request logs of the object through a message middleware.

[0180] Filter out abnormal data from the historical data request logs.

[0181] The historical data request logs obtained after filtering abnormal data are parsed to obtain the data characteristics of different parameters in the historical data request logs.

[0182] The preset verification model is trained offline using the data features to obtain the verification model.

[0183] In one embodiment of this disclosure, before performing abnormal data filtering on the historical data request log, the method further includes:

[0184] The historical data request logs are formatted and normalized based on preset format information.

[0185] In one embodiment of this disclosure, before performing abnormal data filtering on the historical data request log, the method further includes:

[0186] Configure identification information for the historical data request logs for differentiated storage.

[0187] In one embodiment of this disclosure, the step of offline training of the preset verification model using the data features to obtain the verification model can be implemented as follows:

[0188] The data features are used as input to the preset verification model, and the data corresponding to the data features are used as output to the preset verification model for offline training to obtain the verification model.

[0189] In one embodiment of this disclosure, it further includes:

[0190] The controller acquires preset data verification requirement information and sends the preset data verification requirement information to the offline server, so that the offline server adjusts the verification model according to the preset data verification requirement information.

[0191] Figure 5 The technical terms and technical features involved in the illustrated and related embodiments are consistent with Figure 1-4 The technical terms and technical features mentioned in the illustrated and related embodiments are the same or similar, for Figure 5 The explanations and descriptions of the technical terms and features involved in the illustrated and related embodiments can be found in the above-mentioned... Figure 1-4 The explanations of the illustrated and related embodiments are not repeated here.

[0192] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein.

[0193] Figure 6 The diagram shows a structural block diagram of a verification device according to an embodiment of the present disclosure. This device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. Figure 6 As shown, the verification device includes:

[0194] The first acquisition module 601 is configured to acquire data requests from objects;

[0195] The first parsing module 602 is configured to parse the data request of the object to obtain the data to be verified.

[0196] The first verification module 603 is configured to input the data to be verified into a pre-trained verification model to predict the data request verification result of the object, wherein the verification model is obtained by pre-training based on the object's historical data request logs.

[0197] As mentioned above, with the development of data technology, remote data requests and transmissions are becoming increasingly frequent, and the amount of data transmitted is also increasing. When a requester requests data for a specific object, it usually needs to carry parameters related to the requested data. The backend server or service personnel need to determine whether the requested data parameters and the corresponding data are correct and valid. In existing technologies, fixed-value validation and regular expression validation methods are commonly used. Fixed-value validation refers to configuring one or more specific validation values ​​to validate the data when the requester requests it, but this method requires establishing all validation rules in advance, which is complex and time-consuming. Regular expression validation refers to using pre-configured regular expressions to validate the data when the requester requests it, but this method has high maintenance costs in the later stages, and the modification operation is relatively complex when the validation content needs to be changed. Therefore, there is an urgent need for a data validation solution that is simple to operate, saves operation time and complexity, and saves manpower.

[0198] In view of the above problems, this embodiment proposes a verification device. This device trains a verification model offline and uses the model to perform real-time verification of the data to be verified obtained from parsing online object data requests. This technical solution is simple to operate, saves operation time and complexity, reduces manpower investment, and effectively reduces subsequent maintenance costs.

[0199] In one embodiment of this disclosure, the verification device can be implemented as a computer, computing device, electronic device, server, service cluster, etc., capable of performing verification processing.

[0200] In one embodiment of this disclosure, the object refers to an object that contains certain related data, can be requested, can be transmitted, and can be displayed, such as a prompt or explanation of a certain thing or a certain type of thing, or a promotion or advertisement for a certain item or a certain type of item, etc.

[0201] In one embodiment of this disclosure, the object data request refers to a request issued by an object data requester to obtain data of a certain object. The object data requester can be a user, an application, or a device. The object data request may include one or more of the following parameters: applicable application environment information, object data loading location information in the object data requester, object data requester geographical location information, object data requester domain name information, object data request parameters, and their corresponding data. The application environment information of the object is used to characterize the current application environment of the object, such as the operating system version used or applicable to the object. The object data loading location information in the object data requester refers to the location where the object data requester loads the required object data after requesting it. For example, if the object is an advertisement and the object data requester is an application, then the object data loading location information in the object data requester refers to the location where the application loads or displays the advertisement. The domain name information of the object data requester refers to the domain name information of the object data requester, such as Sohu, Sina, Taobao, etc. The object data request parameters and their corresponding data refer to the names, number, types, and numerical ranges of the parameters included in the object data request.

[0202] In one embodiment of this disclosure, the object data request log refers to a log formed based on the object data requests issued by the object data requester. The object data request log may include one or more of the following information: object data request time, object data requester version information, object data requester operating system version information, object data requester application environment information, object data request parameter information, etc. The object data request parameter information may include the name, quantity, type, and corresponding numerical range of the object data request parameters, etc.

[0203] In one embodiment of this disclosure, the object's data request can be sent by the object data requester to the object server, and the object's data request log can be obtained by a logging component requesting it from the object server. In another embodiment of this disclosure, the object's data request log can also be obtained using a pre-configured proxy component.

[0204] In one embodiment of this disclosure, the data to be verified refers to data existing in the data request of the object that needs to be verified to determine whether the data is correct or valid. The data to be verified may be domain name information in the data request of the object, one or more parameter types carried in the data request of the object, or the numerical value corresponding to one or more parameters carried in the data request of the object.

[0205] In one embodiment of this disclosure, the verification model refers to a model used to determine whether the data to be verified is correct or valid. The verification model is obtained through offline training based on the object's historical data request logs. The verification model may include what content a specific piece of data should contain, what type a parameter should be, or what range the value of a parameter should be within, etc.

[0206] In one embodiment of this disclosure, the data request verification result of the object refers to the result obtained after verifying the data to be verified using the verification model, such as whether a certain data to be verified is correct or incorrect, valid or invalid, or whether its value conforms to or does not conform to a preset numerical range, etc.

[0207] In one embodiment of this disclosure, the verification result will be fed back in real time after it is obtained. For example, it can be fed back to the feedback recipient or the proxy component through a web page, table, document, message, etc. The feedback recipient can be the object data requester, the object data request tester or the object data request test device, or any other party that needs to receive the data request verification result.

[0208] In one embodiment of this disclosure, the first parsing module 602 may be configured as follows:

[0209] The data requests for the object are parsed and cached hierarchically to obtain the data to be verified.

[0210] Considering that the number of data requests for an object may be large at a certain time or within a certain time period, resulting in a large amount of data processing, in this embodiment, when parsing the data requests for the object and obtaining the data to be verified, a parsing and hierarchical caching method is used to obtain the data to be verified for the object's data requests, so as to reduce the data processing pressure, reduce the data processing latency, and enhance the real-time performance of data processing.

[0211] In one embodiment of this disclosure, the step of parsing and hierarchically caching the data request for the object to obtain the portion of the data to be verified can be configured as follows:

[0212] The data request for the object is parsed to obtain the valid data of the object's data request;

[0213] The valid data request data of the object is stored in the first-level cache, and the pre-stored first-level comparison data is retrieved from the first-level cache;

[0214] The valid data of the data request of the object is matched with the first comparison data. If there is data in the valid data of the data request of the object that corresponds to the first comparison data, the data corresponding to the first comparison data and its subordinate data are stored as candidate data to be verified in the secondary cache, and the pre-stored secondary comparison data is obtained from the secondary cache.

[0215] Match candidate data to be verified with the secondary comparison data. If there is data in the candidate data to be verified that corresponds to the secondary comparison data, store the data corresponding to the secondary comparison data and its lower-level data as the updated candidate data to be verified in the lower-level cache.

[0216] Traverse the comparison data in each level of cache, and take the candidate data to be verified obtained after matching with the last level comparison data as the data to be verified.

[0217] To alleviate data processing pressure, reduce data processing latency, and enhance the real-time performance of data processing, while considering that not all object data requests will include data requiring verification, this implementation employs parsing and hierarchical caching when acquiring data to be verified. Specifically, firstly, the object's data request is parsed to remove invalid data, obtaining the valid data of the object's data request, such as the requested destination domain name data; then, a multi-level cache space is pre-set to hierarchically store the parsed valid data. This multi-level cache space pre-stores preset multi-level comparison data, which is used to compare with the valid data of the object's data request to determine whether the object's data request contains data that needs subsequent verification; then, the valid data of the object's data request that may include data to be verified is stored in the first-level cache. The pre-stored first-level comparison data is retrieved from the first-level cache and matched against the valid data of the object's data request to determine whether the valid data of the object's data request contains data corresponding to the first-level comparison data. If the valid data in the object's data request contains data corresponding to the primary comparison data, then the valid data in the object's data request is considered to contain data that needs to be verified or further determined to decide whether verification is necessary. The data corresponding to the primary comparison data and its subordinate data are stored as candidate data to be verified in the secondary cache. Pre-stored secondary comparison data is retrieved from the secondary cache and compared with the candidate data to be verified. If the candidate data to be verified contains data corresponding to the secondary comparison data, similarly, the data corresponding to the secondary comparison data and its subordinate data are stored as updated candidate data to be verified in the lower-level cache. This process continues until all comparison data in each level of cache has been traversed. Finally, the latest candidate data to be verified, obtained after matching with the final comparison data, is used as the data to be verified.

[0218] For example, assuming the cache space has two levels, the primary comparison data is preset domain name data, and the valid data of the object's data request contains domain name data corresponding to the preset domain name data, such as domain name data consistent with the preset domain name data: domain name 1 and domain name 2, then the data in the valid data of the object's data request that corresponds to the preset domain name data and its subordinate data domain name 1: parameter 1 and parameter 2, domain name 2: parameter 4 are stored as candidate data to be verified in the secondary cache. The pre-stored secondary comparison data is obtained from the secondary cache. Assuming the secondary comparison data is domain name 1: parameter 1 and parameter 2, domain name 2: parameter 3, it is obvious that the candidate data to be verified includes domain name 1 and its subordinate data parameter 1 and parameter 2. The subordinate data of domain name 2 is inconsistent with the secondary comparison data. At this time, domain name 1: parameter 1 and parameter 2 are used as the final data to be verified.

[0219] In another embodiment of this disclosure, the step of parsing the data request for the object to obtain the valid data of the object's data request can also be configured to be performed after the hierarchical caching step. That is, the step of parsing and hierarchically caching the data request for the object to obtain the data to be verified can be configured as follows:

[0220] The data request for the object is stored in the first-level cache, and the pre-stored first-level comparison data is retrieved from the first-level cache;

[0221] The data request of the object is matched with the first-level comparison data. If the data request of the object contains data corresponding to the first-level comparison data, the data corresponding to the first-level comparison data and its subordinate data are stored as candidate data to be verified in the secondary cache, and the pre-stored secondary comparison data is obtained from the secondary cache.

[0222] Match the candidate data to be verified with the secondary comparison data. If there is data in the candidate data to be verified that corresponds to the secondary comparison data, store the data corresponding to the secondary comparison data and its lower-level data as the updated candidate data to be verified in the lower-level cache.

[0223] Traverse the comparison data in each level of cache and obtain the candidate data to be verified after matching with the last level comparison data;

[0224] The candidate data to be verified is parsed, and the valid data obtained from the candidate data to be verified is used as the data to be verified.

[0225] In one embodiment of this disclosure, before the first verification module 603, the following further includes:

[0226] The training module is configured to determine a preset verification model, and to perform offline training on the preset verification model using the object's historical data request logs to obtain the verification model.

[0227] In order to save online data processing resources and improve online data processing speed, in this embodiment, a preset verification model is first determined, and then the preset verification model is trained offline based on historical data request logs obtained within a preset historical time period to obtain the verification model.

[0228] In one embodiment of this disclosure, the step of using the historical data request logs of an object to perform offline training on the preset verification model to obtain a portion of the verification model can be configured as follows:

[0229] Request logs to retrieve historical data of an object using a message queue;

[0230] Filter out abnormal data from the historical data request logs.

[0231] The historical data request logs obtained after filtering abnormal data are parsed to obtain the data characteristics of different parameters in the historical data request logs.

[0232] The preset verification model is trained offline using the data features to obtain the verification model.

[0233] In this implementation:

[0234] First, historical data request logs of objects within a preset historical time period are obtained through a message middleware. After the historical data request logs are obtained, they can be persisted. The preset historical time period can be set according to the needs of the actual application. The message middleware can be a high-capacity asynchronous streaming log message middleware available on the technology market. The historical data request logs are log data corresponding to the historical data requests of objects. The historical data request refers to a request issued by the historical data requester to obtain data of a certain object. Similar to the data requester, the historical data requester can be a user, an application, or a device.

[0235] Then, abnormal data in the historical data request log is filtered out. Abnormal data refers to non-standard or invalid data, such as noisy data, test data entered by testers that is unrelated to the normal data acquisition request log, etc. Existing abnormal data filtering methods can be used for filtering abnormal data, which will not be elaborated upon in this disclosure.

[0236] Then, the historical data request logs obtained after filtering for abnormal data are parsed to obtain the data characteristics of different parameters in the historical data request logs. These data characteristics may include one or more of the following: parameter type, the data type corresponding to the parameter, the range of data values ​​corresponding to the parameter, the correlation between parameters, and the correlation between the data corresponding to parameters. For example, is a parameter a system-level parameter or an application-level parameter? Is the data type corresponding to a parameter long integer or short integer? What is the range of data values ​​corresponding to a parameter? Does a correlation exist between two or more parameters, and what kind of correlation exists? Does a parameter have a correlation in different application environments? For example, the value of a parameter should be A in operating system 1, while the value should be B in operating system 2, which means that the parameter has a correlation in different application environments. Does a correlation exist between the data corresponding to two or more parameters, and what kind of correlation exists? And so on.

[0237] Finally, the data features obtained from the analysis are used to train the preset verification model offline, thus obtaining the verification model.

[0238] In one embodiment of this disclosure, before performing abnormal data filtering on the historical data request log, the following is further configured:

[0239] The historical data request logs are formatted and normalized based on preset format information.

[0240] To improve data processing efficiency, in this embodiment, the historical data request log is also normalized based on preset format information so that the historical data request log has a unified data structure consistent with the preset format information. The preset format information can be preset according to the needs of actual application and the specific data to be processed.

[0241] In one embodiment of this disclosure, before performing abnormal data filtering on the historical data request log, the following is further configured:

[0242] Configure identification information for the historical data request logs for differentiated storage.

[0243] The identification information refers to information used to distinguish the historical data request logs, such as ID information. The configuration of the identification information enables the historical data request logs to be stored in a distinguishable and orderly manner, and also facilitates the tracing of the historical data request logs.

[0244] In one embodiment of this disclosure, the step of using the data features to perform offline training on the preset verification model to obtain a portion of the verification model can be configured as follows:

[0245] The data features are used as input to the preset verification model, and the data corresponding to the data features are used as output to the preset verification model for offline training to obtain the verification model.

[0246] After obtaining the data features that can be used as training data, an available preset verification model can be determined, such as a decision tree model. Then, the data features are used as the input of the preset verification model, and the data corresponding to the data features, such as parameters, are used as the output of the preset verification model for offline training to obtain the verification model.

[0247] In one embodiment of this disclosure, after offline training of the preset verification model using the data features to obtain the verification model, the following configuration is further provided:

[0248] Obtain preset data verification requirement information, and adjust the verification model according to the preset data verification requirement information.

[0249] Considering that in practical applications, data requesters or other relevant parties may have explicit and specific requirements for the verification of data request logs, which are more effective and targeted for verification, this embodiment needs to simultaneously consider the verification requirements provided by the data requester or other relevant parties, i.e., preset data verification requirement information, to improve the verification model and form an information closed loop of data learning, data verification, requirement feedback, and data relearning. In other words, if the preset data verification requirement information is received, the verification model needs to be adjusted according to it, for example, by modifying or adjusting the training data of the verification model. The preset data verification requirement information may include one or more of the following: preset verification parameters, the type of the preset verification parameters, the corresponding values ​​of the preset verification parameters, preset verification data, the type of the preset verification data, the value of the preset verification data, etc. This allows the training data of the verification model to be modified or adjusted according to the specific content of the preset data verification requirement information.

[0250] In one embodiment of this disclosure, the preset data verification requirement information can be obtained from a preset database. That is, the data requester or other relevant party can first store the preset data verification requirement information in a pre-specified or pre-set database, and then the verification server can obtain it from the preset database.

[0251] In one embodiment of this disclosure, the verification device can be used to verify advertising data requests, where the object is an advertisement. This device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. Figure 7 As shown, when verifying an advertising data request, the verification device includes:

[0252] The second acquisition module 701 is configured to acquire advertising data requests from the advertising request end;

[0253] The second parsing module 702 is configured to parse the advertising data request from the advertising request end to obtain the data to be verified.

[0254] The second verification module 703 is configured to input the data to be verified into a pre-trained verification model to predict the verification result of the advertising data request, wherein the verification model is obtained by pre-training based on the advertising historical data request log.

[0255] The verification device trains a verification model offline and uses this model to perform real-time verification of the data to be verified obtained from parsing online advertising data requests. This technical solution is simple to operate, saves on operation time and complexity, reduces manpower investment, and effectively reduces subsequent maintenance costs.

[0256] In one embodiment of this disclosure, the verification device can be implemented as a computer, computing device, electronic device, server, service cluster, etc., capable of performing advertising data request verification processing.

[0257] Figure 7 The technical terms and technical features involved in the illustrated and related embodiments are consistent with Figure 6 The technical terms and technical features mentioned in the illustrated and related embodiments are the same or similar, for Figure 7 The explanations and descriptions of the technical terms and features involved in the illustrated and related embodiments can be found in the above-mentioned... Figure 6 The explanations of the illustrated and related embodiments are not repeated here.

[0258] Figure 8 The diagram shows a structural block diagram of a verification device according to another embodiment of the present disclosure. This device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. Figure 8 As shown, the verification device includes:

[0259] Controller 801 is configured to receive data requests from an object and forward the data requests from the object to an online server;

[0260] Online server 802 is configured to parse the data request of the object, obtain the data to be verified, input the data to be verified into a pre-trained verification model, predict the data request verification result of the object, and feed back the data request verification result of the object to the controller.

[0261] As mentioned above, with the development of data technology, remote data requests and transmissions are becoming increasingly frequent, and the amount of data transmitted is also increasing. When a requester requests data for a specific object, it usually needs to carry parameters related to the requested data. The backend server or service personnel need to determine whether the requested data parameters and the corresponding data are correct and valid. In existing technologies, fixed-value validation and regular expression validation methods are commonly used. Fixed-value validation refers to configuring one or more specific validation values ​​to validate the data when the requester requests it, but this method requires establishing all validation rules in advance, which is complex and time-consuming. Regular expression validation refers to using pre-configured regular expressions to validate the data when the requester requests it, but this method has high maintenance costs in the later stages, and the modification operation is relatively complex when the validation content needs to be changed. Therefore, there is an urgent need for a data validation solution that is simple to operate, saves operation time and complexity, and saves manpower.

[0262] In view of the above problems, this embodiment proposes a verification device. This device trains a verification model offline and uses the model to perform real-time verification of the data to be verified obtained from parsing online object data requests. This technical solution is simple to operate, saves operation time and complexity, reduces manpower investment, and effectively reduces subsequent maintenance costs.

[0263] In one embodiment of this disclosure, the verification device can be implemented as a verification system including a controller and an online server.

[0264] In one embodiment of this disclosure, the online server parses the data request of the object to obtain a portion of the data to be verified, which can be configured as follows:

[0265] The online server parses the data request of the object to obtain the valid data of the object's data request;

[0266] The valid data request data of the object is stored in the first-level cache, and the pre-stored first-level comparison data is retrieved from the first-level cache;

[0267] The valid data of the data request of the object is matched with the first comparison data. If there is data in the valid data of the data request of the object that corresponds to the first comparison data, the data corresponding to the first comparison data and its subordinate data are stored as candidate data to be verified in the secondary cache, and the pre-stored secondary comparison data is obtained from the secondary cache.

[0268] Match the candidate data to be verified with the secondary comparison data. If there is data in the candidate data to be verified that corresponds to the secondary comparison data, store the data corresponding to the secondary comparison data and its lower-level data as the updated candidate data to be verified in the lower-level cache.

[0269] Traverse the comparison data in each level of cache, and take the candidate data to be verified obtained after matching with the last level comparison data as the data to be verified.

[0270] In one embodiment of this disclosure, it further includes:

[0271] An offline server is configured to obtain historical data request logs of an object, determine a preset verification model, and use the historical data request logs to perform offline training on the preset verification model to obtain the verification model.

[0272] In one embodiment of this disclosure, the offline server may be configured as follows:

[0273] The offline server obtains the historical data request logs of the object through a message middleware.

[0274] Filter out abnormal data from the historical data request logs.

[0275] The historical data request logs obtained after filtering abnormal data are parsed to obtain the data characteristics of different parameters in the historical data request logs.

[0276] The preset verification model is trained offline using the data features to obtain the verification model.

[0277] In one embodiment of this disclosure, before performing abnormal data filtering on the historical data request log, the following configuration may be further included:

[0278] The historical data request logs are formatted and normalized based on preset format information.

[0279] In one embodiment of this disclosure, before performing abnormal data filtering on the historical data request log, the following configuration may be further included:

[0280] Configure identification information for the historical data request logs for differentiated storage.

[0281] In one embodiment of this disclosure, the step of using the data features to perform offline training on the preset verification model to obtain a portion of the verification model can be configured as follows:

[0282] The data features are used as input to the preset verification model, and the data corresponding to the data features are used as output to the preset verification model for offline training to obtain the verification model.

[0283] In one embodiment of this disclosure, the controller may further be configured to:

[0284] Obtain preset data verification requirement information and send the preset data verification requirement information to the offline server so that the offline server can adjust the verification model according to the preset data verification requirement information.

[0285] Figure 8 The technical terms and technical features involved in the illustrated and related embodiments are consistent with Figure 6-7 The technical terms and technical features mentioned in the illustrated and related embodiments are the same or similar, for Figure 8 The explanations and descriptions of the technical terms and features involved in the illustrated and related embodiments can be found in the above-mentioned... Figure 6-7 The explanations of the illustrated and related embodiments are not repeated here.

[0286] This disclosure also discloses an electronic device, which includes a memory and a processor; wherein...

[0287] The memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement any of the above method steps.

[0288] Figure 9 This is a schematic diagram of the structure of a computer system suitable for implementing the verification method according to an embodiment of the present disclosure.

[0289] like Figure 9 As shown, the computer system 900 includes a processing unit 901, which can execute various processes described above based on a program stored in a read-only memory (ROM) 902 or a program loaded from a storage section 908 into a random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the system 900. The processing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0290] The following components are connected to I / O interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to I / O interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 910 as needed so that computer programs read from it can be installed into storage section 908 as needed. The processing unit 901 can be implemented as a CPU, GPU, TPU, FPGA, NPU, etc.

[0291] In particular, according to embodiments of this disclosure, the methods described above can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program tangibly embodied on a readable medium thereof, the computer program containing program code for performing the verification method. In such embodiments, the computer program can be downloaded and installed from a network via communication section 909, and / or installed from removable medium 911.

[0292] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0293] The units or modules described in the embodiments of this disclosure can be implemented in software or hardware. The described units or modules can also be located in a processor, and the names of these units or modules do not necessarily constitute a limitation on the unit or module itself.

[0294] In another aspect, embodiments of this disclosure also provide a computer-readable storage medium, which may be a computer-readable storage medium included in the apparatus described in the above embodiments; or it may be a standalone computer-readable storage medium not assembled into a device. The computer-readable storage medium stores one or more programs, which are used by one or more processors to perform the methods described in embodiments of this disclosure.

[0295] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A verification method, comprising: Data request for retrieving an object; The data request for the object is parsed, and invalid data is removed to obtain the valid data for the object's data request. A multi-level cache space is pre-set to store the parsed valid data in a hierarchical manner. The multi-level cache space pre-stores preset multi-level comparison data. The valid data is stored in the first-level cache, and first-level comparison data is obtained from the first-level cache. The valid data is matched with the first-level comparison data. If there is data in the valid data that corresponds to the first-level comparison data, the data corresponding to the first-level comparison data and its subordinate data are stored as candidate data to be verified in the second-level cache, and second-level comparison data is obtained from the second-level cache. The candidate data to be verified is matched with the second-level comparison data. If there is data in the candidate data to be verified that corresponds to the second-level comparison data, the data corresponding to the second-level comparison data and its subordinate data are stored as updated candidate data to be verified in the second-level cache. Traverse the comparison data in each level of cache, and take the candidate data to be verified after matching with the last level comparison data as the data to be verified. The data to be verified is input into a pre-trained verification model to predict the data request verification result of the object, wherein the verification model is pre-trained based on the object's historical data request logs.

2. The method of claim 1, wherein, Before inputting the data to be verified into the pre-trained verification model to predict the data request verification result of the object, the method further includes: A preset verification model is determined, and the preset verification model is trained offline using the historical data request logs of the object to obtain the verification model.

3. The method according to claim 2, wherein, The offline training of the preset verification model using the historical data request logs of the object to obtain the verification model is implemented as follows: Request logs to retrieve historical data of an object using a message queue; Filter out abnormal data from the historical data request logs. The historical data request logs obtained after filtering abnormal data are parsed to obtain the data characteristics of different parameters in the historical data request logs. The preset verification model is trained offline using the data features to obtain the verification model.

4. The method according to claim 3, further comprising, before performing abnormal data filtering on the historical data request log: The historical data request logs are formatted and normalized based on preset format information.

5. The method according to claim 3 or 4, further comprising, before performing abnormal data filtering on the historical data request log: Configure identification information for the historical data request logs for differentiated storage.

6. The method according to claim 3 or 4, wherein, The step of offline training of the preset verification model using the data features to obtain the verification model is implemented as follows: The data features are used as input to the preset verification model, and the data corresponding to the data features are used as output to the preset verification model for offline training to obtain the verification model.

7. The method according to claim 3 or 4, wherein, After obtaining the verification model by offline training of the preset verification model using the data features, the method further includes: Obtain preset data verification requirement information, and adjust the verification model according to the preset data verification requirement information.

8. A verification method, comprising: Request advertising data from the advertising requester; The advertising data request from the advertising requesting end is parsed, and invalid data is removed to obtain the valid data of the advertising data request from the advertising requesting end. A multi-level cache space is pre-set to store the parsed valid data in a hierarchical manner. The multi-level cache space pre-stores preset multi-level comparison data. The valid data is stored in the first-level cache, and first-level comparison data is obtained from the first-level cache. The valid data is matched with the first-level comparison data. If there is data in the valid data that corresponds to the first-level comparison data, the data corresponding to the first-level comparison data and its subordinate data are stored as candidate data to be verified in the second-level cache, and second-level comparison data is obtained from the second-level cache. The candidate data to be verified is matched with the second-level comparison data. If there is data in the candidate data to be verified that corresponds to the second-level comparison data, the data corresponding to the second-level comparison data and its subordinate data are stored as updated candidate data to be verified in the second-level cache. Traverse the comparison data in each level of cache, and use the candidate data to be verified obtained after matching with the last level comparison data as the data to be verified in the advertising data request of the advertising request end; The data to be verified is input into a pre-trained verification model to predict the verification result of the advertising data request, wherein the verification model is obtained by pre-training based on the advertising historical data request logs.

9. A verification method, comprising: The controller receives a data request from an object and forwards the object's data request to the online server; The online server parses the data request of the object, removes invalid data for verification, and obtains the valid data of the data request of the object. A multi-level cache space is pre-set to store the parsed valid data in a hierarchical manner. The multi-level cache space pre-stores preset multi-level comparison data. The valid data is stored in the first-level cache, and first-level comparison data is obtained from the first-level cache. The valid data is matched with the first-level comparison data. If there is data in the valid data that corresponds to the first-level comparison data, the data corresponding to the first-level comparison data and its subordinate data are stored as candidate data to be verified in the second-level cache, and second-level comparison data is obtained from the second-level cache. The candidate data to be verified is matched with the second-level comparison data. If there is data in the candidate data to be verified that corresponds to the second-level comparison data, the data corresponding to the second-level comparison data and its subordinate data are stored as updated candidate data to be verified in the second-level cache. The comparison data in each level of cache is traversed, and the candidate data to be verified obtained after matching with the last level comparison data is used as the data to be verified for the data request of the object. The data to be verified is input into the pre-trained verification model to predict the data request verification result of the object, and the data request verification result of the object is fed back to the controller.

10. The method of claim 9, further comprising: The offline server obtains the historical data request logs of the object, determines the preset verification model, and uses the historical data request logs to perform offline training on the preset verification model to obtain the verification model.

11. The method according to claim 10, wherein, The offline server obtains historical data request logs of the object, determines a preset verification model, and uses the historical data request logs to perform offline training on the preset verification model to obtain the verification model, which is then implemented as follows: The offline server obtains the historical data request logs of the object through a message middleware. Filter out abnormal data from the historical data request logs. The historical data request logs obtained after filtering abnormal data are parsed to obtain the data characteristics of different parameters in the historical data request logs. The preset verification model is trained offline using the data features to obtain the verification model.

12. The method according to claim 11, further comprising, before performing abnormal data filtering on the historical data request log: The historical data request logs are formatted and normalized based on preset format information.

13. The method according to claim 11 or 12, further comprising, before performing abnormal data filtering on the historical data request log: Configure identification information for the historical data request logs for differentiated storage.

14. The method according to claim 11 or 12, wherein, The step of offline training of the preset verification model using the data features to obtain the verification model is implemented as follows: The data features are used as input to the preset verification model, and the data corresponding to the data features are used as output to the preset verification model for offline training to obtain the verification model.

15. The method according to claim 11 or 12, further comprising: The controller acquires preset data verification requirement information and sends the preset data verification requirement information to the offline server, so that the offline server adjusts the verification model according to the preset data verification requirement information.

16. A verification device, comprising: The first acquisition module is configured to retrieve data requests from objects; The first parsing module is configured to parse the data request of the object, remove invalid data for verification, and obtain the valid data of the data request of the object. A multi-level cache space is pre-set to store the parsed valid data in a hierarchical manner. The multi-level cache space pre-stores preset multi-level comparison data. The valid data is stored in the first-level cache, and first-level comparison data is obtained from the first-level cache. The valid data is matched with the first-level comparison data. If there is data in the valid data that corresponds to the first-level comparison data, the data corresponding to the first-level comparison data and its subordinate data are stored as candidate data to be verified in the second-level cache, and second-level comparison data is obtained from the second-level cache. The candidate data to be verified is matched with the second-level comparison data. If there is data in the candidate data to be verified that corresponds to the second-level comparison data, the data corresponding to the second-level comparison data and its subordinate data are stored as updated candidate data to be verified in the second-level cache. Traverse the comparison data in each level of cache, and take the candidate data to be verified after matching with the last level comparison data as the data to be verified. The first verification module is configured to input the data to be verified into a pre-trained verification model to predict the data request verification result of the object, wherein the verification model is pre-trained based on the object's historical data request logs.

17. A verification device, comprising: The second acquisition module is configured to acquire advertising data requests from the advertising requester. The second parsing module is configured to parse the advertising data request from the advertising requesting end, remove invalid data for verification, and obtain the valid data of the advertising data request from the advertising requesting end. A multi-level cache space is pre-configured to hierarchically store the parsed valid data. The multi-level cache space pre-stores preset multi-level comparison data. The valid data is stored in the first-level cache, and first-level comparison data is retrieved from the first-level cache. The valid data is matched with the first-level comparison data. If data corresponding to the first-level comparison data exists in the valid data, the data corresponding to the first-level comparison data and its subordinate data are stored as candidate data to be verified in the second-level cache, and second-level comparison data is retrieved from the second-level cache. The candidate data to be verified is matched with the second-level comparison data. If data corresponding to the second-level comparison data exists in the candidate data to be verified, the data corresponding to the second-level comparison data and its subordinate data are stored as updated candidate data to be verified in the second-level cache. Traverse the comparison data in each level of cache, and use the candidate data to be verified obtained after matching with the last level comparison data as the data to be verified in the advertising data request of the advertising request end; The second verification module is configured to input the data to be verified into a pre-trained verification model for prediction, and obtain the verification result of the advertising data request, wherein the verification model is pre-trained based on the advertising historical data request log.

18. A verification device, comprising: The controller is configured to receive data requests from an object and forward those requests to an online server. An online server is configured to parse data requests for the object, remove invalid data for verification, and obtain valid data for the object's data requests. A multi-level cache space is pre-configured to hierarchically store the parsed valid data. The multi-level cache space pre-stores preset multi-level comparison data. The valid data is stored in the first-level cache, and first-level comparison data is retrieved from the first-level cache. The valid data is matched with the first-level comparison data. If data corresponding to the first-level comparison data exists in the valid data, the data corresponding to the first-level comparison data and its subordinate data are stored as candidate data to be verified in the second-level cache, and second-level comparison data is retrieved from the second-level cache. The candidate data to be verified is matched with the second-level comparison data. If data corresponding to the second-level comparison data exists in the candidate data to be verified, the data corresponding to the second-level comparison data and its subordinate data are stored as updated candidate data to be verified in the second-level cache. The comparison data in each level of cache is traversed, and the candidate data to be verified obtained after matching with the last level comparison data is used as the data to be verified for the data request of the object. The data to be verified is input into the pre-trained verification model to predict the data request verification result of the object, and the data request verification result of the object is fed back to the controller.

19. The apparatus of claim 18, further comprising: An offline server is configured to obtain historical data request logs of an object, determine a preset verification model, and use the historical data request logs to perform offline training on the preset verification model to obtain the verification model.

20. An electronic device comprising a memory and a processor; wherein, The memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the steps of the method according to any one of claims 1-15.

21. A computer-readable storage medium having computer instructions stored thereon, wherein, When executed by a processor, the computer instructions implement the steps of the method described in any one of claims 1-15.

Images