Data processing method and device, equipment and storage medium

By disassembling the data acquisition request into three types of text and performing automated verification, the problems of low verification efficiency and unstable accuracy in the prior art are solved, and efficient and accurate verification of data acquisition requests are achieved.

CN118982015APending Publication Date: 2024-11-19BEIJING ZITIAO NETWORK TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411186626.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

In the prior art, the verification efficiency of data acquisition requests is low, the accuracy is unstable, and the verification results are inconsistent with manual reliance, making it difficult to adapt to the dynamically changing data application needs.

Method used

By analyzing the user's data acquisition request, it is converted into the first text of data management, the second text of data application, and the third text of auxiliary data request verification, realizing automated verification processing, reducing manual participation, and improving verification efficiency and accuracy sex.

Benefits of technology

It realizes automated verification of data acquisition requests, improves verification efficiency, accuracy and consistency, reduces costs, and adapts to dynamically changing data application needs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982015A_ABST
    Figure CN118982015A_ABST
Patent Text Reader

Abstract

The embodiment of the invention relates to a data processing method and device, equipment and a storage medium. The method comprises the following steps: receiving a data acquisition request; analyzing the data acquisition request, and based on an analysis result, determining a first text for data management, a second text for data application and a third text for assisting data request verification; and based on the first text and / or the third text and the second text, performing verification processing on the data acquisition request to generate a verification result. According to the embodiment of the invention, the data verification cost can be reduced, and the verification efficiency, the verification real-time performance and the verification accuracy of the data acquisition request can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a data processing method, apparatus, device and storage medium. Background Art

[0002] In the data usage scenario, the user requests the data provider to obtain data. The data provider verifies the user's data acquisition request to determine whether the user can use the requested data according to the requested data usage method and data usage scenario.

[0003] Currently, the verification of user data acquisition requests is mainly carried out manually, such as manually analyzing, understanding and evaluating the data acquisition request to determine whether the user's data acquisition request is approved. When the data acquisition request is verified or partially passed, the verification result is manually executed to distribute the data according to the verification opinion. However, manual verification is inefficient, the accuracy is unstable, and the timeliness is poor. The final execution results are also lax or inconsistent. Summary of the invention

[0004] In order to solve the above technical problems, the embodiments of the present disclosure provide a data processing method, apparatus, device and storage medium.

[0005] In a first aspect, an embodiment of the present disclosure provides a data processing method, the method comprising:

[0006] Receive a data acquisition request;

[0007] Parse the data acquisition request, and based on the parsing result, determine a first text for data management, a second text for data application, and a third text for assisting data request verification;

[0008] Based on the first text and / or the third text, and the second text, the data acquisition request is verified to generate a verification result.

[0009] In a second aspect, an embodiment of the present disclosure further provides a data processing device, the device comprising:

[0010] A request receiving module, used for receiving a data acquisition request;

[0011] A text determination module, used to parse the data acquisition request, and based on the parsing result, determine a first text for data management, a second text for data application, and a third text for assisting data request verification;

[0012] A verification module is used to verify the data acquisition request based on the first text and / or the third text, and the second text, and generate a verification result.

[0013] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0014] processor;

[0015] A memory for storing executable instructions;

[0016] The processor is used to read executable instructions from the memory and execute the executable instructions to implement the data processing method described in any embodiment of the present disclosure.

[0017] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the data processing method described in any embodiment of the present disclosure.

[0018] In a fifth aspect, the embodiments of the present disclosure further provide a computer program product, which is used to execute the data processing method described in any embodiment of the present disclosure.

[0019] The data processing method, apparatus, device and storage medium of the embodiments of the present disclosure can convert a user's data acquisition request into a first text for representing data management, a second text for representing data application, and a third text for representing auxiliary data request verification, thereby decomposing a semantically complex data acquisition request into three types of texts with clear semantics, reducing the complexity of understanding the data acquisition request, and providing basic information for the subsequent automatic verification of the data application; then based on the first text and / or the third text, the second text is verified to determine whether the data application is compliant; the data application link, the verification link and the verification result execution link of the data distribution are connected in series to complete the automatic verification of the data acquisition request, thereby reducing the degree of manual participation in the data processing process, and largely avoiding the problems of slow verification, unstable verification results and poor consistency of verification results of similar data applications caused by manual intervention, thereby reducing the cost of data verification, improving the verification efficiency, real-time and accuracy of data acquisition requests, and also improving the accuracy and consistency of verification result execution.

[0020] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0022] Figure 1 A flowchart of a data processing method provided by an embodiment of the present disclosure;

[0023] Figure 2 A flowchart of another data processing method provided by an embodiment of the present disclosure;

[0024] Figure 3 A schematic diagram of text vectorization processing provided by an embodiment of the present disclosure;

[0025] Figure 4 A schematic diagram of the structure of a data processing device provided in an embodiment of the present disclosure;

[0026] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0028] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0029] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0030] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0031] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0032] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0033] There are two main ways to verify the data acquisition request of users for certain data in the related art. One is to verify the data acquisition request of users manually. Each data acquisition request needs to be analyzed, understood and evaluated manually, resulting in low verification efficiency; and due to reliance on manual verification, the accuracy of the verification results is difficult to guarantee. When there are many data acquisition requests, the problem of untimely verification may also occur. In addition, the verification results given manually need to be communicated and understood later, which is prone to errors, resulting in the final verification results being inconsistent with the initial conclusions. The other is to verify the data acquisition request of users through a traditional data verification system. However, the verification rules of the traditional data verification system are statically defined matching rules. It is necessary to determine the specific verification rules corresponding to the data application requirements and verify based on the corresponding verification rules. However, the statically defined matching rules are difficult to cover the verification rules corresponding to all data application requirements. Therefore, the traditional data verification system is difficult to adapt to dynamically changing data application requirements, and the automation processing capability is low.

[0034] Based on the above situation, the disclosed embodiment provides a data processing method to adapt to the dynamically changing data application needs, reduce the manual participation in the data processing process, improve the verification efficiency, real-time and accuracy of data acquisition requests, intercept the data acquisition request when the verification result is not passed, and when the verification result is passed, feedback the data to the user according to the verification opinion. The disclosed embodiment parses the user's data acquisition request and converts the user's data acquisition request into a first text for representing data management, a second text for representing data application, and a third text for representing auxiliary data request verification, thereby realizing the decomposition of semantically complex data acquisition requests into three types of texts with clear semantics and information categories, reducing the complexity of understanding data acquisition requests, so that it can be realized based on the first text and / or the third text, automatically verifying whether the data application of the second text is compliant, avoiding manual intervention, and performing verification processing in real time, thereby improving the verification efficiency, the accuracy and consistency of the execution of the verification results.

[0035] The data processing method provided by the embodiment of the present disclosure can be applied to the scenario of obtaining the data to be processed during the data processing process, for example, it can be applied to the scenario of applying to obtain internal data or external data. The method can be executed by a data processing device, which can be implemented by software and / or hardware, for example, it can be implemented by a data processing platform, and the device can be integrated in an electronic device with data processing function. Among them, the electronic device can include but is not limited to smart phones, laptops, tablet computers (Tablet Personal Computer, Tablet PC), desktop computers, servers, edge computing platforms or server clusters.

[0036] Figure 1 A flow chart of a data processing method provided by an embodiment of the present disclosure is shown.

[0037] like Figure 1 As shown, the data processing method may include the following steps:

[0038] S110: Receive a data acquisition request.

[0039] Among them, the data acquisition request is a request for acquiring data. In the embodiments of the present disclosure, the data acquisition request can be understood as a request triggered by a user to apply for data from a data provider. For example, a request by a user to acquire user access data of data provider A is a data acquisition request; a request by a user to acquire relevant statistical data of resource acquisition of data provider B is also a data acquisition request, and so on. Examples are not given one by one here.

[0040] Specifically, the user inputs the relevant information of the data application (such as the data source, data usage method and data usage scenario corresponding to the data application) through a human-computer interaction interface such as a browser or a web page on a client having a data acquisition function. The client generates a data acquisition request based on the relevant information of the data application, and sends the data acquisition request to the electronic device, so that the electronic device receives the data acquisition request through a data receiving port.

[0041] S120, parsing the data acquisition request, and based on the parsing result, determining a first text for data management, a second text for data application, and a third text for assisting data request verification.

[0042] Among them, the parsing result is the result obtained by parsing the data acquisition request. The text refers to the corresponding text obtained by parsing the data acquisition request to clarify the scope, type, purpose, etc. of the required data. In the embodiment of the present disclosure, the first text is a text that describes the data provider managing data or the data monitoring party monitoring data usage, etc., for example, it can be the relevant regulations on data management / use issued by the authoritative agency, which may include: the authority of the requester, the access rights of the data, etc. The second text is a normative text that describes the specific data applied by the user, which can express the specific type of the user's application data, for example, it can be a text for applying for xx data. The third text is a normative text that describes the relevant situation of data use, which can help understand whether the use of data complies with relevant regulations, thereby assisting the electronic device in verifying the data request. For example, the third text can be a normative text for data use scenarios for internal / external use of data, or it can be a normative text for data use methods such as calculating statistical indicators of batch data, etc.

[0043] Specifically, the user's use and processing of the requested data needs to comply with the data management requirements of the data provider, and needs to comply with the relevant laws, regulations and standards of the relevant countries and regions. Therefore, after receiving the data acquisition request, the electronic device needs to verify the data requested by the user and its usage. In order to enable the electronic device to automatically and accurately verify the data acquisition request and reduce the complexity of understanding the data acquisition request, the embodiment of the present disclosure parses the data acquisition request into three dimensions of text to describe the data acquisition request, namely: a first text for data management, a third text for auxiliary data request verification, and a second text for data application. The data acquisition request is verified based on the three-dimensional text to ensure the accuracy of the automatic verification of the electronic device.

[0044] For example, semantic recognition and parsing are performed on data acquisition requests to obtain text information of the following three dimensions: the first dimension is to obtain the text representing the relevant provisions of data management in the data acquisition request; the second dimension is to obtain the descriptive text related to the specific data requested by the user in the data acquisition request; the third dimension is to obtain the normative descriptive text related to the data usage in the data acquisition request. The descriptive text of the second dimension is directly converted into a standardized text as the second text; when the descriptive text of the first dimension can fully reflect the data management requirements and regulations of the data manager (such as the data provider, data monitor, etc.), the descriptive text of the first dimension is directly converted into a standardized text as the first text; when the descriptive text of the third dimension can fully reflect the data usage specifications, the descriptive text of the third dimension is directly converted into a standardized text as the third text.

[0045] For another example, following the above example, when the description text of the first dimension cannot fully reflect the data management requirements of the data manager, the relevant information of the data manager (such as the name and logo of the data manager) can be determined based on the data acquisition request, and then the determined data manager or related database can be requested to obtain or query its data management regulations, and the relevant text fed back by the data manager or related database can be converted into a standardized first text; when the description text of the third dimension cannot fully reflect the data usage requirements, the text related to the data usage can be queried or obtained from the related database and converted into a standardized third text. In this way, the accuracy of the first and third texts can be guaranteed.

[0046] It should be noted that in order to avoid the phenomenon that the first text and the third text obtained by parsing are inaccurate due to description problems in the user data acquisition request, the relevant regulations on data management and the text of data usage can be directly obtained from the data manager or the relevant database and converted into the first text and the third text.

[0047] On the basis of the above embodiment, as a detailed implementation method of "parsing the data acquisition request, and based on the parsing result, determining the first text for data management, the second text for data application and the third text for auxiliary data request verification", it includes: parsing the data acquisition request, extracting the data source information requested by the data acquisition request, the data description information of the application data requested to be obtained, and the data usage information, determining the first text according to the data source information, determining the second text according to the data description information, and determining the third text according to the data usage information.

[0048] Among them, data source information is relevant information corresponding to the source of the data requested by the data acquisition request. For example, data source information can be relevant information about the application that generates the data, or it can be database information that stores the data, etc. Data description information is information used to describe the specific data to be applied for by the data acquisition request. For example, the details of XXX data, the batch statistical results of YYY, etc. Data usage information is used to explain the scenarios, specific purposes, data processing methods, etc. that the user wants to use the data applied for. For example, the usage scenario of the applied data is internal data analysis of the enterprise, the purpose is to analyze the popularity of ** products, and the data processing method is to calculate certain statistical indicators of batch data, etc.

[0049] Specifically, the electronic device performs semantic recognition and analysis on the data acquisition request, determines the application that generates the application data, the database that stores the application data and other related information, and normalizes the application data, the database that stores the application data and other related information as the first text; determines specific description information of the content of the application data, and normalizes and processes the specific description information of the content of the application data as the second text; determines the usage scenarios, purposes and other information of the application data, and normalizes and processes the usage scenarios, purposes and other information of the application data as the third text.

[0050] For example, the data source information is obtained as A application of company A; the data description information obtained is: XXX data details within a time period of H; the data usage information obtained is: the user intends to use the applied data for internal data analysis of the enterprise. Then the usage specifications, regulations, requirements, etc. formulated by company A for the data generated by A application are queried, and the corresponding usage specifications, regulations, requirements, etc. of A application of company A are normalized as the first text, the data description information is normalized as the second text, and the data usage information for internal data analysis of the enterprise is normalized as the third text.

[0051] S130: Based on the first text and / or the third text, and the second text, verify the data acquisition request and generate a verification result.

[0052] The verification result is used to indicate whether the verification of the data acquisition request is passed, and possible verification results include passed, partially passed, failed, etc. The verification result can be represented by a corresponding result identifier, for example, "1" indicates that the verification result is passed, and "0" indicates that the verification result is failed.

[0053] Specifically, in response to the user's data acquisition request, the electronic device can respond by feeding back the user's requested data or intercepting the data acquisition request, so it can generate a verification result and respond according to the verification result. When the verification result is passed, the application data corresponding to the verified request is fed back to the user, and when the verification result is not passed, the user's data acquisition request is intercepted and no application data is fed back.

[0054] The verification methods may include the following: Verification method one: perform verification based on the first text and second text data acquisition request to generate a verification result; Verification method two: perform verification based on the third text and second text data acquisition request to generate a verification result; Verification method three: perform verification based on the first text, third text and second text data acquisition request to generate a verification result.

[0055] For each of the above verification methods, the text can be identified and analyzed, and then text matching can be performed to generate verification results based on the matching results.

[0056] For example, for verification method 1, the semantics corresponding to the first text and the second text are extracted, and the first text and the second text are matched based on the semantics. When the matching degree is greater than the matching degree threshold, the verification result is determined to be passed, otherwise the verification result is failed. For verification method 2, the semantics corresponding to the third text and the second text are extracted, and the third text and the second text are matched based on the semantics. When the matching degree is greater than the matching degree threshold, the verification result is determined to be passed, otherwise the verification result is failed. For verification method 3, the semantics corresponding to the first text, the second text, and the third text are extracted, and the first text and the second text are matched based on the semantics to obtain the first matching degree, and the third text and the second text are matched to obtain the second matching degree. When the first matching degree and the second matching degree are respectively greater than the corresponding matching degree thresholds, the verification result is passed, otherwise the verification result is failed; or, the first matching degree and the second matching degree can also be weighted summed to calculate the third matching degree. When the third matching degree is greater than the matching degree threshold, the verification result is passed, otherwise the verification result is failed.

[0057] In this example, the text matching process is a process of determining whether the second text complies with the data management rules and data usage rules required by the first text and / or the third text.

[0058] Alternatively, in order to ensure the accuracy of the verification results, for each of the above-mentioned verification methods, the first text, the second text and the third text can be vectorized and converted into three types of vector data recognizable by the model and input into the model respectively, and the three types of vector data can be identified and matched based on the model, and the verification results can be output based on the matching results.

[0059] The above-mentioned technical scheme of the embodiment of the present disclosure can convert the user's data acquisition request into a first text for representing data management, a second text for representing data application, and a third text for representing auxiliary data request verification, thereby realizing the decomposition of semantically complex data acquisition requests into three types of texts with clear semantics, reducing the complexity of understanding data acquisition requests, and providing basic information for the subsequent automatic verification of data applications; then based on the first text and / or the third text, the second text is verified to determine whether the data application is compliant; the data application link, the verification link, and the verification result execution link of data distribution are connected in series to complete the automatic verification of data acquisition requests, thereby reducing the degree of manual participation in the data processing process, and largely avoiding the problems of slow verification, unstable verification results, and poor consistency of verification results of similar data applications caused by manual intervention, thereby reducing the cost of data verification, improving the verification efficiency, real-time and accuracy of data acquisition requests, and also improving the accuracy and consistency of verification result execution.

[0060] Figure 2 is a flowchart of another data processing method provided by an embodiment of the present disclosure. The data processing method further optimizes "based on the first text and / or the third text, and the second text, verifying the data acquisition request and generating a verification result". The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here. Figure 2 As shown, the data processing method specifically includes the following steps S210 to S240.

[0061] S210: Receive a data acquisition request.

[0062] S220, parsing the data acquisition request, and based on the parsing result, determining a first text for data management, a second text for data application, and a third text for assisting data request verification.

[0063] S230 , performing vectorization processing on the first text, the second text, and the third text to generate a first vector, a second vector, and a third vector, respectively.

[0064] Among them, vectorization processing refers to converting text data into a numerical vector. The first vector is a vector obtained by vectorizing the first text, which is used to represent the management requirements of the data provider to manage data or the data monitor to monitor the use of data. The second vector is a vector obtained by vectorizing the second text, which is used to represent the specific data requested by the user. The third vector is a vector obtained by vectorizing the third text, which is used to represent the specifications of the data usage-related situation.

[0065] Specifically, some language models have the ability to understand various natural language expressions, can quickly and efficiently analyze and process input text, extract semantic information therein, and achieve accurate understanding of complex text. Vector data can be effectively processed in mathematical and computational models, so in order to use the model to verify data acquisition, the first text, the second text, and the third text can be processed into corresponding vectors in the embodiment of the present disclosure.

[0066] For example, the first text may be vectorized based on a related algorithm to obtain the first text; the second text may be vectorized to obtain the second text; and the third text may be vectorized to obtain the third text. Alternatively, the same trained language model may be used for vectorization, such as inputting the first text into the language model to obtain the first text; inputting the second text into the language model to obtain the second text; and inputting the third text into the language model to obtain the third text.

[0067] For another example, three different language models may be used to process the first text, the second text, and the third text respectively to obtain a first vector, a second vector, and a third vector.

[0068] In some embodiments, in combination Figure 3 As shown, S230 can be implemented as: inputting the first text into the first generative model to generate a first vector; inputting the second text into the second generative model to generate a second vector; and inputting the first text, the second text and the third text into the third generative model to generate the third vector.

[0069] Among them, the first generative model is a model obtained by training the first language model using the first text sample of data management. The second generative model is a model obtained by training the second language model using the second text sample related to the data application. The third generative model is a model obtained by training the third language model using the third text sample of the auxiliary data request verification and the verification-related text sample. The first language model, the second language model, and the third language model can be different basic language models, or they can be partially or completely the same basic language models. In order to improve the efficiency of model training, the first language model, the second language model, and the third language model can be set to the same basic language model in the embodiment of the present disclosure.

[0070] Specifically, a generative model with larger specifications and more parameters can be used to understand the data acquisition request input by the user, in order to realize the automatic verification of the data acquisition request. In order to accurately interpret the professional knowledge with complex text structure such as the data use-related regulations (such as laws, regulations, and management regulations) issued by the authoritative agency, accurately identify the sensitivity of the user's application data, and accurately identify the data that assists in data verification, the disclosed embodiment can use three targeted generative models to respectively identify the data use-related regulations, application data, and data that assists in data verification, so as to ensure that each type of data can be accurately identified, thereby improving the verification accuracy of the data acquisition request.

[0071] For example, the first text is input into the first generative model, and the first generative model is used to perform semantic recognition and extraction on the first text to obtain and output a first vector; the second text is input into the second generative model, and the second generative model is used to perform semantic recognition and extraction on the second text line to obtain and output a second vector; the first text, the second text, and the third text are input into the third generative model, and the third generative model is used to perform semantic recognition and extraction on the three types of input texts to obtain and output a third vector.

[0072] In some embodiments, the first generative model can be obtained by training the initial generative model using the first text sample of data management. The second generative model can be obtained by training the initial generative model using the second text sample related to the data application. The third generative model can be obtained by training the initial generative model using the third text sample of the auxiliary data request verification and the verification-related text sample.

[0073] In some embodiments, the initial generative model can be first subjected to intensive training based on the first text sample, the second text sample, the third text sample and the verification-related text sample to obtain an intermediate generative model, and then the intermediate generative model can be fine-tuned using the first text sample, the second text sample, the third text sample and the verification-related text sample to obtain the first generative model, the second generative model, and the third generative model with improved recognition accuracy.

[0074] In some embodiments, the first generative model, the second generative model, and the third generative model may be implemented based on the following steps A to B.

[0075] Step A: By using the first text sample of data management, the second text sample of data application, the third text sample of auxiliary data request verification and the verification-related text sample, the initial generative model is reinforced trained to generate an intermediate generative model.

[0076] Among them, the first text sample is a sample set composed of texts related to data management, such as data management regulations and legal documents of multiple companies constitute the first text sample; the second text sample is a sample set composed of text samples corresponding to multiple data applications, such as collecting text samples of M data applications to constitute the second text sample; the third text sample is a sample set composed of texts related to data usage requirements, such as collecting data usage scenarios and purpose description texts corresponding to M data applications to constitute the third text sample; the verification-related text sample is a sample set composed of multiple related texts in other approval fields. The initial generative model is trained by using the first text sample, the second text sample, the third text sample, and the verification-related text sample as training data sets to obtain an intermediate generative model.

[0077] It should be noted that the initial model is a language model with more model parameters and a larger model size. The intermediate generative model obtained can convert text into vector data, but the generated vector data is difficult to accurately represent the corresponding semantics for each type of input text. In order to achieve the purpose of accurately converting text into corresponding vectors based on the model, the embodiment of the present disclosure trains the intermediate generative model based on the following step B to obtain a generative model that accurately recognizes each type of semantics.

[0078] Step B: Use the first text sample and the first vector sample to fine-tune the intermediate generative model to generate a first generative model; use the second text sample and the second vector sample to fine-tune the intermediate generative model to generate a second generative model; use the first text sample and the first vector sample, the second text sample and the second vector sample, and the third text sample and the third vector sample to fine-tune the intermediate generative model to generate a third generative model.

[0079] Specifically, the first text sample is used as the input of the intermediate generative model, the first vector sample corresponding to the first text sample is used as a label to perform supervised training on the intermediate generative model, and the parameters and weights of the generative model are adjusted until the training end condition is met to obtain the first generative model; the second text sample is used as the input of the intermediate generative model, the second vector sample corresponding to the second text sample is used as a label to perform supervised training on the intermediate generative model, and the parameters and weights of the generative model are adjusted until the training end condition is met to obtain the second generative model; the first text sample, the second text sample, and the third text sample are used as the input of the intermediate generative model, the first vector sample, the second vector sample, and the third vector sample are used as labels to perform supervised training on the intermediate generative model, and the parameters and weights of the generative model are adjusted until the training end condition is met to obtain the third generative model.

[0080] The trained first generative model can accurately extract key information and semantic features, correctly understand the professional terms of corporate rules and regulations, data management regulations, laws and regulations, and output a vector that accurately represents the first text. The second generative model can sensitively identify data applications and output a vector that accurately represents the second text. The third generative model can accurately perform semantic recognition and output vectors that accurately represent verification-related texts in other fields. Since the third vector is obtained by semantically recognizing the first text, the second text, and the third text through the third generative model, the third vector can represent the verification risk level. The higher the risk level, the lower the probability that the data acquisition request is verified, and the lower the match between the first text and / or the third text and the second text may be.

[0081] S240: Perform verification processing based on the second vector and the target vector to generate a verification result.

[0082] The target vector includes the first vector and / or the third vector.

[0083] That is, the first vector may be used as the target vector, the third vector may be used as the target vector, or both the first vector and the third vector may be used as the target vector.

[0084] Specifically, the electronic device may perform verification processing based on the second vector and the first vector to generate a verification result, such as using a relevant algorithm or model to perform similarity judgment on the second vector and the first vector, and generating a verification result based on the judgment result. Alternatively, the electronic device may perform verification processing based on the second vector and the third vector to generate a verification result, such as using a relevant algorithm or model to perform similarity judgment on the third vector and the first vector, and generating a verification result based on the judgment result. Alternatively, the electronic device may perform verification processing based on the second vector, the first vector, and the third vector to generate a verification result, such as performing similarity judgment on the second vector and the first vector, performing similarity judgment on the third vector and the second vector, and generating a verification result by combining the two similarity judgment results.

[0085] In some embodiments, the implementation method of verification processing based on the second vector and the target vector may include: determining the vector operation result of the second vector and the target vector; and determining the verification result of whether the data acquisition request has passed the verification based on whether the vector operation result meets the verification threshold.

[0086] Among them, the vector operation result is the result of the vector operation on the second vector and the target vector. For example, the vector operation can be a dot product operation between the second vector and the target vector, a similarity or matching operation between the second vector and the target vector, an operation to determine whether the second vector and the target vector are coplanar, or an operation to determine whether the second vector and the target vector are orthogonal, etc., which will not be listed one by one in the embodiments of the present disclosure. The verification threshold is a threshold condition configured to determine whether the data acquisition request is verified. For example, when the vector operation result meets the verification threshold, it is determined that the data acquisition request is verified and the verification result is passed; when the vector operation result does not meet the verification threshold, it is determined that the data acquisition request is not verified and the verification result is not passed.

[0087] Specifically, determining the vector operation result of the second vector and the target vector may include: performing a vector operation on the second vector and the first vector to obtain a vector operation result, and judging whether the vector operation result meets the verification threshold to determine whether the data acquisition request is verified. For example, performing a dot multiplication operation on the second vector and the first vector, if the dot multiplication result meets the verification threshold, then determining that the data acquisition request is verified, otherwise, it fails to be verified. Alternatively, performing a vector operation on the second vector and the third vector to obtain a vector operation result, and judging whether the vector operation result meets the verification threshold to determine whether the data acquisition request is verified. For example, calculating the similarity between the second vector and the third vector, if the similarity meets the verification threshold, then determining that the data acquisition request is verified, otherwise, it fails to be verified. Alternatively, performing a vector operation on the first vector, the second vector, and the third vector to obtain a vector operation result, and judging whether the vector operation result meets the verification threshold to determine whether the data acquisition request is verified, such as judging whether the first vector, the second vector, and the third vector are coplanar, if the coplanarity meets the verification threshold, then determining that the data acquisition request is verified, otherwise, it fails to be verified.

[0088] It should be noted that the verification threshold is related to the specific application data and the vector operation method between the second vector and the target vector. The vector operation methods between the second vector and the target vector are different, and the verification threshold is also different. For example, when the vector operation method is a similarity operation, the verification threshold can be a set similarity threshold. When the similarity between the second vector and the target vector is greater than the similarity threshold, it is determined that the vector operation result meets the verification threshold; for another example, when the vector operation method is a coplanar operation, when the second vector and the target vector are coplanar, it is determined that the vector operation result meets the verification threshold; for another example, in a certain type of scenario, the first vector, the second vector and the third vector can be set to perform a dot product operation, and it is set that the verification is passed when the dot product result is 0. In other scenarios, the verification requirements can be lowered, and the verification can be passed when the dot product result of the first vector or the third vector and the second vector is 0. Examples are not given one by one here.

[0089] In the disclosed embodiment, verification is no longer a single manual decision, but is transformed into a series of logical rules that can be matched and verified. These logical rules contain the conditions and verification thresholds for verification opinions, which are used to make judgments in dynamic decisions, realize automated verification, reduce manual intervention, and improve verification efficiency.

[0090] In some embodiments, in order to promptly remind the user when the user's data acquisition request fails to pass the verification, so that the user can modify it in time and increase the probability of passing the verification, a prompt message can be generated while intercepting the data acquisition request. One implementation method includes: after determining the verification result of whether the data acquisition request passes the verification based on whether the vector operation result meets the verification threshold, if the verification result is not passed, then based on the target vector, through the fourth generative model, a request modification prompt is generated.

[0091] Among them, the fourth generative model is a model for converting vector data into text data, which is obtained by training the basic language model with text data and vector data as training data. The specific training process will not be described in detail in the embodiment of the present disclosure. The request for modification prompt is a prompt message used to remind the user how to modify the data application to pass the verification, such as "modify the number of data items", "the use of the application data is not compliant", etc. In some embodiments, the specific prompt content of the request for modification prompt is related to the vectorized representation output by the third generative model, and can be set to correspond to the risk level corresponding to the vectorized representation output by the third generative model and the specific prompt content. For example, when the vectorized representation output by the third generative model has the highest risk level, the prompt content can be "Sensitive data of XX data is used, please modify it."

[0092] Specifically, when the verification result is determined to be unsuccessful, the target vector is input into the fourth generative model, the target vector is converted into a prompt text using the fourth generative model and output, and the behavior of obtaining data is intercepted. If the verification result is a successful verification, the electronic device can directly feed back the data requested by the user to the user.

[0093] When it is determined that the verification result of the data acquisition request is failed based on the vector operation result of the second vector and the first vector, the first vector is input into the fourth generative model for text processing, a request modification prompt is output, and the data acquisition request is intercepted; when it is determined that the verification result of the data acquisition request is failed based on the vector operation result of the second vector and the third vector, the third vector is input into the fourth generative model for text processing, a request modification prompt is output, and the data acquisition request is intercepted; when it is determined that the verification result of the data acquisition request is failed based on the vector operation result of the second vector, the first vector and the third vector, the second vector, the first vector and the third vector are input into the fourth generative model for text processing, a request modification prompt is output, and the data acquisition request is intercepted.

[0094] For example, if the internal management specification corresponding to the first text is: the data can be used to analyze the post-party data after desensitization within the company (for example, an average value > 1000, etc.), and the legal and regulatory requirements corresponding to the third text are: certain types of sensitive data details cannot be traded on the market, for the second text a: apply for all details of sensitive data to be provided to external customers, and for the second text b: apply for average calculation of sensitive data for internal analysis, then the data acquisition request corresponding to the second text b can pass the verification, and the data acquisition request corresponding to the second text a cannot pass the verification and is intercepted by the data processing platform.

[0095] Based on the data processing method provided by the embodiment of the present disclosure, after the electronic device generates the verification result, it directly executes the verification result, directly intercepts the data application that does not meet the expectations, and ensures that the approved data application is accurately implemented in accordance with the management regulations, realizing the automatic decision-making and automatic real-time application processing process based on the vector operation results and configuration requirements, and completely connecting the originally decoupled application, verification, and verification result execution links, so that data acquisition requests can be initiated at any time and the verification results and execution results can be obtained in real time, avoiding the verification volatility caused by manual participation and the possibility of different verification conclusions for similar data applications, and improving the stability of verification. Therefore, the compliance of data use is ensured from multiple aspects such as the automation and accuracy of verification, the automation and accuracy of verification result execution, etc., which improves compliance protection and strengthens data security.

[0096] Based on the data processing method provided by the embodiment of the present disclosure, the automated and precise verification process ensures the compliance of data use, releases manpower, reduces execution costs, and ensures the timeliness and accuracy of verification.

[0097] The following is an embodiment of a data processing device provided by an embodiment of the present disclosure. The device and the data processing methods of the above-mentioned embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the data processing device, reference can be made to the embodiments of the above-mentioned data processing methods.

[0098] Figure 4 FIG. 1 is a schematic diagram showing the structure of a data processing device provided by an embodiment of the present disclosure. Figure 4 As shown, the data processing device 400 may include:

[0099] A request receiving module 410, configured to receive a data acquisition request;

[0100] A text determination module 420, for parsing the data acquisition request, and based on the parsing result, determining a first text for data management, a second text for data application, and a third text for assisting data request verification;

[0101] The verification module 430 is used to verify the data acquisition request based on the first text and / or the third text, and the second text, and generate a verification result.

[0102] When executing the data processing method, the data processing device provided by the embodiment of the present disclosure converts the data acquisition request into multiple types of texts for characterizing data management, data application, and auxiliary data request verification for automatic identification by the data management platform, so that it can automatically check whether the data application corresponding to the second text complies with the regulations based on the first text and the third text. Therefore, for dynamically changing data acquisition requests, the first text of data management, the second text of data application, and the third text of auxiliary data request verification can be obtained by parsing the data acquisition request, so that the data acquisition request is automatically verified based on the first text, the second text, and the third text, thereby improving the automatic processing capability and efficiency.

[0103] In some embodiments, the text determination module 420 is specifically used to parse the data acquisition request, determine the data source information, data description information and data usage information as the parsing result; determine the first text based on the data source information; determine the second text based on the data description information; and determine the third text based on the data usage information.

[0104] In some embodiments, the verification module 430 is specifically used to perform vectorization processing on the first text, the second text and the third text to generate a first vector, a second vector and a third vector respectively; perform verification processing based on the second vector and the target vector to generate a verification result; wherein the target vector includes the first vector and / or the third vector.

[0105] In some embodiments, the verification module 430 is specifically used to input the first text into the first generative model to generate a first vector; input the second text into the second generative model to generate a second vector; input the first text, the second text and the third text into the third generative model to generate a third vector; wherein the first generative model, the second generative model and the third generative model are all obtained by intensive training of the initial generative model.

[0106] In some embodiments, the data processing device 400 further includes a training module for:

[0107] Performing intensive training on the initial generative model using the first text sample for data management, the second text sample for data application, the third text sample for assisting data request verification, and the verification-related text sample to generate an intermediate generative model;

[0108] Using the first text sample and the first vector sample, fine-tune the intermediate generative model to generate a first generative model;

[0109] The intermediate generative model is fine-tuned using the second text sample and the second vector sample to generate a second generative model; the intermediate generative model is fine-tuned using the first text sample and the first vector sample, the second text sample and the second vector sample, and the third text sample and the third vector sample to generate a third generative model.

[0110] In some embodiments, the verification module 430 is specifically configured to:

[0111] Determine a vector operation result of the second vector and the target vector; and determine a verification result of whether the data acquisition request is verified to be passed based on whether the vector operation result meets a verification threshold.

[0112] In some embodiments, the verification module 430 is further configured to:

[0113] After determining whether the data acquisition request has passed the verification based on whether the vector operation result meets the verification threshold, if the verification result is failed, a request modification prompt is generated based on the target vector through the fourth generative model.

[0114] The data processing device provided in the embodiments of the present disclosure can execute the data processing method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0115] It is worth noting that in the embodiment of the above-mentioned data processing device, the various modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, the specific names of the various functional modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present disclosure.

[0116] The present disclosure also provides an electronic device, which may include a processor and a memory, wherein the memory may be used to store executable instructions, wherein the processor may be used to read the executable instructions from the memory and execute the executable instructions to implement the data processing method in the above embodiment.

[0117] Figure 5 A schematic structural diagram of an electronic device provided by an embodiment of the present disclosure is shown.

[0118] like Figure 5 As shown, the electronic device 500 may include a processing device 501 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output interface (I / O interface) 505 is also connected to the bus 504.

[0119] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data.

[0120] It should be noted that Figure 5 The electronic device 500 shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure. Figure 5 The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0121] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the data processing method of any embodiment of the present disclosure are executed.

[0122] The embodiments of the present disclosure further provide a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the data processing method in any embodiment of the present disclosure.

[0123] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0124] In some embodiments, the client and server may communicate using any currently known or future developed network protocol such as HTTP, and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0125] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0126] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the steps of the data processing method described in any embodiment of the present disclosure.

[0127] In embodiments of the present disclosure, computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as a separate software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0128] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the equipment, method and computer program product according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0129] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0130] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0131] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0132] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0133] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A data processing method, characterized in that: include: Receive a data acquisition request; Parsing the data acquisition request, and determining, based on the parsing result, a first text for data management, a second text for data application, and a third text for assisting data request verification; Based on the first text and / or the third text, and the second text, the data acquisition request is verified to generate a verification result.

2. The method according to claim 1, characterized in that The step of parsing the data acquisition request and determining, based on the parsing result, a first text for data management, a second text for data application, and a third text for assisting data verification includes: Parsing the data acquisition request to determine data source information, data description information, and data usage information as the parsing result; Determine the first text based on the data source information; Determine the second text based on the data description information; The third text is determined based on the data usage information.

3. The method according to claim 1, characterized in that The verifying the data acquisition request based on the first text and / or the third text, and the second text, and generating a verification result includes: Performing vectorization processing on the first text, the second text, and the third text to generate a first vector, a second vector, and a third vector respectively; A verification process is performed based on the second vector and the target vector to generate the verification result; wherein the target vector includes the first vector and / or the third vector.

4. The method according to claim 3, characterized in that The vectorizing the first text, the second text, and the third text to generate a first vector, a second vector, and a third vector respectively includes: Inputting the first text into a first generative model to generate the first vector; Inputting the second text into a second generative model to generate the second vector; Inputting the first text, the second text and the third text into a third generative model to generate the third vector; Among them, the first generative model, the second generative model and the third generative model are all obtained by intensive training of the initial generative model.

5. The method according to claim 4, characterized in that The first generative model, the second generative model and the third generative model are pre-obtained in the following manner: Performing intensive training on the initial generative model using a first text sample for data management, a second text sample for data application, a third text sample for assisting data request verification, and verification-related text samples to generate an intermediate generative model; Using the first text sample and the first vector sample, fine-tune the intermediate generative model to generate the first generative model; Using the second text sample and the second vector sample, fine-tune the intermediate generative model to generate the second generative model; The intermediate generative model is fine-tuned using the first text sample and the first vector sample, the second text sample and the second vector sample, and the third text sample and the third vector sample to generate the third generative model.

6. The method according to claim 3, characterized in that The performing verification processing based on the second vector and the target vector to generate the verification result includes: Determining a vector operation result of the second vector and the target vector; The verification result of determining whether the data acquisition request passes the verification is based on whether the vector operation result meets the verification threshold.

7. The method according to claim 6, characterized in that After determining the verification result of whether the data acquisition request is verified based on whether the vector operation result satisfies the verification threshold, the method further includes: If the verification result is failure, a modification request prompt is generated based on the target vector through the fourth generative model.

8. A data processing device, characterized in that: include: A request receiving module, used for receiving a data acquisition request; A text determination module, used to parse the data acquisition request, and based on the parsing result, determine a first text for data management, a second text for data application, and a third text for assisting data request verification; A verification module is used to verify the data acquisition request based on the first text and / or the third text, and the second text, and generate a verification result.

9. An electronic device, characterized in that: include: processor; A memory for storing executable instructions; The processor is used to read the executable instructions from the memory and execute the executable instructions to implement the data processing method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the data processing method according to any one of claims 1 to 7.