Data processing method, device, electronic device and computer readable medium
By acquiring and converting value risk data sets, generating structured data sets, determining data characteristics and risk value information, the problems of low human processing efficiency and low accuracy of multimodal data are solved, and efficient value data sets are achieved Process and accurate storage.
Patent Information
- Application Number
- CN202410264419.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-08
- Publication Date
- 2024-12-24
- Estimated Expiration
- 2044-03-08
AI Technical Summary
In the prior art, artificially processing value risk data sets is inefficient, resulting in high pressure to store massive data, frequent server failures, and the accuracy of the multimodal data generation model is not accurate enough.
By obtaining the value risk data set of the target server, converting unstructured data into text modal data, generating structured data sets, determining data characteristics and risk business value information, generating data value information, and performing data storage, encryption, deletion, etc. Operation to slow down storage pressure.
It realizes efficient processing of value risk data sets, slows down the data storage pressure on the server, improves data processing efficiency, and ensures the accuracy and security of data storage.
Smart Images

Figure CN118113703B_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a data processing method, device, electronic device, and computer-readable medium. Background Art
[0002] At present, various fields pay more and more attention to the prediction of value risk. In the process of value risk prediction, a massive value risk data set is required. For the processing of value risk data, the usual method is to manually determine the data value corresponding to the value risk data set, and then perform the corresponding data processing. However, when using the above method to process value risk data, the following technical problems often occur:
[0003] Manual processing of value-risk data sets is inefficient, and as a result, the storage of massive value-risk data sets will cause heavy data storage pressure on the server, leading to server failures.
[0004] In the process of adopting technical solutions to solve the above technical problem one, the following technical problems often arise: the multimodal data generation model is crucial in the data integration process, and how to obtain an accurate multimodal data generation model has become the current main development direction. In response to the above technical problems, the conventional solution is generally to train the multimodal data generation model through conventional model training methods. However, the above conventional solution still has the following problem two: the accuracy of the obtained multimodal data generation model is still not accurate enough.
[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention
[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.
[0007] Some embodiments of the present disclosure propose data processing methods, devices, electronic devices, and computer-readable media to solve one or more of the technical problems mentioned in the above background technology section.
[0008] In a first aspect, some embodiments of the present disclosure provide a data processing method, including: obtaining a value-risk data set stored in a target server, wherein the value-risk data set includes: a first value-risk data set and a second value-risk data set, wherein the first value-risk data in the first value-risk data set is structured data, the second value-risk data in the second value-risk data set is unstructured data, and the second value-risk data set is a data set in a multimodal form; converting the second value-risk data in the second value-risk data set into data in a text mode to generate text-modal data to obtain a text-modal data set; generating a structured value-risk data set for the second value-risk data set according to a data attribute set corresponding to the second value-risk data set and the text-modal data set; determining first data feature value information and first risk business value information corresponding to the first value-risk data set; determining the second value-risk data in the second value-risk data set. The second data feature value information and the second risk business value information corresponding to the data set, and determine the structural conversion value impairment information for the above-mentioned second value risk data set and the above-mentioned structured value risk data set; determine the third data feature value information and the third risk business value information corresponding to the above-mentioned structured value risk data set; generate data value information for the above-mentioned value risk data set according to the above-mentioned first data feature value information, the above-mentioned first risk business value information, the above-mentioned second data feature value information, the above-mentioned second risk business value information, the above-mentioned structural conversion value impairment information, the above-mentioned third data feature value information and the third risk business value information; according to the above-mentioned data value information, execute server data processing operations for the above-mentioned value risk data set to perform data storage decompression for the above-mentioned target server, wherein the above-mentioned server data processing operations include at least one of the following: data storage operation, data encryption operation, data deletion operation, data sending operation.
[0009] In a second aspect, some embodiments of the present disclosure provide a data processing device, including: an acquisition unit, configured to acquire a value-risk data set stored in a target server, wherein the value-risk data set includes: a first value-risk data set and a second value-risk data set, wherein the first value-risk data in the first value-risk data set is structured data, the second value-risk data in the second value-risk data set is unstructured data, and the second value-risk data set is a data set in a multimodal form; a conversion unit, configured to convert the second value-risk data in the second value-risk data set into data in a text modality to generate text modality data and obtain a text modality data set; a first generation unit, configured to generate a structured value-risk data set for the second value-risk data set according to a data attribute set corresponding to the second value-risk data set and the text modality data set; a first determination unit, configured to determine first data feature value information and first risk business value information corresponding to the first value-risk data set; and a second determination unit, configured to Determine the second data feature value information and the second risk business value information corresponding to the above-mentioned second value-risk data set, and determine the structural conversion value impairment information for the above-mentioned second value-risk data set and the above-mentioned structured value-risk data set; a third determination unit is configured to determine the third data feature value information and the third risk business value information corresponding to the above-mentioned structured value-risk data set; a second generation unit is configured to generate data value information for the above-mentioned value-risk data set based on the above-mentioned first data feature value information, the above-mentioned first risk business value information, the above-mentioned second data feature value information, the above-mentioned second risk business value information, the above-mentioned structural conversion value impairment information, the above-mentioned third data feature value information and the third risk business value information; an execution unit is configured to execute a server data processing operation for the above-mentioned value-risk data set according to the above-mentioned data value information, so as to perform data storage decompression for the above-mentioned target server, wherein the above-mentioned server data processing operation includes at least one of the following: data storage operation, data encryption operation, data deletion operation, data sending operation.
[0010] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.
[0011] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner in the first aspect is implemented.
[0012] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the data processing method of some embodiments of the present disclosure, the value risk data set in the target server can be processed quickly and efficiently to relieve the data storage pressure. Specifically, the reason for the high data storage pressure is that the efficiency of artificial processing of the value risk data set is low, and then the storage of a large amount of value risk data sets will cause the data storage pressure of the server to be high, resulting in server failure. Based on this, the data processing method of some embodiments of the present disclosure first obtains the value risk data set stored in the target server. Wherein, the above-mentioned value risk data set includes: a first value risk data set and a second value risk data set, wherein the first value risk data in the above-mentioned first value risk data set is structured data, the second value risk data in the above-mentioned second value risk data set is unstructured data, and the above-mentioned second value risk data set is a data set in a multimodal form. The acquired value risk data set is convenient for determining the corresponding subsequent data value information and realizing the data processing of the target server. Then, the second value risk data in the above-mentioned second value risk data set is converted into data in text mode to generate text mode data, and a text mode data set is obtained to facilitate the determination of subsequent data value information. Then, according to the data attribute set corresponding to the above-mentioned second value risk data set and the above-mentioned text modal data set, a structured value risk data set for the above-mentioned second value risk data set can be accurately generated. Through the structured value risk data set, it is convenient to determine the subsequent data value information. Furthermore, the first data feature value information and the first risk business value information corresponding to the above-mentioned first value risk data set can be accurately determined. Then, the second data feature value information and the second risk business value information corresponding to the above-mentioned second value risk data set are accurately determined, and the structural conversion value impairment information for the above-mentioned second value risk data set and the above-mentioned structured value risk data set is accurately determined. Secondly, the third data feature value information and the third risk business value information corresponding to the above-mentioned structured value risk data set can be accurately determined. Here, by determining the first data feature value information, the above-mentioned first risk business value information, the above-mentioned second data feature value information, the above-mentioned second risk business value information, the above-mentioned structural conversion value impairment information, the above-mentioned third data feature value information and the third risk business value information, for the subsequent structured data, the actual value of the value risk data set is comprehensively considered. Furthermore, based on the above-mentioned first data feature value information, the above-mentioned first risk business value information, the above-mentioned second data feature value information, the above-mentioned second risk business value information, the above-mentioned structural conversion value impairment information, the above-mentioned third data feature value information and the third risk business value information, data value information for the above-mentioned value-risk data set can be accurately generated.Finally, based on the data value information, the server data processing operation for the value risk data set is accurately and efficiently performed to decompress the data storage for the target server. The server data processing operation includes at least one of the following: data storage operation, data encryption operation, data deletion operation, and data sending operation. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.
[0014] Figure 1 are flow charts of some embodiments of the data processing method according to the present disclosure;
[0015] Figure 2 is a schematic diagram of the structure of some embodiments of the data processing device according to the present disclosure;
[0016] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION
[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0018] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.
[0019] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0020] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0021] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0022] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.
[0023] refer to Figure 1 , shows a process 100 of some embodiments of the data processing method according to the present disclosure. The data processing method comprises the following steps:
[0024] Step 101: Acquire a value-risk data set stored in a target server.
[0025] In some embodiments, the execution subject of the above data processing method can obtain the value-risk data set stored in the target server through a wired connection or a wireless connection. The target server may be a server to be processed. The value-risk data set may be a data set related to a value-risk business. In practice, for a credit investigation scenario, the value-risk business may be a credit investigation risk detection business. The above-mentioned value-risk data set includes: a first value-risk data set and a second value-risk data set. The first value-risk data in the above-mentioned first value-risk data set is structured data. The second value-risk data in the above-mentioned second value-risk data set is unstructured data, and the above-mentioned second value-risk data set is a data set in a multimodal form. In practice, the multimodal form may include but is not limited to at least one of the following: an image modality form, a text modality form, an audio modality form, and a video modality form.
[0026] Step 102: convert the second value risk data in the above second value risk data set into data in text mode to generate text modality data and obtain a text modality data set.
[0027] In some embodiments, the execution subject may convert the second value risk data in the second value risk data set into data in text mode to generate text mode data and obtain a text mode data set. The data in text mode may be data in text form.
[0028] As an example, when the second value risk data is data in image mode, the execution entity may use a text recognition algorithm to identify text information corresponding to the second value risk data to obtain text modality data.
[0029] As another example, when the second value risk data is data in audio mode, the execution entity may use an audio-to-text conversion algorithm to identify text information corresponding to the second value risk data to obtain text modality data.
[0030] Step 103: Generate a structured value-risk data set for the second value-risk data set based on the data attribute set corresponding to the second value-risk data set and the text modality data set.
[0031] In some embodiments, the execution entity may generate a structured value risk data set for the second value risk data set based on the data attribute set corresponding to the second value risk data set and the text modality data set, wherein the structured value risk data may be structured value risk data.
[0032] In some optional implementations of some embodiments, generating a structured value-risk data set for the second value-risk data set according to the data attribute set corresponding to the second value-risk data set and the text modality data set may include the following steps:
[0033] The first step is to generate a first initial structured value-risk data set for the text modality data set according to the data attribute set, wherein the first initial structured value-risk data set may be a value-risk data set after the text modality data set is transformed into a structured form.
[0034] The second step is to perform multimodal division on each second value risk data in the above-mentioned second value risk data set to generate a second value risk data set. Among them, the data modality corresponding to each second value risk data in the above-mentioned second value risk data set is the same modality. For example, the second value risk data set includes: second value risk data group A, second value risk data group B and second value risk data group C. The modal form corresponding to each second value risk data in the second value risk data group A is image modality. The modal form corresponding to each second value risk data in the second value risk data group B is text modality. The video form corresponding to each second value risk data in the second value risk data group C is image modality.
[0035] The third step is to generate a summary structured data set corresponding to the second value risk data set, wherein the summary structured data set may be a data set obtained by structurally transforming each second value risk data set in the second value risk data set and summarizing data information.
[0036] The fourth step is to adjust the individual summarized structured data in the summarized structured data set according to the data attribute set to generate an adjusted data set as the second initial structured value-risk data set.
[0037] As an example, for each summarized structured data in the summarized structured data set, a data attribute information set corresponding to the data attribute set is extracted from the summarized structured data as adjustment data.
[0038] The fifth step is to generate the structured value-risk data set according to the first initial structured value-risk data set and the second initial structured value-risk data set.
[0039] As an example, the execution entity may perform a data set union process on the first initial structured value-at-risk data set and the second initial structured value-at-risk data set to generate a unioned data set as the structured value-at-risk data set.
[0040] Optionally, the generating of the summarized structured data set corresponding to the second value-at-risk data set may include the following steps:
[0041] In the first step, for each second value risk data set in the second value risk data set, the following first generation step is performed:
[0042] The first sub-step is to determine the modal information corresponding to the second value risk data group, wherein the modal information may be a model form corresponding to the second value risk data group.
[0043] The second sub-step is to determine the structural conversion method corresponding to the above modal information. The structural conversion method may be a conversion method for converting unstructured data into structured data. For example, when the modal information is in the form of an image modality, a structural conversion method corresponding to the image modality exists. In practice, the structural conversion method corresponding to each modal information may be preset.
[0044] The third sub-step is to perform structural transformation on the second value risk data group according to the structural transformation method to generate a structured data group.
[0045] As an example, the above structural transformation method is executed to perform structural transformation on the above second value risk data group to generate a structured data group.
[0046] The second step is to aggregate the individual structured data in the obtained structured data set according to the above data attribute set to generate an aggregated structured data set.
[0047] As an example, first, for each structured data in the structured data set, a data attribute information group corresponding to the data attribute set is extracted from the structured data. Then, at least one data attribute information group in the obtained data attribute information set is aggregated to generate aggregated structured data, thereby obtaining an aggregated structured data set. Among them, there are target number of data attribute information with the same value between each data attribute information group in at least one data attribute information group.
[0048] Optionally, the generating of the first initial structured value-risk data set for the text modality data set according to the data attribute set may include the following steps:
[0049] In the first step, for each text modality data in the above text modality dataset, the following second generation step is performed:
[0050] The first sub-step is to extract the keyword set corresponding to the above text modal data.
[0051] As an example, the execution entity may extract a keyword set corresponding to the text modality data through a TF-IDF method.
[0052] The second sub-step is to execute the following third generation step for each keyword in the above keyword set:
[0053] Sub-step 1, determining the word semantic importance information corresponding to the above-mentioned keyword, wherein the above-mentioned word semantic importance information represents the importance of the word semantics corresponding to the above-mentioned keyword relative to the data semantics corresponding to the text modal data. The greater the word semantic importance information, the higher the importance of the word semantics corresponding to the above-mentioned keyword relative to the data semantics corresponding to the text modal data.
[0054] As an example, first, the execution subject may extract the word semantics corresponding to the keyword. Then, the data semantics corresponding to the text modal data may be extracted. The word semantics may be information in the form of a vector. The data semantics may also be information in the form of a vector. Finally, the word semantics and the data semantics are input into a multi-layer convolutional neural network to output the word semantic importance information.
[0055] Sub-step 2, input the above keywords into the data attribute identifier generation model to generate the data attribute identifier corresponding to the above keywords as the target data attribute identifier. The data attribute identifier generation model can be a neural network model for generating data attribute identifiers. There is a one-to-one corresponding data attribute for the data attribute identifier. In practice, the data attribute identifier generation model can be a classification model.
[0056] The third sub-step is to filter out keywords whose corresponding target data attribute identifiers are in the corresponding data attribute identifier set of the data attribute set from the above keyword set to obtain a keyword subset.
[0057] The fourth sub-step is to select keywords whose corresponding word semantic importance information is greater than a target value from the above keyword subset to obtain at least one keyword, wherein the target value may be a preset value.
[0058] The fifth sub-step is to establish a knowledge graph for the text modal data using the at least one keyword as a key node, wherein the nodes in the knowledge graph are the words in the text modal data, and the edges in the knowledge graph represent the association between two words.
[0059] In the second step, according to the graph closeness between each two knowledge graphs, the knowledge graphs in the obtained knowledge graph set are fused to generate a fused graph and obtain a fused graph set. The graph closeness between two knowledge graphs can represent the degree of graph overlap between the two knowledge graphs. In practice, the degree of graph overlap can be reflected by the degree of node overlap and the degree of edge overlap.
[0060] Specifically, the graph fusion between knowledge graphs may be the fusion between corresponding graph nodes and graph edges.
[0061] Step 3: For each fusion atlas in the above fusion atlas set, perform the following fourth generation step:
[0062] The first sub-step is to extract the atlas data from the above fused atlas to obtain the atlas dataset.
[0063] As an example, first, the execution subject may extract multiple paths whose corresponding path lengths are greater than the target length from the fused graph, and then extract multiple graph data corresponding to the multiple paths from the fused graph as the graph data set.
[0064] The second sub-step is to supplement and / or screen each atlas data in the atlas data set according to the data attribute set to generate processed atlas data and obtain a processed atlas data group.
[0065] As an example, for each atlas data in the atlas data set, first, determine whether the data attribute subset corresponding to the above atlas data is equivalent to the above data attribute set. Then, in response to determining that the data attribute subset includes the data attribute set, remove the data information whose corresponding attributes do not belong to the above data attribute set from the above atlas data to generate processed atlas data. In response to determining that the data attribute set includes the data attribute subset, add blank data information corresponding to the deviation data set between the data attribute set and the data attribute subset from the above atlas data to generate processed atlas data. Similarly, in the case of whether the data attribute subset is equivalent to the above data attribute set, the blank data information of the corresponding data attributes is supplemented and the data information of the corresponding data attributes is screened to generate processed atlas data.
[0066] The fourth step is to perform structured processing on the obtained processed graph data set to generate a first initial structured value-risk data set.
[0067] As an example, the execution entity may process the graph data in the graph data set into data in a target structured form to generate first initial structured value-risk data, thereby obtaining a first initial structured value-risk data set.
[0068] In some optional implementations of some embodiments, the generating of the summary structured data set corresponding to the second value-at-risk data set may include the following steps:
[0069] The first step is to determine the data keyword set and data semantic information corresponding to each second value risk data in the above second value risk data set. The data semantic information may be the data content semantics corresponding to the second value risk data. The data semantic information may be information in vector form.
[0070] In the second step, data is divided for each second value risk data in the above second value risk data set according to the data keyword set and data semantic information corresponding to each second value risk data, so as to generate a value risk data set. Among them, the number of repetitions of data keywords between every two value risk data in each value risk data set is greater than the target number, and the difference in data semantic information between them is less than the target difference. The difference in data semantic information can be the vector cosine distance between vectors.
[0071] The third step is to perform data group division verification on each value risk data group in the above value risk data group set to generate verification results.
[0072] As an example, the execution subject may perform data clustering processing on each value risk data in the value risk data set to generate a clustering result. Determine whether the clustering result is consistent with the data group division method corresponding to the value risk data set. In response to determining consistency, generate a verification result indicating that the verification has been passed. In response to determining inconsistency, generate a verification result indicating that the verification has not been passed.
[0073] In the fourth step, in response to determining that the verification result indicates that the division of the above-mentioned value-risk data groups is correct, for each value-risk data group in the above-mentioned value-risk data group set, the following seventh generation step is performed:
[0074] In the first sub-step, the value risk data in the value risk data group are divided according to the modal form corresponding to the value risk data to generate a value risk data subset set, wherein the modal form corresponding to each value risk data in each value risk data subset in the value risk data subset set is the same.
[0075] The second sub-step is to obtain a pre-trained multimodal data generation model. The multimodal data generation model includes a feature extraction model set, a feature integration model based on a multi-head attention mechanism, and an attribute data output layer set. The feature extraction models in the feature extraction model set have a one-to-one corresponding modal form, and the attribute data output layer in the attribute data output layer set has a one-to-one corresponding relationship with the data attributes in the data attribute set. The multimodal data generation model can be a neural network model for generating multimodal data. The feature extraction models in the feature extraction model set can be set with a corresponding model structure according to the corresponding modal form. For example, for the image modality, the corresponding feature extraction model can be a multi-layer series convolutional neural network model. The feature integration model based on the multi-head attention mechanism can be: a multi-head attention mechanism model + a convolutional neural network model + a fully connected layer. The attribute data output layer set can be set with a corresponding model structure according to the corresponding modal form. For example, for the image modality, the corresponding attribute data output layer can be a multi-layer series fully connected layer.
[0076] In a third sub-step, in response to determining that the first modal form set corresponding to the value-risk data subset is smaller than the second modal form set corresponding to the feature extraction model set, at least one deviation model form between the first modal form set and the second modal form set is determined. The at least one deviation model form may be a modal form difference between the first modal form set and the second modal form set.
[0077] The fourth sub-step is to perform model pruning on at least one feature extraction model in the feature extraction model set to obtain a pruned feature extraction model set, wherein there is a one-to-one correspondence between the feature extraction model in the at least one feature extraction model and the deviation model form in the at least one deviation model form.
[0078] The fifth sub-step is to input each value-risk data subgroup in the above-mentioned value-risk data subgroup set into the corresponding feature extraction model in the above-mentioned trimmed feature extraction model set to generate a risk feature extraction information subgroup and obtain a risk feature extraction information subgroup set.
[0079] The sixth sub-step is to input the above-mentioned risk feature extraction information subset set into the above-mentioned feature integration model based on the multi-head attention mechanism to generate a data attribute feature information set for the above-mentioned data attribute set.
[0080] The seventh sub-step is to generate a data attribute information set for the data attribute feature information set using the attribute data output layer set.
[0081] An eighth sub-step is to generate summary structured data for the above value-risk data group based on the above data attribute information set.
[0082] As an example, the execution subject may aggregate each piece of data attribute information in the data attribute information set in a target structured form to generate aggregated structured data.
[0083] Considering the problems of the above conventional solutions, facing the above technical problem 2: the accuracy of the obtained multimodal data generation model is still not accurate enough. Combining the advantages / technical status, we can decide to adopt the following solution.
[0084] In some optional implementations of some embodiments, the multimodal data generation model is generated by the following steps:
[0085] The first step is to obtain a training data set, wherein the training data includes a data subset in a multimodal form, wherein a data subset in a plurality of data subsets has a one-to-one correspondence with a modality in the multimodal form.
[0086] The second step is to select target training data from the training data set.
[0087] In the third step, for the target training data, perform the following training steps:
[0088] The first sub-step is to determine multiple data subsets corresponding to the target training data as multiple target data subsets.
[0089] In the second sub-step, each target data subset in the multiple target data subsets is input into the corresponding initial feature extraction model in the initial feature extraction model set to generate a risk feature extraction information subset, thereby obtaining multiple risk feature extraction information subsets. The initial multimodal data generation model includes: an initial feature extraction model set, an initial feature integration model based on a multi-head attention mechanism, an initial keyword generation model set corresponding to the multimodal form, at least one initial attribute association model, and an initial attribute data output layer set. There is a one-to-one correspondence between the initial keyword generation model in the initial keyword generation model set and the modal form in the multimodal form. At least one initial attribute association model may be a model pre-set for at least one attribute pair. There is a one-to-one correspondence between the initial attribute association model in at least one initial attribute association model and the attribute pair in at least one attribute pair. Among them, the initial attribute association model may characterize the numerical association relationship between the attribute pairs.
[0090] The third sub-step is to input each of the multiple risk feature extraction information subsets into an initial feature integration model based on a multi-head attention mechanism to generate an initial data attribute feature information set for the data attribute set.
[0091] The fourth sub-step is to generate an initial data attribute information set for the initial data attribute feature information set using the initial attribute data output layer set.
[0092] The fifth sub-step is to combine the individual initial data attribute characteristic information in the initial data attribute characteristic information set according to the at least one attribute pair to generate at least one initial data attribute characteristic information pair, wherein the initial data attribute characteristic information pairs in the at least one initial data attribute characteristic information pair have a one-to-one correspondence with the attribute pairs in the at least one attribute pair.
[0093] The sixth sub-step is to input the at least one initial data attribute feature information pair into at least one initial attribute association model to generate at least one attribute association information.
[0094] The seventh sub-step is to input each of the multiple risk feature extraction information subsets into the corresponding initial keyword generation model in the initial keyword generation model set to generate a keyword set corresponding to the data subset, thereby obtaining multiple keyword sets.
[0095] An eighth sub-step is to obtain a plurality of actual keyword sets, at least one actual attribute association information and an actual data attribute information set included in the target training data.
[0096] The ninth sub-step is to generate keyword loss information between multiple actual keyword sets and the above multiple keyword sets.
[0097] The tenth sub-step is to generate attribute association loss information between at least one actual attribute association information and at least one attribute association information.
[0098] The eleventh sub-step is to generate data attribute loss information between the actual data attribute information set and the data attribute information set.
[0099] In the twelfth sub-step, weighted summation is performed on the keyword loss information, the attribute association loss information and the data attribute loss information to obtain weighted summation information.
[0100] The thirteenth sub-step is, in response to determining that the weighted sum information is less than the target value, determining the initial multimodal data generation model as the multimodal data generation model.
[0101] In a fourth step, in response to determining that the weighted sum information is greater than or equal to the target value, the model parameters of the initial multimodal data generation model are updated according to the weighted sum information to generate an updated multimodal data generation model.
[0102] The fifth step is to remove the target training data from the training data set to obtain the removed training data set.
[0103] The sixth step is to reselect training data from the removed training data set as candidate training data.
[0104] In the seventh step, the updated multimodal data generation model is used as the initial multimodal data generation model, and the candidate training data is used as the target training data to continue the training step.
[0105] The content of "some optional implementations in some embodiments" as an inventive point of the present disclosure solves the technical problem mentioned in the background technology that "the accuracy of the obtained multimodal data generation model is still not accurate enough." Based on this, the present disclosure can accurately implement the model training of the initial multimodal data generation model by at least one initial attribute association model and an initial keyword generation model set, while effectively ensuring the accuracy of at least one initial data attribute feature information and multiple risk feature extraction information subsets output, so that a more accurate multimodal data generation model can be obtained later.
[0106] Step 104: determine the first data feature value information and the first risk business value information corresponding to the first value risk data set.
[0107] In some embodiments, the execution entity may determine the first data feature value information and the first risk business value information corresponding to the first value risk data set. The first data feature value information may characterize the value importance of the data attribute set corresponding to the first value risk data set. The first risk business value information may characterize the value importance of the first value risk data set in the relevant risk business scenario.
[0108] In some optional implementations of some embodiments, the determining of the first data feature value information and the first risk business value information corresponding to the first value risk data set may include the following steps:
[0109] The first step is to obtain the data attribute set corresponding to the first value risk data set as the target data attribute set. In practice, the data attribute set may include but is not limited to at least one of the following: user information, user loan value, user loan object, user mortgage object, and user assets.
[0110] The second step is to obtain the data usage history records corresponding to the first value risk dataset, wherein the data usage history records may be the dataset usage records corresponding to the first value risk dataset.
[0111] The third step is to generate at least one data domain information and data accuracy evaluation information set for the first value risk data set based on the data usage history.
[0112] As an example, the execution subject may extract domain information related to the usage domain from the dataset usage record to obtain at least one data domain information. For example, the data domain information may be a loan domain. The data accuracy evaluation information in the data accuracy evaluation information set may be the dataset evaluation information of the user on the first value risk dataset.
[0113] Step 4: for each piece of data field information in the at least one piece of data field information, execute the following fifth generation step:
[0114] The first sub-step is to input the data domain information and the target data attribute set into an attribute and domain matching information generation model to generate attribute and domain matching information.
[0115] The second sub-step is to determine the domain importance information corresponding to the data domain information. The domain importance information can represent the importance corresponding to the data domain information. Specifically, the execution subject can use the domain importance association table to determine the domain importance information corresponding to the data domain information.
[0116] The third sub-step is to obtain a complete data attribute set corresponding to the above data domain information.
[0117] The complete data attribute set may be a representative data attribute set required for the data domain information. Specifically, the complete data attribute set corresponding to each data domain information may be preset.
[0118] The fourth sub-step is to determine the data attribute difference between the target data attribute set and the complete data attribute set. The data attribute difference can represent the difference in data attributes between the target data attribute set and the complete data attribute set.
[0119] The fifth sub-step is to generate the first data feature value sub-information for the data domain information according to the attribute and domain matching information, the domain importance information and the data attribute difference.
[0120] The fifth step is to generate candidate data feature value information based on the obtained at least one first data feature value sub-information.
[0121] The sixth step is to generate data accuracy value information for the above-mentioned target data attribute set based on the above-mentioned data accuracy evaluation information set.
[0122] As an example, first, the execution subject may determine the accuracy evaluation score corresponding to each piece of data accuracy evaluation information in the data accuracy evaluation information set to obtain an accuracy evaluation score set. Finally, the average score corresponding to each accuracy evaluation score in the accuracy evaluation score set is determined as the data accuracy value information.
[0123] The seventh step is to perform weighted summation of the candidate data feature value information and the data accuracy value information to generate the first data feature value information.
[0124] Step 8: Obtain a risk-related information generation model set corresponding to the risk business information set, wherein the risk-related information in the risk-related information generation model set and the risk business information in the risk business information set have a one-to-one correspondence.
[0125] In step 9, for each risk-related information generation model in the risk-related information generation model set, the following sixth generation step is performed:
[0126] The first sub-step is to input the first value risk data set into the risk related information generation model to generate initial risk related information.
[0127] The second sub-step is to determine the information gap between the initial risk-related information and the actual risk-related information.
[0128] The tenth step is to generate the first risk business value information for the first value risk data set according to the obtained information difference set.
[0129] Step 105, determining the second data feature value information and the second risk business value information corresponding to the second value-risk data set, and determining the structural conversion value impairment information for the second value-risk data set and the structured value-risk data set.
[0130] In some embodiments, the above-mentioned execution entity can determine the second data feature value information and the second risk business value information corresponding to the above-mentioned second value-risk data set, and determine the structural conversion value impairment information for the above-mentioned second value-risk data set and the above-mentioned structured value-risk data set. Among them, the second data feature value information can characterize the value importance of the data attribute set corresponding to the second value-risk data set. The second risk business value information can characterize the value importance of the second value-risk data set in the relevant risk business scenario. The structural conversion value impairment information can characterize the degree of value impairment caused by the conversion of the second value-risk data set into the structured value-risk data set. In practice, the higher the corresponding numerical value of the structural conversion value impairment information, the higher the value impairment caused by the conversion of the second value-risk data set into the structured value-risk data set. Among them, the structural conversion value impairment information can be a pre-set numerical value, or it can be obtained by training a related neural network model.
[0131] Step 106: Determine the third data feature value information and the third risk business value information corresponding to the structured value-risk data set.
[0132] In some embodiments, the execution subject may determine the third data feature value information and the third risk business value information corresponding to the structured value risk data set. The third data feature value information may characterize the value importance of the data attribute set corresponding to the structured value risk data set. The third risk business value information may characterize the value importance of the structured value risk data set in the relevant risk business scenario.
[0133] Step 107, based on the above-mentioned first data feature value information, the above-mentioned first risk business value information, the above-mentioned second data feature value information, the above-mentioned second risk business value information, the above-mentioned structural conversion value impairment information, the above-mentioned third data feature value information and the third risk business value information, generate data value information for the above-mentioned value-risk data set.
[0134] In some embodiments, the execution subject may generate data value information for the value-risk data set based on the first data feature value information, the first risk business value information, the second data feature value information, the second risk business value information, the structure conversion value impairment information, the third data feature value information, and the third risk business value information. The data value information may characterize the size of the data value corresponding to the value-risk data set. That is, the larger the data value information, the more important the corresponding value-risk data set.
[0135] In some optional implementations of some embodiments, the above-mentioned structural conversion value impairment information includes: data feature value impairment information and risk business value impairment information. Among them, the data feature value impairment information can be the value impairment information of the data feature. The risk business value impairment information can be the value impairment information of the risk business. In practice, the value impairment information can represent the transformation of the value size.
[0136] Optionally, generating the data value information for the value-risk data set according to the first data feature value information, the first risk business value information, the second data feature value information, the second risk business value information, the structure conversion value impairment information, the third data feature value information and the third risk business value information may include the following steps:
[0137] The first step is to multiply the second data feature value information and the data feature value depreciation information to generate first multiplied value information.
[0138] The second step is to multiply the second risk business value information and the risk business value impairment information to generate second multiplied value information.
[0139] The third step is to determine the average value between the first multiplied value information and the third data feature value information to obtain the first average value, and to determine the average value between the second multiplied value information and the third risk business information to obtain the second average value.
[0140] The fourth step is to add the first average value and the first data feature value information to obtain first added information, and to add the second average value and the first risk business value information to obtain second added information.
[0141] The fifth step is to combine the first addition information and the second addition information in the form of a key-value pair to generate the first data value information.
[0142] The sixth step is to divide the third data feature value information by the data feature value depreciation information to generate the first division value information.
[0143] The seventh step is to divide the third risk business value information by the risk business value impairment information to generate the second division value information.
[0144] The eighth step is to determine the average value between the first division value information and the second data feature value information to obtain a third average value, and to determine the average value between the second division value information and the second risk business information to obtain a fourth average value.
[0145] The ninth step is to add the third average value and the first data feature value information to obtain third added information, and to add the fourth average value and the first risk business value information to obtain fourth added information.
[0146] The tenth step is to combine the third addition information and the fourth addition information in the form of a key-value pair to generate second data value information.
[0147] In the eleventh step, the first data value information and the second data value information are packaged in the form of a key-value pair to generate packaged information as the data value information.
[0148] Step 108: Execute a server data processing operation on the value-at-risk data set according to the data value information to decompress the data storage on the target server.
[0149] In some embodiments, the execution subject may perform a server data processing operation on the value risk data set according to the data value information to decompress the data storage on the target server. The server data processing operation includes at least one of the following: data storage operation, data encryption operation, data deletion operation, and data sending operation.
[0150] As an example, the execution subject may execute the server data processing operation on the value-risk data set according to the value of the data value information.
[0151] For example, if the data value information is between [100, 200], the data storage operation is performed. If the data value information is between (200, 400], the data encryption operation is performed.
[0152] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the data processing method of some embodiments of the present disclosure, the value risk data set in the target server can be processed quickly and efficiently to relieve the data storage pressure. Specifically, the reason for the high data storage pressure is that the efficiency of artificial processing of the value risk data set is low, and then the storage of a large amount of value risk data sets will cause the data storage pressure of the server to be high, resulting in server failure. Based on this, the data processing method of some embodiments of the present disclosure first obtains the value risk data set stored in the target server. Wherein, the above-mentioned value risk data set includes: a first value risk data set and a second value risk data set, wherein the first value risk data in the above-mentioned first value risk data set is structured data, the second value risk data in the above-mentioned second value risk data set is unstructured data, and the above-mentioned second value risk data set is a data set in a multimodal form. The acquired value risk data set is convenient for determining the corresponding subsequent data value information and realizing the data processing of the target server. Then, the second value risk data in the above-mentioned second value risk data set is converted into data in text mode to generate text mode data, and a text mode data set is obtained to facilitate the determination of subsequent data value information. Then, according to the data attribute set corresponding to the above-mentioned second value risk data set and the above-mentioned text modal data set, a structured value risk data set for the above-mentioned second value risk data set can be accurately generated. Through the structured value risk data set, it is convenient to determine the subsequent data value information. Furthermore, the first data feature value information and the first risk business value information corresponding to the above-mentioned first value risk data set can be accurately determined. Then, the second data feature value information and the second risk business value information corresponding to the above-mentioned second value risk data set are accurately determined, and the structural conversion value impairment information for the above-mentioned second value risk data set and the above-mentioned structured value risk data set is accurately determined. Secondly, the third data feature value information and the third risk business value information corresponding to the above-mentioned structured value risk data set can be accurately determined. Here, by determining the first data feature value information, the above-mentioned first risk business value information, the above-mentioned second data feature value information, the above-mentioned second risk business value information, the above-mentioned structural conversion value impairment information, the above-mentioned third data feature value information and the third risk business value information, for the subsequent structured data, the actual value of the value risk data set is comprehensively considered. Furthermore, based on the above-mentioned first data feature value information, the above-mentioned first risk business value information, the above-mentioned second data feature value information, the above-mentioned second risk business value information, the above-mentioned structural conversion value impairment information, the above-mentioned third data feature value information and the third risk business value information, data value information for the above-mentioned value-risk data set can be accurately generated.Finally, based on the data value information, the server data processing operation for the value risk data set is accurately and efficiently performed to decompress the data storage for the target server. The server data processing operation includes at least one of the following: data storage operation, data encryption operation, data deletion operation, and data sending operation.
[0153] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a data processing device. These device embodiments are Figure 1 Corresponding to the method embodiments shown, the data processing device can be specifically applied to various electronic devices.
[0154] like Figure 2As shown, a data processing device 200 includes: an acquisition unit 201, a conversion unit 202, a first generation unit 203, a first determination unit 204, a second determination unit 205, a third determination unit 206, a second generation unit 207 and an execution unit 208. Wherein, the acquisition unit 201 is configured to acquire a value-risk data set stored in a target server, wherein the above-mentioned value-risk data set includes: a first value-risk data set and a second value-risk data set, wherein the first value-risk data in the above-mentioned first value-risk data set is structured data, the second value-risk data in the above-mentioned second value-risk data set is unstructured data, and the above-mentioned second value-risk data set is a data set in a multimodal form; the conversion unit 202 is configured to convert the second value-risk data in the above-mentioned second value-risk data set into data in a text mode to generate text-modal data and obtain a text-modal data set; the first generation unit 203 is configured to generate a structured value-risk data set for the above-mentioned second value-risk data set according to the data attribute set corresponding to the above-mentioned second value-risk data set and the above-mentioned text-modal data set; the first determination unit 204 is configured to determine the first data feature value information and the first risk business value information corresponding to the above-mentioned first value-risk data set; the second determination unit 205 is configured to determine the above-mentioned second value-risk data set. The second data feature value information and the second risk business value information corresponding to the data set, and the determination of the structural conversion value impairment information for the above-mentioned second value risk data set and the above-mentioned structured value risk data set; the third determination unit 206 is configured to determine the third data feature value information and the third risk business value information corresponding to the above-mentioned structured value risk data set; the second generation unit 207 is configured to generate the data value information for the above-mentioned value risk data set according to the above-mentioned first data feature value information, the above-mentioned first risk business value information, the above-mentioned second data feature value information, the above-mentioned second risk business value information, the above-mentioned structural conversion value impairment information, the above-mentioned third data feature value information and the third risk business value information; the execution unit 208 is configured to execute the server data processing operation for the above-mentioned value risk data set according to the above-mentioned data value information, so as to perform server data storage decompression for the above-mentioned target server, wherein the above-mentioned data processing operation includes at least one of the following: data storage operation, data encryption operation, data deletion operation, data sending operation.
[0155] It is understood that the units described in the data processing device 200 are similar to those described in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the data processing device 200 and the units included therein, and will not be described in detail here.
[0156] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device (eg, an electronic device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0157] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.
[0158] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.
[0159] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.
[0160] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0161] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0162] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist independently without being assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains a value-risk data set stored in a target server, wherein the above-mentioned value-risk data set includes: a first value-risk data set and a second value-risk data set, wherein the first value-risk data in the above-mentioned first value-risk data set is structured data, the second value-risk data in the above-mentioned second value-risk data set is unstructured data, and the above-mentioned second value-risk data set is a data set in a multimodal form; converts the second value-risk data in the above-mentioned second value-risk data set into data in text mode to generate text modal data and obtain a text modal data set; generates a structured value-risk data set for the above-mentioned second value-risk data set according to the data attribute set corresponding to the above-mentioned second value-risk data set and the above-mentioned text modal data set; determines the first data feature value information and the first risk business value information corresponding to the above-mentioned first value-risk data set. ; Determine the second data feature value information and the second risk business value information corresponding to the above-mentioned second value-risk data set, and determine the structural conversion value impairment information for the above-mentioned second value-risk data set and the above-mentioned structured value-risk data set; Determine the third data feature value information and the third risk business value information corresponding to the above-mentioned structured value-risk data set; Generate data value information for the above-mentioned value-risk data set based on the above-mentioned first data feature value information, the above-mentioned first risk business value information, the above-mentioned second data feature value information, the above-mentioned second risk business value information, the above-mentioned structural conversion value impairment information, the above-mentioned third data feature value information and the third risk business value information; According to the above-mentioned data value information, execute a server data processing operation for the above-mentioned value-risk data set to perform data storage decompression for the above-mentioned target server, wherein the above-mentioned server data processing operation includes at least one of the following: data storage operation, data encryption operation, data deletion operation, data sending operation.
[0163] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0164] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0165] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor including an acquisition unit, a conversion unit, a first generation unit, a first determination unit, a second determination unit, a third determination unit, a second generation unit, and an execution unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the acquisition unit may also be described as a "unit for acquiring a value risk data set stored on a target server".
[0166] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0167] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.
Claims
1. A data processing method, comprising: Acquire a value-risk data set stored in a target server, wherein the value-risk data set includes: a first value-risk data set and a second value-risk data set, wherein the first value-risk data in the first value-risk data set is structured data, the second value-risk data in the second value-risk data set is unstructured data, and the second value-risk data set is a data set in a multimodal form; Converting the second value risk data in the second value risk data set into data in a text mode to generate text mode data, thereby obtaining a text mode data set; Generating a structured value-risk dataset for the second value-risk dataset according to the data attribute set corresponding to the second value-risk dataset and the text modality dataset, wherein generating a structured value-risk dataset for the second value-risk dataset according to the data attribute set corresponding to the second value-risk dataset and the text modality dataset comprises: Generating a first initial structured value-risk data set for the text modality data set according to the data attribute set; Performing multimodal division on each second value risk data in the second value risk data set to generate a second value risk data group set, wherein the data modalities corresponding to each second value risk data in the second value risk data group are the same modality; generating a summary structured data set corresponding to the second value-at-risk data set; According to the data attribute set, performing data adjustment on each summarized structured data in the summarized structured data set to generate an adjusted data set as a second initial structured value-risk data set; The structured value-at-risk data set is generated according to the first initial structured value-at-risk data set and the second initial structured value-at-risk data set, wherein: The generating of a summary structured data set corresponding to the second value-risk data set comprises: For each second value-at-risk data set in the second value-at-risk data set set, the following first generating step is performed: Determining modal information corresponding to the second value risk data group; Determining a structural transformation mode corresponding to the modal information; According to the structural conversion method, the second value risk data group is structurally converted to generate a structured data group; According to the data attribute set, each structured data in the obtained structured data set is aggregated to generate an aggregated structured data set, wherein: The step of generating a first initial structured value-risk data set for the text modality data set according to the data attribute set includes: For each text modality data in the text modality data set, the following second generation step is performed: Extracting a keyword set corresponding to the text modality data; For each keyword in the keyword set, the following third generation step is performed: Determine word semantic importance information corresponding to the keyword, wherein the word semantic importance information represents the importance of the word semantics corresponding to the keyword relative to the data semantics corresponding to the text modal data; Inputting the keyword into a data attribute identifier generation model to generate a data attribute identifier corresponding to the keyword as a target data attribute identifier; Filter out keywords whose corresponding target data attribute identifiers are in the corresponding data attribute identifier set of the data attribute set from the keyword set to obtain a keyword subset; Filter out keywords whose corresponding word semantic importance information is greater than a target value from the keyword subset to obtain at least one keyword; Taking the at least one keyword as a key node, establishing a knowledge graph for the text modality data, wherein the nodes in the knowledge graph are the words in the text modality data, and the edges in the knowledge graph represent the association relationship between two words; According to the graph affinity between every two knowledge graphs, the knowledge graphs in the obtained knowledge graph set are subjected to graph fusion to generate a fused graph, thereby obtaining a fused graph set; for each fused graph in the fused graph set, the following fourth generation step is performed: Extracting atlas data from the fused atlas to obtain an atlas data set; According to the data attribute set, each atlas data in the atlas data set is supplemented and / or screened to generate processed atlas data to obtain a processed atlas data set; the obtained processed atlas data set is structured to generate a first initial structured value-risk data set; Determining first data feature value information and first risk business value information corresponding to the first value risk data set; Determine second data feature value information and second risk business value information corresponding to the second value risk data set, and determine structural conversion value impairment information for the second value risk data set and the structured value risk data set; Determining third data feature value information and third risk business value information corresponding to the structured value risk data set; Generate data value information for the value-risk data set according to the first data feature value information, the first risk business value information, the second data feature value information, the second risk business value information, the structure conversion value impairment information, the third data feature value information and the third risk business value information; According to the data value information, a server data processing operation is performed on the value risk data set to decompress the data storage for the target server, wherein the server data processing operation includes at least one of the following: a data storage operation, a data encryption operation, a data deletion operation, and a data sending operation.
2. The method according to claim 1, wherein: The determining of the first data feature value information and the first risk business value information corresponding to the first value risk data set includes: Acquire a data attribute set corresponding to the first value-risk data set as a target data attribute set; Obtaining a data usage history record corresponding to the first value-at-risk data set; generating at least one data domain information and a data accuracy evaluation information set for the first value-at-risk data set according to the data usage history record; For each piece of data field information in the at least one piece of data field information, the following fifth generating step is performed: Inputting the data domain information and the target data attribute set into an attribute and domain matching information generation model to generate attribute and domain matching information; Determine the domain importance information corresponding to the data domain information; Obtaining a complete data attribute set corresponding to the data domain information; Determining a data attribute difference between the target data attribute set and the complete data attribute set; Generate first data feature value sub-information for the data domain information according to the attribute and domain matching information, the domain importance information and the data attribute difference; Generate candidate data feature value information based on the obtained at least one first data feature value sub-information; Generating data accuracy value information for the target data attribute set according to the data accuracy evaluation information set; Performing a weighted summation on the candidate data feature value information and the data accuracy value information to generate first data feature value information; Obtain risk-related information generation model set corresponding to the risk business information set; For each risk-related information generation model in the risk-related information generation model set, the following sixth generation step is performed: Inputting the first value-risk data set into the risk-related information generation model to generate initial risk-related information; Determine the information gap between the initial risk-related information and the actual risk-related information; According to the obtained information difference set, first risk business value information for the first value-risk data set is generated.
3. The method according to claim 1, wherein: The structure conversion value impairment information includes: data feature value impairment information and risk business value impairment information; and The step of generating data value information for the value-risk data set according to the first data feature value information, the first risk business value information, the second data feature value information, the second risk business value information, the structure conversion value impairment information, the third data feature value information and the third risk business value information includes: multiplying the second data characteristic value information and the data characteristic value depreciation information to generate first multiplied value information; multiplying the second risk business value information and the risk business value impairment information to generate second multiplied value information; Determine an average value between the first multiplied value information and the third data feature value information to obtain a first average value, and determine an average value between the second multiplied value information and the third risk business value information to obtain a second average value; Adding the first average value and the first data feature value information to obtain first added information, and adding the second average value and the first risk business value information to obtain second added information; Combining the first added information and the second added information in the form of a key-value pair to generate first data value information; Dividing the third data feature value information by the data feature value depreciation information to generate first division value information; Dividing the third risk business value information by the risk business value impairment information to generate second division value information; determining an average value between the first phase division value information and the second data feature value information to obtain a third average value, and determining an average value between the second phase division value information and the second risk business value information to obtain a fourth average value; Adding the third average value and the first data feature value information to obtain third added information, and adding the fourth average value and the first risk business value information to obtain fourth added information; Combining the third added information and the fourth added information in the form of a key-value pair to generate second data value information; The first data value information and the second data value information are packaged in the form of a key-value pair to generate packaged information as data value information.
4. The method according to claim 1, wherein: The generating of a summary structured data set corresponding to the second value-risk data set comprises: Determine a data keyword set and data semantic information corresponding to each second value risk data in the second value risk data set; According to the data keyword set and data semantic information corresponding to each second value risk data, data division is performed on each second value risk data set to generate a value risk data set, wherein the number of repeated data keywords between every two value risk data in each value risk data set is greater than the target number, and the difference in data semantic information between the two value risk data is less than the target difference; Performing data group division verification on each value risk data group in the value risk data group set to generate a verification result; In response to determining that the verification result indicates that the division of each value-at-risk data group is correct, for each value-at-risk data group in the value-at-risk data group set, the following seventh generation step is performed: According to the modal form corresponding to the value-risk data, data division is performed on each value-risk data in the value-risk data group to generate a value-risk data subset set; Obtain a pre-trained multimodal data generation model, wherein the multimodal data generation model includes: a feature extraction model set, a feature integration model based on a multi-head attention mechanism, and an attribute data output layer set, wherein the feature extraction models in the feature extraction model set have a one-to-one corresponding modal form, and the attribute data output layers in the attribute data output layer set have a one-to-one corresponding relationship with the data attributes in the data attribute set; In response to determining that a first modal form set corresponding to the value-at-risk data subset set is smaller than a second modal form set corresponding to the feature extraction model set, determining at least one deviation model form between the first modal form set and the second modal form set; Performing model pruning on at least one feature extraction model in the feature extraction model set to obtain a pruned feature extraction model set; Inputting each value-risk data subgroup in the value-risk data subgroup set into the corresponding feature extraction model in the pruned feature extraction model set to generate a risk feature extraction information subgroup, thereby obtaining a risk feature extraction information subgroup set; Inputting the risk feature extraction information subset set into the feature integration model based on the multi-head attention mechanism to generate a data attribute feature information set for the data attribute set; Using the attribute data output layer set, generating a data attribute information set for the data attribute feature information set; According to the data attribute information set, summary structured data for the value-risk data group is generated.
5. A data processing device, comprising: an acquisition unit, configured to acquire a value-risk data set stored in a target server, wherein the value-risk data set includes: a first value-risk data set and a second value-risk data set, wherein the first value-risk data in the first value-risk data set is structured data, the second value-risk data in the second value-risk data set is unstructured data, and the second value-risk data set is a data set in a multimodal form; a conversion unit configured to convert the second value risk data in the second value risk data set into data in a text mode to generate text modality data, thereby obtaining a text modality data set; The first generating unit is configured to generate a structured value risk data set for the second value risk data set according to a data attribute set corresponding to the second value risk data set and the text modal data set, wherein the generating of the structured value risk data set for the second value risk data set according to the data attribute set corresponding to the second value risk data set and the text modal data set comprises: generating a first initial structured value risk data set for the text modal data set according to the data attribute set; performing multimodal division on each second value risk data in the second value risk data set to generate a second value risk data set; wherein the second value risk data set The data modalities corresponding to the respective second value risk data in the data set are the same modality; generating a summarized structured data set corresponding to the second value risk data set; performing data adjustment on the respective summarized structured data in the summarized structured data set according to the data attribute set to generate an adjusted data set as a second initial structured value risk data set; generating the structured value risk data set according to the first initial structured value risk data set and the second initial structured value risk data set, wherein the generating the summarized structured data set corresponding to the second value risk data set comprises: for each second value risk data set in the second value risk data set, executing the following first generation The method comprises the following steps: determining the modal information corresponding to the second value risk data group; determining the structural conversion method corresponding to the modal information; performing a structural conversion on the second value risk data group according to the structural conversion method to generate a structured data group; and performing data aggregation on each structured data in the obtained structured data group according to the data attribute set to generate an aggregated structured data set, wherein the first initial structured value risk data set for the text modal data set is generated according to the data attribute set, including: for each text modal data in the text modal data set, performing the following second generation step: extracting the keyword set corresponding to the text modal data; for the For each keyword in the keyword set, the following third generation step is performed: determining the word semantic importance information corresponding to the keyword, wherein the word semantic importance information represents the importance of the word semantics corresponding to the keyword relative to the data semantics corresponding to the text modal data; inputting the keyword into a data attribute identifier generation model to generate a data attribute identifier corresponding to the keyword as a target data attribute identifier; filtering out keywords whose corresponding target data attribute identifiers are in the data attribute identifier set corresponding to the data attribute set from the keyword set to obtain a keyword subset; filtering out keywords whose corresponding word semantic importance information is greater than a target value from the keyword subset to obtain at least one keyword;Taking the at least one keyword as a key node, a knowledge graph for the text modal data is established, wherein the nodes in the knowledge graph are the words in the text modal data, and the edges in the knowledge graph represent the association between two words; performing graph fusion on the knowledge graphs in the obtained knowledge graph set according to the graph closeness between each two knowledge graphs to generate a fused graph and obtain a fused graph set; for each fused graph in the fused graph set, executing the following fourth generation step: extracting graph data from the fused graph to obtain a graph data set; performing data supplementation and / or screening on each graph data in the graph data set according to the data attribute set to generate processed graph data and obtain a processed graph data set; performing structured processing on the obtained processed graph data set to generate a first initial structured value-risk data set; A first determining unit is configured to determine first data feature value information and first risk business value information corresponding to the first value risk data set; A second determining unit is configured to determine second data feature value information and second risk business value information corresponding to the second value risk data set, and determine structural conversion value impairment information for the second value risk data set and the structured value risk data set; A third determining unit is configured to determine third data feature value information and third risk business value information corresponding to the structured value risk data set; A second generating unit is configured to generate data value information for the value-risk data set according to the first data feature value information, the first risk business value information, the second data feature value information, the second risk business value information, the structure conversion value impairment information, the third data feature value information and the third risk business value information; An execution unit is configured to execute a server data processing operation on the value-risk data set according to the data value information to perform server data storage decompression on the target server, wherein the data processing operation includes at least one of the following: a data storage operation, a data encryption operation, a data deletion operation, and a data sending operation.
6. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.
7. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.
