Data object acquisition method and device based on EAST system, and computer equipment

Abstract

The invention discloses a data object acquisition method and device based on an EAST system, and computer equipment, relates to the technical field of data processing, and mainly aims to solve the problems of data association and insufficient data value of existing data object deep mining. Comprising the following steps: acquiring a data set acquired in an EAST system, and preprocessing the data set according to different business processing demand information; performing first processing on the preprocessed data set based on an unsupervised learning large model to obtain a first processing result, and performing second processing on the first processing result based on a supervised learning model to obtain a second processing result, the unsupervised learning large model being constructed based on at least two unsupervised learning models; and identifying the second processing result based on an application model matched with the business scene demand information to obtain a data object.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data object acquisition method and device based on an EAST system, and a computer device. Background Art

[0002] With the rapid development of big data mining technology, in all walks of life, before the collected data is applied to specific business, it is necessary to mine the data value of useful or valid data objects so that they can be directly transferred to the business for use, such as the large amount of financial institution data in the regulatory analysis platform (EAST, Examination and Analysis System Technology).

[0003] At present, in the process of acquiring a large number of data objects based on the EAST system, the commonly used data object acquisition is usually to classify using known data identifiers or data sources to accurately acquire data objects. However, when some data are highly correlated or have some unknown associations, it is impossible to accurately acquire them based on data identifiers or data sources alone, and it is impossible to acquire data objects with unknown data associations or business associations, which may lead to a large number of errors in subsequent business processing and inaccuracy in business applications. It is also impossible to meet the needs of analyzing and judging the potential value of data, and it is impossible to effectively assist digital transformation, thereby greatly reducing the effectiveness of data object acquisition. Summary of the invention

[0004] In view of this, the present invention provides a data object acquisition method and device, and computer equipment based on the EAST system, the main purpose of which is to solve the problems of deep mining of data associations and insufficient data value of existing data objects.

[0005] According to one aspect of the present invention, a method for acquiring data objects based on an EAST system is provided, comprising:

[0006] Obtain the data set collected in the EAST system and pre-process the data set according to different business processing requirements;

[0007] Performing a first processing on the preprocessed data set based on an unsupervised learning large model to obtain a first processing result, and performing a second processing on the first processing result based on a supervised learning model to obtain a second processing result, wherein the unsupervised learning large model is constructed based on at least two unsupervised learning models;

[0008] The second processing result is identified based on an application model that matches the business scenario requirement information to obtain a data object.

[0009] Furthermore, the first processing is performed on the preprocessed data set based on the unsupervised learning large model to obtain a first processing result, including:

[0010] Determine at least two unsupervised learning models to be processed in the unsupervised learning large model based on the preprocessed data set;

[0011] The data sets are processed separately based on the unsupervised learning model to obtain a plurality of sub-processing results, and the plurality of sub-processing results are integrated to obtain the first processing result.

[0012] Furthermore, before performing a second processing on the first processing result based on the supervised learning model to obtain the second processing result, the method further includes:

[0013] According to different business processing requirement information, the source data samples are labeled using a semantic model, and the supervised learning model is trained using the labeled source data samples, so that the first processing result is subjected to a second processing based on the supervised learning model that has completed the model training.

[0014] Furthermore, the method further comprises:

[0015] If the data object completes business matching verification or business mining verification, the verification tag of the data object after verification is obtained, and the supervised learning model is updated and trained using the verification tag, so that the supervised learning model after the update training can perform the next round of second processing.

[0016] Furthermore, the method further comprises:

[0017] An evaluation strategy is determined according to the output representation of the business scenario requirement information, and an evaluation result of the data object is generated based on the evaluation strategy. The evaluation strategy is used to characterize the rule content for scoring and analyzing the data object to output the data object with the evaluation result.

[0018] Furthermore, the preprocessing of the data set according to different business processing requirement information includes:

[0019] The data set is semantically transformed based on the pre-trained semantic model to obtain a semantic data set, and the semantic data set is cleaned and corrected for abnormal data based on the abnormal information library of the business processing demand information to complete the preprocessing.

[0020] Furthermore, the method further comprises:

[0021] The data source and data type of the data set are parsed, and an application model is matched according to the data source and the data type. The application model includes a compliance semantic model, an anomaly prediction model, and an information classification model.

[0022] According to another aspect of the present invention, a data object acquisition device based on the EAST system is provided, comprising:

[0023] The acquisition module is used to acquire the data set collected in the EAST system and pre-process the data set according to different business processing requirements;

[0024] A processing module, configured to perform a first processing on the preprocessed data set based on an unsupervised learning large model to obtain a first processing result, and perform a second processing on the first processing result based on a supervised learning model to obtain a second processing result, wherein the unsupervised learning large model is constructed based on at least two unsupervised learning models;

[0025] An identification module is used to identify the second processing result based on an application model that matches the business scenario requirement information to obtain a data object.

[0026] Furthermore, the processing module is used to determine at least two unsupervised learning models to be processed in the unsupervised learning large model based on the preprocessed data set; process the data sets separately based on the unsupervised learning models to obtain multiple sub-processing results, and integrate the multiple sub-processing results to obtain the first processing result.

[0027] Furthermore, the device also includes:

[0028] The training module is used to mark the source data samples using the semantic model according to different business processing requirement information, and to train the supervised learning model through the marked source data samples, so as to perform a second processing on the first processing result based on the supervised learning model that has completed the model training.

[0029] Furthermore, the device also includes:

[0030] The updating module is used to obtain the verification label of the data object after verification if the data object completes the business matching verification or the business mining verification, and use the verification label to update the supervised learning model so that the supervised learning model after the update training can perform the next round of second processing.

[0031] Furthermore, the identification module is specifically used to determine an evaluation strategy according to the output representation of the business scenario requirement information, and generate an evaluation result of the data object based on the evaluation strategy. The evaluation strategy is used to characterize the rule content for scoring and analyzing the data object, so as to output the data object with the evaluation result.

[0032] Furthermore, the acquisition module is specifically used to perform semantic conversion on the data set based on the pre-trained semantic model to obtain a semantic data set, and perform abnormal data cleaning and correction processing on the semantic data set based on the abnormal information library of the business processing requirement information to complete preprocessing.

[0033] Furthermore, the device also includes:

[0034] The parsing module is used to parse the data source and data type of the data set, and match the application model according to the data source and the data type. The application model includes a compliance semantic model, an anomaly prediction model, and an information classification model.

[0035] According to another aspect of the present invention, a storage medium is provided, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to execute operations corresponding to the above-mentioned data object acquisition method based on the EAST system.

[0036] According to another aspect of the present invention, there is provided a computer device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0037] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute operations corresponding to the above-mentioned data object acquisition method based on the EAST system.

[0038] By means of the above technical solution, the technical solution provided by the embodiment of the present invention has at least the following advantages:

[0039] The present invention provides a data object acquisition method and device, and a computer device based on an EAST system. Compared with the prior art, the embodiments of the present invention obtain a data set collected in the EAST system, and pre-process the data set according to different business processing demand information; perform a first processing on the pre-processed data set based on an unsupervised learning big model to obtain a first processing result, and perform a second processing on the first processing result based on a supervised learning model to obtain a second processing result, wherein the unsupervised learning big model is constructed based on at least two unsupervised learning models; identify the second processing result based on an application model matching business scenario demand information to obtain a data object, thereby achieving the purpose of splicing self-supervised learning with unsupervised learning, enhancing the value presentation of potential data through in-depth mining of data, and avoiding meaningless training of irrelevant data, thereby reducing the time cost of unsupervised learning, achieving deep mining of data potential value analysis, effectively realizing auxiliary digital transformation, and achieving the purpose of effectively acquiring data objects.

[0040] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0042] Figure 1 A flow chart of a data object acquisition method based on the EAST system provided by an embodiment of the present invention is shown;

[0043] Figure 2 A block diagram of a data object acquisition device based on an EAST system provided by an embodiment of the present invention is shown;

[0044] Figure 3 A schematic structural diagram of a computer device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0045] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0046] The embodiment of the present invention provides a method for acquiring data objects based on the EAST system, such as Figure 1 As shown, the method includes:

[0047] 101. Obtain the data set collected in the EAST system, and pre-process the data set according to different business processing requirement information.

[0048] In the embodiment of the present invention, the current execution end, as the execution end for processing and mining different business data, can be a cloud server or a computer device. By collecting a large amount of financial institution data (such as banking institutions) in the supervision and analysis platform (EAST, Examination and Analysis System Technology) as a data set, the method in steps 101-103 is executed, and the embodiment of the present invention does not make specific restrictions. Among them, the data set is the data content collected from the EAST system and not processed, which can be text data or numerical data, or data content in any form, and the embodiment of the present invention does not make specific restrictions. In addition, the business processing demand information is used to characterize the demand content for processing data for different businesses, for example, the business is a compliance business, a user behavior mining business, etc. Correspondingly, the business processing demand information can be a text compliance judgment on the data, a classification judgment on the behavior data, etc., which can be obtained based on the input of the operator, and the embodiment of the present invention does not make specific restrictions. At this time, the data set can be preprocessed according to the business processing demand information, and the preprocessing can include cleaning of abnormal data in the data set, and can also include correction of deviation data, etc., and the embodiment of the present invention does not make specific restrictions.

[0049] 102. Perform a first processing on the preprocessed data set based on the unsupervised learning large model to obtain a first processing result, and perform a second processing on the first processing result based on the supervised learning model to obtain a second processing result.

[0050] In the embodiment of the present invention, the unsupervised learning large model is constructed based on at least two unsupervised learning models. The unsupervised learning model may include a new clustering algorithm (such as spectral clustering, hierarchical clustering), a dimensionality reduction algorithm (such as unified manifold approximation and projection, t-distribution random neighborhood embedding), an autoencoder, etc. For data sets corresponding to different businesses, multiple unsupervised learning models can be used in parallel for processing, and integrated to obtain a first processing result. Furthermore, by splicing, the first processing result obtained is directly input into the supervised learning model for second processing. At this time, the supervised learning model is to complete the task that usually requires supervised learning through unsupervised learning. At this time, the supervised learning model does not rely on the labeled data set of the supervisory signal, but generates implicit labels from unstructured data to achieve deep mining of the first processing obtained after processing the unsupervised learning large model to obtain a second processing result. Among them, the supervised learning model may include but is not limited to decision numbers, neural networks, and support vector machines, which are not specifically limited in the embodiment of the present invention.

[0051] 103. Identify the second processing result based on an application model that matches the business scenario requirement information to obtain a data object.

[0052] In the embodiment of the present invention, in order to make the result of data processing more compatible with different businesses, in order to mine the relevance and other characteristics of data from the data set, the current execution end matches the application model to be executed for downstream business according to the business scenario demand information. At this time, the application model is a model constructed according to the business process or application scenario architecture, including but not limited to a product matching model, an information push model, etc., which is not specifically limited in the embodiment of the present invention. Among them, the business scenario demand information is used to characterize the specific content required by the application scenario of the data set to match the corresponding application model. For example, if the scenario is a data compliance scenario, the corresponding application model can be identified by the process model such as the extraction of text feature words, comparison of feature words with compliance thesaurus, etc., to obtain data objects. At this time, the data object is an object with relevance and research value obtained by deep mining from a large amount of data, such as illegal words, illegal behaviors, etc., which can be determined based on the input method of the operator. The embodiment of the present invention does not make specific limitations, so as to output the data object to the user or technician through the EAST system.

[0053] It should be noted that in a specific implementation scenario, a data set is collected from an application and pre-processed according to business processing requirement information (such as compliance processing). After deleting abnormal data that cannot be used as a compliance judgment, cluster analysis is performed simultaneously using spectral clustering and hierarchical clustering in the unsupervised learning large model to obtain a first processing result, which is then input into a supervised learning model (such as a deep learning model) for a second processing to obtain a second processing result. After the second processing result is obtained, the second processing result is identified based on the matching application model to obtain a data object.

[0054] In another embodiment of the present invention, for further definition and description, the steps further include:

[0055] The data source and data type of the data set are parsed, and the application model is matched according to the data source and the data type.

[0056] In order to apply the acquired data objects to different business scenarios more efficiently, the current execution end first parses the data source and data type of the data set to match the user model according to the data source and data type. At this time, the application model includes a compliance semantic model, an abnormal prediction model, and an information classification model. Different business scenarios can match different application models. At this time, the data source is the specific starting node where the data set is collected, such as financial product applications, technology product marketing applications, etc. The data categories include text types, image types, numerical types, etc. In the embodiment of the present invention, it can be pre-configured based on business applications, and the embodiment of the present invention does not make specific limitations. Among them, the compliance semantic model can be an intelligent model constructed by natural language processing (NLP), the abnormal prediction model can be a neural network model, a support vector machine model, etc., and the information classification model can be a decision tree model, etc., which are not specifically limited in the embodiment of the present invention.

[0057] In a specific implementation scenario, if the business scenario text compliance judgment, the corresponding data source can be a financial product application, and the data type is text type. After executing steps 101-103, the data objects that can be obtained include expense text objects, user identity text objects, practice text objects, and operation process text objects. According to the financial product application and text type matching, the application model is a compliance semantic model. The expense text object, user identity text object, practice text object, and operation process text object are used as the input of the compliance semantic model, and the text object with a non-compliant label is output, thereby achieving the purpose of mining the data set by splicing the unsupervised learning large model and the supervised learning model, which is convenient for assisting data mining personnel to conduct in-depth research on data correlation, so as to quickly and effectively obtain the true value of data from a large amount of data.

[0058] In a specific implementation scenario, if the business scenario is risk warning, the corresponding data source can be an information push application, and the data type is an image type. After executing steps 101-103, the data objects that can be obtained include product packaging image objects, product structure image objects, and product usage process objects. The application model obtained by matching the information push application and the image type is an abnormality prediction model. The product packaging image object, product structure image object, and product usage process object are used as inputs of the abnormality prediction model, and image objects with abnormal labels are output, thereby realizing the splicing of unsupervised learning large models and supervised learning models to mine data sets, which is convenient for assisting data mining personnel to conduct in-depth research on data correlation, so as to quickly and effectively obtain the true value of data from large amounts of data.

[0059] In a specific implementation scenario, if the business scenario is information security level rating, the corresponding data source may be a tax processing application, and the data type is a numerical type. After executing steps 101-103, the data objects that can be obtained include numerical time objects, numerical upload area objects, and numerical attribute objects. The application model obtained by matching a certain tax processing application and numerical type is an information classification model. The numerical time object, numerical upload area object, and numerical attribute object are used as inputs of the information classification model, and a numerical object with classification results is output, thereby achieving the purpose of mining the data set by splicing the unsupervised learning large model and the supervised learning model, which is convenient for assisting data mining personnel to conduct in-depth research on data correlation, so as to quickly and effectively obtain the true value of data from a large amount of data.

[0060] In the above-mentioned different business scenarios, model training can be performed in advance for different application models based on corresponding prior samples, so that each application model can be accurately and effectively applied to each scenario.

[0061] In another embodiment of the present invention, for further definition and explanation, the step of performing a first processing on the preprocessed data set based on the unsupervised learning large model to obtain a first processing result includes:

[0062] Determine at least two unsupervised learning models to be processed in the unsupervised learning large model based on the preprocessed data set;

[0063] The data sets are processed separately based on the unsupervised learning model to obtain a plurality of sub-processing results, and the plurality of sub-processing results are integrated to obtain the first processing result.

[0064] In order to use unsupervised learning technology to deeply mine the data set to identify the relevance and potential value of the data, the current execution end, when using the unsupervised learning large model for the first processing, specifically, first determines at least two unsupervised learning models to be processed. Specifically, when determining the unsupervised learning model, the unsupervised learning model can be matched based on the data characteristics of the data set. Since the unsupervised learning model can include clustering algorithms, dimensionality reduction algorithms, etc., when determining the unsupervised learning model, the data set can be pre-configured as the input of all unsupervised learning models, and the processed sub-processing results can be obtained, that is, the data set is fully mined through each model to obtain multiple sub-processing results, and the sub-processing results are integrated. Among them, when integrating, the final first processing result can be obtained by weight addition or by direct addition, and the embodiment of the present invention does not make specific limitations. In addition, when adding weights, specific weights can be pre-configured according to the functions of different unsupervised learning models. For example, the model of the new clustering function can be configured with a higher weight value, such as 0.4, and the model of the dimensionality reduction function can be configured with a lower weight value, such as 0.1, and then added. The embodiments of the present invention do not make specific limitations.

[0065] In another embodiment of the present invention, for further definition and explanation, before the step of performing a second processing on the first processing result based on a supervised learning model to obtain the second processing result, the method further includes:

[0066] According to different business processing requirement information, the source data samples are labeled using a semantic model, and the supervised learning model is trained using the labeled source data samples.

[0067] In order to achieve in-depth mining and multi-angle identification of data, before using the supervised learning model for the second processing, in a specific implementation scenario, if the data in the data set is text data, the semantic model can be used to first mark the source data samples used as training sample sets for model training according to different business processing requirements. The semantic model can be pre-trained through a natural language library, and the embodiment of the present invention does not make specific limitations. Among them, the business processing requirement information is used to characterize the requirements to be processed when data mining is performed for different business needs, including but not limited to prediction requirements, abnormal identification requirements, semantic conversion requirements, etc., and is pre-configured according to different business scenarios. The embodiment of the present invention does not make specific limitations. In addition, the source data sample is a data sample used for the self-supervisory model to perform data. If the business processing requirement information is a compliance judgment, the source data sample is marked with illegal semantics using a semantic model. For example, the semantic model is used to automatically detect the dimension name in the data, such as the date format is the date, and the price is the price, and a label is given, and the marked source data sample is used to perform model training on the supervised learning model, so as to perform a second processing on the first processing result based on the supervised learning model that has completed the model training. In the embodiment of the present invention, since the unsupervised learning model itself does not need to add labels to the data, in order to stimulate the mining potential of unsupervised learning, when training the supervised learning model, a labeling method can be used for visualization and to compensate for the shortcomings of unsupervised learning.

[0068] In another embodiment of the present invention, for further definition and description, the steps further include:

[0069] If the data object completes business matching verification or business mining verification, the verification tag of the data object after verification is obtained, and the supervised learning model is updated and trained using the verification tag, so that the supervised learning model after the update training can perform the next round of second processing.

[0070] In order to ensure that the data comparison determined after data processing can be effectively used in downstream businesses, thereby improving the effectiveness of data object acquisition based on the EAST system, the current execution end performs business matching verification or business mining verification on the data object. Among them, business matching verification or business mining verification can be completed based on manual review, or it can be judged based on business rules. Business matching verification is used to determine whether the data object can be applied to the business scenario, and business mining verification is used to determine whether the data object represents business-related content. After passing the verification, the data object can be marked based on manual labeling to obtain a verification label, so that the data object with the verification label is used as a new sample to update the supervised learning model for training, improve the accuracy of self-supervised learning, and thus improve the data correlation processing effect of the unsupervised learning combined with the supervised learning model after data mining.

[0071] In another embodiment of the present invention, for further definition and description, the steps further include:

[0072] An evaluation strategy is determined according to the output representation of the business scenario requirement information, and an evaluation result of the data object is generated based on the evaluation strategy.

[0073] In order to effectively output the data object and improve the visualization effect of the data object data, when the current execution end outputs the object, specifically, it also determines the evaluation strategy according to the output representation corresponding to the business scenario requirement information. At this time, the evaluation strategy is used to characterize the rule content for scoring and analyzing the data object to output the data object with the evaluation result. Among them, the output representation is used to characterize the output form of the output data object, including but not limited to text form, data form, and icon form. At this time, different evaluation strategies can be matched for different output representations, and the embodiment of the present invention does not make specific restrictions. Specifically, the output representation can be pre-set in the business scenario requirement information, and the evaluation strategy matching this output representation is determined based on the pre-corresponding query relationship to generate the evaluation result of the evaluation data object based on this evaluation strategy. Among them, the evaluation strategy includes a semantic analysis scoring strategy and a data association scoring strategy. The semantic analysis scoring strategy can identify the data object based on the natural semantic recognition technology, and construct a scoring system according to whether the predetermined words exist, for example, scoring according to the number of keywords that appear, and the embodiment of the present invention does not make specific new ones. The data relevance scoring strategy can construct a scoring system based on whether there is a correlation between each data in the data object. For example, the relevance of the data objects (reflecting whether the types, business logic and other classifications between the data are associated) is evaluated based on a decision tree model, which is not specifically limited in the embodiments of the present invention.

[0074] In another embodiment of the present invention, for further definition and explanation, the step of preprocessing the data set according to different business processing requirement information includes:

[0075] The data set is semantically transformed based on the pre-trained semantic model to obtain a semantic data set, and the semantic data set is cleaned and corrected for abnormal data based on the abnormal information library of the business processing demand information to complete the preprocessing.

[0076] In order to effectively acquire data objects, so as to know the data relationship in a large number of data sets, when the current execution end pre-processes the data set according to the business processing demand information, specifically, when the data set is in the collected text data scenario, the text in the data set is first semantically converted based on the pre-trained semantic model to obtain a semantic data set, and the semantic model can be BERT (Bidirectional Encoder Representations from Transformers) or GPT (Generative Pre-trained Transformer), which is not specifically limited in the embodiment of the present invention. Further, the abnormal data cleaning and correction processing of the semantic data set are performed based on the abnormal information library corresponding to the business processing demand information, so as to complete the pre-processing, wherein the business processing demand information is used as a compliance business, and an abnormal information library can be pre-configured, and the abnormal information library contains data content that must belong to abnormalities for the collected data, so as to clean the semantic data set using the abnormal information library. At the same time, when there is data missing in the cleaned data set, the default fill data in the abnormal information library can be used for correction, so as to complete the pre-processing, so as to delete data that does not conform to the expected mode, industry standards or regulatory requirements, and data content that may be manifested as data missing, extreme values, format errors, logical errors, etc.

[0077] The embodiment of the present invention provides a method for acquiring data objects based on the EAST system. Compared with the prior art, the embodiment of the present invention obtains a data set collected in the EAST system, and pre-processes the data set according to different business processing demand information; performs a first processing on the pre-processed data set based on an unsupervised learning big model to obtain a first processing result, and performs a second processing on the first processing result based on a supervised learning model to obtain a second processing result, wherein the unsupervised learning big model is constructed based on at least two unsupervised learning models; identifies the second processing result based on an application model matching business scenario demand information to obtain a data object, thereby achieving the purpose of splicing self-supervised learning with unsupervised learning, enhancing the value presentation of potential data through in-depth mining of data, and avoiding meaningless training of irrelevant data, thereby reducing the time cost of unsupervised learning, achieving deep mining of the potential value of data analysis, effectively realizing auxiliary digital transformation, and achieving the purpose of effectively acquiring data objects.

[0078] Furthermore, as a response to the above Figure 1 The implementation of the method shown in the embodiment of the present invention provides a data object acquisition device based on the EAST system, such as Figure 2 As shown, the device comprises:

[0079] The acquisition module 21 is used to acquire the data set collected in the EAST system and pre-process the data set according to different business processing requirement information;

[0080] A processing module 22 is used to perform a first processing on the preprocessed data set based on an unsupervised learning large model to obtain a first processing result, and to perform a second processing on the first processing result based on a supervised learning model to obtain a second processing result, wherein the unsupervised learning large model is constructed based on at least two unsupervised learning models;

[0081] The identification module 23 is used to identify the second processing result based on the application model matched with the business scenario requirement information to obtain a data object.

[0082] Furthermore, the processing module is used to determine at least two unsupervised learning models to be processed in the unsupervised learning large model based on the preprocessed data set; process the data sets separately based on the unsupervised learning models to obtain multiple sub-processing results, and integrate the multiple sub-processing results to obtain the first processing result.

[0083] Furthermore, the device also includes:

[0084] The training module is used to mark the source data samples using the semantic model according to different business processing requirement information, and to train the supervised learning model through the marked source data samples, so as to perform a second processing on the first processing result based on the supervised learning model that has completed the model training.

[0085] Furthermore, the device also includes:

[0086] The updating module is used to obtain the verification label of the data object after verification if the data object completes the business matching verification or the business mining verification, and use the verification label to update the supervised learning model so that the supervised learning model after the update training can perform the next round of second processing.

[0087] Furthermore, the identification module is specifically used to determine an evaluation strategy according to the output representation of the business scenario requirement information, and generate an evaluation result of the data object based on the evaluation strategy. The evaluation strategy is used to characterize the rule content for scoring and analyzing the data object, so as to output the data object with the evaluation result.

[0088] Furthermore, the acquisition module is specifically used to perform semantic conversion on the data set based on the pre-trained semantic model to obtain a semantic data set, and perform abnormal data cleaning and correction processing on the semantic data set based on the abnormal information library of the business processing requirement information to complete preprocessing.

[0089] Furthermore, the device also includes:

[0090] The parsing module is used to parse the data source and data type of the data set, and match the application model according to the data source and the data type. The application model includes a compliance semantic model, an anomaly prediction model, and an information classification model.

[0091] The embodiment of the present invention provides a data object acquisition device based on the EAST system. Compared with the prior art, the embodiment of the present invention obtains a data set collected in the EAST system, and pre-processes the data set according to different business processing demand information; performs a first processing on the pre-processed data set based on an unsupervised learning big model to obtain a first processing result, and performs a second processing on the first processing result based on a supervised learning model to obtain a second processing result, and the unsupervised learning big model is constructed based on at least two unsupervised learning models; identifies the second processing result based on an application model matching business scenario demand information to obtain a data object, thereby achieving the purpose of splicing self-supervised learning with unsupervised learning, and through in-depth mining of data, enhances the value presentation of potential data, avoids meaningless training of irrelevant data, thereby reducing the time cost of unsupervised learning, and realizes deep mining of the potential value of data analysis, effectively realizes auxiliary digital transformation, and achieves the purpose of effectively acquiring data objects.

[0092] According to one embodiment of the present invention, a storage medium is provided, wherein the storage medium stores at least one executable instruction, and the computer executable instruction can execute the data object acquisition method based on the EAST system in any of the above method embodiments.

[0093] Figure 3 A schematic diagram of the structure of a computer device provided according to an embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the computer device.

[0094] like Figure 3 As shown, the computer device may include: a processor (processor) 302 , a communication interface (Communications Interface) 304 , a memory (memory) 306 , and a communication bus 308 .

[0095] The processor 302 , the communication interface 304 , and the memory 306 communicate with each other via the communication bus 308 .

[0096] The communication interface 304 is used to communicate with other devices such as clients or other servers.

[0097] The processor 302 is used to execute the program 310, and specifically can execute the relevant steps in the above-mentioned data object acquisition method embodiment based on the EAST system.

[0098] Specifically, the program 310 may include program codes, which include computer operation instructions.

[0099] The processor 302 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the computer device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0100] The memory 306 is used to store the program 310. The memory 306 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0101] The program 310 may be specifically configured to enable the processor 302 to perform the following operations:

[0102] Obtain the data set collected in the EAST system and pre-process the data set according to different business processing requirements;

[0103] Performing a first processing on the preprocessed data set based on an unsupervised learning large model to obtain a first processing result, and performing a second processing on the first processing result based on a supervised learning model to obtain a second processing result, wherein the unsupervised learning large model is constructed based on at least two unsupervised learning models;

[0104] The second processing result is identified based on an application model that matches the business scenario requirement information to obtain a data object.

[0105] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0106] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data object acquisition method based on the EAST system, characterized in that: include: Obtain the data set collected in the EAST system and pre-process the data set according to different business processing requirements; Performing a first processing on the preprocessed data set based on an unsupervised learning large model to obtain a first processing result, and performing a second processing on the first processing result based on a supervised learning model to obtain a second processing result, wherein the unsupervised learning large model is constructed based on at least two unsupervised learning models; The second processing result is identified based on an application model that matches the business scenario requirement information to obtain a data object.

2. The method according to claim 1, characterized in that The first processing of the preprocessed data set based on the unsupervised learning large model to obtain the first processing result includes: Determine at least two unsupervised learning models to be processed in the unsupervised learning large model based on the preprocessed data set; The data sets are processed separately based on the unsupervised learning model to obtain a plurality of sub-processing results, and the plurality of sub-processing results are integrated to obtain the first processing result.

3. The method according to claim 1, characterized in that Before performing a second processing on the first processing result based on the supervised learning model to obtain the second processing result, the method further includes: According to different business processing requirement information, the source data samples are labeled using a semantic model, and the supervised learning model is trained using the labeled source data samples, so that the first processing result is subjected to a second processing based on the supervised learning model that has completed the model training.

4. The method according to claim 3, characterized in that The method further comprises: If the data object completes business matching verification or business mining verification, the verification tag of the data object after verification is obtained, and the supervised learning model is updated and trained using the verification tag, so that the supervised learning model after the update training can perform the next round of second processing.

5. The method according to claim 1, characterized in that The method further comprises: An evaluation strategy is determined according to the output representation of the business scenario requirement information, and an evaluation result of the data object is generated based on the evaluation strategy. The evaluation strategy is used to characterize the rule content for scoring and analyzing the data object to output the data object with the evaluation result.

6. The method according to claim 1, characterized in that The preprocessing of the data set according to different business processing requirement information includes: The data set is semantically transformed based on the pre-trained semantic model to obtain a semantic data set, and the semantic data set is cleaned and corrected for abnormal data based on the abnormal information library of the business processing demand information to complete the preprocessing.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: The data source and data type of the data set are parsed, and an application model is matched according to the data source and the data type. The application model includes a compliance semantic model, an anomaly prediction model, and an information classification model.

8. A data object acquisition device based on the EAST system, characterized in that: include: The acquisition module is used to acquire the data set collected in the EAST system and pre-process the data set according to different business processing requirements; A processing module, configured to perform a first processing on the preprocessed data set based on an unsupervised learning large model to obtain a first processing result, and perform a second processing on the first processing result based on a supervised learning model to obtain a second processing result, wherein the unsupervised learning large model is constructed based on at least two unsupervised learning models; An identification module is used to identify the second processing result based on an application model that matches the business scenario requirement information to obtain a data object.

9. A storage medium storing at least one executable instruction, wherein the executable instruction enables a processor to execute operations corresponding to the data object acquisition method based on the EAST system as described in any one of claims 1 to 7.

10. A computer device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the data object acquisition method based on the EAST system as described in any one of claims 1-7.

Images