Data object acquisition method and device based on EAST system, and computer device

Abstract

The application discloses a data object acquisition method and device based on an EAST system and computer equipment, relates to the technical field of data processing, and mainly aims to solve the problems of insufficient data correlation and data value of existing data objects. The method comprises the following steps: acquiring a data set collected in the EAST system, and pre-processing the data set according to different business processing demand information; performing first processing on the pre-processed data set based on an unsupervised learning large model to obtain a first processing result, and performing second processing on the first processing result based on a supervised learning model to obtain a second processing result, wherein the unsupervised learning large model is constructed based on at least two unsupervised learning models; and identifying the second processing result based on an application model matched according to business scene demand information to obtain a data object.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data object acquisition method and apparatus, and computer equipment based on the EAST system. Background Technology

[0002] With the rapid development of big data mining technology, various industries need to mine the value of useful or effective data objects before applying the collected data to specific business operations, so that it can be directly used in business. For example, the large amount of financial institution data in the regulatory analysis platform (EAST, Examination and Analysis System Technology).

[0003] Currently, in the process of acquiring a large number of data objects based on the EAST system, the common practice is to classify data objects using known data identifiers or data sources to accurately identify them. However, when some data has high correlation or certain unknown correlations, relying solely on data identifiers or data sources is insufficient for accurate acquisition. This fails to meet the needs of acquiring data objects with unknown or business-related correlations, potentially leading to significant errors in subsequent business processing and inaccuracies in business applications. Furthermore, it fails to meet the analytical needs for deeply exploring the potential value of data and cannot effectively support digital transformation, thus greatly reducing the effectiveness of data object acquisition. Summary of the Invention

[0004] In view of this, the present invention provides a data object acquisition method and apparatus, and a computer device based on the EAST system, the main purpose of which is to solve the problems of insufficient data correlation and data value in existing data object mining.

[0005] According to one aspect of the present invention, a method for acquiring data objects based on the EAST system is provided, comprising:

[0006] Acquire the dataset collected in the EAST system and preprocess the dataset according to different business processing requirements;

[0007] The preprocessed dataset is first processed based on an unsupervised learning large model to obtain a first processing result, and the first processing result is then processed based on a supervised learning model to obtain a second processing result. The unsupervised learning large model is constructed based on at least two unsupervised learning models.

[0008] The application model, based on matching business scenario requirements, identifies the second processing result to obtain a data object.

[0009] Furthermore, the first processing of the preprocessed dataset based on the unsupervised learning large model to obtain the first processing result includes:

[0010] Based on the preprocessed dataset, at least two unsupervised learning models to be processed in the large unsupervised learning model are identified.

[0011] The dataset is processed separately based on the unsupervised learning model to obtain multiple sub-processing results, and the multiple sub-processing results are integrated to obtain the first processing result.

[0012] Furthermore, before performing a second processing on the first processing result based on the supervised learning model to obtain the second processing result, the method further includes:

[0013] According to different business processing requirements, the source data samples are labeled using a semantic model, and the supervised learning model is trained using the labeled source data samples, so as to perform a second processing on the first processing result based on the supervised learning model that has completed model training.

[0014] Furthermore, the method also includes:

[0015] If the data object completes business matching verification or business mining verification, then the verification label of the data object after verification is obtained, and the verification label is used to update and train the supervised learning model so that the supervised learning model after the update and training can perform the next round of second processing.

[0016] Furthermore, the method also includes:

[0017] The evaluation strategy is determined according to the output representation of the business scenario requirement information, and the evaluation result of the data object is generated based on the evaluation strategy. The evaluation strategy is used to characterize the rule content for scoring and analyzing the data object, so as to output the data object with the evaluation result.

[0018] Furthermore, the preprocessing of the dataset based on different business processing requirements includes:

[0019] The dataset is semantically transformed based on the pre-trained semantic model to obtain a semantic dataset. Then, the semantic dataset is cleaned and corrected based on the anomaly information database of the business processing requirements information to complete the preprocessing.

[0020] Furthermore, the method also includes:

[0021] The data source and data type of the dataset are parsed, and an application model is matched according to the data source and data type. The application model includes a compliance semantic model, an anomaly prediction model, and an information classification model.

[0022] According to another aspect of the present invention, a data object acquisition device based on the EAST system is provided, comprising:

[0023] The acquisition module is used to acquire the dataset collected in the EAST system and preprocess the dataset according to different business processing requirements.

[0024] The processing module is used to perform a first processing on the preprocessed dataset based on an unsupervised learning large model to obtain a first processing result, and to perform a second processing on the first processing result based on a supervised learning model to obtain a second processing result, wherein the unsupervised learning large model is constructed based on at least two unsupervised learning models;

[0025] The identification module is used to identify the second processing result based on the application model that matches the business scenario requirement information, and obtain the data object.

[0026] Furthermore, the processing module is used to determine at least two unsupervised learning models to be processed in the large unsupervised learning model based on the preprocessed dataset; to process the dataset separately based on the unsupervised learning models to obtain multiple sub-processing results, and to integrate the multiple sub-processing results to obtain the first processing result.

[0027] Furthermore, the device also includes:

[0028] The training module is used to label source data samples using a semantic model according to different business processing requirements, and to train the supervised learning model using the labeled source data samples, so as to perform a second processing on the first processing result based on the supervised learning model that has completed model training.

[0029] Furthermore, the device also includes:

[0030] The update module is used to obtain the verification label of the data object after verification if the data object completes the business matching verification or business mining verification, and use the verification label to update and train the supervised learning model so that the supervised learning model after the update training can perform the next round of second processing.

[0031] Furthermore, the identification module is specifically used to determine the evaluation strategy according to the output representation of the business scenario requirement information, and generate the evaluation result of the data object based on the evaluation strategy. The evaluation strategy is used to characterize the rule content for scoring and analyzing the data object, so as to output the data object with the evaluation result.

[0032] Furthermore, the acquisition module is specifically used to perform semantic transformation on the dataset based on the pre-trained semantic model to obtain a semantic dataset, and to perform abnormal data cleaning and correction processing on the semantic dataset based on the abnormal information database of the business processing requirement information to complete the preprocessing.

[0033] Furthermore, the device also includes:

[0034] The parsing module is used to parse the data source and data type of the dataset, and match the application model according to the data source and data type. The application model includes a compliance semantic model, an anomaly prediction model, and an information classification model.

[0035] According to another aspect of the present invention, a storage medium is provided, wherein at least one executable instruction is stored therein, the executable instruction causing a processor to perform an operation corresponding to the data object acquisition method based on the EAST system described above.

[0036] According to another aspect of the present invention, a computer device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus;

[0037] The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the data object acquisition method based on the EAST system described above.

[0038] By employing the above-described technical solutions, the technical solutions provided by the embodiments of the present invention have at least the following advantages:

[0039] This invention provides a data object acquisition method, apparatus, and computer device based on the EAST system. Compared with existing technologies, this invention acquires a dataset collected from the EAST system and preprocesses the dataset according to different business processing requirements. It then performs a first processing on the preprocessed dataset based on an unsupervised learning model to obtain a first processing result, and performs a second processing on the first processing result based on a supervised learning model to obtain a second processing result. The unsupervised learning model is constructed based on at least two unsupervised learning models. Finally, an application model matching business scenario requirements identifies the second processing result to obtain data objects. This achieves the goal of combining self-supervised and unsupervised learning, enhancing the value representation of potential data through in-depth data mining, avoiding meaningless training on unrelated data, thereby reducing the time cost of unsupervised learning, achieving in-depth analysis of the potential value of data, effectively assisting digital transformation, and achieving the goal of effectively acquiring data objects.

[0040] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0041] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0042] Figure 1 A flowchart of a data object acquisition method based on the EAST system provided by an embodiment of the present invention is shown;

[0043] Figure 2 This diagram illustrates a data object acquisition device based on the EAST system according to an embodiment of the present invention.

[0044] Figure 3 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention is shown. Detailed Implementation

[0045] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0046] This invention provides a method for acquiring data objects based on the EAST system, such as... Figure 1 As shown, the method includes:

[0047] 101. Obtain the dataset collected in the EAST system and preprocess the dataset according to different business processing requirements.

[0048] In this embodiment of the invention, the current execution end, acting as the execution end for processing and mining different business data, can be a cloud server or a computer device. It collects a large amount of financial institution data (such as banking institutions) from the regulatory analysis platform (EAST, Examination and Analysis System Technology) as the dataset to execute the methods in steps 101-103. This embodiment of the invention does not impose specific limitations. The dataset refers to unprocessed data collected from the EAST system; it can be text data, numerical data, or any other form of data content. This embodiment of the invention does not impose specific limitations. Furthermore, the business processing requirement information is used to characterize the data processing requirements for different businesses. For example, the business could be compliance business, user behavior mining business, etc. Correspondingly, the business processing requirement information could be text compliance judgment, behavioral data classification judgment, etc., and can be obtained based on operator input. This embodiment of the invention does not impose specific limitations. At this time, the dataset can be preprocessed according to the business processing requirement information. Preprocessing can include cleaning abnormal data in the dataset, or correcting biased data, etc. This embodiment of the invention does not impose specific limitations.

[0049] 102. The preprocessed dataset is subjected to a first processing based on an unsupervised learning large model to obtain a first processing result, and the first processing result is subjected to a second processing based on a supervised learning model to obtain a second processing result.

[0050] In this embodiment of the invention, the large unsupervised learning model is constructed based on at least two unsupervised learning models. These unsupervised learning models can include novel clustering algorithms (such as spectral clustering and hierarchical clustering), dimensionality reduction algorithms (such as unified manifold approximation and projection, t-distributed random neighborhood embedding), autoencoders, etc. For datasets corresponding to different business operations, multiple unsupervised learning models can be used in parallel for processing, and then integrated to obtain a first processing result. Furthermore, the first processing result is directly input into a supervised learning model for second processing via concatenation. Here, the supervised learning model performs tasks that typically require supervised learning through unsupervised learning. Instead of relying on labeled datasets of supervised signals, the supervised learning model generates implicit labels from unstructured data to perform deep mining on the first processing result obtained after processing the large unsupervised learning model, thus obtaining the second processing result. The supervised learning model can include, but is not limited to, decision numbers, neural networks, and support vector machines; this embodiment of the invention does not impose specific limitations.

[0051] 103. The application model based on business scenario requirement information matching identifies the second processing result to obtain the data object.

[0052] In this embodiment of the invention, to better match the data processing results with different business needs and to extract data correlations and other characteristics from the dataset, the current execution end matches the application model to be executed for downstream business according to the business scenario requirements. Here, the application model is a model constructed according to the business process or application scenario architecture, including but not limited to product matching models and information push models; this embodiment of the invention does not impose specific limitations. The business scenario requirements information is used to characterize the specific content required by the application scenario of the dataset, in order to match the corresponding application model. For example, if the scenario is a data compliance scenario, the corresponding application model can be a process model for identifying text feature words, comparing feature words with a compliance terminology library, etc., to obtain data objects. Here, the data objects are objects with correlation and research value obtained from in-depth mining of large amounts of data, such as illegal words and illegal behaviors, which can be determined based on the operator's input method; this embodiment of the invention does not impose specific limitations, and the data objects are output to users or technical personnel through the EAST system.

[0053] It should be noted that in specific implementation scenarios, the dataset is collected from the application and preprocessed according to business processing requirements (such as compliance processing). After deleting abnormal data that cannot be used for compliance judgment, cluster analysis is performed simultaneously using spectral clustering and hierarchical clustering in the unsupervised learning model to obtain the first processing result. This result is then input into a supervised learning model (such as a deep learning model) for the second processing to obtain the second processing result. After obtaining the second processing result, the application model based on the matching is used to identify the second processing result to obtain the data object.

[0054] In another embodiment of the invention, for further definition and explanation, the steps further include:

[0055] The data source and data type of the dataset are parsed, and the application model is matched according to the data source and data type.

[0056] To more efficiently apply the acquired data objects to different business scenarios, the current execution end first parses the data source and data type of the dataset to match the appropriate user model according to the data source and data type. At this point, the application model includes a compliance semantic model, an anomaly prediction model, and an information classification model. Different business scenarios can be matched with different application models. The data source is the specific starting node of the collected dataset, such as a financial product application or a technology product marketing application. Data categories include text, image, and numerical types. In this embodiment, pre-configuration based on business applications is possible, but no specific limitations are imposed. The compliance semantic model can be an intelligent model built using Natural Language Processing (NLP), the anomaly prediction model can be a neural network model, a support vector machine model, etc., and the information classification model can be a decision tree model, etc. No specific limitations are imposed in this embodiment.

[0057] In a specific implementation scenario, if the business scenario involves text compliance judgment, the corresponding data source can be a financial product application with a text data type. After executing steps 101-103, the data objects obtained include fee text objects, user identity text objects, professional practice text objects, and operation process text objects. The application model obtained by matching the financial product application and text type is a compliance semantic model. Then, the fee text objects, user identity text objects, professional practice text objects, and operation process text objects are used as inputs to the compliance semantic model, and the output is text objects with non-compliance labels. This achieves the purpose of using unsupervised learning large models and supervised learning models to concatenate and mine the dataset, which is convenient for assisting data mining personnel in conducting in-depth research on data correlation, so as to quickly and effectively obtain the true value of data from a large amount of data.

[0058] In a specific implementation scenario, if the business scenario is risk warning, the corresponding data source can be an information push application with image data. After executing steps 101-103, the data objects obtained include product packaging image objects, product structure image objects, and product usage process objects. The application model obtained by matching the information push application and image type is an anomaly prediction model. Then, the product packaging image objects, product structure image objects, and product usage process objects are used as inputs to the anomaly prediction model, and the output is image objects with anomaly labels. This achieves the purpose of mining the dataset by splicing unsupervised learning large models and supervised learning models, which is convenient for assisting data mining personnel in conducting in-depth research on data correlation, so as to quickly and effectively obtain the true value of data from a large amount of data.

[0059] In a specific implementation scenario, if the business scenario is information security level rating, the corresponding data source can be a tax processing application with a numeric data type. After executing steps 101-103, the data objects obtained include numeric time objects, numeric upload area objects, and numeric attribute objects. The application model obtained by matching the tax processing application and the numeric type is an information classification model. Then, the numeric time objects, numeric upload area objects, and numeric attribute objects are used as inputs to the information classification model, and the output is a numeric object with classification results. This achieves the purpose of mining the dataset by combining unsupervised learning large models and supervised learning models, which is convenient for assisting data mining personnel in conducting in-depth research on data correlations, so as to quickly and effectively obtain the true value of data from a large amount of data.

[0060] In the different business scenarios mentioned above, different application models can be pre-trained based on corresponding prior samples to ensure that each application model is accurately and effectively applied to each scenario.

[0061] In another embodiment of the invention, for further definition and explanation, the step of performing a first processing on the preprocessed dataset based on an unsupervised learning large model to obtain a first processing result includes:

[0062] Based on the preprocessed dataset, at least two unsupervised learning models to be processed in the large unsupervised learning model are identified.

[0063] The dataset is processed separately based on the unsupervised learning model to obtain multiple sub-processing results, and the multiple sub-processing results are integrated to obtain the first processing result.

[0064] To leverage unsupervised learning techniques for in-depth dataset mining to identify data correlations and potential value, the current execution end, when performing initial processing using a large unsupervised learning model, first determines at least two unsupervised learning models to be processed. Specifically, when determining the unsupervised learning models, the models can be matched based on the characteristics of the dataset. Since unsupervised learning models can include clustering algorithms, dimensionality reduction algorithms, etc., the dataset can be pre-configured as the input to all unsupervised learning models, yielding processed sub-processing results. That is, each model performs comprehensive data mining on the dataset, obtaining multiple sub-processing results, which are then integrated. During integration, the final first processing result can be obtained through weighted summation or direct summation; this embodiment of the invention does not impose specific limitations. In addition, when adding weights, specific weights can be pre-configured according to the function of different unsupervised learning models. For example, a model with a novel clustering function can be configured with a higher weight value, such as 0.4, while a model with a dimensionality reduction function can be configured with a lower weight value, such as 0.1, and then added together. This embodiment of the invention does not impose specific limitations.

[0065] In another embodiment of the invention, for further definition and explanation, before the step of performing a second processing on the first processing result based on a supervised learning model to obtain the second processing result, the method further includes:

[0066] According to different business processing requirements, the source data samples are labeled using a semantic model, and the supervised learning model is trained using the labeled source data samples.

[0067] To achieve in-depth data mining and multi-faceted identification, before using the supervised learning model for the second processing, in a specific implementation scenario where the data in the dataset is text data, a semantic model can be used to first label the source data samples used as the training sample set for different business processing needs. The semantic model can be pre-trained using a natural language library; this embodiment of the invention does not impose specific limitations. The business processing requirement information is used to characterize the processing needs when performing data mining for different business needs, including but not limited to prediction needs, anomaly identification needs, and semantic conversion needs. This is pre-configured according to different business scenarios; this embodiment of the invention does not impose specific limitations. Furthermore, the source data samples are the data samples used to train the self-supervised model. For example, if the business processing requirement information is compliance judgment, the semantic model is used to label the source data samples with non-compliant semantics. Alternatively, the semantic model can automatically detect dimension names in the data, such as "date" for "date" and "price" for "price," and label them. The labeled source data samples are then used to train the supervised learning model, and the second processing is performed on the first processing result based on the trained supervised learning model. In this embodiment of the invention, since the unsupervised learning model itself does not need to add labels to the data, in order to stimulate the potential of unsupervised learning, a labeling method can be used for visualization and to compensate for the shortcomings of unsupervised learning when training the supervised learning model.

[0068] In another embodiment of the invention, for further definition and explanation, the steps further include:

[0069] If the data object completes business matching verification or business mining verification, then the verification label of the data object after verification is obtained, and the verification label is used to update and train the supervised learning model so that the supervised learning model after the update and training can perform the next round of second processing.

[0070] To ensure that the data comparisons determined after data processing can be effectively used in downstream business operations, thereby improving the effectiveness of data object acquisition based on the EAST system, the current execution end performs business matching verification or business mining verification on the data objects. Business matching verification or business mining verification can be based on manual review or on business rules. Business matching verification determines whether the data object can be applied to a business scenario, while business mining verification determines whether the data object represents business-related content. After passing verification, the data objects can be labeled manually to obtain verification labels. Data objects with verification labels are then used as new samples to update and train the supervised learning model, improving the accuracy of self-supervised learning. This, in turn, improves the data correlation processing effect of unsupervised learning combined with the supervised learning model after data mining.

[0071] In another embodiment of the invention, for further definition and explanation, the steps further include:

[0072] The evaluation strategy is determined based on the output representation of the business scenario requirements information, and the evaluation result of the data object is generated based on the evaluation strategy.

[0073] To effectively output data objects and improve their visualization, the current execution end, when outputting objects, specifically determines an evaluation strategy based on the output representation corresponding to the business scenario requirements. This evaluation strategy characterizes the rules for scoring and analyzing the data objects, resulting in the output of the data objects with the evaluation results. The output representation characterizes the output format of the data objects, including but not limited to text, data, and icon formats. Different evaluation strategies can be matched to different output representations, and this embodiment does not impose specific limitations. Specifically, the output representation can be pre-defined in the business scenario requirements, and an evaluation strategy matching this output representation is determined based on a pre-defined query relationship to generate an evaluation result for the data objects. The evaluation strategies include semantic analysis scoring strategies and data association scoring strategies. Semantic analysis scoring strategies can identify data objects based on natural language processing technology and construct a scoring system based on the presence or absence of predetermined words, for example, scoring based on the number of keywords appearing. This embodiment does not introduce any new specific strategies. Data correlation scoring strategies can construct a scoring system based on whether there is correlation between various data in a data object. For example, a decision tree model can be used to evaluate the correlation of data objects (reflecting whether there is correlation between data types, business logic, and other classifications). This embodiment of the invention does not impose specific limitations.

[0074] In another embodiment of the invention, for further definition and explanation, the preprocessing of the dataset according to different business processing requirements includes:

[0075] The dataset is semantically transformed based on the pre-trained semantic model to obtain a semantic dataset. Then, the semantic dataset is cleaned and corrected based on the anomaly information database of the business processing requirements information to complete the preprocessing.

[0076] To effectively acquire data objects and understand the data relationships within a large dataset, the current execution end preprocesses the dataset according to business processing requirements. Specifically, in scenarios where the dataset consists of collected text data, the text in the dataset is first semantically transformed based on a pre-trained semantic model to obtain a semantic dataset. The semantic model can be BERT (Bidirectional Encoder Representations from Transformers) or GPT (Generative Pre-trained Transformer), and this embodiment of the invention does not impose specific limitations. Furthermore, based on an anomaly information database corresponding to the business processing requirements, the semantic dataset undergoes anomaly cleaning and correction, thus completing the preprocessing. The business processing requirements, such as compliance-related business information, can be pre-configured with an anomaly information database containing data content that is definitely abnormal in the collected data. This database is used to clean the semantic dataset. Simultaneously, if the cleaned dataset contains missing data, it can be corrected using default filler data from the anomaly information database, thus completing the preprocessing and deleting data that does not conform to expected patterns, industry standards, or regulatory requirements, as well as data content that may exhibit missing data, extreme values, format errors, logical errors, etc.

[0077] This invention provides a method for acquiring data objects based on the EAST system. Compared with existing technologies, this invention acquires a dataset collected from the EAST system and preprocesses the dataset according to different business processing requirements. It then performs a first processing on the preprocessed dataset based on an unsupervised learning model to obtain a first processing result, and performs a second processing on the first processing result based on a supervised learning model to obtain a second processing result. The unsupervised learning model is constructed based on at least two unsupervised learning models. Finally, an application model matching business scenario requirements identifies the second processing result to obtain data objects. This method combines self-supervised and unsupervised learning, enhances the value representation of potential data through in-depth data mining, avoids meaningless training on unrelated data, reduces the time cost of unsupervised learning, and enables in-depth analysis of the potential value of data. This effectively assists in digital transformation and achieves the goal of effectively acquiring data objects.

[0078] Furthermore, as a response to the above Figure 1 The implementation of the method shown in this invention provides a data object acquisition device based on the EAST system, such as... Figure 2 As shown, the device includes:

[0079] The acquisition module 21 is used to acquire the dataset collected in the EAST system and preprocess the dataset according to different business processing requirements.

[0080] The processing module 22 is used to perform a first processing on the preprocessed dataset based on an unsupervised learning large model to obtain a first processing result, and to perform a second processing on the first processing result based on a supervised learning model to obtain a second processing result. The unsupervised learning large model is constructed based on at least two unsupervised learning models.

[0081] The identification module 23 is used to identify the second processing result based on the application model that matches the business scenario requirement information to obtain the data object.

[0082] Furthermore, the processing module is used to determine at least two unsupervised learning models to be processed in the large unsupervised learning model based on the preprocessed dataset; to process the dataset separately based on the unsupervised learning models to obtain multiple sub-processing results, and to integrate the multiple sub-processing results to obtain the first processing result.

[0083] Furthermore, the device also includes:

[0084] The training module is used to label source data samples using a semantic model according to different business processing requirements, and to train the supervised learning model using the labeled source data samples, so as to perform a second processing on the first processing result based on the supervised learning model that has completed model training.

[0085] Furthermore, the device also includes:

[0086] The update module is used to obtain the verification label of the data object after verification if the data object completes the business matching verification or business mining verification, and use the verification label to update and train the supervised learning model so that the supervised learning model after the update training can perform the next round of second processing.

[0087] Furthermore, the identification module is specifically used to determine the evaluation strategy according to the output representation of the business scenario requirement information, and generate the evaluation result of the data object based on the evaluation strategy. The evaluation strategy is used to characterize the rule content for scoring and analyzing the data object, so as to output the data object with the evaluation result.

[0088] Furthermore, the acquisition module is specifically used to perform semantic transformation on the dataset based on the pre-trained semantic model to obtain a semantic dataset, and to perform abnormal data cleaning and correction processing on the semantic dataset based on the abnormal information database of the business processing requirement information to complete the preprocessing.

[0089] Furthermore, the device also includes:

[0090] The parsing module is used to parse the data source and data type of the dataset, and match the application model according to the data source and data type. The application model includes a compliance semantic model, an anomaly prediction model, and an information classification model.

[0091] This invention provides a data object acquisition device based on the EAST system. Compared with the prior art, this invention acquires a dataset collected from the EAST system and preprocesses the dataset according to different business processing requirements. It then performs a first processing on the preprocessed dataset based on an unsupervised learning model to obtain a first processing result, and performs a second processing on the first processing result based on a supervised learning model to obtain a second processing result. The unsupervised learning model is constructed based on at least two unsupervised learning models. Finally, an application model matching business scenario requirements identifies the second processing result to obtain data objects. This achieves the purpose of combining self-supervised learning and unsupervised learning, enhancing the value presentation of potential data through in-depth data mining, avoiding meaningless training on unrelated data, thereby reducing the time cost of unsupervised learning, achieving in-depth analysis of the potential value of data, effectively assisting digital transformation, and achieving the goal of effectively acquiring data objects.

[0092] According to one embodiment of the present invention, a storage medium is provided, the storage medium storing at least one executable instruction, the computer-executable instruction being able to execute the data object acquisition method based on the EAST system in any of the above method embodiments.

[0093] Figure 3 The diagram illustrates a structural schematic of a computer device according to an embodiment of the present invention. The specific embodiments of the present invention do not limit the specific implementation of the computer device.

[0094] like Figure 3 As shown, the computer device may include: a processor 302, a communications interface 304, a memory 306, and a communications bus 308.

[0095] The processor 302, communication interface 304, and memory 306 communicate with each other via communication bus 308.

[0096] Communication interface 304 is used to communicate with other network elements such as clients or other servers.

[0097] The processor 302 is used to execute program 310, specifically to execute the relevant steps in the above-described embodiment of the data object acquisition method based on the EAST system.

[0098] Specifically, program 310 may include program code that includes computer operation instructions.

[0099] Processor 302 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The computer device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0100] Memory 306 is used to store program 310. Memory 306 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0101] Specifically, program 310 can be used to cause processor 302 to perform the following operations:

[0102] Acquire the dataset collected in the EAST system and preprocess the dataset according to different business processing requirements;

[0103] The preprocessed dataset is first processed based on an unsupervised learning large model to obtain a first processing result, and the first processing result is then processed based on a supervised learning model to obtain a second processing result. The unsupervised learning large model is constructed based on at least two unsupervised learning models.

[0104] The application model, based on matching business scenario requirements, identifies the second processing result to obtain a data object.

[0105] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0106] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for acquiring data objects based on the EAST system, characterized in that, include: Acquire the dataset collected in the EAST system and preprocess the dataset according to different business processing requirements; The preprocessed dataset is first processed based on an unsupervised learning large model to obtain a first processing result, and the first processing result is then processed based on a supervised learning model to obtain a second processing result. The unsupervised learning large model is constructed based on at least two unsupervised learning models. The application model based on business scenario requirement information matching identifies the second processing result to obtain a data object. The business scenario requirement information is used to characterize the specific content required by the application scenario of the dataset. The application model includes a compliance semantic model, an anomaly prediction model, and an information classification model. Different business scenarios can be matched with different application models. The business scenarios include text compliance judgment, risk warning, and information security level rating. The first processing of the preprocessed dataset based on the unsupervised learning large model to obtain the first processing result includes: Based on the preprocessed dataset, at least two unsupervised learning models to be processed in the large unsupervised learning model are identified. The dataset is processed separately based on the unsupervised learning model to obtain multiple sub-processing results, and the multiple sub-processing results are integrated to obtain the first processing result; Before performing a second processing on the first processing result based on a supervised learning model to obtain the second processing result, the method further includes: According to different business processing requirements, the source data samples are labeled using a semantic model, and the supervised learning model is trained using the labeled source data samples, so as to perform a second processing on the first processing result based on the supervised learning model that has completed model training. The method further includes: If the data object completes business matching verification or business mining verification, then the verification label of the data object after verification is obtained, and the verification label is used to update and train the supervised learning model so that the supervised learning model after the update and training can perform the next round of second processing. The method further includes: The evaluation strategy is determined according to the output representation of the business scenario requirement information, and the evaluation result of the data object is generated based on the evaluation strategy. The evaluation strategy is used to characterize the rule content for scoring and analyzing the data object, so as to output the data object with the evaluation result.

2. The method according to claim 1, characterized in that, The preprocessing of the dataset based on different business processing requirements includes: The dataset is semantically transformed based on the pre-trained semantic model to obtain a semantic dataset. Then, the semantic dataset is cleaned and corrected based on the anomaly information database of the business processing requirements information to complete the preprocessing.

3. The method according to claim 1 or 2, characterized in that, The method further includes: The data source and data type of the dataset are parsed, and an application model is matched according to the data source and data type. The application model includes a compliance semantic model, an anomaly prediction model, and an information classification model.

4. A data object acquisition device based on the EAST system, characterized in that, include: The acquisition module is used to acquire the dataset collected in the EAST system and preprocess the dataset according to different business processing requirements. The processing module is used to perform a first processing on the preprocessed dataset based on an unsupervised learning large model to obtain a first processing result, and to perform a second processing on the first processing result based on a supervised learning model to obtain a second processing result, wherein the unsupervised learning large model is constructed based on at least two unsupervised learning models; The identification module is used to identify the second processing result based on the application model matching the business scenario requirement information to obtain the data object. The business scenario requirement information is used to characterize the specific content required by the application scenario of the dataset. The application model includes a compliance semantic model, an anomaly prediction model, and an information classification model. Different business scenarios can be matched with different application models. The business scenarios include text compliance judgment, risk warning, and information security level rating. The processing module is used to determine at least two unsupervised learning models to be processed in the large unsupervised learning model based on the preprocessed dataset. The dataset is processed separately based on the unsupervised learning model to obtain multiple sub-processing results, and the multiple sub-processing results are integrated to obtain the first processing result; The device further includes: The training module is used to label source data samples using a semantic model according to different business processing requirements, and to train the supervised learning model using the labeled source data samples, so as to perform a second processing on the first processing result based on the supervised learning model that has completed model training. The device further includes: The update module is used to obtain the verification label of the data object after verification if the data object completes the business matching verification or business mining verification, and use the verification label to update and train the supervised learning model so that the supervised learning model after the update training can perform the next round of second processing. The identification module is specifically used to determine the evaluation strategy according to the output representation of the business scenario requirement information, and generate the evaluation result of the data object based on the evaluation strategy. The evaluation strategy is used to characterize the rule content for scoring and analyzing the data object, so as to output the data object with the evaluation result.

5. A storage medium storing at least one executable instruction that causes a processor to perform an operation corresponding to the data object acquisition method based on the EAST system as described in any one of claims 1-3.

6. A computer device, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the data object acquisition method based on the EAST system as described in any one of claims 1-3.

Images