Data classification method and apparatus, electronic device, and computer-readable storage medium

Abstract

This application discloses a data classification method, comprising: acquiring interface data samples; constructing single-field structured samples corresponding to each single-field sample according to a preset single-field structure template for each single-field sample in each interface data sample; grouping the single-field structured samples under each interface data sample according to a preset grouping method to obtain multi-field structured samples corresponding to the corresponding interface data samples; training a model using each single-field structured sample and each multi-field structured sample to obtain a multi-field aggregation classification model; and classifying target interface data using the multi-field aggregation classification model. Applying the technical solution provided in this application allows for faster and more efficient classification of interface data. This application also discloses a data classification device, electronic device, and computer-readable storage medium, which also have the above-mentioned technical effects.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data classification method, as well as a data classification device, electronic device, and computer-readable storage medium. Background Technology

[0002] For API return data that is multi-industry and dynamically changing, existing solutions mostly perform classification prediction on a single field basis to achieve classification prediction for the entire API return data. However, an API return data typically contains multiple single fields, which greatly reduces the classification efficiency of the API return data. Therefore, how to perform faster and more efficient classification processing of API data is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0003] The purpose of this application is to provide a data classification method that can classify interface data more quickly and efficiently; another purpose of this application is to provide a data classification device, electronic device, and computer-readable storage medium, all of which have the above-mentioned beneficial effects.

[0004] Firstly, this application provides a data classification method, including: Obtain interface data samples, and for each single field sample in each interface data sample, construct a single field structured sample corresponding to each single field sample according to a preset single field structure template. According to the preset packaging method, each single-field structured sample under each interface data sample is packaged to obtain the multi-field structured sample corresponding to the corresponding interface data sample. The model is trained using the single-field structured samples and the multi-field structured samples to obtain a multi-field aggregation classification model; The target interface data is classified using the multi-field aggregation classification model.

[0005] Preferably, for each single-field sample in each of the interface data samples, a single-field structured sample corresponding to each single-field sample is constructed according to a preset single-field structure template, including: For each single-field sample in each interface data sample, the teacher model is used to construct the single-field structured sample corresponding to each single-field sample according to the preset single-field structure template; Each of the single-field structured samples includes at least the classification label, classification confidence, and field path of the corresponding single-field sample.

[0006] Preferably, after constructing a single-field structured sample corresponding to each single-field sample according to a preset single-field structure template for each single-field sample in each interface data sample, the method further includes: Each of the single-field structured samples is format-validated to determine whether the data format of the single-field structured sample meets the target data parsing format. A consistency check is performed on each of the single-field structured samples to determine whether the single-field structured sample matches the corresponding single-field sample. When the format verification fails and / or the consistency verification fails, the single-field structured sample and the corresponding single-field sample are either removed, regenerated, or manually corrected.

[0007] Preferably, according to a preset packaging method, the single-field structured samples under each interface data sample are packaged to obtain the multi-field structured sample corresponding to the corresponding interface data sample, including: Determine the various package size specifications and the proportion of multi-field structured samples corresponding to each package size specification in all multi-field structured samples; According to the package size specifications and sample proportions, the single-field structured samples under each interface data sample are packaged to obtain the multi-field structured samples corresponding to the interface data samples. The order of the single-field structured samples in each multi-field structured sample is consistent with the order of the corresponding single-field samples in the corresponding interface data sample.

[0008] Preferably, a multi-field aggregation classification model is obtained by training the model using each of the single-field structured samples and each of the multi-field structured samples, including: The basic language model is trained using the single-field structured samples described above to obtain the initial language model; The initial language model is fine-tuned using the multi-field structured samples to obtain the multi-field aggregated classification model.

[0009] Preferably, after classifying the target interface data using the multi-field aggregation classification model, the method further includes: Perform a legality check on the classification results; If the legality check of the classification result fails, the target interface data will be reclassified, or a classification mismatch message or an error message will be output.

[0010] Preferably, after classifying the target interface data using the multi-field aggregation classification model, the method further includes: The target interface data and the classification results of the target interface data are saved to a preset storage area. The multi-field aggregation classification model is optimized and updated using the stored data in the preset storage area at preset time intervals.

[0011] Secondly, this application also discloses a data classification device, comprising: The construction module is used to obtain interface data samples and, for each single field sample in each interface data sample, construct a single field structured sample corresponding to each single field sample according to a preset single field structure template. The packaging module is used to package each single-field structured sample under each interface data sample according to a preset packaging method to obtain a multi-field structured sample corresponding to the interface data sample. The training module is used to train the model using the single-field structured samples and the multi-field structured samples to obtain a multi-field aggregated classification model. The classification module is used to classify the target interface data using the multi-field aggregation classification model.

[0012] Thirdly, this application also discloses an electronic device, comprising: Memory, used to store computer programs; A processor for executing the computer program to implement any of the data classification methods described above.

[0013] Fourthly, this application also discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the data classification methods described above.

[0014] This application provides a data classification method comprising: acquiring interface data samples; constructing single-field structured samples corresponding to each single-field sample according to a preset single-field structure template for each single-field sample in each interface data sample; grouping the single-field structured samples under each interface data sample into packages according to a preset packaging method to obtain multi-field structured samples corresponding to the corresponding interface data samples; training a model using each single-field structured sample and each multi-field structured sample to obtain a multi-field aggregation classification model; and classifying target interface data using the multi-field aggregation classification model.

[0015] As can be seen, the data classification method provided in this application first collects interface data samples to extract single-field samples from each interface data sample, and then obtains single-field structured samples and multi-field structured samples through structured processing and packaging, respectively, to train a multi-field aggregation classification model. This multi-field aggregation classification model can directly classify data for a complete target interface data, which has higher classification efficiency than the traditional technology of sequentially identifying each single field in the target interface data. Therefore, this technical solution can perform faster and more efficient classification processing of interface data.

[0016] The data classification system, apparatus, electronic device, and computer-readable storage medium provided in this application all have the aforementioned beneficial effects, which will not be elaborated further here. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the prior art and the embodiments of this application, the accompanying drawings used in the description of the prior art and the embodiments of this application will be briefly introduced below. Of course, the accompanying drawings described below with respect to the embodiments of this application are only a part of the embodiments in this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort, and such other drawings also fall within the protection scope of this application.

[0018] Figure 1 A flowchart illustrating a data classification method provided in this application; Figure 2 A flowchart illustrating another data classification method provided in this application; Figure 3 A schematic diagram of a data classification device provided in this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0019] The core of this application is to provide a data classification method that can classify interface data more quickly and efficiently; another core aspect of this application is to provide a data classification device, apparatus, and computer-readable storage medium, which also have the aforementioned beneficial effects.

[0020] To provide a clearer and more complete description of the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0021] This application provides a data classification method.

[0022] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a data classification method provided in this application, which may include the following steps S101 to S104.

[0023] S101: Obtain interface data samples. For each single-field sample in each interface data sample, construct the single-field structured sample corresponding to each single-field sample according to the preset single-field structure template.

[0024] This step aims to construct single-field structured samples. Specifically, a large number of interface data samples can be collected first. For example, interface data samples corresponding to different industries can be collected based on the classification directories and business interface specifications of different industries. For any industry category, single-field sample extraction can be performed on each interface data sample under that industry category, that is, extracting the single-field data that can participate in classification and recognition from the corresponding interface data sample. Furthermore, the single-field structured sample corresponding to each single-field sample can be constructed according to the pre-set single-field structure template.

[0025] In one possible implementation, the extraction of individual field samples from the interface data sample can be achieved by referring to the following extraction rules: (1) a field is a leaf field or can be split into atomic fields; (2) the field value of a field is not empty, not a pure placeholder, and not a meaningless constant; (3) the field value or field label of a field can reflect the business semantics related to the category directory; (4) a field does not belong to a pure technical control field, a structural placeholder field, or a metadata field used only for internal routing of the program; (5) a field is within the scope of the preset fields to be governed. Therefore, when a field in the interface data sample meets the above requirements, it can be used as a single field sample in the interface data sample.

[0026] Furthermore, the preset single-field structure template can be a chain structure, which specifies the relevant information required to be included in the single-field structured sample. Its specific content can be set according to actual needs. The following embodiment provides a single-field structured sample, whose specific content can at least include the classification label, classification confidence score, and field path of the corresponding single-field sample.

[0027] In one embodiment of this application, for each single-field sample in each interface data sample, constructing a single-field structured sample corresponding to each single-field sample according to a preset single-field structure template may include: for each single-field sample in each interface data sample, using a teacher model to construct a single-field structured sample corresponding to each single-field sample according to a preset single-field structure template; wherein, each single-field structured sample includes at least the classification label, classification confidence, and field path of the corresponding single-field sample.

[0028] Specifically, the teacher model can be a large language model with both text parsing and structure generation capabilities, deployable offline and directly invoked during use. Specifically, after obtaining the individual field samples from each interface data sample, these samples are input into the teacher model for text parsing and structure generation. The final model output is the structured sample corresponding to each individual field sample. Furthermore, this structured sample can at least include the classification label, classification confidence score, and field path of the corresponding individual field sample. The classification label indicates the category to which the individual field sample belongs, the classification confidence score indicates the degree of confidence that the individual field sample belongs to the corresponding category, and the field path indicates the position information of the individual field sample in the corresponding interface data sample, specifically, the path representation of the individual field's position in the interface return body (interface data sample) hierarchical structure. Understandably, the purpose of field paths is to facilitate accurate differentiation of identical fields across different levels and contexts during subsequent model training and inference, thereby ensuring accuracy. For example, although both passenger.contact.phone and emergency.contact.phone contain "phone," their paths differ, and their corresponding business semantics may also differ, thus requiring path information for distinction. Of course, the data information required in a single-field structured sample is not limited to this. For instance, it can further include the industry name, interface address, access intent, category directory, classification evidence, and complete interface return body corresponding to the single-field sample.

[0029] In addition, before constructing the corresponding single-field structured sample according to the preset single-field structure template for each single-field sample in each interface data sample, it may also include: standardizing each interface data sample according to the preset data format to obtain each standard interface data sample; wherein, each standard interface data sample includes at least the interface return body, a single-field list, and the field paths of each single-field sample in the single-field list.

[0030] Understandably, standardizing interface data samples yields more standardized samples, facilitating the subsequent construction of single-field structured samples. Here, the standardization of interface data samples can be implemented using a pre-defined data format, as exemplified above: each standard interface data sample includes at least the interface return body, a list of single fields, and the field paths for each single field sample within the list. Furthermore, it can include industry name, interface address, access intent, etc.

[0031] Furthermore, after constructing the corresponding single-field structured sample according to the preset single-field structure template for each single-field sample in each interface data sample, the process may further include: performing format validation on each single-field structured sample to determine whether the data format of the single-field structured sample meets the target data parsing format; performing consistency validation on each single-field structured sample to determine whether the single-field structured sample matches the corresponding single-field sample; and when the format validation fails and / or the consistency validation fails, removing, regenerating, or manually correcting the single-field structured sample and the corresponding single-field sample.

[0032] To effectively ensure the accuracy of the data samples used for model training and further guarantee model precision, after constructing each single-field structured sample, format validation and consistency validation can be performed on each sample. Only when both validations pass can the corresponding single-field structured sample be used for subsequent model training. Of course, single-field structured samples that fail validation can be removed, regenerated, or manually corrected according to actual needs.

[0033] Among them, format validation aims to determine whether the data format of a single-field structured sample meets the target data parsing format. In one possible implementation, format validation may include, but is not limited to: (1) being parsable by JSON; (2) having all required fields at the top and sub-levels. For example, the top level only contains fields used to carry the reasoning process and decision results, and is further refined internally into field name, classification decision (referring to the judgment result of the final category to which the corresponding single-field sample belongs, for example, it may be hitting a certain leaf category; when the evidence is insufficient or the directory is not covered, it may output "no suitable category matched"), discrete confidence, and sample value (referring to the field example value of the corresponding single-field sample directly extracted from the original interface return body, which may be the original value or the representative value after desensitization). (3) The field type meets the preset requirements, for example, array fields must be arrays and string fields must be strings; (4) The enumeration value is valid, for example, the confidence level can only be a preset level; (5) The output of undefined keys is prohibited (if necessary).

[0034] Among them, consistency verification aims to determine whether a single-field structured sample matches the corresponding single-field sample. In one possible implementation, consistency verification may include, but is not limited to: (1) field_name and target_field_path are consistent with the input single field; (2) sample_value can indeed be located from the original interface return body; (3) category_decision must belong to a valid path in the category directory, or be a preset "unmatched appropriate category"; (4) when the output is "unmatched appropriate category", its "unmatched main reason" must come from a preset reason set; (5) rule hit, morphological information and sample value cannot contradict each other; (6) context fields must actually exist in the original interface return body; (7) confidence and evidence completeness should satisfy a preset consistency relationship.

[0035] S102: According to the preset packaging method, package the single-field structured samples under each interface data sample to obtain the multi-field structured sample corresponding to the interface data sample.

[0036] This step aims to package single-field structured samples into multi-field structured samples, so that in subsequent steps, these single-field and multi-field structured samples can be used to train a multi-field aggregation classification model. The packaging method for single-field structured samples is not unique and can be set according to actual needs. This application does not limit this method; the following embodiment provides a specific method for packaging single-field structured samples.

[0037] In one embodiment of this application, according to a preset packaging method, the single-field structured samples under each interface data sample are packaged to obtain the multi-field structured sample corresponding to the corresponding interface data sample. This may include: determining multiple packaging size specifications and the sample proportion of the multi-field structured sample corresponding to each packaging size specification in all multi-field structured samples; according to each packaging size specification and each sample proportion, the single-field structured samples under each interface data sample are packaged to obtain the multi-field structured sample corresponding to the corresponding interface data sample; wherein, the arrangement order of the single-field structured samples in each multi-field structured sample is consistent with the arrangement order of the corresponding single-field samples in the corresponding interface data sample.

[0038] In other words, the packet assembly rules include packet size specifications, packet proportions, and the order of single-field samples within a packet. Firstly, packets of different sizes can be constructed, such as five-field packets, ten-field packets, fifteen-field packets, and complete packets. Secondly, the proportions of different sized packets within all packets can be set. For example, the proportions of five-field, ten-field, fifteen-field, and complete packet samples in the total sample can be determined. These proportions do not necessarily have to be exactly equal; the aim is to balance the learning stability of small-scale packet assembly with the generalization ability of large-scale complete packet output. One possible implementation is to use an approximately balanced ratio, or to perform unbalanced sampling of packets of different sizes based on the distribution of the number of fields, error rate distribution, or business importance. For example, in scenarios with a large number of fields, the five-field, ten-field, fifteen-field, and complete packets could be set to 30%, 30%, 20%, and 20%, respectively. Thirdly, the arrangement order of each single-field structured sample in the multi-field structured sample should be consistent with the arrangement order of each corresponding single-field sample in the corresponding interface data sample. Here, the arrangement order can come from the traversal order after parsing the interface return body, or from the field path arrangement order of each single-field sample in the interface return body, or be generated based on other repeatable deterministic rules. It should be noted that since the multi-field structured sample is used to train the multi-field aggregation classification model, and the multi-field aggregation classification model is used to classify the complete interface data, the training phase and the inference phase of the model should follow the same order rule.

[0039] S103: Use single-field structured samples and multi-field structured samples to train the model and obtain a multi-field aggregated classification model.

[0040] This step aims to train a multi-field aggregation classification model. The training samples include both single-field structured samples and multi-field structured samples. In practice, the basic language model can be trained first using the single-field structured samples to obtain an initial language model. Then, the initial language model can be fine-tuned using the multi-field structured samples to obtain the multi-field aggregation classification model. It should be noted that the basic language model is the student model corresponding to the teacher model mentioned above, and can also be a large language model. This means that after generating a high-quality training dataset containing single-field structured samples based on the teacher model, this high-quality training dataset can be used to perform supervised training on the corresponding student model, i.e., the basic language model, to obtain the initial language model. Then, the initial language model can be further trained using a high-quality training dataset containing multi-field structured samples to obtain the multi-field aggregation classification model. Alternatively, a mixed training approach using single-field and multi-field structured samples can be used to maintain the structured output habit while improving the model's adaptability to multi-field environments.

[0041] S104: Use a multi-field aggregation classification model to classify the target interface data.

[0042] This step aims to perform classification processing (model inference) on the target interface data. The target interface data refers to the interface data that needs to be classified. After obtaining the target interface data, the same pre-processing steps as those used in the model training phase, such as standardization and structuring, can be performed on the target interface data. The processing results are then input into a multi-field aggregation classification model for further processing. The model's output consists of each individual field in the target interface data and its classification result.

[0043] Furthermore, after classifying the target interface data using a multi-field aggregation classification model, the process can also include: performing a validity check on the classification results; and if the validity check fails, reclassifying the target interface data, outputting a classification mismatch message, or outputting an error message. In other words, to effectively ensure the accuracy of the classification results, after obtaining the classification results for the target interface data, a validity check can be performed on the classification results, including but not limited to detecting whether there are missing fields, incorrect order, or invalid enumeration values. For classification results that fail the validity check, the target interface data can be reclassified according to actual needs, or a classification mismatch message / error message can be directly output.

[0044] Furthermore, after classifying the target interface data using a multi-field aggregation classification model, the process can also include: saving the target interface data and its classification results to a preset storage area; and optimizing and updating the multi-field aggregation classification model using the stored data in the preset storage area at preset time intervals. In other words, the multi-field aggregation classification model can be optimized and updated periodically or irregularly using actual inference data to effectively ensure model accuracy and thus improve the accuracy of data classification results.

[0045] As can be seen, the data classification method provided in this application first collects interface data samples to extract single-field samples from each interface data sample, and then obtains single-field structured samples and multi-field structured samples through structured processing and packaging, respectively, to train a multi-field aggregation classification model. This multi-field aggregation classification model can directly classify data for a complete target interface data, which has higher classification efficiency than the traditional technology of sequentially identifying each single field in the target interface data. Therefore, this technical solution can perform faster and more efficient classification processing of interface data.

[0046] Based on the above embodiments, this application provides another data classification method.

[0047] Please refer to Figure 2 , Figure 2 The flowchart of another data classification method provided in this application is shown below, and its specific implementation process is as follows: 1. Data preparation.

[0048] Historical API data samples were collected according to the target industry's classification directory and business interface specifications, and then standardized into a unified input format that includes the industry name, classification directory text, API address, access intent, API return body, field tags, and a list of target single fields. The classification directory can be in "path:description" text format or an equivalent structured format, constructing a multi-level category tree. Simultaneously, the API return body was parsed to identify the set of single fields and dotted paths (field paths) that can participate in the classification, providing a foundation for the subsequent construction of single-field and multi-field samples.

[0049] 2. Construction of single-field structured data.

[0050] For each standardized interface record and each target single field within it, a chained inference input is constructed according to a single-field structure template. This input can include, but is not limited to, industry name, category directory, interface address, access intent, complete return body, single-field label, and single-field path. This input can be strictly constrained by pre-designed structured chained inference hints, such as allowing only fixed-structure JSON objects, with the top level containing only fields for the inference process and decision results, and further subdivided into subfields such as field name, classification decision, discrete confidence score, sample value, rule hit, morphological information, context fields, directory keywords, main cause of non-match, and Chinese reason. This process can be implemented using a teacher model. Furthermore, after generating structured output for each single-field sample, format and consistency checks are performed to ensure JSON structure closure, valid enumeration values, and traceable sample values. Samples that do not meet the constraints are removed or regenerated, ultimately resulting in a high-quality, controllable single-field structured chained inference dataset.

[0051] 3. Multi-field grouping.

[0052] Based on the single-field dataset, using the same interface record as the granularity, multiple target single fields are combined into multi-field samples according to a preset package size. Specifically, depending on the number of fields and business requirements, multi-field samples of different specifications can be constructed, such as five-field packages, ten-field packages, fifteen-field packages, and full packages. An appropriate ratio can also be set so that the training data covers both full-field scenarios and smaller field sets, enhancing the model's generalization ability across different field sizes. When constructing multi-field samples, using the target field array as a baseline, the single-field structured outputs corresponding to each target field in the same record are sequentially merged into a single multi-field decision array. The length of the decision array (multi-field structured sample) must be exactly the same as the length of the target field array, and the field name in each decision object (single-field structured sample) must match the target field at its corresponding index character by character. Simultaneously, a consistency check can be performed again on the merged multi-field structured output, especially regarding the semantic relationships between the unmatched main reasons, Chinese reasons, and evidence information, to effectively ensure that the overall structure is consistent at both the field and record levels.

[0053] 4. Model training.

[0054] After preparing single-field and multi-field bundled datasets, supervised fine-tuning of the basic language model is performed. This training process can be divided into stages: In the first stage, single-field structured data is mainly used to allow the model to fully learn the controlled JSON output format, the correspondence between category directories and field semantics, the handling of invalid values ​​and unmatched scenarios, and the correlation between confidence and evidence; in the second stage, multi-field bundled samples are introduced, allowing the model to generate multi-field decisions and evidence simultaneously during a single decoding process, strengthening capabilities such as field order alignment and overall package output integrity. Alternatively, a mixed training approach using single-field and multi-field samples can be adopted to improve the model's adaptability to multi-field environments while maintaining structured output habits. During training, in addition to the standard language modeling loss, auxiliary constraints for illegal enumerations and structural errors can be added to the implementation, enabling the model to improve its adherence to structure and protocols while learning content semantics.

[0055] Among them, "auxiliary constraints" refer to additional constraint mechanisms introduced in addition to the standard language modeling loss to improve the model's compliance with the structured output protocol. These constraints may include at least one or more of the following: (1) Syntactic structure constraint: The constraint output satisfies the predetermined JSON syntax and hierarchical structure; (2) Key set constraint: The constraint must output the preset fields and avoid outputting undefined keys; (3) Enumeration legality constraint: Only preset enumerations are allowed to be output for controlled fields such as confidence level, unmatched main cause, and morphological type; (4) Directory path constraint: The classification result must fall within the given directory tree, or output the preset unmatched class; (5) Field alignment constraint: The length of the decision array must be consistent with the target field array, and the corresponding subscripts must match item by item; (6) Evidence backreference constraint: Evidence such as sample values ​​and context fields should be able to be traced back to the original input; (7) Consistency constraint: The confidence level, unmatched main cause, Chinese reason and evidence completeness should satisfy the preset logical relationship. All of the above auxiliary constraints can be implemented through constraint decoding, rule verification, loss penalty, illegal sample regeneration or a combination thereof.

[0056] 5. Multi-field aggregation classification reasoning.

[0057] After the model training is complete and it passes validation set evaluation, it is deployed to the online service to perform inference for new interface requests. During the online phase, the system constructs inference requests according to the input specifications from the training phase. It combines the industry name, category directory, interface address, access intent, complete return body, and list of fields to be classified into a single package input, which is then fed into the fine-tuned model for one inference iteration. The model then outputs a chained inference result with a unified structure, including a description of fixed inference steps, decision arrays corresponding one-to-one with the target fields, the classification result for each field, the confidence level, and evidence and reasons. Furthermore, the system can perform structural and enumeration validity checks on the output. If missing fields, incorrect order, or invalid enumeration values ​​are detected, the system can choose to retry, downgrade to simply outputting "No suitable category matched," or record it as a problem sample for subsequent analysis, depending on the configuration.

[0058] 6. Results auditing and closed-loop optimization.

[0059] The aggregated classification results of multiple fields, along with corresponding evidence information, are stored in the audit system or data warehouse for field-level visualization checks, compliance reviews, and model behavior analysis. Understandably, by statistically analyzing the non-match ratio under different categories, the distribution of main causes outside the catalog, and the relationship between confidence and error rates, gaps in the classification catalog, conflicts between rules and samples, and weaknesses in model decision-making can be identified. For misjudged fields or frequently non-matched samples, data can be fed back to the data preparation and template design stages to supplement or revise the classification catalog, construct new single-field and packaged samples, and retrain or fine-tune the model when necessary, thus forming a continuous optimization loop around structured chain reasoning.

[0060] As can be seen, the data classification method provided in this application has the following technical advantages: (1) In a single inference process, multiple target single fields in the same interface record can be uniformly modeled and aggregated and classified, which can improve coverage while ensuring the completeness of the output structure, no omission of fields, and no cross-positioning. (2) The reasoning steps, evidence composition and reasons for non-matching of each target single field are explicitly expressed in the form of structured chain reasoning, so that the classification results can not only be used as the basis for final decision-making, but also for subsequent model iteration and manual review; (3) By combining single-field data construction with multi-field group training, the robustness and overall accuracy of the model in multi-field output scenarios can be enhanced while controlling training costs; (4) Under a unified output structure, complex situations such as those outside the catalog and those with insufficient evidence can be characterized in a standardized manner, and discrete confidence levels can be given to support the upper-level business to carry out differentiated processing and strategy decisions according to risk level.

[0061] This application provides a data classification device.

[0062] Please refer to Figure 3 , Figure 3 This application provides a schematic diagram of the structure of a data classification device, which may include: Module 1 is used to obtain interface data samples. For each single field sample in each interface data sample, a single field structured sample corresponding to each single field sample is constructed according to a preset single field structure template. Package module 2 is used to package the single-field structured samples under each interface data sample according to a preset package method to obtain the multi-field structured sample corresponding to the interface data sample. Training module 3 is used to train the model using single-field structured samples and multi-field structured samples to obtain a multi-field aggregated classification model. Classification module 4 is used to classify target interface data using a multi-field aggregation classification model.

[0063] As can be seen, the data classification device provided in this application first collects interface data samples to extract single-field samples from each interface data sample, and then obtains single-field structured samples and multi-field structured samples through structured processing and packaging, respectively, to train a multi-field aggregation classification model. This multi-field aggregation classification model can directly classify data for a complete target interface data, which has higher classification efficiency than the traditional technology of sequentially identifying each single field in the target interface data. Therefore, this technical solution can perform faster and more efficient classification processing on interface data.

[0064] In one embodiment of this application, the above-mentioned construction module can be specifically used to construct a single-field structured sample corresponding to each single-field sample in each interface data sample using a teacher model according to a preset single-field structure template; wherein, each single-field structured sample includes at least the classification label, classification confidence and field path of the corresponding single-field sample.

[0065] In one embodiment of this application, the data classification device may further include a verification module, which is used to perform format verification on each single-field structured sample after constructing a single-field structured sample corresponding to each single-field sample according to a preset single-field structure template for each single-field sample in each interface data sample, to determine whether the data format of the single-field structured sample meets the target data parsing format; to perform consistency verification on each single-field structured sample to determine whether the single-field structured sample matches the corresponding single-field sample; and when the format verification fails and / or the consistency verification fails, to perform elimination processing, regeneration processing, or manual correction processing on the single-field structured sample and the corresponding single-field sample.

[0066] In one embodiment of this application, the above-mentioned package module 2 can be specifically used to determine multiple package size specifications and the sample proportion of multi-field structured samples corresponding to each package size specification in all multi-field structured samples; according to each package size specification and each sample proportion, the single-field structured samples under each interface data sample are packaged to obtain the multi-field structured sample corresponding to the corresponding interface data sample; wherein, the arrangement order of each single-field structured sample in each multi-field structured sample is consistent with the arrangement order of each corresponding single-field sample in the corresponding interface data sample.

[0067] In one embodiment of this application, the training module 3 described above can be specifically used to train the basic language model using each single-field structured sample to obtain an initial language model; and to fine-tune the initial language model using each multi-field structured sample to obtain a multi-field aggregated classification model.

[0068] In one embodiment of this application, the data classification device may further include a checking module, which is used to perform a legality check on the classification result after classifying the target interface data using a multi-field aggregation classification model; when the legality check of the classification result fails, the target interface data is reclassified or a classification mismatch prompt or an error prompt is output.

[0069] In one embodiment of this application, the data classification device may further include an update module, which is used to save the target interface data and the classification result of the target interface data to a preset storage area after classifying the target interface data using a multi-field aggregation classification model; and to optimize and update the multi-field aggregation classification model using the stored data in the preset storage area at preset time intervals.

[0070] For a description of the apparatus provided in the embodiments of this application, please refer to the above method embodiments; further details will not be repeated here.

[0071] This application provides an electronic device.

[0072] Please refer to Figure 4 , Figure 4 This application provides a schematic diagram of the structure of an electronic device, which may include: Memory 11 is used to store computer programs; The processor 10 is configured to execute computer programs to implement the steps of any of the data classification methods described above.

[0073] like Figure 4 The diagram shows the structural composition of an electronic device, which may include a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, memory 11, and communication interface 12 all communicate with each other through the communication bus 13.

[0074] In this embodiment, the processor 10 may be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic devices.

[0075] The processor 10 can call programs stored in the memory 11. Specifically, the processor 10 can execute operations in the embodiments of the data classification method.

[0076] The memory 11 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment, the memory 11 stores at least a program for implementing the following functions: Obtain interface data samples. For each single-field sample in each interface data sample, construct a single-field structured sample corresponding to each single-field sample according to a preset single-field structure template. Package the single-field structured samples under each interface data sample according to a preset packaging method to obtain the multi-field structured sample corresponding to the interface data sample. Use each single-field structured sample and each multi-field structured sample to train the model and obtain a multi-field aggregation classification model. Use the multi-field aggregation classification model to classify the target interface data.

[0077] In one possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; and the data storage area may store data created during use.

[0078] In addition, memory 11 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0079] Communication interface 12 can be an interface for the communication module, used to connect with other devices or systems.

[0080] Of course, it should be noted that, Figure 4 The structure shown does not constitute a limitation on the electronic device in the embodiments of this application. In practical applications, the electronic device may include more than Figure 4 More or fewer components as shown, or combinations of certain components.

[0081] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of any of the data classification methods described above.

[0082] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0083] For a description of the computer-readable storage medium provided in this application, please refer to the above method embodiments; further details will not be repeated here.

[0084] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0085] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0086] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0087] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of this application. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A data classification method, characterized in that, include: Obtain interface data samples, and for each single field sample in each interface data sample, construct a single field structured sample corresponding to each single field sample according to a preset single field structure template. According to the preset packaging method, each single-field structured sample under each interface data sample is packaged to obtain the multi-field structured sample corresponding to the corresponding interface data sample. The model is trained using the single-field structured samples and the multi-field structured samples to obtain a multi-field aggregation classification model; The target interface data is classified using the multi-field aggregation classification model.

2. The data classification method according to claim 1, characterized in that, For each single-field sample in each of the interface data samples, construct a single-field structured sample corresponding to each single-field sample according to a preset single-field structure template, including: For each single-field sample in each interface data sample, the teacher model is used to construct the single-field structured sample corresponding to each single-field sample according to the preset single-field structure template; Each of the single-field structured samples includes at least the classification label, classification confidence, and field path of the corresponding single-field sample.

3. The data classification method according to claim 1, characterized in that, For each single-field sample in each of the interface data samples, after constructing the corresponding single-field structured sample according to the preset single-field structure template, the method further includes: Each of the single-field structured samples is format-validated to determine whether the data format of the single-field structured sample meets the target data parsing format. A consistency check is performed on each of the single-field structured samples to determine whether the single-field structured sample matches the corresponding single-field sample. When the format verification fails and / or the consistency verification fails, the single-field structured sample and the corresponding single-field sample are either removed, regenerated, or manually corrected.

4. The data classification method according to claim 1, characterized in that, According to a preset packaging method, the single-field structured samples under each interface data sample are packaged to obtain the multi-field structured sample corresponding to the corresponding interface data sample, including: Determine the various package size specifications and the proportion of multi-field structured samples corresponding to each package size specification in all multi-field structured samples; According to the package size specifications and sample proportions, the single-field structured samples under each interface data sample are packaged to obtain the multi-field structured samples corresponding to the interface data samples. The order of the single-field structured samples in each multi-field structured sample is consistent with the order of the corresponding single-field samples in the corresponding interface data sample.

5. The data classification method according to claim 1, characterized in that, The model is trained using the single-field structured samples and the multi-field structured samples to obtain a multi-field aggregation classification model, including: The basic language model is trained using the single-field structured samples described above to obtain the initial language model; The initial language model is fine-tuned using the multi-field structured samples to obtain the multi-field aggregated classification model.

6. The data classification method according to claim 1, characterized in that, After classifying the target interface data using the multi-field aggregation classification model, the process further includes: Perform a legality check on the classification results; If the legality check of the classification result fails, the target interface data will be reclassified, or a classification mismatch message or an error message will be output.

7. The data classification method according to claim 1, characterized in that, After classifying the target interface data using the multi-field aggregation classification model, the process further includes: The target interface data and the classification results of the target interface data are saved to a preset storage area. The multi-field aggregation classification model is optimized and updated using the stored data in the preset storage area at preset time intervals.

8. A data classification device, characterized in that, include: The construction module is used to obtain interface data samples and, for each single field sample in each interface data sample, construct a single field structured sample corresponding to each single field sample according to a preset single field structure template. The packaging module is used to package each single-field structured sample under each interface data sample according to a preset packaging method to obtain a multi-field structured sample corresponding to the interface data sample. The training module is used to train the model using the single-field structured samples and the multi-field structured samples to obtain a multi-field aggregated classification model. The classification module is used to classify the target interface data using the multi-field aggregation classification model.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the data classification method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the data classification method as described in any one of claims 1 to 7.

Images