Intelligent Matching Method and System for SMS AI Training Model
Through structured instructions and acquisition of multi-dimensional feature data and dynamic weight matching algorithm, the shortcomings of SMS AI model matching in the existing technology are solved, more accurate and efficient model selection is achieved, and self-optimization is provided.
Patent Information
- Application Number
- CN202510436872.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-09
- Publication Date
- 2025-08-05
- Estimated Expiration
- 2045-04-09
AI Technical Summary
In the prior art, the matching method of SMS AI model relies on a single natural language input and lacks a standardized model capability description system, resulting in the matching results being insufficiently comprehensive and dynamically optimized, and inefficient.
By obtaining natural language descriptions, file names and file contents, performing semantic chunking and feature extraction, forming structured instructions and multi-dimensional feature data, using feature quadruple dynamic weight matching algorithm, and combining user feedback to optimize the model feature image library.
It realizes self-optimization of model matching accuracy, reduces manual intervention, improves the accuracy and efficiency of data processing, and the system has the ability to evolve itself.
Smart Images

Figure CN119939199B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of natural language processing, and particularly to a method and system for intelligent matching of SMS AI training models. Background Art
[0002] The field of intelligent matching of SMS AI models involves multiple technical directions such as natural language processing, feature recognition, and model matching. The aim of this field is to achieve precise matching between data processing requirements and the functions of AI models, improving data processing efficiency and accuracy.
[0003] Currently, common technical solutions mainly adopt fixed rule matching or simple keyword matching methods. For example, manual selection is carried out based on a preset model classification directory, or literal matching is performed between keywords in the task description and model tags. Although this method is simple to implement, it cannot meet the precise matching requirements in complex scenarios, often requiring professionals to make multiple attempts and adjustments, resulting in low efficiency.
[0004] The closest prior art adopts a model recommendation scheme based on semantic understanding. This scheme performs semantic analysis on the natural language description input by the user, calculates the similarity in combination with a model function description library, and recommends possible applicable AI models. Its technical principle is to construct a model function vector space, map the user's requirements to this space for nearest neighbor search, thereby achieving model matching.
[0005] However, this technology still has the following main problems: First, relying only on a single natural language input for matching fails to fully utilize multi-dimensional information such as file features, resulting in incomplete matching results; Second, there is a lack of a standardized model ability description system, unable to precisely quantify model capabilities, resulting in low matching accuracy; Third, adopting a static matching strategy, it cannot be dynamically optimized and adjusted according to actual usage effects, and the system cannot self-evolve. Summary of the Invention
[0006] In view of this, this application provides a method and system for intelligent matching of SMS AI training models, which solves the problems in the prior art of relying only on a single natural language input, lacking a standardized model ability description system, and adopting a static matching strategy.
[0007] An embodiment of this application provides a method for intelligent matching of SMS AI training models, including:
[0008] Obtain the natural language description, file name, file content, and file format of the data to be processed, and use the natural language description to perform semantic chunking to form structured instructions, and perform feature extraction on the file name, the file content, and the file format to form multi-dimensional feature data;
[0009] According to a preset model feature portrait library, a feature quadruple dynamic weight matching algorithm is used to process the structured instruction and the multi-dimensional feature data, and a candidate AI training model is output;
[0010] According to the candidate AI training model, an intelligent auxiliary decision-making interface is constructed, and the processing logic flow chart, feature analysis report and historical usage records of the candidate AI training model are analyzed and displayed to generate user feedback results;
[0011] Using the user feedback results, update the weight parameters in the feature quadruple dynamic weight matching algorithm, and re-evaluate and classify the AI training models in the preset model feature portrait library according to the updated weight parameters to generate an optimized model feature portrait library.
[0012] Optionally, the performing semantic chunking on the natural language description to form a structured instruction includes: dividing the natural language description into multiple semantic chunks by calling a pre-trained scenario-based intent recognition model; identifying and extracting the core action instruction, target object and data carrier of each semantic chunk to obtain preliminary semantic information containing the core action instruction, target object and data carrier; according to the preliminary semantic information, mapping the core action instruction, the target object and the implementation data carrier to a preset intent knowledge graph to form the structured instruction.
[0013] Optionally, implementing feature extraction on the file name, the file content and the file format to form multi-dimensional feature data includes: calling a dynamic rule engine to parse the file name, extract time, content, status and format information, and obtain file name features; performing first-line detection on the file content to identify column header features, and using random sampling to judge the data pattern, and performing format recognition on specific fields to generate file content features; according to a preset format decoding matrix, parsing the file format, determining the processing method for encrypted files, compressed packages or unstructured data, and forming file format features; using a feature fusion algorithm to process the file name features, file content features and file format features to form the multi-dimensional feature data.
[0014] Optionally, by means of standardized description, a preset model feature portrait library is constructed, including: obtaining the functional data of an initial candidate AI training model, and using a scenario tree structure to describe the applicable scenarios to determine the specific application field and processing target of the initial candidate AI training model; extracting technical parameters including required fields, data formats, and data volume limits according to the technical specifications of the initial candidate AI training model to form input requirement features; processing the functional characteristics of the initial candidate AI training model using a standardized capability description language, refining specific functions into measurable processing units, and generating processing capability features; integrating the specific application field, the processing target, the input requirement features, and the processing capability features to generate a three-dimensional feature profile of the candidate AI training model as the preset model feature portrait library.
[0015] Optionally, the execution process of the feature quadruple dynamic weight matching algorithm includes: calculating the semantic similarity between the structured instruction and the function description of the candidate AI training model using a first deep learning model to generate a natural language weight score; processing the key information of the file name feature through regular expressions and a rule engine, and matching it with the target data type of the candidate AI training model to form a file name feature weight score; calculating the matching degree between the file content feature and the input requirements of the candidate AI training model, and outputting a content feature weight score; analyzing the data format conversion cost corresponding to the file format feature to generate a format adaptation weight score; using a preset weight coefficient to perform weighted calculation on the natural language weight score, the file name feature weight score, the content feature weight score, and the format adaptation weight score to obtain a comprehensive matching degree, and using the comprehensive matching degree as the final scoring result of the candidate AI training model.
[0016] Optionally, before executing the feature quadruple dynamic weight matching algorithm, it further includes: collecting historical matching records, identifying typical subtask combinations that frequently appear through data mining, constructing a high-frequency subtask optimization pool, and outputting a task combination pattern; analyzing the execution process of each high-frequency subtask using the task combination pattern, extracting data dependency relationships, execution order requirements, and resource consumption limits, and generating a constraint equation set; processing the search space of the candidate AI training model according to the constraint equation set and performing pruning operations to form an optimized candidate AI model set.
[0017] Optionally, the execution process of each high-frequency subtask is analyzed using the task combination mode to extract data dependency relationships, execution order requirements, and resource consumption limits, and a constraint equation set is generated, including: calling a first deep learning model to process the model combination mode in historical successful cases, and outputting a model link prediction strategy through pattern recognition and feature learning; analyzing the high-frequency subtasks according to the model link prediction strategy, extracting data flow transfer rules and calculation resource limit constraint elements, and forming formalized constraint conditions; converting the constraint conditions into a calculation model, and establishing a quantifiable constraint equation set for guiding subsequent model search and optimization processes.
[0018] Optionally, the intelligent matching method further includes: adopting a feature compression mechanism in the feature quadruple dynamic weight matching algorithm, including: obtaining original feature space data, constructing a feature pyramid structure through multi-level feature extraction, and outputting feature combinations at different granularity levels; performing a top-down iterative matching operation according to the feature combinations, and screening candidate AI training models using the compressed feature data to generate a preliminary matching result; constructing a feature weight adjustment algorithm according to the preliminary matching result, dynamically optimizing the feature matching process, and outputting an optimized weight matching algorithm.
[0019] Optionally, the top-down iterative matching operation is adopted to screen candidate AI training models using the compressed feature data, including: performing preliminary screening through the compressed features of the highest layer of the feature pyramid structure to determine the category range of candidate AI training models and outputting model category data; using the model category data to perform feature matching refinement layer by layer in each level of the feature pyramid, and determining candidate AI training models through layer-by-layer screening to generate a screening result; evaluating and verifying the screening result, and storing the feature weight combination and matching strategy with the screening effect reaching a preset threshold in the algorithm library to form a reusable feature matching scheme.
[0020] Correspondingly, the present invention further provides a smart matching system for a short message AI training model, including: a feature acquisition module, configured to acquire the natural language description, file name, file content and file format of the data to be processed, perform semantic chunking on the natural language description to form a structured instruction, and extract features from the file name, the file content and the file format to form multi-dimensional feature data; a model matching module, configured to process the structured instruction and the multi-dimensional feature data according to a preset model feature portrait library by using a feature quadruple dynamic weight matching algorithm, and output a candidate AI training model; a decision interface module, configured to construct a smart assisted decision interface according to the candidate AI training model, generate a user feedback result by analyzing and displaying the processing logic flow chart, feature analysis report and historical usage record of the candidate AI training model; an optimization and update module, configured to update the weight parameters in the feature quadruple dynamic weight matching algorithm by using the user feedback result, re-evaluate and classify the AI training models in the preset model feature portrait library according to the updated weight parameters, and output an optimized model feature portrait library.
[0021] An embodiment of the present application further provides a computer system, where the computer system includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned smart matching method for a short message AI training model.
[0022] An embodiment of the present application further provides a computer-readable storage medium, which stores computer instructions for causing a computer to execute the above-mentioned smart matching method for a short message AI training model.
[0023] An embodiment of the present application further provides a computer program product, including computer instructions, where the computer instructions implement the steps of the above-mentioned smart matching method for a short message AI training model when executed by a processor.
[0024] The present application has the following technical effects: By constructing a multi-dimensional input parsing system, data feature information is comprehensively captured, improving the input quality of model matching; by using a feature quadruple dynamic weight matching algorithm, the accuracy self-optimization of model matching is achieved, reducing manual intervention; a standardized three-dimensional file system of model feature portraits is designed, effectively quantifying the model ability boundary and improving the matching accuracy; a feedback learning closed-loop mechanism is established, and the matching effect is continuously optimized through human-machine collaboration, realizing the self-evolution ability of the system. Description of the Drawings
[0025] To more clearly illustrate the technical solutions of the embodiments of the present disclosure, the accompanying drawings required for the embodiments will be briefly introduced below. The accompanying drawings herein are incorporated into the specification and constitute a part of this specification. These accompanying drawings illustrate the embodiments consistent with the present disclosure and, together with the specification, are used to explain the technical solutions of the present disclosure. It should be understood that the following accompanying drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related accompanying drawings can also be obtained based on these accompanying drawings.
[0026] Figure 1 It is a schematic flowchart of the intelligent matching method for the SMS AI training model provided by the embodiments of the present application;
[0027] Figure 2 It is a schematic flowchart of the multi-dimensional feature analysis process provided by the embodiments of the present application;
[0028] Figure 3 It is a schematic flowchart of the calculation process of the feature quadruple dynamic weight matching algorithm provided by the embodiments of the present application;
[0029] Figure 4 It is a schematic diagram of the intelligent auxiliary decision-making interface provided by the embodiments of the present application;
[0030] Figure 5 It is a block diagram of the structure of the SMS AI training model intelligent matching system provided by the embodiments of the present application. Specific Embodiments
[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only some of the embodiments of the present disclosure, rather than all the embodiments. Usually, the components of the embodiments of the present disclosure described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the present disclosure claimed, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present disclosure.
[0032] It should be noted that similar reference numerals and letters denote similar items in the following accompanying drawings. Therefore, once an item is defined in one accompanying drawing, it does not need to be further defined and explained in subsequent accompanying drawings.
[0033] In this article, the term "and / or" merely describes an association relationship, indicating three possible relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. Additionally, the term "at least one" in this article means any one of multiple items or any combination of at least two of multiple items. For example, including at least one of A, B, and C can represent any one or more elements selected from the set composed of A, B, and C.
[0034] As Figure 1 shown, an embodiment of this application provides a method for intelligent matching of a short message AI training model, including:
[0035] S1: Obtain the natural language description, file name, file content, and file format of the data to be processed, and use the natural language description to perform semantic chunking to form structured instructions, and form multi-dimensional feature data by extracting features from the file name, the file content, and the file format.
[0036] This step constructs an all-round input feature parsing framework. In actual implementation, an embodiment of this application first receives the natural language description and the data file to be processed provided by the user. For the processing of the natural language description, an embodiment of this application calls a pre-trained scenario-based intent recognition model to divide the complex user description into semantic chunks containing core semantics.
[0037] For example, when the user inputs "I need to process a batch of short message numbers and check their validity", an embodiment of this application will decompose it into semantic units such as "I need", "process a batch of short message numbers", and "check their validity".
[0038] Subsequently, an embodiment of this application identifies and extracts elements such as core action instructions (such as "process", "check"), target objects (such as "short message numbers"), and data carriers (implied as files) from these semantic chunks to form preliminary semantic information. An embodiment of this application maps these semantic elements to a pre-constructed intent knowledge graph to convert natural language into computer-processable structured instructions, such as "Action: ValidateObject: SMS_NumberProperty: Validity".
[0039] At the same time, an embodiment of this application performs multi-dimensional feature extraction on the uploaded file. An embodiment of this application calls a dynamic rule engine to parse the file name, and extracts time information (such as "2024Q3" representing the third quarter of 2024), content information (such as "number list" representing the data type), status information (such as "FINAL version" representing the final version), and format information from it.
[0040] This information provides important clues for understanding the business background of the data. In the embodiments of this application, a progressive intelligent sniffing technology is adopted for the file content. The column header features are identified through the first-line detection to understand the structural organization of the data; then the distribution characteristics of the data are analyzed through random sampling; finally, special format recognition is performed for specific fields (such as short signal codes, content, etc.).
[0041] For example, the embodiments of this application can automatically identify the format type, distribution characteristics, and possible abnormal patterns of short signal codes. In addition, the embodiments of this application also parse the file format according to a preset format decoding matrix to determine whether special processing such as decryption, decompression, or OCR is required.
[0042] The embodiments of this application integrate these multi-dimensional features into a unified feature representation through a feature fusion algorithm, providing rich input information for subsequent model matching. This multi-dimensional feature acquisition method solves the limitation of relying only on a single dimension for matching in the prior art, enabling the embodiments of this application to more comprehensively understand user needs and data characteristics.
[0043] Such as Figure 2 shown, semantic chunking is performed using the natural language description to form structured instructions, including:
[0044] A1: By calling a pre-trained scenario-based intent recognition model, the natural language description is divided into multiple semantic chunks.
[0045] First, the natural language description input by the user is received, such as "I need to analyze this batch of short signal code data and check its validity, and at the same time extract the invalid numbers to generate a report". For such complex descriptions, the embodiments of this application call a pre-trained scenario-based intent recognition model for processing.
[0046] This model is based on pre-trained language models such as improved BERT or RoBERTa, and through the attention mechanism and semantic understanding ability, it identifies the semantic boundaries and turning points in the sentence.
[0047] The embodiments of this application will split the original description into multiple independent semantic chunks according to semantic integrity. For example, the above description is split into three semantic chunks: "analyze short signal code data", "check the validity of the numbers", and "extract invalid numbers to generate a report". This semantic chunking processing enables the embodiments of this application to decompose complex and multi-task descriptions into basic units that can be independently understood and processed, laying a foundation for subsequent accurate intent recognition.
[0048] In practical applications, the embodiments of the present application also consider domain-specific language habits and expression patterns, such as professional terms and common expression patterns in the field of short message data processing, and improve the chunking accuracy through domain adaptation fine-tuning. In addition, the embodiments of the present application can also process descriptions of implicit relationships. For example, the user may not explicitly indicate the data source, but the embodiments of the present application can understand its implication in the uploaded file through the context. This advanced semantic chunking ability greatly improves the ability of the embodiments of the present application to understand complex natural language instructions and lays a foundation for subsequent accurate matching.
[0049] A2: Identify and extract the core action instructions, target objects, and data carriers of each semantic chunk to obtain preliminary semantic information including the core action instructions, target objects, and data carriers.
[0050] The embodiments of the present application adopt structured semantic analysis technology to deeply analyze each semantic chunk.
[0051] Specifically, the embodiments of the present application identify the core elements in each semantic chunk through semantic role labeling technology: action instructions (such as "analyze", "check", "extract", "generate") represent the operations that the user hopes to perform; target objects (such as "short message number data", "validity", "invalid number", "report") represent the recipients of the operations; data carriers (such as the implicit "uploaded file" or the explicitly mentioned "Excel table") represent the storage forms of the data. To improve the extraction accuracy, the embodiments of the present application construct a domain-specific action dictionary and object ontology library, which contain common operation types and data objects in the field of short message data processing.
[0052] For example, for the semantic chunk "check the validity of the number", the embodiments of the present application identify "check" as the action instruction (classified as a "verification" type operation), "number validity" as the target object (including the subject "number" and the attribute "validity"), and the data carrier is inferred from the context as the uploaded file. The embodiments of the present application can also process composite actions and nested objects. For example, "extract invalid numbers to generate a report" contains two consecutive actions of "extract" and "generate", and two related objects of "invalid number" and "report". Through this structured semantic parsing, the embodiments of the present application convert the natural language description into preliminary semantic information with a clear subject-predicate-object structure, preparing for subsequent standardization processing.
[0053] A3: According to the preliminary semantic information, map the core action instructions, the target objects, and the implementation data carriers to a preset intention knowledge graph to form the structured instruction.
[0054] The embodiments of this application achieve the conversion from preliminary semantic information to standardized structured instructions. The embodiments of this application construct a complete intention knowledge graph, which contains the standardized representations of common operations in the domain, the hierarchical classification of object types, and the legal relationships between them.
[0055] For example, in the knowledge graph, it is defined that operations of the "Verify" class can be applied to attributes such as "validity", "format", and "status" of objects of the "Number" class. When the embodiments of this application obtain the preliminary semantic information, they will map the action instructions, target objects, and data carriers therein to the standard nodes of the knowledge graph.
[0056] This process involves semantic equivalence judgment, such as mapping different expressions such as "Check", "Verify", and "Confirm" to the same standard operation "Validate".
[0057] At the same time, the embodiments of this application will also handle the relationships of synonyms, hypernyms, and hyponyms, such as mapping "mobile phone number", "telephone number", etc. to the standard object type "TelephoneNumber".
[0058] Through this mapping, the embodiments of this application convert various forms of natural language descriptions into a unified structured instruction format:
[0059] Such as "Action: ValidateObject: TelephoneNumberProperty: Validity".
[0060] This standardized structured instruction eliminates the ambiguity and diversity of natural language expressions, enabling the embodiments of this application to accurately understand the user's intention and precisely match it with the function descriptions in the model library.
[0061] In addition, when the mapping process encounters new concepts not defined in the knowledge graph, the embodiments of this application will also activate the concept extension mechanism, find the closest known concepts through semantic similarity calculation, or create new concept nodes to keep the knowledge graph updated and extended. This semantic standardization mechanism based on the knowledge graph is the key link for the embodiments of this application to accurately understand the user's intention and provides a reliable basis for subsequent model matching.
[0062] Among them, the processing of natural language descriptions is based on scenario-based intent recognition technology. In the embodiments of this application, the description text of the user is first divided into multiple semantic chunks, such as "I need to + process a batch of short signal code data + check its validity". This chunking processing helps to more accurately understand the user's expression intention. Then, core elements are extracted from these semantic chunks, including action instructions ("process", "check"), target objects ("short signal code data"), and data carriers (implied as files). This structured analysis method can standardize the same intention in different expression ways and improve the accuracy of subsequent matching.
[0063] In the embodiments of this application, the extracted semantic elements are mapped to a pre-constructed intent knowledge graph. This knowledge graph contains standardized representations of common operations in the field. Through the mapping operation, natural language is converted into structured instructions that can be processed by a computer. For example, "check the validity of short signal codes" may be mapped to a structured representation such as "Action: ValidateObject: SMS_NumberProperty: Validity".
[0064] In addition, by extracting features from the file name, the file content, and the file format, multi-dimensional feature data is formed, including:
[0065] B1: Call a dynamic rule engine to parse the file name, extract time, content, status, and format information, and obtain file name features.
[0066] For the extraction of file features, the embodiments of this application adopt a multi-level feature parsing strategy:
[0067] For the file name, the embodiments of this application use a dynamic rule engine for parsing. This engine contains parsing rules for different naming patterns and can extract multi-dimensional information such as time features (such as "2024Q3" representing the third quarter of 2024), content features (such as "number details table" representing the data type), and status features (such as "FINAL version" representing the file status) from the file name.
[0068] The embodiments of this application implement an intelligent file name parsing mechanism. File names usually contain rich implicit information, which is of great value for understanding the business background of data. The embodiments of this application construct a dynamic rule engine, which contains multiple sets of parsing rules for different naming specifications and can adaptively identify and extract key information in the file name.
[0069] The embodiments of this application can identify the time information in the file name. For example, "2024Q3" represents the third quarter of 2024, "20240315" represents March 15, 2024, and "Jan-Feb" represents January to February. The embodiments of this application extract the content information in the file name, such as "Short Message Number List", "Customer Information Table", "A Certain Record", etc., and identify the basic types and uses of the data.
[0070] Meanwhile, the embodiments of this application also analyze the status information, such as "FINAL", "V2", "Draft Version", etc., to understand the version and completion degree of the file. The embodiments of this application identify the format suffix and special identifiers to determine the technical format of the file and possible special processing requirements. To improve the parsing accuracy, the dynamic rule engine adopts a hierarchical rule organizational structure, from general rules to specific domain rules, tries to match layer by layer, and can learn and remember new naming patterns.
[0071] For example, when processing a file named "2024Q3_SMS_NumList_Corp_FINAL.xlsx", the embodiments of this application can extract that the time is "the third quarter of 2024", the content is "Enterprise Short Message Number List", the status is "Final Version", and the format is "Excel Spreadsheet". These features extracted from the file name provide important clues for understanding the business background and usage scenarios of the data and play a key role in subsequent model matching.
[0072] B2: Perform first-line detection on the file content to identify the column header features, use random sampling to judge the data pattern, and perform format recognition for specific fields to generate file content features.
[0073] For the file content, the embodiments of this application adopt a progressive intelligent sniffing technology. First, identify the column header features through first-line detection to understand the data structure organization; then analyze the distribution features of the data through random sampling; finally, perform specialized format recognition for specific fields (such as short message numbers, content, etc.). This multi-level content analysis ensures a comprehensive grasp of the file content features.
[0074] The embodiments of this application adopt a multi-level intelligent content analysis strategy. The embodiments of this application perform first-line detection, analyze the header or column name information of the file, and understand the basic structure and organization method of the data.
[0075] The embodiments of the present application can identify keyword field names, such as "mobile phone number", "text message content", "sending status", etc., and initially judge the field and use of the data. To cope with different header naming habits, the embodiments of the present application construct a domain-specific field name synonym library, which can identify the same semantics under different expressions. The embodiments of the present application analyze the data content through random sampling technology to understand the data type, value range and distribution characteristics of each field.
[0076] For example, the embodiments of the present application can identify that a certain column contains mobile phone numbers in the 11-digit number format, another column contains text content within 200 words, and another column contains date and time information. Through this distributed sampling analysis, the embodiments of the present application can quickly understand the overall characteristics of a large data file without fully scanning all the content.
[0077] Subsequently, the embodiments of the present application conduct in-depth format recognition for specific fields, especially for keyword fields. For example, for the short signal number field, the embodiments of the present application will detect whether its format conforms to the domestic mobile phone number specification, whether it contains an international area code, whether it is mixed with a fixed phone number, etc.; for the text message content, the embodiments of the present application will analyze features such as text length distribution, keyword frequency, language type, etc.
[0078] The embodiments of the present application synthesize these analysis results to generate a structured description of the file content features, including data structure information (such as the number and type of fields), content feature information (such as data distribution, abnormal patterns), and domain feature information (such as whether it is marketing text message data, whether it is notification text message, etc.). This comprehensive and in-depth content analysis provides an accurate data feature description for subsequent model matching and is an important basis for achieving accurate matching.
[0079] B3: According to the preset format decoding matrix, parse the file format, determine the processing method of encrypted files, compressed packages or unstructured data, and form file format features.
[0080] For file formats, the embodiments of the present application construct a format decoding matrix, which can identify and process various common formats. For encrypted files, compressed packages or unstructured data (such as PDF, pictures), the embodiments of the present application will start the corresponding processing process to ensure that effective information can be extracted.
[0081] The embodiments of the present application implement an intelligent file format processing mechanism. The embodiments of the present application construct a decoding matrix containing various common data formats, which can not only judge the format by the file extension, but also verify the format through file header information and content features, solving the problem that the file extension may be modified.
[0082] Based on the recognized file format, the embodiments of this application adopt a differentiated processing strategy: for standard structured data formats (such as CSV, Excel, SQL database files, etc.), the embodiments of this application directly call the corresponding parser to extract the data structure and content; for encrypted files, the embodiments of this application will detect the encryption type, and if the user provides access credentials, automatically call the decryption module for processing; for compressed packages (such as ZIP, RAR, etc.), the embodiments of this application will analyze the content structure of the compressed package, automatically decompress and recursively process the internal files; for unstructured data such as PDF or pictures, the embodiments of this application will start the OCR text extraction process to convert the tables and text in the image into processable structured data.
[0083] In addition, the embodiments of this application can also process compound format files, such as Excel files containing multiple worksheets or PDF documents containing multiple media types. Through this intelligent format processing, the embodiments of this application form a detailed description of file format features, including file type, structural complexity, parsing difficulty, and possible preprocessing operations. These format features are crucial for subsequent selection of appropriate processing models. Especially when evaluating data conversion costs and processing compatibility, the format features can provide key decision-making basis.
[0084] B4: Use the feature fusion algorithm to process the file name features, file content features, and file format features to form the multi-dimensional feature data.
[0085] The embodiments of this application integrate feature information of different dimensions into a unified multi-dimensional feature data through the feature fusion algorithm. This fusion process is not a simple feature splicing, but takes into account the relevance and importance between different features to form a more expressive feature representation.
[0086] Specifically, when processing a file named "2024Q3_Customer Short Message Number List_V2.xlsx", the embodiments of this application will parse it from multiple dimensions: first, identify that this is an Excel file (format feature), containing customer short message number data (content feature) in the third quarter of 2024 (time feature), and it is the second version (status feature); then, by analyzing the file content, it is found that it contains fields such as mobile phone numbers, operators, status, etc. (content structure feature); finally, combined with the user's natural language description "need to check the validity of this batch of numbers", a complete task understanding is formed. This multi-dimensional feature acquisition lays a solid foundation for subsequent accurate model matching.
[0087] The embodiments of this application implement an advanced feature integration mechanism. Feature fusion is not simply the splicing of features, but rather takes into account the correlation and complementarity between different features, forming a more expressive unified feature representation. The embodiments of this application use feature correlation analysis to identify the mutual support or conflict relationships between features in different dimensions.
[0088] For example, when the file name contains "Short Signal Code List" and the content analysis shows that there is indeed a mobile phone number field, these two features support each other, increasing the confidence of the relevant features; while when the file name implies "Final Version" but the content analysis finds a large number of missing values, this may indicate that the file status is inconsistent with the content, and the embodiments of this application will accordingly adjust the feature weights.
[0089] The embodiments of this application implement a feature supplementation mechanism, using high-confidence features in one dimension to infer missing features in another dimension. For example, when the file content clearly shows that it is SMS data but the file name does not contain relevant information, the embodiments of this application will automatically supplement the content type feature.
[0090] At the same time, the embodiments of this application also perform feature unification processing, converting features in different dimensions into a consistent representation format for subsequent matching calculations. In addition, the embodiments of this application implement multi-scale feature extraction, retaining both fine-grained specific features (such as specific field names and formats) and generating coarse-grained general features (such as the overall type and domain of the data). Through this multi-level feature fusion processing, the embodiments of this application finally generate structured multi-dimensional feature data, comprehensively describing the various aspect features of the data to be processed, providing rich input information for subsequent accurate model matching. This feature fusion method solves the problem of one-sided understanding caused by relying on a single feature dimension in traditional methods, significantly enhancing the depth and comprehensiveness of the embodiments of this application's understanding of data features.
[0091] S2: According to the preset model feature portrait library, use the feature quadruple dynamic weight matching algorithm to process the structured instruction and the multi-dimensional feature data, and output a candidate AI training model.
[0092] This step realizes the core model matching function. In the embodiment of this application, a standardized model feature profile library is constructed. This library uses a three-dimensional modeling method to describe the ability boundaries of each AI training model. The applicable scenario dimension is described using a multi-level classification scenario tree structure, such as "Data cleaning > Phone number verification > Number validity detection", to accurately locate the application field of the model. The input requirement dimension defines detailed data specification descriptions, including technical parameters such as required fields, data formats, and data volume limits, clarifying the prerequisite conditions for the model to process data. The processing ability dimension uses a standardized ability description language to refine the functions of the model into measurable processing units, such as accuracy rate, processing speed, supported exceptions, etc. The features of these three dimensions are integrated into a complete feature profile of the model through a unified representation method, constituting the model feature profile library. This standardized model description system solves the problem that it is difficult to accurately describe the model capabilities in the prior art and provides a basis for accurate matching.
[0093] On this basis, the embodiment of this application realizes the feature quadruple dynamic weight matching algorithm. This algorithm decomposes the matching process into four dimensions: natural language matching, file name feature matching, content feature matching, and format adaptation matching. The embodiment of this application uses a deep learning model to calculate the semantic similarity between the structured instruction and the model function description, generating a natural language weight score. This semantic-based matching method can better understand the true intention of the user than simple keyword matching.
[0094] The embodiment of this application processes the file name features through a rule engine, analyzes their matching degree with the target data type of the candidate model, and forms a file name feature weight score. Then, the embodiment of this application calculates the matching degree between the file content features and the model input requirements, considering factors such as data structure and field type, and outputs a content feature weight score.
[0095] At the same time, the embodiment of this application also analyzes the compatibility between the file format and the model-supported formats, evaluates the possible conversion costs, and generates a format adaptation weight score. The embodiment of this application uses a preset weight coefficient (such as 40% for natural language, 20% for file name, 30% for content, and 10% for format) to perform weighted calculation on these four scores to obtain the comprehensive matching degree as the final scoring result of the candidate model.
[0096] In the embodiment of this application, the first deep learning model adopts a hierarchical hybrid architecture design and is specifically used to calculate the semantic similarity between the structured instruction and the model function description. The basic layer of this model uses a pre-trained BERT-base model (12-layer Transformer encoder, hidden layer dimension 768, 12 attention heads) and is further pre-trained on a corpus in the field of text message processing.
[0097] Domain adaptation pre-training used approximately 2 million text data related to SMS processing, including task descriptions, processing instructions, and result reports. The pre-training adopted a dual-task approach of masked language model and next sentence prediction. The learning rate was set to 2e-5, the batch size was 32, and it was trained for 8 epochs. On this basis, the model added an attention enhancement layer specifically for structured instruction processing. This layer adopted a multi-head cross-attention mechanism (8 attention heads), enabling the model to better capture the correspondence between structured instructions and natural language descriptions. The top layer of the model was a task-specific fine-tuning layer, including two fully connected layers (with dimensions of 768 and 256 respectively), using the GELU activation function, and finally calculating the semantic matching score through cosine similarity.
[0098] The model training adopted a two-stage strategy: First, it performed supervised learning on a labeled semantic similarity dataset (about 100,000 pairs of labeled samples), using the AdamW optimizer, with a learning rate of 3e-5 and a weight decay of 0.01; then it performed online fine-tuning through a feedback learning mechanism, using user feedback as a signal and adopting a contrastive learning method to further optimize the model performance. The evaluation metrics included semantic matching accuracy, ranking quality (NDCG@5), and consistency score. On the internal test set, this model achieved an 89.7% matching accuracy, which was 15.3 percentage points higher than the baseline model. In addition, the model also achieved an incremental learning function, capable of adapting to new matching patterns and domain terms without complete retraining and maintaining long-term effectiveness.
[0099] To improve the matching efficiency, the embodiments of this application also implemented two important optimization mechanisms. On the one hand, the embodiments of this application identified typical sub-task combinations that frequently appeared through data mining, such as a task chain like "mobile phone number verification → address normalization → customer classification", and extracted key constraint conditions for these high-frequency sub-tasks to form a system of constraint equations, which were used to pre-prune the search space and improve the matching efficiency.
[0100] On the other hand, the embodiments of this application introduced a feature compression mechanism, constructed a feature pyramid structure, and implemented a top-down iterative matching strategy. This strategy first used highly compressed features for rough screening to quickly determine possible model categories; then it used more detailed features layer by layer for refined screening, and finally obtained the most suitable model. These two optimization mechanisms greatly improved the matching speed and accuracy of the embodiments of this application, especially in the case of processing complex tasks and a large number of candidate models.
[0101] As Figure 3 shown, through standardized descriptions, the preset model feature portrait library is constructed, including:
[0102] C1: Obtain the functional data of the initial candidate AI training model, describe the applicable scenarios using a scenario tree structure, and determine the specific application fields and processing objectives of the initial candidate AI training model.
[0103] The embodiments of this application implement a standardized model capability description mechanism. The embodiments of this application collect the original functional descriptions of AI training models, including materials such as development documents, technical specifications, and usage instructions. These original descriptions usually exist in the form of unstructured text, with inconsistent content and are difficult to directly use for precise matching.
[0104] To address this issue, the embodiments of this application construct a multi-level scenario tree structure as a framework for standardizing the description of model applicable scenarios. The scenario tree adopts a hierarchical classification organization method, gradually refining from the top-level major categories to the bottom-level specific application scenarios. For example, the top level may be "SMS data processing", the second level is further divided into "data cleaning", "content analysis", "number management", etc., and the third level is further refined into specific scenarios such as "number validity detection", "number location identification", "operator classification", etc.
[0105] Based on this scenario tree structure, the embodiments of this application locate each AI training model, clarify the specific scenario path it applies to, such as "SMS data processing > number management > number validity detection". This standardized scenario description method solves the problem of inconsistent descriptions between different models, enabling the applicable scope of the model to be accurately positioned and compared.
[0106] In addition, the embodiments of this application also identify the processing objectives of the model, clarify the specific problems it aims to solve, such as "detecting invalid numbers", "identifying spam messages", etc. Through this description method of the scenario tree structure, the embodiments of this application can clearly define the application fields and processing objectives of each model, laying a foundation for subsequent precise matching. This method is particularly suitable for processing domain-specific AI models and can accurately capture their professional characteristics and applicable boundaries.
[0107] C2: According to the technical specifications of the initial candidate AI training model, extract technical parameters including required fields, data formats, and data volume limitations to form input requirement features.
[0108] Each AI training model has specific input requirements, and only data that meets these requirements can be effectively processed. The embodiments of this application first analyze the technical documents of the model, extract information about the input data requirements, and then standardize this information into structured input requirement features.
[0109] The embodiments of the present application identify the fields necessary for model processing. For example, the number verification model may require the "mobile phone number" field, and the SMS classification model may require the "SMS content" field. The embodiments of the present application not only record the field names but also include the semantic meanings of the fields to cope with the differences in field naming in different datasets. The embodiments of the present application clarify the data format requirements for each necessary field. For example, the "mobile phone number" field must be 11 digits, and the "SMS content" field should be of text type and non-empty. These format requirements are important bases for evaluating the compatibility between data and the model.
[0110] In addition, the embodiments of the present application also extract the limit parameters of the model for the data volume, such as the minimum processing unit, the maximum processing capacity, the batch processing ability, etc. For example, some models may require at least 1000 records to be provided for effective training, while for other models, the performance may significantly decline when processing more than 100,000 records. At the same time, the embodiments of the present application also identify the requirements of the model for data quality, such as the missing value ratio limit, the outlier handling ability, etc. These parameters together constitute the input requirement characteristics of the model, which describe in detail the data types and conditions that the model can process. Through this standardized description of input requirements, the embodiments of the present application can accurately evaluate the compatibility between the data to be processed and each model, avoid selecting models with mismatched input requirements, and thus improve the accuracy and effectiveness of the matching.
[0111] C3: Process the functional characteristics of the initial candidate AI training model using a standardized ability description language, refine the specific functions into measurable processing units, and generate processing ability characteristics.
[0112] Different from the traditional fuzzy function descriptions, the embodiments of the present application adopt a standardized ability description language to transform the functional characteristics of the model into specific and measurable processing ability indicators.
[0113] The embodiments of the present application define a set of unified ability measurement dimensions, including the function dimension (the specific operation types that the model can perform), the performance dimension (execution efficiency and resource consumption), the accuracy dimension (the reliability of the results), and the adaptability dimension (the ability to handle edge cases), etc.
[0114] In the function dimension, the embodiments of the present application decompose the complex functions of the model into basic processing units, such as "number format verification", "operator identification", "active status detection", etc., and record the support level (fully supported, partially supported or not supported) for each unit. In the performance dimension, the embodiments of the present application quantify indicators such as processing speed (e.g., the number of records processed per second), memory consumption, and parallel processing ability. In the accuracy dimension, the embodiments of the present application record evaluation indicators such as precision, recall rate, and F1 score of the model in various tasks, as well as the confidence intervals of these indicators.
[0115] In the adaptability dimension, the evaluation model of the embodiments of the present application processes the ability of non-standard situations, such as the tolerance of abnormal data, missing values, and noise. These metrics together constitute the processing ability characteristics of the model, comprehensively and specifically describing the performance of the model in all aspects.
[0116] Through this standardized ability description, the embodiments of the present application can quantitatively compare the ability differences between different models, providing an objective basis for model selection.
[0117] For example, when facing a dataset containing a large number of non-standard format numbers, the embodiments of the present application can select a model with stronger abnormal processing ability according to the score of the adaptability dimension, rather than simply based on general performance metrics. This fine-grained ability description is the key link to achieving accurate model matching, greatly improving the decision-making quality of the embodiments of the present application.
[0118] C4: Integrate the specific application field, the processing target, the input requirement characteristics, and the processing ability characteristics to generate a three-dimensional feature profile of the candidate AI training model, which is used as a preset model feature portrait library.
[0119] In the previous steps, the embodiments of the present application described the model characteristics from three dimensions: applicable scenarios, input requirements, and processing ability. This step integrates the characteristics of these three dimensions to form a complete model feature profile.
[0120] The embodiments of the present application construct a unified feature representation structure to ensure that the characteristics of the three dimensions can be stored and accessed in a consistent format. This unified structure adopts a multi-level nested structure, which not only retains the independence of the characteristics of each dimension but also establishes the association relationship between them. The embodiments of the present application implement the association mapping between features, clarifying the logical relationship between the characteristics of the three dimensions.
[0121] For example, "number validity detection" in the applicable scenario is closely related to "number format verification" and "operator identification" in the processing ability. The embodiments of the present application will establish these association relationships for overall consideration in the matching process.
[0122] In addition, the embodiments of the present application will also calculate the integrity and reliability scores of the characteristics of each dimension as a measure of feature quality. For features with incomplete or uncertain information, the embodiments of the present application will mark their confidence levels and adjust their weights accordingly in subsequent matching. Through this integration process, the embodiments of the present application finally generate a three-dimensional feature profile of each AI training model, comprehensively and accurately describing the characteristics of all aspects of the model. These feature profiles together constitute a model feature portrait library, providing a standardized reference basis for subsequent model matching.
[0123] Compared with traditional methods, this three-dimensional feature profile not only provides a more comprehensive model description, but also realizes the associated representation between features, greatly improving the expression ability of model representation and the accuracy during matching. Through this standardized model capability description system, the embodiments of the present application solve the problems of inconsistent and incomplete model descriptions in the prior art, providing a solid foundation for achieving accurate model matching.
[0124] It should be noted that the execution process of the feature quadruple dynamic weight matching algorithm includes:
[0125] D1: Calculate the semantic similarity between the structured instruction and the function description of the candidate AI training model using the first deep learning model, and generate a natural language weight score.
[0126] The embodiments of the present application adopt a specially trained deep learning model as the core engine for semantic understanding. This model is based on advanced language understanding architectures such as BERT, RoBERTa or their domain-adapted versions, and through large-scale pre-training and specific task fine-tuning, it has the ability to understand the professional language in the field of short message processing.
[0127] During the matching process, the embodiments of the present application first obtain the structured instruction generated in step A, such as "Action: ValidateObject: TelephoneNumberProperty: Validity", and convert it into a natural language expression, such as "Verify the validity of the telephone number". At the same time, the embodiments of the present application extract the function descriptions of the candidate models from the model feature portrait library, and these descriptions have been standardized, such as "Detect the validity of the mobile phone number and identify invalid numbers".
[0128] Then, the embodiments of the present application use the deep learning model to calculate the semantic similarity between these two expressions. Different from simple lexical matching or vector space models, this deep learning method can understand semantic equivalence and can identify descriptions with the same semantics even if the expression methods are different.
[0129] For example, although the words used in "Verify the number" and "Detect the validity of the number" are different, their semantics are very close. The embodiments of the present application calculate such semantic similarities for each candidate model and generate a natural language weight score between 0 and 1. The higher the score, the higher the semantic matching degree between the function description of the model and the user requirements. This matching method based on deep semantic understanding overcomes the limitations of traditional keyword matching, can understand semantic similarities and equivalences, and significantly improves the accuracy of semantic matching. Especially for complex and multi-level task descriptions, this semantic matching method can capture deep Figure 1 consistency and find the model that truly meets the user's needs.
[0130] In the embodiments of the present application, the first deep learning model adopts a hierarchical hybrid architecture design and is specifically used to calculate the semantic similarity between structured instructions and model function descriptions. The basic layer of this model uses a pre-trained BERT-base model (12-layer Transformer encoder, hidden layer dimension 768, 12 attention heads) and is further pre-trained on a corpus in the field of short message processing.
[0131] Domain adaptation pre-training uses approximately 2 million text data related to short message processing, including task descriptions, processing instructions, and result reports. The pre-training adopts a dual-task method of masked language model and next sentence prediction, with a learning rate set to 2e-5, a batch size of 32, and 8 epochs of training. On this basis, the model adds an attention enhancement layer specifically for structured instruction processing, which adopts a multi-head cross-attention mechanism (8 attention heads), enabling the model to better capture the correspondence between structured instructions and natural language descriptions. The top layer of the model is a task-specific fine-tuning layer, which includes two fully connected layers (dimensions are 768 and 256 respectively), uses the GELU activation function, and finally calculates the semantic matching score through cosine similarity.
[0132] The model training adopts a two-stage strategy: first, supervised learning is carried out on a labeled semantic similarity dataset (about 100,000 pairs of labeled samples), using the AdamW optimizer, with a learning rate of 3e-5 and a weight decay of 0.01; then, online fine-tuning is carried out through a feedback learning mechanism, using user feedback as a signal, and a contrastive learning method is used to further optimize the model performance. The evaluation metrics include semantic matching accuracy, ranking quality (NDCG@5), and consistency score. On the internal test set, the model achieves an 89.7% matching accuracy, which is 15.3 percentage points higher than the baseline model.
[0133] D2: Process the key information of the file name features through regular expressions and a rule engine, and match it with the target data type of the candidate AI training model to form a file name feature weight score.
[0134] The embodiments of the present application implement an intelligent matching mechanism based on file names. File names usually contain rich business information and can reflect the type, use, and processing requirements of data.
[0135] The embodiments of the present application first start from the file name features extracted in step B1, which already contain information in multiple dimensions such as time, content, status, and format. Subsequently, the embodiments of the present application process these features through regular expressions and a rule engine, extract the key information in the file name and standardize it.
[0136] For example, for a file name like "2024Q3_SMS_NumList_Corp_FINAL.xlsx", the embodiments of the present application will recognize its relevance to key concepts such as "short message numbers" and "enterprise users".
[0137] Next, the embodiments of the present application will match these key information with the target data types of each candidate model in the model feature portrait library. The target data type is an important part of the model feature portrait, which describes the data categories that the model is designed to process. The embodiments of the present application will evaluate the matching degree between the file name features and the model target data types and calculate the matching score. This process not only considers direct matches (such as the file name clearly contains the same keywords as the model target), but also indirect matches (such as the data type implied by the file name is compatible with the model target).
[0138] For example, a file named "Number List" has a high matching degree with the "Number Validity Detection Model", while the matching degree with the "SMS Content Analysis Model" is relatively low. Finally, the embodiments of the present application generate a file name feature weight score between 0 and 1 based on the matching evaluation, which reflects the degree of fit between the processing requirements implied by the file name and the model functions. This matching mechanism based on file names makes full use of the business semantics hidden in file naming and provides additional decision-making basis for model selection. Especially when the user's natural language description is not detailed enough, the file name features can provide important supplementary information.
[0139] D3: Calculate the matching degree between the file content features and the input requirements of the candidate AI training model, and output the content feature weight score.
[0140] The embodiments of the present application obtain the file content features generated in step B2, which describe multiple aspects such as the structural organization, field characteristics, and content distribution of the data. Then, the embodiments of the present application extract the input requirements of each candidate model from the model feature portrait library, which clearly define the data conditions that the model can effectively process, including required fields, data formats, and quality requirements, etc. Subsequently, the embodiments of the present application perform multi-level matching degree calculations to comprehensively evaluate the compatibility between the data content and the model input requirements.
[0141] At the field matching level, the embodiments of the present application check whether the data contains all the necessary fields required by the model. For example, the number verification model requires the "mobile phone number" field, and the SMS classification model requires the "SMS content" field.
[0142] The embodiments of the present application can handle variants and synonymous expressions of field names. For example, "mobile phone number", "phone number", "telephone", etc. may refer to the same semantic field.
[0143] At the format matching level, embodiments of the present application evaluate whether the format of the data field conforms to the processing requirements of the model, such as whether the number is in the standard 11-digit number format, whether the text is in the expected language type, etc. At the quality matching level, embodiments of the present application analyze the integrity, consistency, and noise level of the data, and evaluate whether it meets the quality requirements of the model, such as whether the proportion of missing values is within an acceptable range, whether there are too many outliers, etc. At the scale matching level, embodiments of the present application check whether the data volume meets the processing capacity of the model, which should neither be less than the minimum processing unit of the model nor exceed the maximum processing capacity of the model.
[0144] Through this comprehensive matching analysis, embodiments of the present application finally calculate the content feature weight score between 0 and 1, reflecting the overall compatibility degree between the file content and the model input requirements. This matching mechanism based on content features ensures that the selected model can effectively process actual data, avoiding processing failures caused by data incompatibility, and is an important link to achieve accurate model matching.
[0145] D4: Analyze the data format conversion cost corresponding to the file format features, and generate a format adaptation weight score.
[0146] Embodiments of the present application obtain the file format features identified in step B3, and clarify the storage format and structural characteristics of the data. At the same time, extract the list of input formats supported by each candidate model from the model feature portrait library. Embodiments of the present application then perform format compatibility analysis to evaluate the matching situation between the data format and the model input format.
[0147] When the data format is directly supported by the model, such as the data is in CSV format and the model accepts CSV input, embodiments of the present application will give the highest format adaptation score because no format conversion is required.
[0148] However, in practical applications, the data format and the model's expected format often do not match. At this time, embodiments of the present application will further analyze the complexity and cost of format conversion.
[0149] Embodiments of the present application construct a detailed format conversion mapping table, which records the feasibility, complexity, and potential risks of conversion between various formats. For example, the conversion from Excel to CSV is relatively simple and almost lossless, while the conversion from a PDF table to structured data is much more complex and may introduce errors.
[0150] Based on this conversion analysis, embodiments of the present application calculate the format conversion cost, considering factors including: the technical complexity of the conversion (whether special tools or processing steps are required), the risk of data loss (the amount of information that may be lost during the conversion process), the processing time cost (the computing resources and time required for the conversion), and the reliability of the conversion result (the expected quality of the data after conversion).
[0151] Finally, the embodiments of the present application generate a format adaptation weight score between 0 and 1. The higher the score, the easier the format adaptation and the lower the cost. This format adaptability analysis ensures that the selected model can efficiently process data in a given format, avoiding complex or unreliable format conversion processes, and optimizing the efficiency and reliability of the overall processing flow. In practical applications, this consideration is particularly important because inappropriate format conversion not only increases processing time but may also introduce data errors, affecting the accuracy of the final result.
[0152] D5: Using a preset weight coefficient, perform weighted calculation on the natural language weight score, file name feature weight score, content feature weight score, and format adaptation weight score to obtain a comprehensive matching degree, and use the comprehensive matching degree as the final scoring result of the candidate AI training model.
[0153] In the previous steps, the embodiments of the present application evaluated the matching degree of the candidate model from four dimensions: natural language description, file name, content feature, and format adaptation. This step integrates the scores of these four dimensions to generate a final comprehensive matching degree.
[0154] The embodiments of the present application perform weighted calculation on the scores of the four dimensions using preset weight coefficients. These weight coefficients reflect the relative importance of each dimension in model matching. The initial settings may be 40% for natural language description, 30% for file content features, 20% for file name features, and 10% for format adaptation. This weight allocation takes into account the information value and reliability provided by each dimension, focusing on the user's clearly expressed needs (natural language description) and the actual characteristics of the data (content features).
[0155] Then, the embodiments of the present application perform a weighted summation calculation, adding the weighted scores of the four dimensions to obtain a comprehensive matching degree between 0 and 1. For example, if a model's scores in the four dimensions are 0.9 (natural language), 0.7 (file name), 0.8 (content), and 0.6 (format), then its comprehensive matching degree is 0.9×40% + 0.7×20% + 0.8×30% + 0.6×10% = 0.8. This comprehensive matching degree is used as the final scoring result of the candidate model, reflecting the overall fit degree of the model with the current task requirements.
[0156] It should be noted that although the embodiments of the present application use preset weights for initial calculation, these weights are not fixed but will be dynamically adjusted according to the user feedback in step S4. For example, if it is found that the user's decision is more inclined to consider content features, the embodiments of the present application will gradually increase the weight of content features. This dynamic weight adjustment mechanism enables the embodiments of the present application to adapt to different user preferences and decision-making patterns, and continuously optimize the accuracy of the matching algorithm. Through this multi-dimensional comprehensive scoring mechanism, the embodiments of the present application can comprehensively consider various factors, find the model that is truly most suitable for the current task, and achieve accurate intelligent matching.
[0157] Optionally, the core calculation formula of the feature quadruple dynamic weight matching algorithm in the embodiments of the present application is as follows:
[0158] MatchScore = α·NL + β·FN + γ·FC + δ·FF,
[0159] Where:
[0160] NL represents the natural language weight score, which is obtained by calculating the semantic similarity through a deep learning model:
[0161] NL = cosine_similarity(E(Instruction), E(ModelDesc)),
[0162] Where E() represents the semantic encoding function, which is implemented by the first deep learning model.
[0163] FN represents the file name feature weight score, which is calculated through a rule engine:
[0164] FN = Σ(wi·match(ri, filename)) / Σwi,
[0165] Where ri represents the rule pattern related to the model, and wi represents the corresponding weight.
[0166] FC represents the content feature weight score, which is calculated through the field matching degree and data compatibility:
[0167] FC = λ1·FieldMatch + λ2·DataCompat,
[0168] Where FieldMatch calculates the coverage rate of necessary fields, and DataCompat evaluates the compatibility of data formats.
[0169] FF represents the format adaptation weight score, which is calculated based on the format conversion cost:
[0170] FF = 1 - normalize(ConversionCost),
[0171] where ConversionCost represents the computational cost of converting from the current format to the format required by the model.
[0172] The weight coefficients α, β, γ, δ (corresponding to initial values of 40%, 20%, 30%, and 10% respectively) are dynamically adjusted based on user feedback:
[0173] α' = α + η·Δα,
[0174] β' = β + η·Δβ,
[0175] γ' = γ + η·Δγ,
[0176] δ' = δ + η·Δδ,
[0177] where η is the learning rate (usually set to 0.05), and the Δ values are calculated based on user feedback: when the user accepts the recommendation model, the current weight configuration is enhanced; when the user selects a different model, the weights are adjusted according to the differences. To ensure that the sum of the weights is 1, normalization is performed after each update:
[0178] [α',β',γ',δ'] = [α',β',γ',δ'] / Σ[α',β',γ',δ'],
[0179] In addition, before executing the feature quadruple dynamic weight matching algorithm, the following is also included:
[0180] E1: Collect historical matching records, identify typical subtask combinations that frequently occur through data mining, construct a high-frequency subtask optimization pool, and output task combination patterns.
[0181] Before executing the matching algorithm, the embodiment of the present application also implements a high-frequency subtask optimization mechanism. Through data mining of historical matching records, the embodiment of the present application identifies typical subtask combinations that often occur, such as a task chain like "mobile phone number verification → address normalization → customer classification".
[0182] The embodiment of the present application implements an intelligent optimization mechanism based on historical experience. The embodiment of the present application continuously records the model matching and usage situations of users, including the complete information chain such as input data features, selected models, processing results, and user feedback. These historical records constitute a valuable experience database, reflecting the model application patterns in actual business scenarios.
[0183] Subsequently, the embodiments of the present application use advanced data mining techniques to analyze these historical records and identify frequently occurring model usage patterns. The embodiments of the present application adopt an improved sequence pattern mining algorithm, which not only focuses on the usage frequency of individual models but also pays more attention to the patterns of combined use of multiple models.
[0184] For example, the embodiments of the present application may find that a processing chain such as "mobile phone number verification → address normalization → customer classification" is often used together to form a typical task combination.
[0185] For the identified high-frequency patterns, the embodiments of the present application further analyze their business backgrounds, triggering conditions, and application effects, and extract the key features and constraint conditions in the patterns. Based on these analyses, the embodiments of the present application construct an optimization pool for high-frequency subtasks, structurally store common task combination patterns, and mark their applicable conditions and expected effects. These task combination patterns become important references for subsequent model matching. Especially when dealing with complex tasks, the embodiments of the present application can directly recommend verified model combination solutions instead of constructing a processing chain from scratch.
[0186] Through this optimization mechanism based on historical experience, the embodiments of the present application can make full use of the accumulated knowledge to improve the matching efficiency and accuracy. Especially for common business scenarios, the embodiments of the present application can quickly identify the best processing solutions, significantly shortening the decision-making time. This method reflects the learning ability of the embodiments of the present application. Through continuous experience accumulation, the matching performance of the embodiments of the present application will continue to improve with the increase in usage time.
[0187] E2: Analyze the execution process of each high-frequency subtask using the task combination pattern, extract data dependency relationships, execution order requirements, and resource consumption limitations, and generate a system of constraint equations, including:
[0188] For these high-frequency subtasks, the embodiments of the present application further analyze their execution processes, extract key constraint conditions, and formalize these constraint conditions into a system of constraint equations. This process includes using a deep learning model to analyze historical successful cases, extracting data flow rules and resource limitations, and converting the constraint conditions into a computational model.
[0189] E2.1: Invoke the first deep learning model to process the model combination pattern in the historical successful cases, and output a model link prediction strategy through pattern recognition and feature learning.
[0190] The embodiments of the present application implement an advanced pattern learning and prediction mechanism.
[0191] In the embodiments of this application, successful cases with good processing effects are screened from historical records. These cases contain complete data features, model selections, and processing result information. Then, the embodiments of this application call a specially designed deep learning model to analyze these successful cases. This model adopts a composite architecture, combining sequence modeling capabilities (such as LSTM or Transformer) and graph structure learning capabilities (such as graph neural networks), and is able to capture both the temporal patterns and interdependencies used by the model simultaneously.
[0192] During the learning process, the embodiments of this application first represent each successful case as a structured feature sequence, including data features, task requirements, and context information, etc. Then, the deep learning model learns the mapping relationship between the input features and the optimal model combination from these cases through self-supervised learning.
[0193] Through the training of a large number of cases, the model gradually masters the implicit rules of model selection in different scenarios and is able to predict the most likely successful model combination plan based on new input features.
[0194] After learning, the embodiments of this application output a model link prediction strategy, which is a decision-making embodiment that can quickly generate model combination suggestions according to input features. This strategy not only considers the applicability of individual models, but also considers the cooperation relationship and sequence dependence between models, and is able to recommend an overall optimal processing flow.
[0195] For example, when facing short signal code data containing multiple abnormal situations, this strategy may recommend first using a "data cleaning model" to process abnormal formats, then using a "number validity verification model" to detect validity, and finally using a "number classification model" for classification and statistics. This prediction strategy based on deep learning greatly improves the ability of the embodiments of this application to handle complex tasks, can automatically construct an optimal model processing chain based on historical experience, and reduces the complexity of manually designing the processing flow.
[0196] The neural routing solver in the embodiments of this application is constructed based on the graph neural network architecture and includes three key components: a feature encoder, a routing predictor, and a search strategy generator.
[0197] The feature encoder adopts a multi-layer perceptron structure, including 3 hidden layers (with dimensions of 256, 512, and 256 respectively), uses the ReLU activation function, and maps the input features into high-dimensional representations. The specific implementation is as follows:
[0198] h1 = ReLU(W1x + b1),
[0199] h2 = ReLU(W2h1 + b2),
[0200] h3 = ReLU(W3h2 + b3),
[0201] where x is the input feature, and W and b are the weight matrix and bias vector respectively.
[0202] The routing predictor is based on the Transformer architecture, includes 4 attention heads and 2 encoder layers, and can capture the complex association patterns between features and the model. The calculation formula of the attention mechanism is:
[0203] Attention(Q, K, V) = softmax(QK T / dk)V
[0204] where Q, K, and V are the query, key-value, and value matrices respectively, and dk is the dimension of the key-value vector.
[0205] The search strategy generator uses a policy gradient network, and the output layer uses the Softmax function to generate the probability distribution of different search paths:
[0206] π(a|s) = softmax(Wπhπ + bπ),
[0207] where s is the current state, a is the possible action, and hπ is the hidden representation of the policy network.
[0208] E2.2: Analyze the high-frequency subtasks according to the model link prediction strategy described above, extract the data flow rules and calculation resource limit constraint elements, and form formal constraint conditions.
[0209] The embodiment of this application deeply analyzes the identified high-frequency subtasks based on the model link prediction strategy output in the previous step. The focus of the analysis is to understand the dependencies and execution conditions between tasks, and extract the constraint elements that can be formally expressed.
[0210] In terms of data flow, the embodiment of this application analyzes the data transfer rules between models, including the necessary input-output matching relationship, data format conversion requirements, and dependencies of intermediate results. For example, the embodiment of this application may identify that the "SMS content analysis model" needs to receive the valid records filtered by the "short message number verification model" as input, which constitutes a data flow constraint.
[0211] In terms of computing resources, the embodiment of this application evaluates the resource requirements and performance characteristics of each model, including processing time, memory consumption, and parallel capabilities, etc. These resource constraints are crucial for evaluating the feasibility of the entire processing chain, especially in large-scale data processing scenarios.
[0212] In addition, the embodiment of this application also analyzes the business logic constraints, such as the rules that certain models must be executed under specific conditions or must be executed in a specific order.
[0213] For example, "customer information matching" must be executed after "number validity verification" to avoid unnecessary matching operations on invalid numbers.
[0214] Through this comprehensive constraint analysis, the embodiments of the present application transform various implicit dependencies and restrictions into explicit formal constraint expressions. These constraint expressions adopt standardized formats. For example, "ModelAoutput->ModelBinput" represents a data flow constraint, and "ModelC.time<5min" represents a time constraint, etc. These formal constraint conditions provide clear evaluation criteria for subsequent model search and optimization, ensuring that the model combinations recommended by the embodiments of the present application not only match in function but are also feasible under actual operating conditions.
[0215] E2.3: Transform the constraint conditions into a computational model, establish a quantifiable system of constraint equations, and use it to guide the subsequent model search and optimization process.
[0216] The embodiments of the present application implement an advanced constraint modeling and quantification calculation mechanism.
[0217] The embodiments of the present application transform the formal constraint conditions extracted in the previous step into a strict mathematical representation and construct a computable system of constraint equations. This transformation process uses a hybrid constraint solving technique to uniformly represent different types of constraints as a computational model.
[0218] For data flow constraints, the embodiments of the present application establish a model dependency representation based on a directed graph, regard each model as a node in the graph, the data flow relationship as a directed edge, and assign a compatibility weight to each edge to represent the matching degree of data transfer.
[0219] For resource constraints, the embodiments of the present application establish a representation based on linear inequalities, represent resource requirements such as processing time and memory consumption as linear combinations of model selection variables, and set corresponding upper threshold values. For business logic constraints, the embodiments of the present application use predicate logic expressions for formalization, transforming relationships such as conditional execution and sequential dependencies into logical constraints. After normalization, these constraint equations form a unified system of constraint equations, describing all the conditions that an effective model combination must satisfy.
[0220] The embodiments of this application also design a solution priority for the constraint equations, distinguishing necessary constraints (such as data compatibility) from secondary constraints (such as performance optimization), to ensure a reasonable trade-off in case of constraint conflicts. The finally constructed constraint equations not only describe the boundaries of feasible solutions but also contain an optimization objective function, such as minimizing the total processing time or maximizing the result accuracy. This quantifiable constraint model provides a clear evaluation criterion for subsequent model search, enabling quick judgment of the feasibility and pros and cons of candidate solutions, and greatly improving the search efficiency. By transforming business requirements and technical limitations into precise mathematical models, the embodiments of this application achieve the precise expression and efficient processing of complex constraint conditions, laying a foundation for optimizing the model combination.
[0221] E3: Process the search space of candidate AI training models according to the constraint equations, and perform pruning operations to form an optimized set of candidate AI models.
[0222] Based on these constraint equations, the embodiments of this application perform pruning operations on the search space of candidate models, excluding model combinations that obviously do not meet the conditions, thereby improving the search efficiency. This optimization mechanism is particularly applicable to complex task scenarios that require multiple models to cooperate in processing.
[0223] To further optimize the search efficiency in the model matching process, the embodiments of this application implement a high-frequency subtask constraint optimization system. First, by performing data mining on historical matching records, the embodiments of this application identify typical subtask combinations that often occur during processing, such as a task chain like'mobile phone number verification → address normalization → customer classification'. These high-frequency subtask combinations are stored in an optimization pool, and a standardized feature description is established for each combination.
[0224] Secondly, the embodiments of this application implement a function for automatically extracting constraint conditions. For each high-frequency subtask, the embodiments of this application analyze the key constraint conditions during its execution process, including data dependency relationships, execution order requirements, resource consumption limitations, etc. These constraint conditions are formalized into computable constraint equations. For example, for the'mobile phone number verification' task, the embodiments of this application will extract specific constraints such as 'the input must be 11 digits' and 'the processing delay does not exceed 100 ms'. These constraint equations are used to narrow the subsequent search space.
[0225] The embodiments of this application implement an efficient search space optimization mechanism.
[0226] In practical applications, the number of possible model combinations grows exponentially with the increase in the scale of the model library, and it is unrealistic to evaluate all combinations in detail. To solve this problem, the embodiments of this application use the previously constructed constraint equations to perform intelligent pruning on the search space.
[0227] In the embodiments of this application, a preliminary screening is performed on all candidate models to exclude models that obviously do not meet the basic requirements, such as models with completely incompatible input formats or lacking necessary functions. This preliminary screening process significantly reduces the basic search space. Subsequently, the embodiments of this application apply a constraint propagation algorithm to perform deep pruning based on the hard constraints in the constraint equations (such as data flow relationships and necessary conditions).
[0228] For example, if the constraint requires that the output of model A must be able to serve as the input of model B, the embodiments of this application will exclude all variants of model A whose outputs are incompatible with model B, greatly reducing the number of combinations that need to be considered.
[0229] For complex model chains, the embodiments of this application adopt a dynamic programming method to gradually construct the solution space starting from simple sub-problems, avoiding repeated exploration of infeasible sub-structures.
[0230] On this basis, the embodiments of this application also apply a heuristic search strategy to preferentially explore model combination patterns with high success rates in history, further improving the search efficiency.
[0231] Through this series of optimization operations, the embodiments of this application finally form a significantly reduced set of candidate AI models. These model combinations all meet the basic requirements of the constraint equations and have high feasibility.
[0232] For example, for an embodiment of this application that contains hundreds of basic models, the original possible combinations may be in the tens of thousands. After constraint pruning, only dozens of high-quality candidate combinations may need to be considered. This optimization of the search space not only significantly improves the speed of model matching, but also enhances the matching quality by excluding unreasonable combinations, enabling the embodiments of this application to quickly recommend the optimal model processing solution in complex scenarios.
[0233] In the feature quadruple dynamic weight matching algorithm, a feature compression mechanism is adopted, including:
[0234] F1: Obtain the original feature space data, construct a feature pyramid structure through multi-level feature extraction, and output feature combinations at different granularity levels.
[0235] The feature pyramid structure in the embodiments of this application adopts a multi-level design, usually including 3 to 5 levels. At the bottom layer (layer 1), the complete original features are retained, and the dimension is usually between 200 and 500, including all fine-grained information such as specific field names, data formats, distribution features, etc. At the middle layer (layers 2-3), the dimension is reduced to 50-100 through principal component analysis (PCA) and feature aggregation, retaining the main feature patterns. At the top layer (layers 4-5), it is further compressed to highly abstract features of 10-20 dimensions, only including core information such as data types and main tasks.
[0236] A non - linear mapping relationship is adopted between layers, and an auto - encoder network is used to achieve feature compression and reconstruction. Specifically, the encoder network Ei is used for compression from the i - th layer to the (i + 1) - th layer, and the decoder network Di is used for reconstruction from the (i + 1) - th layer to the i - th layer. Both the encoder and the decoder adopt a fully - connected neural network structure, which includes a BatchNormalization layer and a LeakyReLU activation function in the middle to ensure the non - linear expression ability of feature transformation. These networks are trained by minimizing the reconstruction error ||x - Di(Ei(x))||² to ensure that the most informative features are retained during the compression process.
[0237] The embodiment of this application realizes an innovative multi - scale feature representation mechanism.
[0238] During the model matching process, although the full - scale original features contain complete information, they have high computational complexity and are easily affected by noise.
[0239] To solve this problem, the embodiment of this application adopts a feature pyramid structure to achieve feature representations at different abstraction levels.
[0240] The embodiment of this application obtains the complete feature space data extracted in step S1, including all the original information such as natural language features, file name features, content features, and format features. Then, the embodiment of this application constructs a feature pyramid from bottom - layer details to top - layer generalizations through a multi - level feature extraction algorithm.
[0241] At the bottom layer of the pyramid, the embodiment of this application retains the complete original features, such as detailed field names, specific data distributions, and complete file format information. The feature representation at this layer is the most detailed but also the most complex.
[0242] At the middle layer, the embodiment of this application generates more abstract feature representations through feature aggregation and dimensionality reduction operations. For example, detailed field information is aggregated into a general feature such as "contains phone number information", and the specific data distribution is simplified to an evaluation result such as "good data integrity". These middle - layer features retain the main characteristics of the original data while significantly reducing the feature dimension.
[0243] At the top layer of the pyramid, the embodiment of this application generates highly abstract feature generalizations, such as core feature descriptions like "short signal code data" and "needs validity verification". Although the information volume of these top - layer features is small, they capture the essential features of the task and are suitable for fast matching.
[0244] Through this pyramid structure, the embodiments of the present application represent the characteristics of the same data at different granularity levels, which is conducive to quick screening and retains the possibility of detailed verification. There is a clear hierarchical association between the features of each layer in the feature pyramid. The upper-layer features are effective generalizations of the lower-layer features, and the lower-layer features are detailed expansions of the upper-layer features. This multi-level feature representation greatly improves the flexibility and efficiency of feature processing and provides a solid foundation for subsequent iterative matching.
[0245] F2: According to the feature combination, perform a top-down iterative matching operation, and use the compressed feature data to screen candidate AI training models to generate a preliminary matching result, including:
[0246] F2.1: Perform preliminary screening through the compressed features of the highest layer of the feature pyramid structure to determine the category range of candidate AI training models and output model category data.
[0247] Before full-scale matching, the embodiments of the present application first use the highly compressed features at the top layer of the feature pyramid to perform preliminary screening to quickly determine potentially relevant model categories.
[0248] This process is similar to the thinking mode of human experts. First, determine the general direction according to the general characteristics of the task, and then perform detailed analysis.
[0249] Specifically, the embodiments of the present application obtain the compressed features at the top layer of the feature pyramid. These features highly summarize the core properties of the data and tasks, such as basic labels like "text message data", "number verification", "content classification", etc. Then, the embodiments of the present application quickly match these compressed features with the top-layer features of each model in the model feature portrait library. Since the top-layer features have low dimensions and are highly abstract, this matching process has a small computational amount and can be completed quickly. Based on the matching results, the embodiments of the present application calculate the preliminary relevance of each model category to the current task.
[0250] The "model category" here refers to a set of models with similar functions, such as "number verification models" and "text message classification models", etc.
[0251] The embodiments of the present application screen out several model categories with higher scores as the candidate set according to the relevance score. For example, for the task of "checking the validity of short message numbers", the embodiments of the present application may preferentially select "number verification" and "data verification" models and exclude "content analysis" and "customer segmentation" models. This preliminary screening greatly reduces the scope of models that need to be examined in detail and improves the efficiency of subsequent matching.
[0252] Meanwhile, the embodiments of this application record the preliminary matching evidence and confidence scores for each candidate category, which serve as a reference for subsequent refined matching. Through this top-down screening strategy, the embodiments of this application can quickly focus on the most likely relevant subset of models, avoiding ineffective calculations for models that are obviously irrelevant, and significantly improving the matching efficiency, especially when the model library is large.
[0253] F2.2: Using the model category data, perform feature matching refinement layer by layer in each level of the feature pyramid, and determine the candidate AI training models through layer-by-layer screening to generate the screening results.
[0254] After determining the initial range of model categories, the embodiments of this application start to perform step-by-step refined matching analysis on each level of the feature pyramid. This hierarchical matching strategy is similar to the "gradual approximation" method in scientific research. By continuously increasing the accuracy and depth of analysis, the best solution is finally determined.
[0255] The embodiments of this application perform the second round of matching in the middle layer of the feature pyramid. The features in the middle layer are more specific than those in the top layer, but still maintain a moderate degree of abstraction, such as more detailed task descriptions like "data set containing standard format phone numbers" and "requiring double verification of validity and place of origin".
[0256] The embodiments of this application match the middle layer features with each specific model in the candidate model categories screened in the previous step to further narrow down the candidate range. For example, in the "number verification class" models, some models may focus on format verification, while some may include the function of identifying the place of origin. The embodiments of this application will make distinctions based on the matching results of the middle layer features. Subsequently, the embodiments of this application enter the bottom layer of the feature pyramid and use the most detailed original features for the final matching. The bottom layer features contain all the detailed information, such as specific field names, data format requirements, and exception handling capabilities.
[0257] The embodiments of this application conduct a comprehensive and detailed evaluation of the remaining high-potential candidate models, considering all relevant factors, and generate the final matching scores. Through this top-down and layer-by-layer refined matching process, the embodiments of this application effectively combine the advantages of rapid screening and precise matching. In each layer of matching, the embodiments of this application will exclude models that obviously do not meet the requirements, resulting in a decreasing number of models that need to be evaluated in detail layer by layer, greatly improving the matching efficiency.
[0258] Finally, the embodiments of this application generate a screening result containing a small number of high-quality candidate models. These models have passed multiple levels of screening and have a high matching credibility. Compared with the traditional one-time full-scale matching, this hierarchical matching strategy not only improves the efficiency but also ensures the matching quality, and is especially suitable for scenarios dealing with complex tasks and large-scale model libraries.
[0259] F2.3: Evaluate and verify the screening results, store the feature weight combinations and matching strategies whose screening effects reach the preset threshold in the algorithm library, and form a reusable feature matching scheme.
[0260] The embodiments of the present application implement an advanced algorithm learning and knowledge accumulation mechanism. After completing multi-level screening, the embodiments of the present application not only focus on the final matching results, but also pay more attention to the effectiveness evaluation and experience extraction of the entire screening process. The embodiments of the present application conduct a comprehensive quality evaluation of the screening results and adopt a multi-dimensional evaluation index system. These indexes include accuracy index (the coincidence degree between the screening result and the optimal selection), efficiency index (the computational cost and time consumption of the screening process), stability index (the sensitivity of the screening result to input changes), and interpretability index (whether the basis for screening decisions is clear), etc. The embodiments of the present application will set reasonable evaluation criteria and weights for each evaluation index in combination with historical verification data and the expert knowledge base.
[0261] During the evaluation process, the embodiments of the present application will focus on analyzing the contribution degree of the feature weight combination. The feature weight combination refers to the weight configuration of different feature dimensions in the matching algorithm. For example, a higher weight may be given to the field format feature while a lower weight is given to the file name feature.
[0262] The embodiments of the present application analyze the influence of different weight combinations on the matching results through the control variable method, and identify the weight configuration modes that are particularly effective for the current task type. At the same time, the embodiments of the present application will also evaluate the effectiveness of the entire matching strategy, including the construction method of the feature pyramid, the decision point setting of hierarchical screening, and the pruning strategy, etc.
[0263] For those feature weight combinations and matching strategies whose evaluation effects are significantly better than the benchmark level and reach the preset quality threshold, the embodiments of the present application will standardize them and store them in the algorithm library. This storage process not only includes the recording of weight parameters and strategy rules, but also includes the clear definition of usage conditions and applicable scenarios.
[0264] For example, the embodiments of the present application may identify that a specific weight combination is particularly effective when dealing with the task of "cleaning SMS data containing a large number of invalid numbers", and will save this combination together with its applicable conditions. These conditions are usually defined based on data feature patterns (such as data types, structural characteristics, abnormal patterns, etc.) and task semantic characteristics (such as operation types, processing objectives, etc.).
[0265] Optionally, during the execution of the top-down iterative matching process, the embodiments of the present application adopt an advanced neural routing solver to optimize the model search path. The neural routing solver is constructed based on an improved graph neural network architecture and includes three key components: a feature encoder, a routing predictor, and a search strategy generator.
[0266] The feature encoder adopts a multi-layer perceptron structure, including 3 hidden layers (with dimensions of 256, 512, and 256 respectively), uses the ReLU activation function, and maps the input features into high-dimensional representations.
[0267] The routing predictor is based on the Transformer architecture, includes 4 attention heads and 2 encoder layers, and can capture complex association patterns between features and the model.
[0268] The search strategy generator uses a policy gradient network, and the output layer uses the Softmax function to generate the probability distribution of different search paths. The training of the neural routing solver adopts a combination of supervised learning and reinforcement learning. Historical successful matching cases are used as supervision signals, and at the same time, the search efficiency is optimized through a reward function, which is designed as a weighted combination of matching accuracy and search steps. In this way, the neural routing solver can intelligently guide the search process, significantly reduce the search space, and improve the matching efficiency.
[0269] In addition, the embodiments of this application will also establish an evaluation record for the stored matching solutions, including historical usage, success rate statistics, and applicable case analysis, etc. These records are continuously updated as the solutions are continuously used, forming a dynamic performance evaluation mechanism. As time goes by and data accumulates, the embodiments of this application can more and more accurately evaluate the actual effects of each matching solution and adjust its priority in future tasks accordingly.
[0270] By establishing such an algorithm library that continuously learns and optimizes, the embodiments of this application achieve the accumulation and reuse of matching experience. When facing a new matching task, the embodiments of this application first check whether there is a suitable stored solution. If a solution with a high matching degree is found, it can be directly applied, greatly improving the decision-making efficiency. Even if no completely matching solution is found, the embodiments of this application can also refer to the successful experiences in similar scenarios and make reasonable strategy adjustments. This learning mechanism based on experience accumulation enables the embodiments of this application to continuously improve its own performance. Especially for common task types, the matching accuracy and efficiency will increase significantly with the increase in the number of uses.
[0271] Finally, this step establishes a self-evolving feature matching knowledge base, making the embodiments of this application not just a static matching tool, but an intelligent embodiment that can continuously learn and optimize. This mechanism is particularly suitable for enterprise-level application scenarios, can adapt to the business models and data characteristics of specific organizations, and provide more and more accurate model matching services. By continuously accumulating and optimizing matching strategies, the embodiments of this application can maintain a high matching accuracy while continuously improving the processing efficiency, providing a smoother intelligent matching experience for users.
[0272] F3: Based on the preliminary matching results, construct a feature weight adjustment algorithm to dynamically optimize the feature matching process and output an optimized weight matching algorithm.
[0273] After completing the preliminary multi-level matching, the embodiments of the present application analyze the key decision points and influencing factors in the matching process and identify the links where there may be room for optimization. The embodiments of the present application evaluate the actual contribution of different features in the current task, identify which features play a decisive role in correct matching, and which features are relatively less important.
[0274] For example, the embodiments of the present application may find that in the task of number verification, the data field format feature is more critical than the file name feature. Based on these analyses, the embodiments of the present application dynamically construct a feature weight adjustment algorithm to optimize the feature weight allocation for the current task type. This weight adjustment is context-aware and will change dynamically according to the task characteristics and data features, rather than a simple fixed rule.
[0275] In addition, the embodiments of the present application also analyze the key decision paths in the matching process and identify which matching steps have the greatest impact on the final result.
[0276] For example, for some tasks, the preliminary screening at the top layer of the feature pyramid may already be highly accurate, while for other tasks, it may be necessary to rely on the detailed features at the bottom layer to make a correct judgment. Based on these patterns, the embodiments of the present application optimize the selection strategy of the feature hierarchy and customize the most efficient matching path for different types of tasks.
[0277] At the same time, the embodiments of the present application will also learn effective feature combination patterns from the preliminary matching results and identify which feature combinations are particularly effective for specific types of tasks. For example, for the short signal number verification task, a feature combination such as "field format + data distribution + model accuracy" may be particularly discriminative.
[0278] The embodiments of the present application encode these effective combinations as optimized matching rules to further improve the matching efficiency. Through this series of adaptive optimizations, the embodiments of the present application output weight matching algorithms highly customized for the current task, which can identify the most suitable model more accurately and efficiently. This dynamic optimization mechanism enables the embodiments of the present application to continuously learn and improve, and always maintain the best matching performance for different types of tasks.
[0279] In one of the embodiments, an adaptive heuristic algorithm generator is designed to dynamically generate optimized strategies for new matching problems. The generator consists of three parts: a pattern recognition engine, a strategy synthesizer, and a performance evaluator.
[0280] The pattern recognition engine adopts a densely connected convolutional neural network architecture, including 5 convolutional layers (the convolutional kernel sizes are 3×3, 5×5, 3×3, 3×3, 1×1 respectively) and 3 pooling layers, and can identify typical problem patterns and challenge points from the matching history.
[0281] The strategy synthesizer is implemented based on program synthesis technology. It uses a predefined library of basic algorithm components (including 15 basic sorting algorithms, 8 filtering methods, and 12 feature transformation functions) as building blocks, and automatically combines and generates new heuristic algorithms through genetic programming methods.
[0282] The performance evaluator adopts an A / B test framework, compares and evaluates the newly generated algorithms with the benchmark algorithms on historical data, and calculates the accuracy improvement, efficiency improvement, and stability metrics.
[0283] When the performance of the newly generated heuristic algorithm exceeds the preset threshold (usually requiring at least a 5% increase in accuracy or a 30% increase in efficiency), the algorithm will be added to the algorithm library and assigned applicable scenario labels. These dynamically generated heuristic algorithms can provide optimized solutions for specific types of matching problems, such as scenarios like processing high-noise SMS data, parsing complex nested file structures, or processing multi-language mixed content, greatly enhancing the adaptability and performance of the embodiments of this application.
[0284] S3: According to the candidate AI training model, construct an intelligent assisted decision-making interface, analyze and display the processing logic flow chart, feature analysis report, and historical usage records of the candidate AI training model, and generate user feedback results.
[0285] This step implements a human-machine collaborative verification mechanism, transforming the technical capabilities of the matching engine into an available interaction experience.
[0286] The embodiments of this application design a clear and intuitive three-pane comparison view interface: the left pane displays the processing logic flow chart of the recommended model, presenting the working principle and processing steps of the model in a visual manner; the middle pane shows the feature analysis report of the data to be processed, including key information such as data structure, distribution characteristics, and outliers; the right pane provides the historical usage records of the model, such as usage frequency, success rate, typical application scenarios, etc. This multi-dimensional information display method takes into account the cognitive needs of user decision-making, enabling users without technical backgrounds to understand the reasons and expected effects of model selection.
[0287] For example, when the embodiments of this application recommend the "Short Signal Number Validity Verification Model", users can understand the processing flow of the model that first checks the number format and then verifies the operator validity through the flow chart; view the format distribution and possible outliers of the numbers in the current data through the feature analysis report; and understand the accuracy and processing speed of the model in similar tasks through the historical records.
[0288] Based on this information, the user can make a decision to confirm or adjust the recommendation of the embodiments of this application and provide corresponding feedback. This auxiliary decision-making interface greatly reduces the professional threshold for model selection, enabling business personnel to independently complete model selection without relying on technical experts. At the same time, the embodiments of this application record the user's decision-making process and feedback opinions, providing a data basis for subsequent optimization.
[0289] In this step, the embodiments of this application design an intelligent auxiliary decision-making interface, adopting a layout of a three-pane comparison view. As Figure 4 shown, the left pane displays the processing logic flow chart of the recommended model, intuitively showing the working principle of the model; the middle pane shows the feature analysis report of the data to be processed, helping the user understand the data characteristics; the right pane provides the historical usage records of this model, including information such as usage frequency and success rate.
[0290] This visual display design fully considers the cognitive needs of user decision-making. By providing multi-dimensional decision-making support information, it reduces the user's cognitive burden and improves decision-making efficiency. The user can confirm or adjust the model recommended by the embodiments of this application based on this information and provide feedback.
[0291] For example, when the embodiments of this application recommend the "Short Signal Number Validity Verification Model", the user can understand the processing flow of this model through the interface (such as first checking the number format and then verifying the operator validity), view the data feature analysis (such as the number format distribution and possible abnormal points), and the historical application situation (such as the accuracy rate in similar tasks). These information helps the user make a more informed decision.
[0292] When implementing the intelligent auxiliary decision-making interface, the embodiments of this application adopt a visualization architecture based on Web components. The front-end visualization engine is built based on the D3.js and ECharts libraries, adopting a component-based design pattern, and decomposing the interface into reusable view components.
[0293] The core view components include a model flow chart renderer, a data feature visualizer, and a historical record analyzer. The model flow chart renderer uses a directed acyclic graph layout algorithm (specifically adopting a hierarchical layout strategy with automatic adjustment of node spacing), and realizes a highly interactive process display through SVG vector graphics technology, supporting operations such as zooming, node expansion, and path highlighting.
[0294] The data feature visualizer integrates multiple chart types (including heat maps, radar charts, and parallel coordinate charts), and can automatically select the most suitable visualization method according to the data characteristics. For example, bar charts are used for categorical features and box plots are used for numerical distributions.
[0295] The historical record analyzer uses time series visualization technology to display the changing trends of model application frequency, success rate, and user satisfaction.
[0296] The interface communicates with the backend via RESTful API, transmits data in JSON format, and the front end implements a data caching mechanism to improve the response speed.
[0297] In terms of user experience design, the interface follows the principle of progressive information disclosure. Initially, only key decision-making information is displayed, and users can expand detailed content as needed.
[0298] The interaction design adopts a consistent feedback mechanism. All user operations have clear visual and functional feedback, and the perceptibility of state changes is enhanced through animation transitions. In terms of accessibility design, the interface supports keyboard navigation and screen readers, meeting the WCAG 2.1 AA standard. This highly interactive and information-rich decision-making interface significantly reduces the cognitive burden of users, making complex model matching decisions intuitive and controllable.
[0299] S4: Utilize the user feedback results to update the weight parameters in the feature quadruple dynamic weight matching algorithm, and re-evaluate and classify the AI training models in the preset model feature portrait library according to the updated weight parameters to generate an optimized model feature portrait library.
[0300] This step establishes a complete feedback learning closed-loop mechanism, realizing the self-evolution ability of the embodiment of this application. When the user confirms or modifies the recommendation result of the embodiment of this application, the embodiment of this application will record the differences between the manual decision and the recommendation of the embodiment of this application, and analyze the characteristic patterns of these differences. For example, if the user repeatedly selects a model different from the recommendation of the embodiment of this application when processing short signal code data, the embodiment of this application will analyze the common characteristics of these situations, such as whether they are related to specific data formats or content features.
[0301] Based on these analyses, the embodiment of this application automatically adjusts the weight parameters in the feature quadruple dynamic weight matching algorithm. If it is found that the user's decision-making is more dependent on content features, the embodiment of this application may increase the proportion of the weight score of content features; if it is found that natural language descriptions are not sufficient to accurately express the user's intention, the embodiment of this application may reduce the proportion of the natural language weight score. This dynamic adjustment mechanism enables the embodiment of this application to gradually adapt to the preferences and decision-making patterns of specific users or organizations, improving the personalization degree and accuracy of matching.
[0302] Meanwhile, the embodiments of the present application will also update and optimize the model feature portrait library based on user feedback. If a certain model is frequently used for a specific type of task, the embodiments of the present application will strengthen the association degree of the model in the corresponding scenario; if there is a deviation between the actual performance of a certain model and its feature description, the embodiments of the present application will adjust its feature description to make it more accurate.
[0303] For example, if a model that claims to be able to handle various format numbers performs poorly on certain formats during actual use, the embodiments of the present application will modify its processing ability description accordingly. This dynamic update of model features enables the feature portrait library to be continuously improved and more accurately reflect the true capabilities of the model.
[0304] Through this closed-loop optimization mechanism, the embodiments of the present application achieve the self-evolution ability of "getting more accurate with use", overcoming the limitation that the static matching strategy in the prior art cannot adapt. This mechanism is particularly suitable for enterprise-level application scenarios with long-term operation, and can continuously improve the performance of the embodiments of the present application as the usage increases, forming a positive feedback loop. Actual tests show that after about 100 user feedbacks, the matching accuracy rate of the embodiments of the present application can be increased by 15-20 percentage points, greatly reducing the need for manual intervention.
[0305] This step is one of the key innovation points of the entire embodiments of the present application, realizing the self-evolution ability of the embodiments of the present application. When the user confirms or modifies the recommended results of the embodiments of the present application, the embodiments of the present application will record the differences between the manual decision and the suggestions of the embodiments of the present application, and accordingly adjust the weight parameters in the feature quadruple dynamic weight matching algorithm.
[0306] For example, if the user repeatedly selects a model different from the recommended model of the embodiments of the present application in a specific scenario, the embodiments of the present application will analyze the characteristic pattern of this difference and adjust the weight coefficient accordingly. For example, it may increase the weight of the file content feature or reduce the weight of the file name feature. This dynamic adjustment mechanism enables the embodiments of the present application to continuously learn the user's preferences and decision-making patterns, and gradually improve the matching accuracy rate.
[0307] Meanwhile, the embodiments of the present application will also update and optimize the model feature portrait library based on user feedback. If a certain model is frequently used for a specific type of task, the embodiments of the present application will strengthen the association degree of the model in the corresponding scenario; if there is a deviation between the actual performance of a certain model and its feature description, the embodiments of the present application will adjust its feature description to make it more accurate.
[0308] This closed-loop optimization mechanism solves the problem that the static matching strategy cannot self-evolve, enabling the embodiments of the present application to continuously improve performance with use and form a virtuous cycle.
[0309] For example, a telecommunications enterprise needs to process a batch of SMS marketing data, including customer mobile phone numbers, sending content, and status records. The user uploaded an Excel file named "2023Q4_SMS_Marketing_Activity_Result_Statistics_V3.xlsx" and provided a natural language description: "It is necessary to analyze this batch of SMS sending records, identify invalid numbers, generate a detailed report, and at the same time, count the conversion rates of various types of SMS."
[0310] In the feature acquisition stage, the embodiments of this application first analyze the natural language description. By calling a pre-trained scenario-based intent recognition model, the description is divided into four semantic blocks: "Analyze SMS sending records", "Identify invalid numbers", "Generate a detailed report", and "Count the conversion rate". From these semantic blocks, the embodiments of this application extract core action instructions ("Analyze", "Identify", "Generate", "Count"), target objects ("SMS sending records", "Invalid numbers", "Detailed report", "Conversion rate"), etc. to form preliminary semantic information. Then, the embodiments of this application map these semantic elements to a preset intent knowledge graph to form a standardized structured instruction:
[0311] "Action: Analyze + Identify + GenerateObject:
[0312] SMS_Records + Invalid_Numbers + ReportProperty: Conversion_Rate".
[0313] At the same time, the embodiments of this application perform multi-dimensional feature extraction on the uploaded file. By analyzing the file name through a dynamic rule engine, time information (the fourth quarter of 2023), content information (SMS marketing activity result statistics), status information (V3 version), and format information (Excel) are extracted. The file content is probed to identify keyword fields such as "mobile phone number", "SMS content", "sending status", "customer feedback", etc., and the data format and distribution characteristics are determined through random sampling, such as the mobile phone number is in the 11-digit numerical format, the average length of the SMS content is 70 characters, the sending status includes two values of "success" / "failure", etc.
[0314] In the model matching stage, the embodiments of this application call the feature quadruple dynamic weight matching algorithm. First, the system calculates the semantic similarity between the structured instruction and the candidate model function description, and the "SMS analysis and report generation model" obtains a semantic similarity score of 0.88. Then, the file name features are processed through the rule engine to determine the matching degree with the SMS marketing analysis task as 0.85. The file content features are matched with the model input requirements, and a content matching score of 0.92 is calculated. The file format features are analyzed to determine that the format is fully compatible, with a score of 1.0. Through weighted calculation (semantics 40%, file name 20%, content 30%, format 10%), the comprehensive matching degree of this model is obtained as 0.895.
[0315] In the decision interface presentation stage, the system constructs a three-pane comparison view interface. The left pane shows the processing flow chart of the "SMS analysis and report generation model", including four main steps: data preprocessing, number validity detection, classification statistics, and report generation. The middle pane displays the feature analysis report of the uploaded data, highlighting that approximately 8% of the detected records may contain invalid numbers and marking several main types of SMS. The right pane provides the historical usage records of this model, showing an average accuracy rate of 93.5% and an average processing time of 2.5 minutes per 10,000 records for similar tasks.
[0316] The user confirmed and selected this model through the interface, and fine-tuned the automatically detected invalid number types, adding the detection requirement for a specific category of "unregistered numbers". The system recorded this feedback and used it to update the weight matching algorithm. Specifically, the system increased the weight of the short message number field format recognition and added the ability label of "supporting unregistered number recognition" to the "SMS analysis and report generation model" in the model feature portrait library.
[0317] Through this complete process, the system successfully matches the user's data processing requirements with the most suitable AI model, and further optimizes the matching algorithm through user feedback, reflecting the practicality and effectiveness of the technical solution of this application.
[0318] As Figure 5 shown, this application also provides an intelligent matching system for SMS AI training models, including a feature acquisition module, a model matching module, a decision interface module, and an optimization and update module. These modules respectively implement each step in the above method, forming a complete technical implementation architecture.
[0319] In summary, this application comprehensively solves the main problems in the prior art by constructing a multi-dimensional input parsing system, implementing a dynamic weight matching algorithm, designing a standardized model feature portrait system, and establishing a feedback learning closed-loop mechanism, achieving intelligent and accurate matching of SMS AI training models, and greatly improving the data processing efficiency and accuracy.
[0320] For example, when a businessperson without a technical background needs to process a batch of customer short message number data, they only need to upload the data file and simply describe "need to check whether this batch of numbers is valid". The system can automatically analyze the file characteristics, understand the user's intention, and recommend the most suitable "number validity verification model". Through the intuitive decision-making interface, the user can quickly confirm or adjust the system's suggestions without having to understand the details of complex AI models. This greatly reduces the usage threshold and improves work efficiency.
[0321] The embodiments of the present disclosure also provide a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the short message AI training model intelligent matching method described in the above method embodiments. Among them, the storage medium can be a volatile or non-volatile computer-readable storage medium.
[0322] In addition, the embodiments of the present disclosure also provide a computer program product, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the short message AI training model intelligent matching method provided in any of the above embodiments of the present disclosure. For details, please refer to the above method embodiments and will not be elaborated here.
[0323] Among them, the above computer program product can be specifically implemented in the form of hardware, software, or a combination thereof. In an optional embodiment, the computer program product is specifically embodied as a computer storage medium, which can be a volatile or non-volatile computer-readable storage medium. In another optional embodiment, the computer program product is specifically embodied as a software product, such as a Software Development Kit (SDK), etc.
[0324] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described devices and apparatuses can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here. In the several embodiments provided by the present disclosure, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some communication interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.
[0325] The unit described as a separation component may or may not be physically separated. The component presented as a unit may or may not be a physical unit, that is, it may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0326] In addition, in each embodiment of the present disclosure, each functional unit may be integrated into one processing unit, may exist separately as individual physical units, or two or more units may be integrated into one unit.
[0327] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such an understanding, the technical solution of the present disclosure, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.
[0328] Finally, it should be noted that the above-described embodiments are only specific implementation manners of the present disclosure, used to illustrate the technical solutions of the present disclosure, and are not intended to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present disclosure can still modify the technical solutions recorded in the foregoing embodiments, or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes, or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should all be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.
Claims
1. A SMS AI training model intelligent matching method, characterized by: include: Obtaining a natural language description, file name, file content, and file format of the data to be processed, performing semantic segmentation using the natural language description to form structured instructions, and performing feature extraction on the file name, the file content, and the file format to form multi-dimensional feature data; Based on a preset model feature profile library, a feature quadruple dynamic weight matching algorithm is used to process the structured instructions and the multi-dimensional feature data, and output a candidate AI training model; Based on the candidate AI training model, an intelligent decision-making support interface is constructed to analyze and display the processing logic flow chart, feature analysis report, and historical usage records of the candidate AI training model to generate user feedback results; The method further includes: adopting a feature compression mechanism in the feature quadruple dynamic weight matching algorithm, including: Obtain the original feature space data, construct a feature pyramid structure through multi-level feature extraction, and output feature combinations at different granularity levels; Based on the feature combination, a top-down iterative matching operation is performed, and the compressed feature data is used to screen candidate AI training models to generate preliminary matching results; Based on the preliminary matching results, a feature weight adjustment algorithm is constructed to dynamically optimize the feature matching process and output an optimized weight matching algorithm; The top-down iterative matching operation is used to screen candidate AI training models using compressed feature data, including: Performing preliminary screening through the highest level compressed features of the feature pyramid structure to determine the category range of candidate AI training models and outputting model category data; Using the model category data, feature matching is refined at each level of the feature pyramid, and candidate AI training models are determined by screening layer by layer to generate screening results; Evaluate and verify the screening results, and store the feature weight combinations and matching strategies that meet the preset thresholds in the algorithm library to form a reusable feature matching solution; Before executing the feature quadruple dynamic weight matching algorithm, the method further includes: Collect historical matching records, identify subtask combinations through data mining, build a subtask optimization pool, and output task combination patterns; Analyzing the execution process of each subtask using the task combination pattern, extracting data dependencies, execution sequence requirements, and resource consumption constraints, and generating a set of constraint equations; The search space of the candidate AI training models is processed according to the set of constraint equations, and a pruning operation is performed to form an optimized set of candidate AI models.
2. The method according to claim 1, characterized in that The performing semantic segmentation using the natural language description to form structured instructions includes: By calling a pre-trained scenario-based intent recognition model, the natural language description is divided into multiple semantic blocks; Identifying and extracting the core action instructions, target objects, and data carriers of each semantic block to obtain preliminary semantic information including the core action instructions, target objects, and data carriers; Based on the preliminary semantic information, the core action instructions, the target object and the implementation data carrier are mapped to a preset intention knowledge graph to form the structured instructions.
3. The method according to claim 1, characterized in that The feature extraction is performed on the file name, the file content and the file format to form multi-dimensional feature data, including: Calling a dynamic rule engine to parse the file name, extract time, content, status and format information, and obtain file name features; Performing first-row detection on the file content to identify column header features, using random sampling to determine data patterns, performing format recognition on specific fields, and generating file content features; Parsing the file format according to a preset format decoding matrix, determining the processing method of the encrypted file, compressed package or unstructured data, and forming the file format characteristics; The file name features, file content features and file format features are processed using a feature fusion algorithm to form the multi-dimensional feature data.
4. The method according to claim 1, wherein Through standardized description, the preset model feature portrait library is constructed, including: Obtain functional data of the initial candidate AI training model, and use the scenario tree structure to describe the applicable scenario, and determine the specific application field and processing target of the initial candidate AI training model; Extracting technical parameters including required fields, data formats, and data volume limits based on the technical specifications of the initial candidate AI training model to form input requirement features; Using a standardized capability description language to process the functional characteristics of the initial candidate AI training model, breaking down specific functions into measurable processing units, and generating processing capability characteristics; The specific application field and the processing target, the input requirement characteristics and the processing capability characteristics are integrated to generate a three-dimensional feature file of the candidate AI training model as a preset model feature portrait library.
5. The method according to claim 3, characterized in that The execution process of the feature quadruple dynamic weight matching algorithm includes: Calculate the semantic similarity between the structured instruction and the candidate AI training model function description using the first deep learning model to generate a natural language weight score; Processing key information of the file name features through regular expressions and rule engines, matching it with the target data type of the candidate AI training model to form a file name feature weight score; Calculate the matching degree between the file content features and the candidate AI training model input requirements, and output the content feature weight score; Analyzing the data format conversion cost corresponding to the file format features to generate a format adaptation weight score; Using preset weight coefficients, a weighted calculation is performed on the natural language weight score, file name feature weight score, content feature weight score and format adaptation weight score to obtain a comprehensive matching degree, which is used as the final scoring result of the candidate AI training model.
6. An SMS AI training model intelligent matching system, characterized by: include: a feature acquisition module, configured to acquire a natural language description, file name, file content, and file format of the data to be processed, perform semantic segmentation using the natural language description to form structured instructions, and perform feature extraction on the file name, file content, and file format to form multi-dimensional feature data; A model matching module is used to process the structured instructions and the multi-dimensional feature data using a feature quadruple dynamic weight matching algorithm based on a preset model feature profile library, and output a candidate AI training model; A decision interface module is used to build an intelligent auxiliary decision interface based on the candidate AI training model, and generate user feedback results by analyzing and displaying the processing logic flow chart, feature analysis report and historical usage records of the candidate AI training model; The model matching module is further configured to adopt a feature compression mechanism in the feature quadruple dynamic weight matching algorithm, including: Obtain the original feature space data, construct a feature pyramid structure through multi-level feature extraction, and output feature combinations at different granularity levels; Based on the feature combination, a top-down iterative matching operation is performed, and the compressed feature data is used to screen candidate AI training models to generate preliminary matching results; Based on the preliminary matching results, a feature weight adjustment algorithm is constructed to dynamically optimize the feature matching process and output an optimized weight matching algorithm; The top-down iterative matching operation is used to screen candidate AI training models using compressed feature data, including: Performing preliminary screening through the highest level compressed features of the feature pyramid structure to determine the category range of candidate AI training models and outputting model category data; Using the model category data, feature matching is refined at each level of the feature pyramid, and candidate AI training models are determined by screening layer by layer to generate screening results; Evaluate and verify the screening results, and store the feature weight combinations and matching strategies that meet the preset thresholds in the algorithm library to form a reusable feature matching solution; Before executing the feature quadruple dynamic weight matching algorithm, the method further includes: Collect historical matching records, identify subtask combinations through data mining, build a subtask optimization pool, and output task combination patterns; Analyzing the execution process of each subtask using the task combination pattern, extracting data dependencies, execution sequence requirements, and resource consumption constraints, and generating a set of constraint equations; The search space of the candidate AI training models is processed according to the set of constraint equations, and a pruning operation is performed to form an optimized set of candidate AI models.
Citation Information
Patent Citations
Data processing method and device, electronic equipment and storage medium
CN117370373A
Neural network model structure determination method and device, equipment, medium and product
CN117454959A