Material data standardization method based on vertical field large model and related device

Abstract

The invention relates to the technical field of data processing, and discloses a material data standardization method and related device based on a vertical field large model, and the method comprises the steps: obtaining a material description data set; performing normalization processing on each piece of material description data in the material description data set through a global normalization module to obtain a material specification data set; performing category identification processing on each piece of material specification data in the material specification data set through a category identification module to obtain a material category data set; carrying out attribute identification processing on each piece of material category data in the material category data set through an attribute identification module to obtain a material attribute data set; through the automatic learning module, system learning optimization processing is performed according to the material specification data set, the material category data set and the material attribute data set, so that a more accurate material standardization data set can be obtained, and a more efficient and more accurate material data standardization process can be realized.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a material data standardization method and related devices based on a vertical field big model. Background Art

[0002] In today's material management field, accurate processing and standardization of material data play a vital role. Traditional material attribute recognition methods are often highly dependent on manual labor, and the rule configuration process is extremely cumbersome. Due to the wide variety of material categories and complex attributes, manual writing and maintenance of rules and lexicons consumes a lot of manpower and time, and is prone to errors. As the business develops, the types and attributes of materials are constantly updated, and the continuous maintenance costs of rules and lexicons are high. At the same time, its recognition accuracy is low and its generalization ability is poor. It is difficult for manually written rules to fully cover all types of material categories and attributes, especially when facing long-tail materials and newly emerging attributes. The recognition accuracy is greatly reduced, and traditional models rely on shallow features. Limited by limited training data, it is difficult to learn rich patterns and features. Furthermore, existing technologies are difficult to adapt to the diversity of materials. Different suppliers and industries have different ways of describing materials, and there are also problems such as language differences. Adding new attributes also faces the dilemma of high expansion costs and delayed introduction. In addition, in terms of data processing, storage and query efficiency are low, data analysis capabilities are insufficient, and it is impossible to deeply explore the value of data to provide strong support for corporate decision-making. In addition, the business process is complex, the operating cost is high, the overall efficiency is low, the user experience is not good, and recognition errors and slow response speeds are common. Therefore, how to achieve a more efficient and accurate material data standardization process has become a problem that needs to be solved urgently. Summary of the invention

[0003] The present invention provides a material data standardization method and related devices based on a vertical field big model to solve the technical problem of how to achieve a more efficient and accurate material data standardization process, and to obtain a more accurate material standardization data set.

[0004] In the first aspect, a material data standardization method based on a vertical field big model is provided, which is applied to a material data standardization system based on a vertical field big model. The material data standardization system based on a vertical field big model includes a global normalization module, a category recognition module, an attribute recognition module and an automatic learning module; the material data standardization method based on a vertical field big model includes: Get the material description data set; Through the global normalization module, each material description data in the material description data set is normalized to obtain a material standard data set; Through the category identification module, each material specification data in the material specification data set is subjected to category identification processing to obtain a material category data set; Through the attribute recognition module, attribute recognition processing is performed on each material category data in the material category data set to obtain a material attribute data set; Through the automatic learning module, system learning and optimization processing are carried out according to the material specification data set, material category data set and material attribute data set to obtain the material standardized data set.

[0005] In the second aspect, a material data standardization device based on a vertical field big model is provided, including: An acquisition unit, used for acquiring a material description data set; A first processing unit, configured to perform normalization processing on each material description data in the material description data set through the global normalization module to obtain a material standard data set; A second processing unit is used to perform category identification processing on each material specification data in the material specification data set through the category identification module to obtain a material category data set; A third processing unit is used to perform attribute recognition processing on each material category data in the material category data set through the attribute recognition module to obtain a material attribute data set; The fourth processing unit is used to perform system learning optimization processing according to the material specification data set, the material category data set and the material attribute data set through the automatic learning module to obtain a material standardized data set.

[0006] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the steps of the above-mentioned material data standardization method based on the vertical field big model are implemented.

[0007] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned material data standardization method based on the vertical field big model are implemented.

[0008] In the scheme implemented by the above-mentioned material data standardization method and related devices based on the vertical field big model, a material description data set is obtained, and each material description data in the material description data set is normalized through the global normalization module to obtain a material specification data set, and then the category identification module is further used to perform category identification processing on each material specification data in the material specification data set to obtain a material category data set, so that the attribute identification module can be used to perform attribute identification processing on each material category data in the material category data set to obtain a material attribute data set, and then the automatic learning module can be used to perform system learning and optimization processing based on the material specification data set, the material category data set and the material attribute data set, so as to obtain a more accurate material standardization data set, which is conducive to achieving a more efficient and accurate material data standardization process. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0010] Figure 1 It is a schematic diagram of an application environment of a material data standardization method based on a vertical field big model in one embodiment of the present invention; Figure 2 It is a flow chart of a material data standardization method based on a vertical field big model in one embodiment of the present invention; Figure 3 It is a schematic diagram of the architecture of a material data standardization system based on a vertical field big model in one embodiment of the present invention; Figure 4 It is a structural schematic diagram of a material data standardization device based on a vertical field big model in one embodiment of the present invention; Figure 5 is a schematic diagram of a structure of a computer device in one embodiment of the present invention; Figure 6 It is another structural schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0011] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0012] The material data standardization method based on the vertical field big model provided by the embodiment of the present invention can be applied in the following aspects: Figure 1 In an application environment, the client communicates with the server through a network. Exemplarily, taking the scenario of material data standardization as an example, the target user (such as R&D personnel or management personnel) can upload a material description data set through the client. Correspondingly, the server can obtain the material description data set through the client, and perform normalization processing on each material description data in the material description data set through the global normalization module to obtain a material specification data set, and further perform category recognition processing on each material specification data in the material specification data set through the category recognition module to obtain a material category data set, so that each material category data in the material category data set can be attributed through the attribute recognition module to obtain a material attribute data set, and then the automatic learning module can perform system learning optimization processing according to the material specification data set, the material category data set and the material attribute data set, so as to obtain a more accurate material standardization data set, and the material standardization data set can be fed back to the client. Correspondingly, the client can receive the material standardization data set from the server, and can display the material standardization data set on the client for the target user to query or browse. By adopting the material data standardization method based on the vertical field big model provided in this application, a more accurate material standardization data set can be obtained, which is conducive to achieving a more efficient and accurate material data standardization process.

[0013] The client may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices. The server may be implemented by an independent server or a server cluster consisting of multiple servers. The present invention is described in detail below through specific embodiments.

[0014] It should be noted that a vertical field refers to a field that focuses on a specific industry or market segment.

[0015] Optionally, vertical areas include the area of ​​material data governance.

[0016] The vertical domain big model, which has been trained in the general domain, has in-depth understanding and professional knowledge in the specific domain, and can better handle various tasks and problems in the domain. For example, the vertical domain big model of material data governance mainly includes accurate analysis of material data to support decision-making, data standardization and normalization, improving data quality, intelligent classification and identification of materials, optimizing resource allocation, risk warning and promoting automation of related processes, thereby improving the efficiency and effectiveness of material data governance. The big model trained in the general domain is trained on data in a specific domain to provide high-quality services and solutions that better meet the needs of the domain.

[0017] See also Figure 2 As shown, Figure 2 A flow chart of a material data standardization method based on a vertical field big model provided in an embodiment of the present invention. The method is applied to a material data standardization system based on a vertical field big model, and the material data standardization system based on a vertical field big model includes a global normalization module, a category identification module, an attribute identification module and an automatic learning module. Among them, the material data standardization system based on a vertical field big model can be the carrier applied by the entire material data standardization method, and each module thereof assumes different responsibilities, cooperates with each other, and jointly completes the entire processing flow of material data from input to final output of standardized results. The material data standardization method based on a vertical field big model includes the following steps: S10: Obtain a material description data set.

[0018] The material description data set can be the initial input content of the entire material data standardization process, and can be a collection of descriptive data about materials from different channels and in various formats. The material description data set can include but is not limited to basic information, attribute description, usage scenarios, and other aspects of the material.

[0019] It is understandable that the various material description data in the material description data set may usually be uneven in terms of format, degree of standardization, etc., and need to be processed in sequence by subsequent modules to gradually convert them into data forms that meet standardization requirements.

[0020] S20: performing normalization processing on each material description data in the material description data set through the global normalization module to obtain a material normalization data set.

[0021] Among them, the global normalization module can be a module built based on the global standardization model (Holistic StandardizationModel), which can also be called the H module, and can be responsible for normalizing each material description data in the input material description data set. This normalization process can cover multiple aspects, such as unifying data formats, cleaning impurities in data (such as special characters, extra spaces, etc.), and standardizing different measurement units, attribute names, etc., so that the material description data that originally differed in format, content expression, etc. can become uniform, forming a standardized material specification data set, providing a high-quality, unified standard data foundation for subsequent more accurate category identification and other processing links, avoiding interference in subsequent analysis due to the chaos of the data.

[0022] The material specification data set may be a result set obtained after the global standardization module normalizes each material description data in the material description data set. Each material specification data may be data with a unified data format and standardized content description.

[0023] It should be understood that the server can use the global normalization module to perform multi-format data analysis, intelligent data cleaning and data normalization processing on each material description data in the material description data set, thereby obtaining the above-mentioned material specification data set.

[0024] Multi-format data analysis can support multiple data formats, such as text files, Microsoft spreadsheets (Excel tables), Comma-Separated Values ​​(CSV) files, JavaScript Object Notation (JSON), eXtensible Markup Language (XML), database exports, etc., to meet the needs of different data sources; and through adaptive analysis algorithms, such as using machine learning and pattern recognition technology, automatically identify data formats and encoding methods, and accurately extract resource description information; to further process data extraction and mapping, such as mapping original data fields to standardized data fields according to preset mapping rules.

[0025] Intelligent data cleaning may include but is not limited to noise filtering, such as removing special characters, emoticons, extra spaces, HyperText Markup Language (HTML) tags, etc. to improve the purity of data; spelling correction, such as using natural language processing technology to automatically correct common spelling errors and input errors to improve text quality; synonym processing, such as establishing a synonym dictionary and semantic network to unify different expressions into a standard form to solve the problem of polysemy and synonyms; missing value filling, such as using data mining and statistical methods to intelligently fill in missing key attributes to improve data integrity.

[0026] Data normalization may include but is not limited to unified measurement units, such as converting and unifying different measurement units, for example, unifying "kg" and "kilogram" into "kilogram", and performing unit conversion; attribute name normalization, such as establishing an attribute name mapping table to map different attribute names to standard attribute names, for example, "memory" and "memory capacity" are unified into "memory capacity"; format normalization, such as unifying the representation format of dates, quantities, specifications, models, etc. to ensure data consistency and comparability; encoding normalization, such as unified processing of special characters and encodings to ensure data readability and compatibility.

[0027] S30: performing category identification processing on each material specification data in the material specification data set through the category identification module to obtain a material category data set.

[0028] Among them, the category recognition module can be a module built based on the Attribute Recognition LargeModel, which can also be called module A. It can be used to perform category recognition processing on each data in the material specification data set obtained after processing by the global normalization module to determine the specific category to which each material belongs.

[0029] The material category data set can be the final set obtained after the category identification module performs category identification processing on each material specification data in the material specification data set. The material category data set clarifies the exact category to which each material belongs, eliminates the ambiguity and uncertainty in the classification process, and provides an accurate category basis for the subsequent attribute identification module, so that attribute identification can extract the corresponding material attribute information based on accurate category classification, ensuring the orderly advancement of the material data standardization process.

[0030] Optionally, the server can base on a pre-trained large language model, such as a model with strong semantic understanding capabilities such as Bidirectional Encoder Representations from Transformers (BERT) and Generative Pretrained Transformer (GPT); and use low-rank adaptation (LoRA) fine-tuning technology to perform low-rank adaptation fine-tuning on the large model to quickly adapt to the task of material category recognition while maintaining the original knowledge; and further use large-scale training data for model training, such as using a large amount of data with a wide coverage, such as 398,000 material annotation data covering 1,312 categories, to ensure the generalization ability of the model, thereby obtaining a large model for category recognition.

[0031] It should be understood that the process of performing category identification processing on each material specification data in the material specification data set by the category identification module to obtain the material category data set refers to the process of performing category identification processing to obtain the material category data set. In step S30, that is, performing category identification processing on each material specification data in the material specification data set by the category identification module to obtain the material category data set includes the following steps: S31: performing semantic analysis on each material specification data in the material specification data set to obtain a material semantic data set; S32: performing multi-level classification processing on each material semantic data in the material semantic data set to obtain a material multi-level classification data set; S33: Perform high-precision category identification processing on each material multi-level classification data in the material multi-level classification data set to obtain a material category data set.

[0032] Among them, the material semantic data set can be a set obtained by the category recognition module after semantic analysis of each material specification data in the material specification data set. Compared with the data in the material specification data set, the data in the material semantic data set focuses more on the mining and presentation of its semantic content. For example, the semantic relationship between words in the material description, the overall meaning of the sentence, and other deep semantic information can be parsed through natural language processing and other technologies, providing an analysis basis based on semantic understanding for subsequent multi-level classification processing and high-precision category recognition processing, which helps to more accurately determine the category to which the material belongs.

[0033] Optionally, natural language processing (NLP) technology, such as lexical analysis, can be used to segment the text in the material specification data, identify the nouns, verbs, adjectives and other words of different parts of speech, and clarify the role of each word in describing the material. For example, for a description such as "this laptop is equipped with a high-performance processor and large-capacity memory", it will be split into "laptop" (noun, core description object), "equipped" (verb, reflecting the configuration relationship), "high performance" (adjective, modifying the processor attribute), "processor" (noun, component name), "large capacity" (adjective, modifying the memory attribute), "memory" (noun, component name), etc.

[0034] Then we can do syntactic analysis, and analyze the grammatical structure relationship between words by building a syntax tree, etc., to determine the subject, predicate, object, attributive, adverbial, complement and other components of the sentence, and further clarify the priority and logical order of each part of the description. Taking the above sentence as an example, we can analyze that "laptop" is the subject, "equipped" is the predicate, and "high-performance processor and large-capacity memory" is the object, and we can see that "high performance" modifies "processor" and "large capacity" modifies "memory". In this way, we can accurately grasp that the focus of the description is on the configuration of the laptop.

[0035] Furthermore, semantic role labeling and other methods can be used to determine the roles played by different words at the semantic level of sentences, such as whether a word is the executor, recipient, or subject of the attribute of the description. In the above example, it is clear that "laptop" is the subject of the configuration behavior, "processor" and "memory" are the objects to be configured, and have corresponding attribute descriptions. Through these analyses, pure text is converted into content with clear semantic understanding. All material specification data after such semantic analysis can constitute a material semantic data set.

[0036] The material multi-level classification data set can be a set obtained by the category recognition module after multi-level classification processing of each material semantic data in the material semantic data set. The material multi-level classification data set reflects the multi-level structure of materials in category classification. Materials can be classified into different levels of categories according to the pre-set classification system and the relevant features extracted from the semantic data. Among them, multi-level classification can support multi-level category classification such as primary, secondary, and tertiary to meet fine-grained classification needs.

[0037] For example, we can first base on the pre-set material classification system framework, which is usually constructed in combination with industry standards, business needs and past experience, and can cover multiple levels of classification standards from major categories to minor categories. For example, in the field of electronic products, there may be a first-level major category such as "electronic equipment", which is further subdivided into second-level categories such as "computers", "communication equipment", and "consumer electronics". "Computers" can be further divided into third-level categories such as "desktop computers", "laptop computers", and "tablet computers".

[0038] Then, relevant semantic features can be extracted from the material semantic data set. These semantic features may include key noun concepts (such as material names, core component names, etc.) and characteristic words describing attributes (such as functional characteristics, applicable scenario-related words, etc.). Taking the material semantic data of a smart watch as an example, the key name "smart watch" and attribute characteristic words such as "with health monitoring function" and "can be connected to a mobile phone" are extracted.

[0039] Then, we can use classification algorithms (such as decision trees, support vector machines, etc.) or deep learning-based classification models (such as fine-tuned BERT, GPT and other large models) to determine the categories to which each material semantic data belongs at different levels in turn according to the extracted semantic features and the hierarchical order of the classification system, and classify them into corresponding hierarchical classifications, and finally form a multi-level classification data set of materials. For example, the smart watch will first be classified into the second-level category of "consumer electronics" under the general category of "electronic equipment", and then further divided into the third-level category of "smart wearable devices", etc. BERT, whose full name is Bidirectional Encoder Representations from Transformers, is a pre-trained natural language processing (NLP) model. GPT, whose full name is Generative Pre-trained Transformer, is a natural language processing (NLP) model based on deep learning.

[0040] On the basis of the aforementioned multi-level classification, there may still be some situations where the boundaries are blurred, easy to confuse, or the classification is not accurate enough. These uncertainties can be further eliminated through high-precision category identification and processing, using more precise strategies and methods, thereby ensuring that each material can be accurately classified into its corresponding final category, which can provide a reliable and accurate category basis for subsequent attribute identification and other processes.

[0041] Optionally, a more sophisticated classification model or algorithm optimization method can be used, such as further adjusting the model parameters (such as increasing the number of training rounds, optimizing the weight parameters in the loss function, etc.) of the classification model that has been trained based on the previous steps to make it more sensitive to the subtle differences between similar categories and better able to distinguish easily confused categories of materials. For example, for two electronic products with similar appearances but different functions, the optimized model can accurately determine the category based on the key differences in their functional descriptions.

[0042] Optionally, business rules and expert knowledge can be combined for identification. In some specific industry fields, there are professional classification rules and empirical knowledge, which can assist model judgment. For example, in the field of medical supplies, for some drugs or devices with similar names but different uses, according to the medical professional usage specifications, applicable diseases and other business rules, manual sorting and setting of some discrimination conditions can help the system accurately determine the category of the material in the final stage. After such high-precision processing, an accurate data set of material categories is finally obtained, clearly defining the specific category of each material.

[0043] S40: performing attribute recognition processing on each material category data in the material category data set through the attribute recognition module to obtain a material attribute data set.

[0044] Among them, the attribute recognition module can be a module built based on the Item Recognition LargeModel, which can also be called the I module. It can perform attribute recognition processing on each material category data in the material category data set, aiming to extract various specific material attribute information from these data, such as the material specifications, models, performance parameters and other related attribute names and attribute values, so as to finally form a material attribute data set. During the processing, the attribute recognition module can fully consider different types of complex attributes, such as nested attributes, multi-value attributes, list attributes, etc., and use appropriate technologies and methods to accurately extract these attribute information, further enrich and refine the content of the material data, so that it can more comprehensively reflect the specific characteristics of the material.

[0045] The material attribute data set can be a set formed by the attribute recognition module after performing attribute recognition processing on each material category data in the material category data set. The material attribute data set can contain various specific attribute names and attribute values ​​extracted from the material description, and can cover detailed attribute information of the material specifications, models, performance parameters, etc. Furthermore, by accurately extracting different types of complex attributes (such as nested attributes, multi-valued attributes, list attributes, etc.), the specific characteristics of the material can be comprehensively and meticulously portrayed, and the content of the material data can be further improved so that it can more fully reflect the actual situation of the material, providing a richer data foundation for subsequent business applications, data analysis, etc.

[0046] Optionally, in the attribute recognition module, since the material attribute recognition task requires accurate extraction of various attribute-related information in the material category data, the server can fine-tune the material attribute recognition task based on the large language model to give the model powerful semantic understanding and information extraction capabilities. With the help of pre-trained model fine-tuning, the attribute recognition module does not need to train the model from scratch to learn semantic knowledge, which greatly saves training time and data resources. At the same time, it can use the powerful semantic understanding and text representation capabilities of the large language model to quickly adapt to the material attribute recognition scenario and improve the accuracy and efficiency of recognition. Especially for complex material description texts, it can better reflect its advantages, so that the model can accurately grasp the semantic associations between different words and sentences and accurately identify the attribute content.

[0047] Optionally, the server can build a sequence labeling model in combination with the Conditional Random Field (CRF) model structure, and build a sequence-to-sequence model in combination with model structures such as Bidirectional Long Short-Term Memory (BiLSTM) and Transformer to improve the accuracy and robustness of model attribute recognition. Among them, since the material category data is essentially a sequence of characters or words, attribute recognition can be regarded as labeling each element in this sequence (such as each word) with its corresponding attribute category, such as whether it is an attribute name, attribute value, etc. Among them, CRF is a probabilistic graph model commonly used in sequence labeling tasks. It can consider the dependency between adjacent elements in a text sequence, and combine the features extracted after fine-tuning the large language model mentioned above to conduct a comprehensive analysis of the entire text sequence, so as to more accurately mark which parts are related to the attributes of the materials.

[0048] In some cases, material attribute recognition is not only a simple labeling problem, but may also involve converting the input material category data into another standardized attribute representation form, which can adopt a sequence-to-sequence model structure. Among them, the bidirectional long short-term memory network (BiLSTM) can capture information in a bidirectional manner on the text sequence, taking into account both the previous contextual information and the impact of subsequent content on the current position, which is very helpful for understanding the complete semantics of the attributes in the text. The Transformer architecture has the advantages of strong parallel computing capabilities and the ability to capture long-distance dependencies. It can better handle the long text and complex semantic structures that may exist in the material description, and effectively convert the input original material description text into the output structured attribute information, such as converting a complex description text about the detailed parameters of a certain device into a clear standardized expression in the form of attribute name-attribute value pairs, thereby improving the completeness and accuracy of attribute recognition.

[0049] By combining model structures such as CRF, BiLSTM, and Transformer, the attribute recognition module can give full play to the advantages of different models in processing sequence data and cope with different complex situations in material description texts. Whether it is a simple attribute labeling or a complex text-to-attribute structure conversion task, it can improve the accuracy and robustness of attribute recognition, so that when faced with material descriptions of different styles, lengths, and complexities, the model can extract attribute information stably and accurately.

[0050] Optionally, in the process of material attribute identification, there are multiple related subtasks, such as entity identification (identifying key entities in the material description, such as material name, component name, etc., which are often closely related to the attributes), relationship extraction (clarifying the relationship between different entities and between entities and attributes, such as the correspondence between a component and its attributes), and attribute normalization (unifying attributes with different expressions but the same actual meaning, such as unifying "memory size" and "memory capacity" into a standard expression).

[0051] The multi-task learning mechanism allows the model to learn these related tasks at the same time. By sharing some parameters and feature representations of the model, different tasks can promote and complement each other. For example, when learning the entity recognition task, the "processor" entity is identified, and the relationship between the "processor" and the attributes such as "main frequency" and "number of cores" is further clarified in the relationship extraction task. At the same time, the expression form of these attributes is standardized in the attribute normalization task. This helps the model to understand the material description text more comprehensively and deeply, so as to obtain a more accurate category recognition model, and then more accurately identify attributes.

[0052] Multi-task learning can make full use of the data distribution characteristics and internal connections between different tasks, so that the model can learn richer and more universal feature representations, avoiding the waste of resources for training models for each individual task separately. In addition, through the collaborative learning of multiple tasks, the overall comprehensive ability of the model is improved, and the performance of the attribute recognition module is improved when facing complex and diverse material descriptions and multi-dimensional attribute recognition needs, making the attribute recognition results more accurate, comprehensive and standardized.

[0053] It should be understood that the process of performing attribute recognition processing on each material category data in the material category data set by the attribute recognition module to obtain the material attribute data set refers to the process of performing attribute recognition processing to obtain the material attribute data set. In step S40, that is, performing attribute recognition processing on each material category data in the material category data set by the attribute recognition module to obtain the material attribute data set includes the following steps: S41: performing context-aware processing on each material category data in the material category data set to obtain a material context data set; S42: extracting nested attributes from each material context data in the material context data set to obtain a material nested attribute data set; S43: extracting multi-valued attributes from each material context data in the material context data set to obtain a material multi-valued attribute data set; S44: extracting list attributes from each material context data in the material context data set to obtain a material list attribute data set; S45: Determine a material attribute data set according to the material nested attribute data set, the material multi-valued attribute data set and the material list attribute data set.

[0054] The attribute information of materials is often closely related to the context in which they are located. The same attribute expression may have different meanings or qualifications in different contexts. Context-aware processing can fully understand the relevant information around each description in the material category data, so as to more accurately grasp the true meaning of the attribute, eliminate ambiguity, and clarify its relationship with other related elements, laying the foundation for the accurate extraction of various attributes in the future.

[0055] Optionally, a text window setting method can be used. For example, for each text description in the material category data, a text window of a certain range can be set to define the context content. The window size can be flexibly determined according to the actual situation. For example, a text range of several sentences or a certain number of words can be taken forward and backward, and all related sentences around the location of the target attribute description are taken into consideration. For example, for the text describing an electronic product, "This mobile phone is equipped with a high-performance processor with a main frequency of 3.0GHz. It can run various large-scale applications smoothly, and the battery life is also strong. It adopts a dual-cell structure design and can meet the needs of one day under normal use." To identify the attribute "battery life", in addition to paying attention to the description of the attribute itself, the surrounding sentences mentioned before and after about the running of mobile phone applications and battery-related technologies are included in the analysis as context.

[0056] Optionally, a semantic association analysis method may be used, for example, semantic analysis techniques in natural language processing, such as semantic role labeling, entity relationship extraction, etc., to analyze the semantic associations between words, phrases, and sentences in the context text. Optionally, a context feature extraction method may be used to extract key feature information from the context that helps understand the attribute, such as specific qualifiers, modifiers, other associated entities, etc., which is not limited in this application.

[0057] The nested attribute data set of materials can be a data set with nested attributes in the material description. The nested structure can be a large attribute concept containing multiple sub-attributes, and the multiple sub-attributes can each have corresponding values ​​or further subdivided descriptions. The nested attribute extraction is to deeply analyze the attribute content with a hierarchical structure, accurately mine the attributes of each level and their corresponding values, so that the material attribute information is more detailed and complete in structure, and more comprehensively reflects the specific characteristics of the material.

[0058] Optionally, the structure of nested attributes in the material context data can be identified first by grammatical analysis and specific pattern matching. Then, based on the identified structure, a recursive algorithm or a tree-structure-based traversal method can be used to gradually parse the attributes of each level from the outer layer to the inner layer, and extract the corresponding attribute values, which is not limited in this application.

[0059] Some properties of materials may have multiple values. Accurately extracting these multi-valued properties can more comprehensively display the characteristics and applicable scope of materials in a certain aspect. By accurately identifying properties with multiple values ​​from the material context data and extracting each value completely, we can ensure that no valid information is missed and further improve the property data content of materials.

[0060] Optionally, you can use text pattern recognition, semantic analysis and other technologies to determine which attributes in the material context data have multiple values. For example, for the sentence describing an office supply, "This folder is suitable for storing A4, A3 and B5 size paper", by analyzing the expression "suitable for storage" and the multiple paper size listings that appear, it is determined that "suitable paper size" is a multi-valued attribute. After determining the multi-valued attribute, you can use methods such as text segmentation and regular expression matching to separate and extract each specific attribute value from the text and organize it into a standardized data form. For the above example, you can extract the paper size values ​​of "A4", "A3", and "B5", and unify these extracted multi-valued attributes and their corresponding values ​​to form a material multi-valued attribute data set, which provides multi-dimensional information support for the comprehensive presentation of material attributes.

[0061] Sometimes, some attribute information is presented in the form of a list in the material description, such as a list of accessories, a list of functions, etc. List attributes often reflect multiple related elements of a material in a certain aspect. List attribute extraction can accurately parse out key information such as the structure of the list, the specific content of each item, and the order, so as to extract the attribute values ​​in the list completely and orderly, and further refine the attribute data of the material so that it can more fully reflect the actual situation of the material.

[0062] Optionally, text parsing technology can be used to identify the attribute description part in the material context data that is presented in the form of a list to determine the structural features of the list, such as the start and end marks, separators, etc. Then, according to the identified structural features, the text segmentation operation can be used to extract the elements (i.e., attribute values) in the list in sequence according to the separators, and maintain their original order. Because the order may also contain certain business significance in some cases, such as the importance ranking of accessories. For example, in the example of the electronic product accessories list, the names of accessories such as "power adapter", "data cable", "headphones", and "instructions" can be extracted in sequence to form a material list attribute data set, which fully presents the attribute information of the material in terms of accessories.

[0063] Furthermore, after the aforementioned extraction operations have been performed on attributes of different complex types and the corresponding attribute data sets have been obtained, these different types of attribute data can be integrated to form a comprehensive and complete material attribute data set, which covers various forms of attribute information of materials, from simple to complex, from single value to multiple values ​​and lists, thereby providing a rich and detailed data foundation for subsequent business applications, data analysis and decision support.

[0064] It should be noted that the attribute identification big model (I module) on which the attribute identification module in the embodiment of the present application is based and the category identification big model (A module) on which the category identification module is based can be understood as the vertical field big model involved in the embodiment of the present application, which can accurately analyze material data to support decision-making, realize data standardization and normalization, improve data quality, and perform intelligent attribute classification and category identification of materials, thereby optimizing resource allocation, conducting risk warnings, and promoting the automation of related processes, thereby improving the efficiency and effectiveness of material data governance. This application does not impose any restrictions on this.

[0065] S50: Through the automatic learning module, system learning optimization processing is performed according to the material specification data set, the material category data set and the material attribute data set to obtain a material standardization data set.

[0066] Among them, the automatic learning module can be a module built based on the automatic learning model (Meta-Learning Auto-LearningModel), which can also be called I module. It can mainly play the role of system learning optimization, and use a variety of learning mechanisms and strategies to continuously optimize and improve the entire system based on the data in the material specification data set, material category data set and material attribute data set. For example, through self-learning mechanisms such as meta-learning, active learning, and online learning, the system's cognition of material data characteristics is continuously updated, and through continuous optimization strategies, model parameters are adjusted and system performance is improved according to user feedback, model performance evaluation, etc., and cross-domain transfer learning strategies can also be used to learn from other fields and adapt to the characteristics of specific fields. Through the above-mentioned comprehensive learning optimization processing, the accuracy, generalization ability and adaptability of the system in material data standardization processing are continuously improved, and finally the optimized material standardization data set is output to ensure that the system can effectively respond to the changing material data situation and business needs in the long term.

[0067] The material standardization data set can be the final output result set of the material data standardization method based on the vertical field big model, and can be obtained after the automatic learning module performs system learning and optimization processing based on the material specification data set, material category data set and material attribute data set. The material standardization data set combines the high-quality, standardized data information obtained after processing in the aforementioned links, and after systematic learning and optimization, it has reached a high level in terms of accuracy, standardization and adaptability, which can better serve subsequent business processes such as material management, supply chain optimization, and market analysis, and can better meet the requirements of actual business for material data standardization.

[0068] It should be understood that the process of performing system learning optimization processing according to the material specification data set, the material category data set and the material attribute data set through the automatic learning module to obtain the material standardized data set refers to the process of performing system learning optimization processing to obtain the material standardized data set. Among them, step S50, that is, performing system learning optimization processing according to the material specification data set, the material category data set and the material attribute data set through the automatic learning module to obtain the material standardized data set, includes the following steps: S51: using a self-learning mechanism, performing self-learning processing according to the material specification data set, the material category data set and the material attribute data set to obtain a self-learning strategy; S52: Perform continuous optimization processing according to the material specification data set, the material category data set and the material attribute data set to obtain a continuous optimization strategy; S53: performing cross-domain transfer learning processing according to the material specification data set, the material category data set and the material attribute data set to obtain a cross-domain transfer learning strategy; S54: Perform material data learning optimization processing according to the self-learning strategy, the continuous optimization strategy and the cross-domain transfer learning strategy to obtain a material standardized data set.

[0069] The automatic learning module may first use a self-learning mechanism to carry out a series of processing according to the material specification data set, the material category data set and the material attribute data set to generate a self-learning strategy. It should be understood that using the self-learning mechanism to carry out self-learning processing according to the material specification data set, the material category data set and the material attribute data set to obtain a self-learning strategy may include the following steps: S511: performing metadata extraction processing according to the material specification data set, the material category data set and the material attribute data set to obtain material metadata; S512: using meta-learning technology to perform meta-learning processing on the material metadata to obtain a meta-learning strategy; S513: Performing value assessment processing according to the material specification data set, the material category data set, and the material attribute data set to obtain the most valuable material data; S514: Determine an active learning strategy according to the most valuable material data; S515: performing new data determination processing according to the material specification data set, the material category data set, and the material attribute data set to obtain new material data; S516: Using an incremental learning method, performing incremental learning processing on the newly added material data to obtain an online learning strategy; S517: Perform strategy fusion processing according to the meta-learning strategy, the active learning strategy and the online learning strategy to obtain a self-learning strategy.

[0070] Optionally, the automatic learning module can deeply analyze the data in the material specification data set, material category data set, and material attribute data set, and extract the key parts that can reflect the overall data characteristics and potential laws, that is, the above-mentioned material metadata. For example, key attribute values ​​and their combinations that appear frequently and have important indicative effects on material category and attribute judgment can be screened out from the material attribute data set, or representative text expression patterns can be extracted from the material specification data as material metadata.

[0071] Furthermore, meta-learning technology can be used to process material metadata. Meta-learning aims to quickly master the ability to learn new tasks from a small number of samples. By analyzing the common patterns and rules contained in material metadata, the system can learn more efficiently when faced with new material categories, attributes, or data of different levels of standardization, and quickly adjust the internal recognition and processing logic to obtain a meta-learning strategy.

[0072] For the above three data sets, the automatic learning module can perform value assessment processing, such as using specific evaluation algorithms and business rules to comprehensively consider factors such as data scarcity, potential impact on model improvement, and data uncertainty to screen out the most valuable material data. Then, the active learning strategy can be determined based on the most valuable material data. Among them, the active learning strategy usually involves actively requesting external parties (such as manual labelers or relevant experts) to more accurately label or confirm these data in order to obtain more accurate supervision information, thereby optimizing the system learning process in a targeted manner, improving learning effects, and enabling the model to better handle similar complex or error-prone situations.

[0073] With the continuous development of business and the continuous introduction of new materials, new data may enter the system. The automatic learning module can identify new material data by judging and processing the material specification data set, material category data set, and material attribute data set, such as using data timestamps, comparing differences with existing data features, etc. After determining the new material data, it can be processed using incremental learning. Incremental learning enables the system to receive and learn new data in real time to dynamically adjust its own state, such as updating the parameters of the model to adapt to the characteristic distribution of new data, and then obtaining an online learning strategy, thereby ensuring that the system can continue to adapt to the changing data environment and always maintain the ability to learn and process new material data.

[0074] Finally, the automatic learning module can integrate the meta-learning strategy, active learning strategy and online learning strategy to fully integrate the advantages of each strategy. For example, the meta-learning strategy provides the ability to quickly adapt to new situations based on the existing data rules, the active learning strategy supplements the way to obtain accurate supervision information for key uncertain data, and the online learning strategy ensures the system's dynamic learning ability for real-time new data. By integrating these strategies to form a self-learning strategy, the system can have comprehensive and flexible self-learning capabilities, so that it can use existing data from different angles for self-improvement, so as to better cope with various changes and challenges in the process of material data standardization.

[0075] It should be understood that performing continuous optimization processing according to the material specification data set, the material category data set and the material attribute data set to obtain a continuous optimization strategy may include the following steps: S521: Acquire material feedback data for the material specification data set, the material category data set, and the material attribute data set; S522: performing root-source tracing processing according to the material feedback data to obtain a feedback learning strategy; S523: Performing model performance evaluation according to the material specification data set, the material category data set, and the material attribute data set to obtain model parameter data; S524: Perform parameter adjustment processing according to the model parameter data to obtain a parameter adjustment strategy; S525: Performing strategy fusion processing according to the feedback learning strategy and the parameter adjustment strategy to obtain a continuous optimization strategy.

[0076] The automatic learning module can carry out continuous optimization processing based on the same three data sets to generate continuous optimization strategies. Optionally, various types of information feedback from users on material specification data sets, material category data sets, and material attribute data sets during use can be collected. These material feedback data cover feedback on issues such as category identification errors, inaccurate attribute identification, and data specifications that do not meet actual needs. Based on these material feedback data, the automatic learning module can perform root cause tracing, such as using data analysis and business logic combing methods to deeply analyze the root causes of the problem, such as analyzing whether the problem is caused by unreasonable model parameter settings, data feature understanding deviations, or improper application of business rules. In this way, feedback learning strategies can be obtained to clarify how to adjust the learning direction and focus based on feedback information, thereby improving system performance to avoid the recurrence of similar problems.

[0077] Optionally, a comprehensive model performance evaluation can be conducted on the material specification data set, material category data set, and material attribute data set. The performance of relevant models in the current system (such as category recognition models, attribute recognition models, etc.) in processing these data can be analyzed using multiple evaluation indicators such as accuracy, recall, and F1 value to obtain model parameter data, that is, to clarify the performance of each model parameter and the degree of influence on the overall result. Then, parameter adjustment processing is performed based on the model parameter data, such as determining appropriate parameter adjustment strategies through optimization algorithms, simulation experiments, etc., such as optimizing model weights, thresholds, learning rates, and other parameters, to improve the model's processing capabilities and accuracy for material data, so that it can better fit the data characteristics and task requirements in actual business.

[0078] Furthermore, the feedback learning strategy and parameter adjustment strategy can be integrated to generate a continuous optimization strategy by integrating the improvement direction and adjustment methods of the two. The feedback learning strategy focuses on guiding system optimization based on the actual problems reported by users, while the parameter adjustment strategy focuses on scientifically adjusting parameters based on model performance data. After the two are integrated, the continuous optimization strategy can ensure that the system can continuously optimize its own performance based on actual operating conditions and feedback information, and continuously improve the quality and effect of standardized material data processing to better serve business needs.

[0079] It should be understood that performing cross-domain transfer learning processing according to the material specification data set, the material category data set, and the material attribute data set to obtain a cross-domain transfer learning strategy may include the following steps: S531: Acquire relevant field characteristics according to the material specification data set, the material category data set and the material attribute data set to obtain relevant field characteristic data; S532: Perform transfer learning processing according to the related field feature data to obtain a transfer learning strategy; S533: extracting domain-adaptive features according to the material specification data set, the material category data set, and the material attribute data set to obtain domain-adaptive feature data; S534: Performing domain adaptation training according to the domain adaptation feature data to obtain a domain adaptation strategy; S535: Performing strategy fusion processing according to the transfer learning strategy and the domain adaptation strategy to obtain a cross-domain transfer learning strategy.

[0080] The automatic learning module can perform cross-domain transfer learning processing for material specification data sets, material category data sets, and material attribute data sets, and generate cross-domain transfer learning strategies. Optionally, the automatic learning module can mine and obtain relevant field features from these three data sets, analyze the common feature information that can be transferred and referenced between material data and other related fields (such as similar material fields, upstream and downstream industrial material fields, etc.) in different dimensions, such as similarities in vocabulary usage, data structure, category division logic, etc., so as to obtain relevant field feature data. Then, transfer learning processing is performed based on the relevant field feature data to transfer the existing effective knowledge, model structure, processing methods, etc. in other fields to the current material data standardization system based on the vertical field large model, such as borrowing the mature category division model structure in the general commodity classification field, and applying it to its own category recognition module after appropriate adjustment, and then obtaining a transfer learning strategy, so as to enhance the versatility and learning ability of the system and broaden the system's processing capabilities for different types of material data.

[0081] In a possible implementation, domain adaptive feature extraction is performed based on the material specification data set, the material category data set and the material attribute data set to obtain domain adaptive feature data, which may include the following steps: performing a vocabulary usage feature analysis on each material specification data in the material specification data set to obtain vocabulary usage feature data; performing a category classification habit analysis on each material category data in the material category data set to obtain category classification habit feature data; performing an attribute description analysis on each material attribute data in the material attribute data set to obtain attribute description feature data; and determining the domain adaptive feature data based on the vocabulary usage feature data, the category classification habit feature data and the attribute description feature data.

[0082] It should be noted that domain adaptive feature extraction can be performed on material specification data sets, material category data sets, and material attribute data sets. Vocabulary usage feature analysis can be performed, such as analyzing the vocabulary usage of each material specification data in the material specification data set, and statistically analyzing vocabulary usage feature data such as professional vocabulary, high-frequency vocabulary, and vocabulary collocation specific to different categories of materials, to understand the semantic role and importance of each vocabulary in the material description. For example, in the field of electronic materials, words such as "chip model" and "memory capacity" often appear and are critical to attribute identification. These vocabulary usage feature data can help the system better understand and process material description texts.

[0083] It is possible to analyze the habit of category division, such as exploring the category division habits of each material category data in the material category data set, and sorting out the classification basis, boundaries, and mutual relationships of different levels of categories. For example, in the field of medical supplies, there are large categories such as drugs, medical devices, and medical consumables, and drugs are further divided into chemical drugs, biological drugs, etc. This helps the system to more accurately identify the category to which the materials belong and avoid classification confusion.

[0084] Attribute description analysis can be performed, such as performing attribute description analysis on each material attribute data in the material attribute data set, to clarify the attribute description feature data of different material attributes in terms of description format, value range, and relationship with other attributes. For example, the "power" attribute of a certain type of equipment has a specific value range and unit requirements, and this feature data helps the system to extract and standardize attribute information more accurately.

[0085] By integrating the above-mentioned vocabulary usage feature data, category classification habit feature data and attribute description feature data, we can determine the domain adaptive feature data, and then conduct domain adaptive training based on the domain adaptive feature data to make the system better fit the inherent laws of material data in a specific field. For example, we can adjust the model structure to make it more suitable for the strict attribute description specifications in the field of medical materials, thereby obtaining a domain adaptive strategy and improving the professionalism and accuracy of the system in this field.

[0086] Furthermore, the transfer learning strategy and the domain adaptation strategy can be integrated to take into account the advantages of knowledge transfer in different fields and the adaptability to specific fields, and generate a cross-domain transfer learning strategy. The transfer learning strategy enables the system to learn from the useful experience and knowledge of external fields, and the domain adaptation strategy ensures that the system can play a precise role in the specific field of its own concern. After the integration of the two, the cross-domain transfer learning strategy enables the system to play a good role in cross-domain applications and specific field optimization, and can better cope with the standardized processing requirements of material data in different fields.

[0087] After generating the self-learning strategy, continuous optimization strategy and cross-domain transfer learning strategy respectively, the automatic learning module conducts comprehensive learning and optimization processing on the material data according to these three strategies. The self-learning strategy ensures that the system has the ability to continuously learn and self-improve, the continuous optimization strategy promotes the system to continuously improve according to actual feedback and performance evaluation, and the cross-domain transfer learning strategy broadens the adaptability and versatility of the system. By comprehensively applying these strategies, the system mines more valuable information from the material specification data set, material category data set and material attribute data set, continuously optimizes the data processing process and results, and finally obtains a fully optimized and high-standard material standardization data set, so that it can better serve subsequent business application scenarios such as material management, supply chain optimization, and data analysis, and meet the diverse needs of actual business for material data standardization.

[0088] Optionally, after the aforementioned series of processes, including global normalization, category identification, attribute identification, and system learning optimization of the automatic learning module, the final output can be standardized material data containing complete categories, attribute names, and attribute values, so that each material has a clear, accurate, and comprehensive standardized description, which is convenient for the direct use of material data in subsequent different business links. Optionally, these standardized material data can be stored in a high-performance database. High-performance databases have fast data reading and writing capabilities, strong data storage capacity, and good data security and stability. For example, common relational databases (such as MySQL, Oracle, etc.) or non-relational databases (such as MongoDB, Redis, etc.), select according to actual business needs and data characteristics. Optionally, real-time query can be supported, that is, relevant personnel can obtain the latest and accurate material data information at any time, and this application does not limit this.

[0089] Optionally, data visualization can be further performed and corresponding reports can be automatically generated. Pure text data makes it difficult for people to quickly understand and grasp the laws and trends in the data and discover abnormal situations. However, through data visualization, material data can be displayed in an intuitive way such as charts (such as bar charts, line charts, pie charts, etc.) and dashboards, which allows business personnel, managers, etc. to see the key information of the data at a glance.

[0090] For example, in inventory management, you can visualize the inventory ratio of various materials in different warehouses (using pie charts) and the trend of each material inventory over time (using line charts), so that warehouse managers can intuitively grasp the inventory dynamics and arrange replenishment or adjust the inventory layout in a timely manner. In terms of supply chain optimization, visualizing the circulation time of materials in each supply link (using bar charts to compare the duration of different links) can help find the links that take a long time and optimize them in a targeted manner. For market analysis, using bar charts to show the sales share of material products with different attribute configurations in the market can assist in analyzing consumers' preferences for different attributes and provide a reference for product improvement and new product development.

[0091] The automatic report generation function can generate comprehensive data analysis reports based on in-depth analysis of material data according to preset cycles (such as daily, weekly, monthly, etc.). These reports cover many important information of material data, such as category distribution, attribute statistics, key indicator change trends, etc., and are presented in a standardized and well-organized document form, providing strong data support for business decision-making and strategic planning.

[0092] See also Figure 3 , Figure 3 The following is an exemplary diagram of the material data standardization system architecture based on the vertical field big model. Figure 3 As shown, starting from the data input module, the data (such as the material description data set) can first enter the H module for normalization processing. If the normalization processing passes, it will enter the A module. If the normalization processing fails, it will be re-parsed and cleaned; after entering the A module, category recognition processing can be performed; then enter the I module for attribute recognition processing; and further enter the M module for learning optimization processing; finally, data storage and management can be performed, and enter the data output and application module and the data visualization and reporting module respectively.

[0093] It can be seen that in the above scheme, by acquiring a material description data set, and through the global normalization module, normalizing each material description data in the material description data set, a material specification data set is obtained, and further through the category identification module, each material specification data in the material specification data set is subjected to category identification processing to obtain a material category data set, so that through the attribute identification module, each material category data in the material category data set can be subjected to attribute identification processing to obtain a material attribute data set, and then through the automatic learning module, system learning and optimization processing can be performed according to the material specification data set, the material category data set and the material attribute data set, so as to obtain a more accurate material standardization data set, which is conducive to achieving a more efficient and accurate material data standardization process.

[0094] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0095] In one embodiment, a material data standardization device based on a vertical field big model is provided, and the material data standardization device based on a vertical field big model corresponds one-to-one to the material data standardization method based on a vertical field big model in the above embodiment. Figure 4 As shown, the material data standardization device based on the vertical field big model includes an acquisition module 101, a first processing module 102, a second processing module 103, a third processing module 104 and a fourth processing unit 105. The detailed description of each functional module is as follows: An acquisition unit 101 is used to acquire a material description data set; The first processing unit 102 is used to perform normalization processing on each material description data in the material description data set through the global normalization module to obtain a material standard data set; The second processing unit 103 is used to perform category identification processing on each material specification data in the material specification data set through the category identification module to obtain a material category data set; The third processing unit 104 is used to perform attribute recognition processing on each material category data in the material category data set through the attribute recognition module to obtain a material attribute data set; The fourth processing unit 105 is used to perform system learning optimization processing according to the material specification data set, the material category data set and the material attribute data set through the automatic learning module to obtain a material standardized data set.

[0096] In one embodiment, the second processing unit 103 is used to perform category identification processing on each material specification data in the material specification data set to obtain a material category data set, specifically for: Performing semantic analysis on each material specification data in the material specification data set to obtain a material semantic data set; Performing multi-level classification processing on each material semantic data in the material semantic data set to obtain a material multi-level classification data set; High-precision category identification processing is performed on each material multi-level classification data in the material multi-level classification data set to obtain a material category data set.

[0097] In one embodiment, the third processing unit 104 is used to perform attribute recognition processing on each material category data in the material category data set to obtain a material attribute data set, specifically for: Performing context-aware processing on each material category data in the material category data set to obtain a material context data set; Extracting nested attributes of each material context data in the material context data set to obtain a material nested attribute data set; Extracting multi-valued attributes from each material context data in the material context data set to obtain a material multi-valued attribute data set; Extracting list attributes from each material context data in the material context data set to obtain a material list attribute data set; A material attribute data set is determined according to the material nested attribute data set, the material multi-value attribute data set and the material list attribute data set.

[0098] In one embodiment, the fourth processing unit 105 is used to perform system learning optimization processing according to the material specification data set, the material category data set and the material attribute data set to obtain a material standardized data set, specifically for: Using a self-learning mechanism, performing self-learning processing according to the material specification data set, the material category data set and the material attribute data set, to obtain a self-learning strategy; Perform continuous optimization processing according to the material specification data set, the material category data set and the material attribute data set to obtain a continuous optimization strategy; Perform cross-domain transfer learning processing according to the material specification data set, the material category data set, and the material attribute data set to obtain a cross-domain transfer learning strategy; According to the self-learning strategy, the continuous optimization strategy and the cross-domain transfer learning strategy, material data learning optimization processing is performed to obtain a material standardized data set.

[0099] In one embodiment, the fourth processing unit 105 is used to use a self-learning mechanism to perform self-learning processing according to the material specification data set, the material category data set and the material attribute data set to obtain a self-learning strategy, which is specifically used to: Performing metadata extraction processing according to the material specification data set, the material category data set and the material attribute data set to obtain material metadata; Using meta-learning technology, meta-learning processing is performed on the material metadata to obtain a meta-learning strategy; Performing value assessment processing according to the material specification data set, the material category data set, and the material attribute data set to obtain the most valuable material data; Determine an active learning strategy based on the most valuable material data; Performing new data determination processing according to the material specification data set, the material category data set, and the material attribute data set to obtain new material data; Adopting an incremental learning method, performing incremental learning processing on the newly added material data to obtain an online learning strategy; A strategy fusion process is performed according to the meta-learning strategy, the active learning strategy and the online learning strategy to obtain a self-learning strategy.

[0100] In one embodiment, the fourth processing unit 105 is used to perform continuous optimization processing according to the material specification data set, the material category data set and the material attribute data set to obtain a continuous optimization strategy, which is specifically used to: Acquire material feedback data for the material specification data set, the material category data set, and the material attribute data set; Perform root-source tracing processing based on the material feedback data to obtain a feedback learning strategy; Performing a model performance evaluation based on the material specification data set, the material category data set, and the material attribute data set to obtain model parameter data; Perform parameter adjustment processing according to the model parameter data to obtain a parameter adjustment strategy; A strategy fusion process is performed according to the feedback learning strategy and the parameter adjustment strategy to obtain a continuous optimization strategy.

[0101] In one embodiment, the fourth processing unit 105 is used to perform cross-domain transfer learning processing according to the material specification data set, the material category data set and the material attribute data set to obtain a cross-domain transfer learning strategy, which is specifically used to: Acquire relevant field characteristics according to the material specification data set, the material category data set and the material attribute data set to obtain relevant field characteristic data; Perform transfer learning processing according to the characteristic data of the relevant field to obtain a transfer learning strategy; Performing domain adaptive feature extraction according to the material specification data set, the material category data set and the material attribute data set to obtain domain adaptive feature data; Performing domain adaptation training according to the domain adaptation feature data to obtain a domain adaptation strategy; A strategy fusion process is performed according to the transfer learning strategy and the domain adaptation strategy to obtain a cross-domain transfer learning strategy.

[0102] In one embodiment, the fourth processing unit 105 is used to perform domain adaptive feature extraction according to the material specification data set, the material category data set and the material attribute data set to obtain domain adaptive feature data, which is specifically used to: Performing vocabulary usage feature analysis on each material specification data in the material specification data set to obtain vocabulary usage feature data; Performing a category classification habit analysis on each material category data in the material category data set to obtain category classification habit feature data; Performing attribute description analysis on each material attribute data in the material attribute data set to obtain attribute description feature data; Domain adaptive feature data is determined based on the vocabulary usage feature data, the category classification habit feature data and the attribute description feature data.

[0103] The present invention provides a material data standardization device based on a vertical field big model, which obtains a material description data set and normalizes each material description data in the material description data set through the global normalization module to obtain a material standard data set, and further performs category identification processing on each material standard data in the material standard data set through the category identification module to obtain a material category data set, so that the attribute identification module can be used to perform attribute identification processing on each material category data in the material category data set to obtain a material attribute data set, and then the automatic learning module can be used to perform system learning and optimization processing according to the material standard data set, the material category data set and the material attribute data set, so as to obtain a more accurate material standardization data set, which is conducive to realizing a more efficient and accurate material data standardization process.

[0104] For the specific definition of the material data standardization device based on the vertical field big model, please refer to the definition of the material data standardization method based on the vertical field big model above, which will not be repeated here. Each module in the above-mentioned material data standardization device based on the vertical field big model can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0105] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client through a network connection. When the computer program is executed by the processor, it implements the functions or steps of a service-side method for material data standardization based on a vertical field big model.

[0106] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the client side of a material data standardization method based on a vertical field large model.

[0107] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: Get the material description data set; By means of the global normalization module, each material description data in the material description data set is normalized to obtain a material standard data set; By means of the category identification module, each material specification data in the material specification data set is subjected to category identification processing to obtain a material category data set; By means of the attribute recognition module, attribute recognition processing is performed on each material category data in the material category data set to obtain a material attribute data set; Through the automatic learning module, system learning optimization processing is performed according to the material specification data set, the material category data set and the material attribute data set to obtain a material standardization data set.

[0108] The present invention provides a computer device, which obtains a material description data set and performs normalization processing on each material description data in the material description data set through the global normalization module to obtain a material standard data set, and further performs category identification processing on each material standard data in the material standard data set through the category identification module to obtain a material category data set, so that the attribute identification module can perform attribute identification processing on each material category data in the material category data set to obtain a material attribute data set, and then the automatic learning module can perform system learning and optimization processing according to the material standard data set, the material category data set and the material attribute data set, so as to obtain a more accurate material standardization data set, which is conducive to realizing a more efficient and accurate material data standardization process.

[0109] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: Get the material description data set; By means of the global normalization module, each material description data in the material description data set is normalized to obtain a material standard data set; By means of the category identification module, each material specification data in the material specification data set is subjected to category identification processing to obtain a material category data set; By means of the attribute recognition module, attribute recognition processing is performed on each material category data in the material category data set to obtain a material attribute data set; Through the automatic learning module, system learning optimization processing is performed according to the material specification data set, the material category data set and the material attribute data set to obtain a material standardization data set.

[0110] The present invention provides a computer-readable storage medium, which obtains a material description data set and performs normalization processing on each material description data in the material description data set through the global normalization module to obtain a material specification data set, and further performs category identification processing on each material specification data in the material specification data set through the category identification module to obtain a material category data set, so that the attribute identification module can be used to perform attribute identification processing on each material category data in the material category data set to obtain a material attribute data set, and then the automatic learning module can be used to perform system learning and optimization processing according to the material specification data set, the material category data set and the material attribute data set, so as to obtain a more accurate material standardization data set, which is conducive to realizing a more efficient and accurate material data standardization process.

[0111] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0112] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0113] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0114] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A material data standardization method based on a vertical field big model, characterized in that: Applied to a material data standardization system based on a vertical field big model, the material data standardization system based on a vertical field big model includes a global normalization module, a category recognition module, an attribute recognition module and an automatic learning module; The material data standardization method based on the vertical field big model includes: Get the material description data set; By means of the global normalization module, each material description data in the material description data set is normalized to obtain a material standard data set; By means of the category identification module, each material specification data in the material specification data set is subjected to category identification processing to obtain a material category data set; By means of the attribute recognition module, attribute recognition processing is performed on each material category data in the material category data set to obtain a material attribute data set; Through the automatic learning module, system learning optimization processing is performed according to the material specification data set, the material category data set and the material attribute data set to obtain a material standardization data set.

2. The material data standardization method based on the vertical field big model according to claim 1 is characterized in that: The performing category identification processing on each material specification data in the material specification data set to obtain the material category data set includes: Performing semantic analysis on each material specification data in the material specification data set to obtain a material semantic data set; Performing multi-level classification processing on each material semantic data in the material semantic data set to obtain a material multi-level classification data set; High-precision category identification processing is performed on each material multi-level classification data in the material multi-level classification data set to obtain a material category data set.

3. The material data standardization method based on the vertical domain big model according to claim 2 is characterized in that: The performing attribute recognition processing on each material category data in the material category data set to obtain a material attribute data set includes: Performing context-aware processing on each material category data in the material category data set to obtain a material context data set; Extracting nested attributes of each material context data in the material context data set to obtain a material nested attribute data set; Extracting multi-valued attributes from each material context data in the material context data set to obtain a material multi-valued attribute data set; Extracting list attributes from each material context data in the material context data set to obtain a material list attribute data set; A material attribute data set is determined according to the material nested attribute data set, the material multi-valued attribute data set and the material list attribute data set.

4. The material data standardization method based on the vertical domain big model according to claim 3 is characterized in that: The system learning optimization processing is performed according to the material specification data set, the material category data set and the material attribute data set to obtain the material standardization data set, including: Using a self-learning mechanism, performing self-learning processing according to the material specification data set, the material category data set and the material attribute data set, to obtain a self-learning strategy; Perform continuous optimization processing according to the material specification data set, the material category data set and the material attribute data set to obtain a continuous optimization strategy; Perform cross-domain transfer learning processing according to the material specification data set, the material category data set, and the material attribute data set to obtain a cross-domain transfer learning strategy; According to the self-learning strategy, the continuous optimization strategy and the cross-domain transfer learning strategy, material data learning optimization processing is performed to obtain a material standardized data set.

5. The material data standardization method based on the vertical domain big model according to claim 4 is characterized in that: The self-learning mechanism is used to perform self-learning processing according to the material specification data set, the material category data set and the material attribute data set to obtain a self-learning strategy, including: Performing metadata extraction processing according to the material specification data set, the material category data set and the material attribute data set to obtain material metadata; Using meta-learning technology, meta-learning processing is performed on the material metadata to obtain a meta-learning strategy; Performing value assessment processing according to the material specification data set, the material category data set, and the material attribute data set to obtain the most valuable material data; Determine an active learning strategy based on the most valuable material data; Performing new data determination processing according to the material specification data set, the material category data set, and the material attribute data set to obtain new material data; Adopting an incremental learning method, performing incremental learning processing on the newly added material data to obtain an online learning strategy; A strategy fusion process is performed according to the meta-learning strategy, the active learning strategy and the online learning strategy to obtain a self-learning strategy.

6. The material data standardization method based on the vertical domain big model according to claim 5 is characterized in that: The continuous optimization process is performed according to the material specification data set, the material category data set and the material attribute data set to obtain a continuous optimization strategy, including: Acquire material feedback data for the material specification data set, the material category data set, and the material attribute data set; Perform root-source tracing processing based on the material feedback data to obtain a feedback learning strategy; Performing a model performance evaluation based on the material specification data set, the material category data set, and the material attribute data set to obtain model parameter data; Perform parameter adjustment processing according to the model parameter data to obtain a parameter adjustment strategy; A strategy fusion process is performed according to the feedback learning strategy and the parameter adjustment strategy to obtain a continuous optimization strategy.

7. The material data standardization method based on a vertical field big model according to any one of claims 4 to 6, characterized in that: The cross-domain transfer learning process is performed according to the material specification data set, the material category data set and the material attribute data set to obtain a cross-domain transfer learning strategy, including: Acquire relevant field characteristics according to the material specification data set, the material category data set and the material attribute data set to obtain relevant field characteristic data; Perform transfer learning processing according to the characteristic data of the relevant field to obtain a transfer learning strategy; Performing domain adaptive feature extraction according to the material specification data set, the material category data set and the material attribute data set to obtain domain adaptive feature data; Performing domain adaptation training according to the domain adaptation feature data to obtain a domain adaptation strategy; A strategy fusion process is performed according to the transfer learning strategy and the domain adaptation strategy to obtain a cross-domain transfer learning strategy.

8. The material data standardization method based on the vertical domain big model according to claim 7 is characterized in that: The extracting domain-adaptive features according to the material specification data set, the material category data set and the material attribute data set to obtain domain-adaptive feature data includes: Performing vocabulary usage feature analysis on each material specification data in the material specification data set to obtain vocabulary usage feature data; Performing a category classification habit analysis on each material category data in the material category data set to obtain category classification habit feature data; Performing attribute description analysis on each material attribute data in the material attribute data set to obtain attribute description feature data; Domain adaptive feature data is determined based on the vocabulary usage feature data, the category classification habit feature data and the attribute description feature data.

9. A material data standardization device based on a vertical field big model, characterized in that: Applied to a material data standardization system based on a vertical field big model, the material data standardization system based on a vertical field big model includes a global normalization module, a category recognition module, an attribute recognition module and an automatic learning module; The material data standardization device based on the vertical field big model includes: An acquisition unit, used for acquiring a material description data set; A first processing unit, configured to perform normalization processing on each material description data in the material description data set through the global normalization module to obtain a material standard data set; A second processing unit is used to perform category identification processing on each material specification data in the material specification data set through the category identification module to obtain a material category data set; A third processing unit is used to perform attribute recognition processing on each material category data in the material category data set through the attribute recognition module to obtain a material attribute data set; The fourth processing unit is used to perform system learning optimization processing according to the material specification data set, the material category data set and the material attribute data set through the automatic learning module to obtain the material standardized data set.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the material data standardization method based on the vertical field big model as described in any one of claims 1 to 8.

Images