Multi-source supply chain component data processing system and method, electronic equipment and medium

Abstract

The invention provides a multi-source supply chain component data processing system and method, electronic equipment and a medium, and belongs to the technical field of data processing, and the method comprises the steps: collecting original component data from at least two types of heterogeneous supplier systems through a heterogeneous system adaptive API gateway; cleaning the original component data by adopting a data cleaning module to obtain cleaned component data; inputting the cleaned component data into a semantic mapping engine, and automatically converting non-standard attributes in the cleaned component data into standard attribute semantics based on a preset standard attribute dictionary database and a preset machine learning mapping model in the semantic mapping engine, according to the invention, the efficiency and accuracy of multi-source supply chain component data processing are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a multi-source supply chain component data processing system and method, electronic equipment and medium. BACKGROUND

[0002] In the current building decoration industry, especially in the field of customized and refined delivery of residential and commercial space, the design party needs to obtain and integrate tens of thousands of component (such as tiles, sanitary wares, doors and windows, etc.) data from numerous suppliers, including their specifications, materials, environmental protection levels, prices, inventories and delivery cycles, and other key attributes, to complete the scheme design, cost accounting and construction docking. However, different suppliers often use different enterprise resource planning (ERP) systems, data formats and attribute description habits, resulting in significant semantic heterogeneity of component data from multiple sources.

[0003] At present, there are mainly two typical paths in the existing technical solutions for processing semantic heterogeneity of multi-source supply chain component data in the industry. The first type of solution focuses on centralized storage and display of data, for example, by building a central database and requiring each supplier to manually enter or convert their component information according to a unified data template. This type of solution can achieve centralized query of data, but it relies heavily on manual format translation and entry, which is not only inefficient, but also prone to errors due to subjective understanding bias. The second type of solution attempts to introduce some automation and pre-set simple rules for data cleaning (such as text replacement). However, this type of solution can usually only adapt to limited types of systems and cannot flexibly cope with the diverse heterogeneous data sources in the market. When suppliers update data formats or add new suppliers, a large amount of manpower needs to be invested in rule adjustment and system adaptation, resulting in high implementation and maintenance costs.

[0004] In summary, the existing technology generally relies on manual semantic conversion and mapping, uses static and limited cleaning rules, and lacks a self-adaptive iteration mechanism, resulting in technical problems such as low data processing efficiency, difficulty in ensuring accuracy, and difficulty in adapting to new data sources. SUMMARY

[0005] Therefore, it is necessary to provide a multi-source supply chain component data processing system, method, electronic equipment and storage medium to solve the technical problems of low efficiency, low accuracy and poor scalability of heterogeneous data processing caused by the existing methods relying on manual semantic conversion and mapping, using static and limited cleaning rules, and lacking a self-adaptive iteration mechanism.

[0006] To solve the above technical problems, in a first aspect, the present application provides a multi-source supply chain component data processing system, comprising: a heterogeneous system adaptation API gateway for automatically identifying and accessing component data from at least two types of heterogeneous supplier systems; The data cleaning module is in communication connection with the heterogeneous system adaptive API gateway, and is configured to perform repeated value filtering and missing value completion processing on the received component data. The semantic mapping engine is in communication connection with the data cleaning module, and is configured to automatically convert non-standard component attributes that have been cleaned into standard attribute semantics based on a preset standard attribute dictionary library and a preset machine learning mapping model. The mapping rule iteration module is in communication connection with the heterogeneous system adaptive API gateway and the semantic mapping engine, and is configured to update the mapping model in the semantic mapping engine according to component data of a newly added supplier system and manual annotation results.

[0007] In a possible implementation, the data cleaning module comprises: The repeated value filtering unit is configured to identify and delete or mark repeated component data entries based on a double-key hash index constructed based on component codes and specification information. The missing value completion unit is configured to perform a filling operation based on average values of similar historical data or initiate a completion request to a corresponding supplier system for component data with missing attribute values.

[0008] In a possible implementation, the machine learning mapping model preset in the semantic mapping engine is a model trained based on a logistic regression algorithm; and the preset standard attribute dictionary library stores standard attribute semantics in multiple dimensions.

[0009] In a possible implementation, the multi-source supply chain component data processing system comprises a normalization database in communication connection with the semantic mapping engine, and configured to store standardized component data after semantic conversion, and store mapping logs between component data of heterogeneous supplier systems and the standardized component data.

[0010] In another aspect, the present application also provides a multi-source supply chain component data processing method, comprising: Collecting original component data from at least two types of heterogeneous supplier systems through a heterogeneous system adaptive API gateway; Cleaning the original component data using a data cleaning module to obtain cleaned component data; Inputting the cleaned component data into a semantic mapping engine, and automatically converting non-standard attributes in the cleaned component data into standard attribute semantics based on a preset standard attribute dictionary library and a preset machine learning mapping model in the semantic mapping engine.

[0011] In a possible implementation, the cleaning of the original component data using the data cleaning module to obtain cleaned component data comprises: acquire part codes and specification information in the original part data; construct a double-key hash index based on the part codes and specification information; identify and process duplicate entries in the original part data according to the double-key hash index to obtain de-duplicated part data; perform a filling operation based on the mean value of similar historical data or initiate a completion request to a corresponding supplier system for missing attribute values in the de-duplicated part data to obtain the cleaned part data.

[0012] In a possible implementation, the automatic conversion of non-standard attributes in the cleaned part data into standard attribute semantics includes: inputting the non-standard attributes in the cleaned part data into the preset machine learning mapping model for semantic recognition to obtain an attribute recognition result; according to the attribute recognition result, matching and outputting corresponding standard attribute semantics from the preset standard attribute dictionary library.

[0013] In a possible implementation, the multi-source supply chain part data processing method further includes: in response to the access of a new supplier system, collecting sample part data of the new supplier system as incremental part data through the mapping rule iteration module; obtaining a manual standard attribute semantic annotation result of the incremental part data through a manual annotation interactive interface in the mapping rule iteration module; updating the machine learning mapping model in the semantic mapping engine according to the annotation result.

[0014] In a third aspect, the present application further provides an electronic device including a memory and a processor, wherein the memory is configured to store a program; the processor is coupled to the memory and configured to execute the program stored in the memory to implement the steps of the multi-source supply chain part data processing method in any of the above implementation modes.

[0015] In a fourth aspect, the present application further provides a computer readable storage medium for storing computer readable programs or instructions, which can implement the steps of the multi-source supply chain part data processing method in any of the above implementation modes when executed by a processor.

[0016] The beneficial effects of this invention are as follows: The multi-source supply chain component data processing method provided by this invention collects raw component data from at least two types of heterogeneous supplier systems through a heterogeneous system adaptation API gateway; then, a data cleaning module is used to clean the raw component data to obtain cleaned component data, thereby improving data collection efficiency and data quality. At the same time, the cleaned component data is input into a semantic mapping engine. Based on the preset standard attribute dictionary and preset machine learning mapping model in the semantic mapping engine, the non-standard attributes in the cleaned component data are automatically converted into standard attribute semantics, enabling automatic matching and semantic normalization of heterogeneous data, and timely updating of the mapping model, thereby improving the efficiency and accuracy of multi-source supply chain component data processing. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic flowchart of an embodiment of the multi-source supply chain component data processing system provided by the present invention; Figure 2 For the present invention Figure 1 A schematic diagram of the structure of an embodiment of the data cleaning module; Figure 3 A flowchart illustrating another embodiment of the multi-source supply chain component data processing system provided by the present invention; Figure 4 A schematic flowchart of an embodiment of the multi-source supply chain component data processing method provided by the present invention; Figure 5 For the present invention Figure 4 A schematic diagram of an embodiment of S402; Figure 6 For the present invention Figure 4 A schematic diagram of an embodiment of S403; Figure 7 A schematic flowchart of another embodiment of the multi-source supply chain component data processing method provided by the present invention; Figure 8 A schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0020] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0021] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0022] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0023] This invention provides a multi-source supply chain component data processing system, method, electronic device, and storage medium. The technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] Figure 1 This is a schematic diagram of an embodiment of the multi-source supply chain component data processing system provided by the present invention, as shown below. Figure 1 As shown, the multi-source supply chain component data processing system includes: Heterogeneous system adapter API gateway 101 is used to automatically identify and access component data from at least two types of heterogeneous supplier systems; The data cleaning module 102 communicates with the heterogeneous system adapter API gateway 101 and is used to filter duplicate values ​​and fill in missing values ​​in the received component data. The semantic mapping engine 103 is connected in communication with the data cleaning module 102 and is used to automatically convert the cleaned non-standard parts attributes into standard attribute semantics based on a preset standard attribute dictionary and a preset machine learning mapping model. The mapping rule iteration module 104 is connected to the heterogeneous system adaptation API gateway 101 and the semantic mapping engine 103 respectively, and is used to update the mapping model in the semantic mapping engine 103 based on the component data of the newly added supplier system and the manual annotation results.

[0025] Specifically, the Heterogeneous System Adaptor API Gateway 101 is used to connect to external vendor systems. It has a built-in protocol identification module (supporting HTTP, MQTT, and WebSocket) and a data format conversion module, automatically identifying the communication protocols of different vendor systems. It can connect to at least three types of heterogeneous systems without secondary development, with a protocol conversion response time of ≤100ms, achieving unified access to multi-source data. These external vendor systems include, but are not limited to: SAP (System Applications and Products), external vendors, and self-developed systems.

[0026] The input end of the data cleaning module 102 is connected to the heterogeneous system adapter API gateway 101 for filtering duplicate values ​​and filling missing values ​​in the received component data, so that the obtained data attribute fields meet the application requirements.

[0027] The semantic mapping engine 103 includes a pre-defined standard attribute dictionary and a pre-defined machine learning mapping model.

[0028] Specifically, this pre-defined standard attribute dictionary is a structured semantic knowledge base pre-built and stored during the system deployment or initialization phase, based on industry-standard data, enterprise specifications, or project requirements. This dictionary clarifies several key dimensions in the supply chain component data that need to be standardized, along with their corresponding unique and standardized values. For example, in the high-end interior decoration field, typical dimensions and their standard semantics may include: Size dimensions: The standard unit is "millimeters (mm)", with a format such as "1200mm×600mm".

[0029] Environmental protection level dimensions: The standard values ​​are uniformly "E0", "E1", "E2", etc.

[0030] Waterproof rating dimensions: The standard values ​​are uniformly "IPX1" to "IPX8", etc.

[0031] The purpose of this dictionary is to provide a clear and unambiguous standardized target reference for all non-standard inputs.

[0032] Furthermore, the machine learning mapping model in this embodiment is an intelligent analysis model used to understand and determine which standard semantic in a preset standard attribute dictionary a non-standard attribute should belong to. In this embodiment, the model is preferably a classification model trained based on a logistic regression algorithm. The model construction process includes a data acquisition phase and a model training phase.

[0033] During the data acquisition phase, a large-scale sample dataset (e.g., no less than 1,000 data points) needs to be collected and labeled. Each sample contains the original non-standard attribute descriptions provided by the supplier (e.g., "superior waterproof", "cycle 2 weeks") and their corresponding manually labeled correct standard semantics (e.g., "IPX6", "14 days").

[0034] During the model training phase, this labeled data is input into the logistic regression algorithm for training. The algorithm learns the complex mapping relationship between non-standard text features (such as keywords, phrase patterns, and numerical units) and the target standard semantic classification, and generates a model with discriminative capabilities.

[0035] In one specific implementation, when the cleaned component data (such as "waterproof excellent") is input into the semantic mapping engine 103, the machine learning mapping model first analyzes and identifies it to determine the standard semantic category to which it belongs. Then, based on the output of the model, the engine retrieves and outputs the corresponding standard value (such as "IPX6") from the preset standard attribute dictionary.

[0036] By using the aforementioned pre-defined standard attribute dictionary and pre-defined machine learning mapping model, heterogeneous attribute data from multiple sources can be efficiently and accurately converted into standard semantics, thereby improving the accuracy and efficiency of heterogeneous data semantic normalization.

[0037] The mapping rule iteration module 104 is a functional unit that ensures continuous learning and adaptive expansion capabilities. When a new supplier system is added or the existing supplier data description changes, it can quickly update the mapping model in the semantic mapping engine 103 at minimal cost, thereby improving the overall semantic normalization accuracy of the system.

[0038] The mapping rule iteration module 104 is structurally bidirectional: it is connected to the heterogeneous system adaptation API gateway 101 to actively acquire or receive raw component data samples from the new supplier system as a source of new knowledge for model iteration learning; and it is connected to the semantic mapping engine 103 to inject the learned new knowledge into and update its internal preset machine learning mapping model so that the preset machine learning mapping model can optimize or reconstruct its model parameters.

[0039] In this embodiment, the multi-source supply chain component data processing system adopts a heterogeneous system adaptation API gateway 101 to automatically identify and access component data from at least two types of heterogeneous supplier systems. Then, the data cleaning module 102 performs duplicate value filtering and missing value completion on the received component data, improving data collection efficiency and data quality. Simultaneously, using a preset standard attribute dictionary and a preset machine learning mapping model in the semantic mapping engine 103, the cleaned non-standard component attributes are automatically converted into standard attribute semantics. Finally, based on the mapping rule iteration module 104, the mapping model in the semantic mapping engine 103 is updated according to the component data from newly added supplier systems and manual annotation results, enabling automatic matching and semantic normalization of heterogeneous data, and timely updating of the mapping model, thereby improving the efficiency and accuracy of multi-source supply chain component data processing.

[0040] In some embodiments of the present invention, such as Figure 2 As shown, the data cleaning module 102 includes: The duplicate value filtering unit 201 is used to identify, delete or mark duplicate part data entries based on a dual-key hash index constructed based on part codes and specification information. The missing value completion unit 202 is used to perform a filling operation based on the average of similar historical data for part data with missing attribute values ​​or to initiate a completion request to the corresponding supplier system.

[0041] In some embodiments of the present invention, such as Figure 3 As shown, the multi-source supply chain component data processing system includes a normalized database 301, which is communicatively connected to a semantic mapping engine 103. It is used to store standardized component data after semantic transformation, as well as to store mapping logs between component data from heterogeneous supplier systems and standardized component data.

[0042] Specifically, all component data processed by the semantic mapping engine 103 and converted into unified standard semantics (e.g., tile data with specifications of "1200mm×600mm", environmental protection level of "E1", and waterproof level of "IPX6") is then stored in a normalized database 301 in real time and in a structured manner. This supports subsequent efficient querying and retrieval (e.g., response time ≤ 50ms), enabling stable and rapid provision of standard data services to external systems (such as CAD design systems, ERP procurement systems, and project management systems). This allows downstream business processes such as design, procurement, and cost accounting to obtain a unified and accurate data source, eliminating collaboration barriers caused by heterogeneous data attribute semantics.

[0043] Furthermore, the normalized database 301 is also used to store the mapping log between component data of heterogeneous supplier systems and standardized component data. This embodiment uses this method to achieve traceability.

[0044] Specifically, the normalized database 301 synchronously saves a detailed "mapping log" for each successfully transformed data record. This log records at least the following key information: Raw data: Initial non-standard descriptions obtained from the supplier's system (such as "size 1.2m × 0.6m", "superior waterproof").

[0045] Conversion result: The final corresponding standard semantic value (such as "1200mm×600mm", "IPX6").

[0046] Transformation basis: The specific rule or machine learning model version that triggered this transformation.

[0047] Spatiotemporal information: data source (supplier identifier), conversion timestamp, processing engine identifier, etc.

[0048] The core value of mapping logs lies in providing a complete data traceability chain. When there is doubt about a standardized result, or when it is necessary to audit the compliance and accuracy of the data transformation process, the mapping log can be queried to clearly reconstruct the complete transformation path and decision basis of the data from its original form to the standard form. For example, in the event of a quality dispute, it is possible to trace and confirm whether the original supplier's description of a board material labeled "E1 grade" was "National Standard E1," "European Standard E1," or "Environmentally Friendly," thereby clarifying the attribution of responsibility.

[0049] Figure 4 This is a schematic diagram of an embodiment of the multi-source supply chain component data processing method provided by the present invention, applied to the aforementioned multi-source supply chain component data processing system, such as... Figure 4 As shown, the multi-source supply chain component data processing method includes: S401. Collect raw component data from at least two types of heterogeneous supplier systems through a heterogeneous system adapter API gateway. S402. Use a data cleaning module to clean the original component data to obtain cleaned component data. S403. Input the cleaned component data into the semantic mapping engine. Based on the preset standard attribute dictionary and preset machine learning mapping model in the semantic mapping engine, the non-standard attributes in the cleaned component data are automatically converted into standard attribute semantics.

[0050] This embodiment, through steps S401 to S403, constitutes a complete technical link from the access of raw heterogeneous data to the final output of standardized semantics, aiming to solve problems such as heterogeneous data semantics, low processing efficiency, and poor scalability.

[0051] Step S401 is the data entry point and unified access layer for the entire process. This heterogeneous system adaptation API gateway is a software middleware with intelligent protocol and format recognition capabilities. It can actively or passively interface with at least two types of supplier systems (such as SAP, external vendor private systems, and enterprise-developed systems) with varying technical architectures, communication protocols (such as HTTP and MQTT), and data formats. Through automatic adaptation, it eliminates the need for customized development for each vendor, enabling real-time and accurate collection and aggregation of component data streams from different sources into the system, forming the initial raw component data set. This avoids issues such as multi-source data failing to enter or entering with disorganized formats.

[0052] Step S402 is the preprocessing stage to improve data quality. Raw data typically contains a lot of noise, such as duplicate records (the same part being entered multiple times in different systems) and missing key attribute values ​​(e.g., missing "environmental protection level"). The data cleaning module in this embodiment does not simply filter, but executes an intelligent cleaning strategy. This module first performs high-precision deduplication based on composite keywords such as "part code + specification," identifying and eliminating redundant entries. Subsequently, missing values ​​are processed using a dual mechanism: statistical inference based on historical data (e.g., mean filling) or automatically initiating a completion request to the source system. Through this step, the raw data is transformed into cleaned part data with a unified data format and easy identification.

[0053] In step S403, the cleaned data is sent to the semantic mapping engine. This engine has a built-in standard attribute dictionary and a pre-defined machine learning mapping model.

[0054] The preset standard attribute dictionary is a standard terminology library that stores unified standards within the industry or enterprise (such as the unit of size being "mm", and the environmental protection level being clearly defined as "E0 / E1 / E2").

[0055] The pre-defined machine learning mapping model (e.g., a logistic regression model trained on a large amount of labeled data) can understand and learn the complex mapping relationship between non-standard expressions (such as "waterproof and excellent" and "cycle 2 weeks") and standard semantics (such as "IPX6" and "14 days"). The engine uses this model to intelligently identify and classify the non-standard attributes of the input, and extracts the corresponding standard values ​​from the dictionary based on the classification results for output, thereby achieving high-accuracy (e.g., ≥98%) batch semantic conversion without human intervention.

[0056] This embodiment automates the entire process of data processing for multi-source heterogeneous components, from acquisition to standardized output, thereby improving the efficiency and accuracy of multi-source supply chain component data processing in terms of efficiency, accuracy, and system scalability.

[0057] In some embodiments of the present invention, such asFigure 5 As shown, step S402 uses a data cleaning module to clean the original component data, resulting in cleaned component data including: S501. Obtain the part code and specification information from the original part data; S502. Construct a dual-key hash index based on part codes and specification information; S503. Identify and process duplicate entries in the original component data based on the double-key hash index to obtain deduplicated component data. S504. For entries with missing attribute values ​​in the deduplicated part data, perform a fill operation based on the average of similar historical data or send a completion request to the corresponding supplier system to obtain the cleaned part data.

[0058] This embodiment employs a data cleaning module to automate and intelligently clean the original component data. This cleaning process aims to address the common problems of duplication and missing information in multi-source data, thereby improving the quality and efficiency of subsequent semantic mapping. The specific implementation process is as follows: Step S501 first parses and extracts two key fields from the raw component data stream accessed through the API gateway, which can uniquely identify and describe a component entity: component code (usually a unique ID within the supplier's system) and specification information (text describing the component's specific model, size, material, and other characteristics). These two fields together form the basis for determining data uniqueness and performing subsequent association operations.

[0059] In step S502, to improve the efficiency and accuracy of duplicate identification, a composite key field (dual key) of "part code + specification information" is used. The data cleaning module uses a hash algorithm to generate a unique hash value index for this composite key. This ensures that the same part may have different temporary codes in different systems, but if its specification description is exactly the same, it may still point to the same entity; or, parts with the same code may have different specification variations, which are distinguished into different entries. Building a hash index helps improve the efficiency of subsequent comparison and search.

[0060] In step S503, a pre-built hash index is used to compare each incoming data entry in real time. When a new data entry's hash value is found to completely match an existing record in the index, it is identified as a duplicate entry (including completely duplicate records within the same system and semantically duplicate records across different systems). The specific handling method can be to automatically delete later-arriving duplicate data or mark it as duplicate for auditing purposes, thereby ensuring that each deduplicated part data entry entering subsequent processing is unique. This step isolates invalid and redundant data, reducing the burden on subsequent operations.

[0061] In step S504, for entries with missing attribute values ​​in the deduplicated part data, a filling operation based on the average of similar historical data is performed, or a completion request is sent to the corresponding supplier system to obtain the cleaned part data.

[0062] Specifically, after deduplication, each record is further checked for missing values ​​in key attributes (such as "delivery cycle" and "environmental protection level"). This embodiment employs a dual-mode completion strategy for this purpose: If the system determines that the supplier and component category to which the currently missing attribute belongs has accumulated a sufficient amount of valid data in the historical database (e.g., ≥10 records of the same type of data), it will automatically calculate the average (or mode, etc.) of the historical values ​​of that attribute and fill it in, and record this as the system's inferred value.

[0063] If historical data is insufficient or cannot be inferred, a structured attribute completion request will be automatically initiated to the supplier's system via the API gateway. After the supplier returns information, the entry will be updated based on that information.

[0064] This embodiment uses an automated process to replace the manual investigation and completion work that takes several days in the traditional solution, compressing the data processing time of a single project to a very short time (e.g., within 1 hour), providing high-quality and usable input for the semantic mapping engine, which is conducive to improving the efficiency of semantic normalization processing of heterogeneous data.

[0065] In some embodiments of the present invention, such as Figure 6 As shown, step S403 automatically converts non-standard attributes in the cleaned component data into standard attribute semantics, including: S601. Input the non-standard attributes in the cleaned parts data into a preset machine learning mapping model for semantic recognition to obtain attribute recognition results; S602. Based on the attribute recognition results, match and output the corresponding standard attribute semantics from the preset standard attribute dictionary.

[0066] Specifically, step S601 is a crucial step in achieving intelligent understanding and classification. Each non-standard attribute value (e.g., the text "waterproof excellent," the phrase "cycle 2 weeks," or the mixed expression "European Standard E1") in the cleaned component data is extracted and fed as an input vector into a pre-defined machine learning mapping model. This pre-defined machine learning mapping model is a logistic regression classification model trained on a large amount of labeled data (e.g., ≥1000 "non-standard-standard" paired samples). It has internally learned and encoded rich semantic features and mapping patterns. When a new non-standard attribute is input, the model performs real-time analysis and calculation on its textual features (e.g., keywords, word order, numerical values, unit symbols, etc.), outputting an attribute recognition result. This result is a probability distribution vector or a definitive classification label, indicating the standard semantic category to which the input is most likely to belong (e.g., determining that "waterproof excellent" has a 98% probability of belonging to the "waterproof rating" dimension, further pointing to the category "IPX6").

[0067] It should be noted that the machine learning mapping model preset in this embodiment can be selected and configured according to the specific application scenario, data scale, and different emphases on accuracy and efficiency, including but not limited to Logistic Regression model, Decision Tree model, Random Forest model, or Gradient Boosting Decision Tree (such as XGBoost, LightGBM) model.

[0068] As a preferred approach, this embodiment employs a logistic regression model as the pre-defined machine learning mapping model. The choice of logistic regression is primarily based on its advantages in text classification and semantic mapping tasks: the model's principles are clear, training efficiency is high, and it is suitable for rapid iterative training and online updates on massive amounts of supply chain component data; simultaneously, the output results have probabilistic significance, providing credibility to mapping decisions and facilitating subsequent manual review or rule optimization; furthermore, with sufficient labeled data (e.g., over 1000 high-quality samples), it can learn the complex nonlinear mapping relationship between non-standard attributes and standard semantics, achieving high accuracy (e.g., ≥98%) in classification. In addition, the logistic regression model is relatively lightweight, easily enabling efficient incremental learning and meeting the need for rapid adaptation to new suppliers.

[0069] Step S602 is the precise execution step for completing the standardized output. Based on the attribute recognition results (i.e., the target semantic category) output by the model in the previous step, the engine queries a pre-defined standard attribute dictionary. This dictionary is a structured, predefined set of standards that clearly specifies a unique and unambiguous enumeration of standard values ​​for each dimension that needs to be standardized (such as size, environmental protection level, and waterproof level). The engine retrieves and extracts the corresponding standardized values ​​from the corresponding dimension in the dictionary based on the category indicated by the recognition result.

[0070] The semantic mapping engine in this embodiment is based on a preset standard attribute dictionary and a preset machine learning mapping model. It efficiently converts non-standard attributes from different sources into unified machine-readable standard attribute semantics in batches, thereby improving the efficiency and accuracy of semantic normalization.

[0071] In some embodiments of the present invention, such as Figure 7 As shown, this multi-source supply chain component data processing method also includes: S701. In response to the access of a new supplier system, the sample component data of the new supplier system is collected through the mapping rule iteration module as incremental component data. S702. Obtain the manual standard attribute semantic annotation results for incremental component data through the manual annotation interactive interface in the mapping rule iteration module; S703. Update the machine learning mapping model in the semantic mapping engine based on the annotation results.

[0072] Specifically, the multi-source supply chain component data processing method also includes the mapping model adaptation and expansion process shown in steps S701 to S703. This process is designed to address the inevitable scenario of new supplier access in business practice, enabling the semantic understanding capability of the mapping model to continuously evolve, thereby quickly and accurately adapting to new data sources without large-scale redevelopment or lengthy manual configuration. The specific steps are as follows: When the heterogeneous system adaptation API gateway detects or is configured by the administrator to connect to a new supplier system (e.g., a cabinet supplier that previously used Kingdee ERP), the mapping rule iteration module is automatically triggered. This module proactively collects a certain number (e.g., 50) of representative component data samples from the newly connected system via the API gateway. These sample data constitute incremental component data, unique in that they include the supplier's unique attribute description habits, terminology system, or coding rules (e.g., describing "surface treatment" as "skin-feel membrane pressing" instead of the general "PET film"). This step achieves proactive perception and acquisition of new data patterns.

[0073] Since the attribute descriptions in this incremental component data are not pre-stored in the current model, domain knowledge needs to be introduced for parsing. To this end, the mapping rule iteration module provides a manual annotation interface. The collected incremental component data is presented to the annotators. On this interface, for each non-standard attribute in the incremental data, the annotator, based on their professional judgment, selects or confirms the correct standard semantics from the standard attribute dictionary of the semantic mapping engine (for example, annotating "skin-feel membrane pressure" as "surface treatment: PET coating" in the standard library).

[0074] This manual annotation process provides a clear "question-answer" pair for the machine learning mapping model. After the data semantic normalization process is completed, the "question-answer" pair is updated to the machine learning mapping model, thereby increasing the mapping data pairs in the machine learning mapping model. When processing the component data collected from the supplier system in the future, the updated machine learning mapping model can be used for rapid identification and processing.

[0075] This embodiment updates the mapping model in a timely manner, improving compatibility with new supplier systems and enhancing the efficiency and accuracy of multi-source supply chain component data processing.

[0076] like Figure 8 As shown, the present invention also provides an electronic device 800. The electronic device 800 includes a processor 801, a memory 802, and a display 803. Figure 8 Only some components of the electronic device 800 are shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0077] In some embodiments, processor 801 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 802 or process data, such as the multi-source supply chain component data processing method of the present invention.

[0078] In some embodiments, processor 801 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 801 may be local or remote. In some embodiments, processor 801 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, intranet, multi-cloud, etc., or any combination thereof.

[0079] In some embodiments, memory 802 may be an internal storage unit of electronic device 800, such as a hard disk or memory of electronic device 800. In other embodiments, memory 802 may also be an external storage device of electronic device 800, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 800.

[0080] Furthermore, the memory 802 may include both internal storage units of the electronic device 800 and external storage devices. The memory 802 is used to store application software and various types of data installed on the electronic device 800.

[0081] In some embodiments, display 803 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 803 is used to display information from electronic device 800 and to display a visual user interface. Components 801-803 of electronic device 800 communicate with each other via a system bus.

[0082] In one embodiment, when processor 801 executes the multi-source supply chain component data processing program in memory 802, the following steps can be implemented: Collect raw component data from at least two types of heterogeneous supplier systems through a heterogeneous system adaptation API gateway; The original component data is cleaned using a data cleaning module to obtain cleaned component data; The cleaned component data is input into the semantic mapping engine. Based on the preset standard attribute dictionary and preset machine learning mapping model in the semantic mapping engine, the non-standard attributes in the cleaned component data are automatically converted into standard attribute semantics.

[0083] It should be understood that when the processor 801 executes the multi-source supply chain component data processing program in the memory 802, in addition to the functions mentioned above, it can also perform other functions, as detailed in the description of the corresponding method embodiments above.

[0084] Furthermore, this embodiment of the invention does not specifically limit the type of electronic device 800 mentioned. Electronic device 800 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the invention, electronic device 800 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0085] Accordingly, this application also provides a computer-readable storage medium for storing computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can implement the steps or functions of the multi-source supply chain component data processing method provided in the above-described method embodiments.

[0086] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0087] The multi-source supply chain component data processing system, method, electronic device, and storage medium provided by the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A multi-source supply chain component data processing system, characterized in that, include: A heterogeneous system adapter API gateway is used to automatically identify and access component data from at least two types of heterogeneous supplier systems; The data cleaning module communicates with the heterogeneous system's API gateway and is used to filter duplicate values ​​and fill in missing values ​​in the received component data. The semantic mapping engine, which is connected in communication with the data cleaning module, is used to automatically convert the cleaned non-standard parts attributes into standard attribute semantics based on a preset standard attribute dictionary and a preset machine learning mapping model. The mapping rule iteration module communicates with the heterogeneous system adaptation API gateway and the semantic mapping engine respectively, and is used to update the mapping model in the semantic mapping engine based on the component data of the newly added supplier system and the manual annotation results.

2. The multi-source supply chain component data processing system according to claim 1, characterized in that, The data cleaning module includes: The duplicate value filtering unit is used to identify, delete, or mark duplicate part data entries based on a dual-key hash index built on part codes and specification information. The missing value completion unit is used to perform a filling operation based on the average of similar historical data for part data with missing attribute values, or to initiate a completion request to the corresponding supplier system.

3. The multi-source supply chain component data processing system according to claim 1, characterized in that, The preset machine learning mapping model in the semantic mapping engine is a model trained based on the logistic regression algorithm; the preset standard attribute dictionary stores standard attribute semantics in multiple dimensions.

4. The multi-source supply chain component data processing system according to claim 1, characterized in that, The multi-source supply chain component data processing system includes a normalized database, which is communicatively connected to the semantic mapping engine. The normalized database is used to store standardized component data after semantic transformation, as well as to store mapping logs between component data from heterogeneous supplier systems and standardized component data.

5. A method for processing component data in a multi-source supply chain, characterized in that, The multi-source supply chain component data processing system applied to any one of claims 1 to 4 includes: Collect raw component data from at least two types of heterogeneous supplier systems through a heterogeneous system adaptation API gateway; The original component data is cleaned using a data cleaning module to obtain cleaned component data; The cleaned component data is input into the semantic mapping engine. Based on the preset standard attribute dictionary and preset machine learning mapping model in the semantic mapping engine, the non-standard attributes in the cleaned component data are automatically converted into standard attribute semantics.

6. The multi-source supply chain component data processing method according to claim 5, characterized in that, The original component data is cleaned using a data cleaning module to obtain cleaned component data, including: Obtain the component code and specification information from the original component data; A dual-key hash index is constructed based on the component codes and specifications. The duplicate entries in the original component data are identified and processed according to the dual-key hash index to obtain the deduplicated component data. For entries with missing attribute values ​​in the deduplicated component data, a fill operation based on the average of similar historical data is performed, or a completion request is sent to the corresponding supplier system to obtain the cleaned component data.

7. The multi-source supply chain component data processing method according to claim 5, characterized in that, The step of automatically converting non-standard attributes in the cleaned component data into standard attribute semantics includes: The non-standard attributes in the cleaned component data are input into the preset machine learning mapping model for semantic recognition to obtain attribute recognition results; Based on the attribute recognition results, the corresponding standard attribute semantics are matched and output from the preset standard attribute dictionary.

8. The multi-source supply chain component data processing method according to claim 5, characterized in that, include: In response to the access of a new supplier system, the sample component data of the new supplier system is collected through the mapping rule iteration module as incremental component data. The manual annotation interactive interface in the mapping rule iteration module is used to obtain the manual standard attribute semantic annotation results of the incremental component data; Based on the annotation results, update the machine learning mapping model in the semantic mapping engine.

9. An electronic device, characterized in that, Including memory and processor, among which, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the multi-source supply chain component data processing method according to any one of claims 5 to 8.

10. A computer-readable storage medium, characterized in that, Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps in the multi-source supply chain component data processing method according to any one of claims 5 to 8.

Images