Purchase demand generation method and system based on multi-source data and AI

Abstract

The invention provides a procurement demand generation method and system based on multi-source data and AI in the technical field of digital procurement, and the method comprises the steps: S1, collecting a large amount of procurement data, and carrying out the preprocessing of each piece of procurement data; s2, storing each piece of pre-processed purchase data, and constructing a purchase knowledge graph based on each piece of stored purchase data; s3, creating a purchase entity recognition model and a purchase demand generation model, and performing training; s4, acquiring an input purchase appeal text, inputting the purchase appeal text into the purchase entity recognition model to obtain a purchase entity, and matching associated purchase knowledge from the purchase knowledge graph based on the purchase entity; step S5, performing compliance verification and supplementary perfection on the associated purchase knowledge; and S6, inputting the purchase knowledge into the purchase demand generation model, and generating a purchase demand text. The method has the advantages that the accuracy, the efficiency, the adaptability and the expansibility of purchase demand generation are greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of digital procurement, and particularly to a procurement requirement generation method and system based on multi-source data and AI. Background Art

[0002] During the operation process of each enterprise, unit or organization, it is inevitable to conduct some procurements. Especially for some large enterprises, procurement requirements need to be generated first. However, the generation of procurement requirements faces many severe challenges.

[0003] Firstly, the data sources are extensive and heterogeneous, and the integration difficulty is extremely high. Procurement public information is scattered in procurement networks across different regions, including tender documents, result announcements, etc.; at the same time, data formats such as requirement standards, industry standards, and manufacturer product parameters are diverse and have different structures. For example, the tender document formats released by procurement networks in different regions are not unified. Some are presented in Word documents, while others are in PDF format, and the content layout and expression methods are also very different; the product parameters provided by manufacturers may vary greatly in terms of the indicators and description methods emphasized due to different industries. It is difficult to directly utilize these massive heterogeneous data in a structured manner, which makes procurement personnel spend a lot of time and effort in obtaining and organizing data, and it is easy to have information omissions or errors.

[0004] Secondly, traditional procurement requirement generation methods mainly rely on manual market research and compilation. Procurement personnel need to consult a large amount of information, communicate with numerous suppliers, understand the situation of various goods and services in the market, and manually prepare procurement requirement documents based on their own experience and relevant regulations. This method not only consumes time and effort, but also due to the limitations of manual operations, it is difficult to ensure the comprehensiveness, accuracy, and compliance of procurement requirements. For example, when preparing procurement requirements for large and complex projects, manual research may not cover all relevant market information, resulting in the omission of certain key technical or service requirements in the requirements; at the same time, due to misunderstandings of regulations and industry standards, compliance issues may also occur in procurement requirements.

[0005] Therefore, there is a need to integrate multi-source data and automatically generate procurement requirements according to rules. However, for the automatic generation of procurement requirements, traditionally, it mainly retrieves keywords to find matching procurement documents, and then obtains procurement requirements from these procurement documents, which has the following disadvantages:

[0006] I. Insufficient requirement accuracy:

[0007] 1. Limitations of keyword matching: Relying solely on keyword retrieval can only simply identify whether a specific word is contained in the text, and it cannot understand the exact meaning of the word in the context. For example, in procurement documents, the keyword "high-performance computer" may have different references in different contexts. It could refer to a supercomputer used for scientific research or a computer with relatively high performance in an ordinary office scenario. Simply relying on keyword retrieval is difficult to distinguish, resulting in the retrieved procurement documents may not truly meet the requirements, and thus the obtained requirement information is also inaccurate.

[0008] 2. Lack of semantic understanding: It lacks the ability to conduct in-depth semantic analysis of the text and cannot grasp the logical relationships and potential meanings between the clauses in the procurement documents. For example, for complex technical specification descriptions such as "This device needs to meet the requirement of continuous and stable operation for more than 24 hours in high-temperature and high-humidity environments", keyword retrieval may only be able to identify words such as "device", "high temperature", and "high humidity", but it cannot understand the specific requirements of the device performance when these conditions are combined, resulting in the extracted requirements may miss key information or have a deviation in understanding the requirements.

[0009] II. Insufficient utilization of data:

[0010] 1. Difficulty in integrating multi-source data: Procurement-related data comes from a wide range of sources, including procurement official websites, multi-department industry standard libraries, manufacturer product specification data, etc. These data have diverse formats and different structures. Traditionally, keyword retrieval can usually only be carried out for a single type or a limited number of data sources, and it is difficult to comprehensively integrate and utilize massive heterogeneous data. For example, the bidding documents obtained from the procurement website are in PDF format, while the product parameters provided by manufacturers may be in Excel spreadsheets or unstructured text. Traditionally, there is a lack of effective means to uniformly process and conduct correlation analysis on these different formats of data, resulting in a large amount of valuable data that cannot be fully mined and utilized.

[0011] 2. Lack of knowledge association: The procurement data has not been deeply mined and knowledge association construction has not been carried out. It is just simple data storage and retrieval. For example, for similar items in different procurement projects, the internal connections between their technical parameters and standard specifications have not been established, and an organic knowledge system cannot be formed. When generating new procurement requirements, it is impossible to use the existing knowledge associations to conduct more accurate and comprehensive requirement derivation, and only information can be obtained from a single procurement document in isolation, which limits the quality of requirement generation.

[0012] III. Low generation efficiency:

[0013] 1. Heavy burden of manual screening: After retrieving a large number of procurement documents through keyword search, it is necessary to manually search and extract requirement information from these procurement documents one by one. As the number and length of procurement documents increase, the workload of manual screening is huge and time-consuming. For example, in a large-scale procurement project, hundreds or thousands of relevant procurement documents may be retrieved, and it may take several days or even weeks to manually read and extract requirements, seriously affecting the progress speed of the procurement process.

[0014] 2. Lack of automated generation ability: It is essentially only an auxiliary for data search and does not truly achieve the automated generation of procurement requirements. For complex procurement requirements, such as projects involving multiple items, various technical and commercial requirements, procurement personnel still need to manually sort and compile requirement documents based on the retrieved procurement documents, which is not only inefficient but also prone to omissions and errors.

[0015] IV. Poor adaptability and scalability:

[0016] 1. Difficulty in adapting to changes: The market environment, regulations, and procurement business requirements are constantly changing, and new procurement items, technical requirements, and specification standards are emerging continuously. The traditional method based on fixed keyword search and simple data processing is difficult to quickly adapt to these changes. For example, when new regulations have new provisions on commodity parameters, it is impossible to update the search and analysis rules in a timely manner, resulting in procurement requirements not meeting the latest requirements.

[0017] 2. Limited system scalability: The architecture is relatively closed, lacking flexibility and scalability. When it is necessary to introduce new data sources, adopt new data analysis technologies, or add new functional modules, it is difficult to conveniently upgrade and transform the existing system. For example, if it is necessary to integrate new industry standard data sources or apply more advanced natural language processing technologies to improve semantic understanding ability, the existing system may not be able to achieve it due to architecture limitations, restricting the further development and application of technologies.

[0018] Therefore, how to provide a procurement requirement generation method and system based on multi-source data and AI to improve the accuracy, efficiency, adaptability, and scalability of procurement requirement generation has become an urgent technical problem to be solved. Summary of the Invention

[0019] The technical problem to be solved by the present invention is to provide a procurement requirement generation method and system based on multi-source data and AI to improve the accuracy, efficiency, adaptability, and scalability of procurement requirement generation.

[0020] In the first aspect, the present invention provides a procurement requirement generation method based on multi-source data and AI, including the following steps:

[0021] Step S1: Collect a large amount of multi-source procurement data through API interfaces or data collection tools, and perform preprocessing on each piece of the procurement data, including at least data cleaning, formatting, and structuring.

[0022] Step S2: Store each piece of the preprocessed procurement data, and build a procurement knowledge graph based on the stored procurement data.

[0023] Step S3: Create a procurement entity recognition model and a procurement requirement generation model, and train the procurement entity recognition model and the procurement requirement generation model.

[0024] Step S4: Obtain the input procurement requirement text, input the procurement requirement text into the procurement entity recognition model to obtain the procurement entities carried by the procurement requirement text, and match and associate the procurement knowledge from the procurement knowledge graph based on the procurement entities.

[0025] Step S5: Perform compliance verification and supplementation and improvement on each piece of the associated procurement knowledge.

[0026] Step S6: Input each piece of the procurement knowledge into the procurement requirement generation model to generate a procurement requirement text, and verify the procurement requirement text.

[0027] Further, in Step S1, the data cleaning is specifically as follows: For numerical data in the procurement data, identify outliers by calculating the interquartile range, and correct or delete the outliers; for text data in the procurement data, use regular expressions to remove special characters and stop words, and use a pre-trained language model for semantic error correction.

[0028] The formatting is specifically as follows: Based on the file format of the procurement data, use the corresponding data processing library for formatting to convert it into a unified text format, and segment it according to punctuation marks and paragraph marks; specifically, use PyPDF2 to process PDF-format procurement data, use python-docx to process Word-format procurement data, use BeautifulSoup to process HTML-format procurement data, use the pandas library to process tabular-format procurement data, and use the dateutil library to process date-format procurement data.

[0029] The structuring is specifically as follows:

[0030] Convert semi-structured procurement data into structured procurement data through the ElementTree library; identify entities in unstructured procurement data through a deep learning-based named entity recognition model, and determine entity relationships through dependency syntax analysis to convert unstructured procurement data into structured procurement data.

[0031] Further, step S2 is specifically as follows:

[0032] Store each of the preprocessed procurement data in a relational database, construct a procurement knowledge graph based on the Neo4j graph database, extract procurement knowledge including procurement entities and entity relationships from the stored procurement data, and store the procurement knowledge in the procurement knowledge graph.

[0033] Further, in step S3, the procurement entity recognition model is used to recognize procurement entities from the input text, and is constructed based on a language model, a classifier, a part-of-speech tagging tool, named entity recognition technology, a part-of-speech information assistance model, and an attention mechanism;

[0034] The procurement demand generation model is used to generate a procurement demand text according to the input procurement knowledge, and is constructed based on the Transformer module.

[0035] Further, step S5 is specifically as follows:

[0036] Perform compliance verification on each of the associated procurement knowledge through a rule engine and semantic role labeling technology, and supplement and improve each of the procurement knowledge through an association rule mining algorithm and a generative adversarial network.

[0037] In a second aspect, the present invention provides a procurement demand generation system based on multi-source data and AI, including the following modules:

[0038] A procurement data preprocessing module, configured to collect a large amount of multi-source procurement data through an API interface or a data collection tool, and perform preprocessing on each of the procurement data including at least data cleaning, formatting, and structuring;

[0039] A procurement knowledge graph construction module, configured to store each of the preprocessed procurement data, and construct a procurement knowledge graph based on each of the stored procurement data;

[0040] A model creation module, configured to create a procurement entity recognition model and a procurement demand generation model, and train the procurement entity recognition model and the procurement demand generation model;

[0041] A procurement knowledge matching module, configured to obtain an input procurement requirement text, input the procurement requirement text into the procurement entity recognition model, obtain the procurement entities carried by the procurement requirement text, and match the associated procurement knowledge from the procurement knowledge graph based on the procurement entities;

[0042] A procurement knowledge verification module, configured to perform compliance verification and supplement and improve each of the associated procurement knowledge;

[0043] A procurement requirement generation module, which is used to input each of the procurement knowledge into a procurement requirement generation model, generate a procurement requirement text, and verify the procurement requirement text.

[0044] Further, in the procurement data preprocessing module, the data cleaning is specifically as follows: for numerical data in the procurement data, the interquartile range is calculated to identify outliers, and the outliers are corrected or deleted; for text data in the procurement data, special characters and stop words are removed using regular expressions, and a pre-trained language model is used for semantic error correction.

[0045] The formatting is specifically as follows: based on the file format of the procurement data, the corresponding data processing library is used for formatting to convert it into a unified text format, and segmentation is performed according to punctuation marks and paragraph marks; specifically, PyPDF2 is used to process procurement data in PDF format, python-docx is used to process procurement data in Word format, BeautifulSoup is used to process procurement data in HTML format, the pandas library is used to process procurement data in table format, and the dateutil library is used to process procurement data in date format.

[0046] The structuring is specifically as follows:

[0047] The semi-structured procurement data is converted into structured procurement data through the ElementTree library; for unstructured procurement data, entities are identified through a deep learning-based named entity recognition model, and entity relationships are determined through dependency syntactic analysis to convert the unstructured procurement data into structured procurement data.

[0048] Further, the procurement knowledge graph construction module is specifically used for:

[0049] Each of the preprocessed procurement data is stored through a relational database, a procurement knowledge graph is constructed based on the Neo4j graph database, procurement knowledge including procurement entities and entity relationships is extracted from the stored procurement data, and the procurement knowledge is stored in the procurement knowledge graph.

[0050] Further, in the model creation module, the procurement entity recognition model is used to identify procurement entities from the input text, and is constructed based on a language model, a classifier, a part-of-speech tagging tool, named entity recognition technology, a part-of-speech information assistance model, and an attention mechanism.

[0051] The procurement requirement generation model is used to generate a procurement requirement text based on the input procurement knowledge, and is constructed based on the Transformer module.

[0052] Further, the procurement knowledge verification module is specifically used for:

[0053] Perform compliance verification on each of the associated procurement knowledge through a rules engine and semantic role labeling technology, and supplement and improve each of the procurement knowledge through an association rule mining algorithm and a generative adversarial network.

[0054] The advantages of the present invention are as follows:

[0055] 1. Collect a large amount of multi-source procurement data through an API interface or a data collection tool, and perform preprocessing on each procurement data including at least data cleaning, formatting, and structuring; store the preprocessed procurement data, construct a procurement knowledge graph based on the stored procurement data, create and train a procurement entity recognition model and a procurement demand generation model; then obtain the input procurement requirement text, input the procurement requirement text into the procurement entity recognition model to obtain the procurement entities carried by the procurement requirement text, match and associate the procurement knowledge from the procurement knowledge graph based on the procurement entities, perform compliance verification and supplement and improvement on the associated procurement knowledge, then input each procurement knowledge into the procurement demand generation model to generate a procurement demand text, and verify the procurement demand text; that is, perform the recognition of procurement entities and the generation of procurement demands based on the AI-based procurement entity recognition model and procurement demand generation model, overcome the limitations of traditional keyword matching and the lack of semantic understanding, construct a procurement knowledge graph by obtaining multi-source procurement data, effectively integrate multi-source procurement data and associate each procurement knowledge to make full use of procurement data. The whole process is automatically executed without manual intervention, and subsequently, the procurement knowledge graph, procurement entity recognition model, and procurement demand generation model can be continuously updated and optimized based on the latest data or requirements to adapt to current needs, ultimately greatly improving the accuracy, efficiency, adaptability, and scalability of procurement demand generation.

[0056] 2. By leveraging automated procurement data collection technology, it is possible to quickly obtain the required procurement data from many channels such as government official websites, multi-department industry standards, and manufacturer data, and through an efficient data preprocessing process, convert massive multi-source (heterogeneous) data into an available form; for example, in the generation link, based on a powerful procurement demand generation model and a procurement knowledge graph, it is possible to quickly generate a procurement demand text automatically according to the procurement requirement text, greatly shortening the preparation time and enabling the procurement process to proceed more quickly, greatly improving the overall procurement efficiency.

[0057] 3. By leveraging advanced NLP technologies and procurement knowledge graphs, in-depth semantic analysis and element extraction are performed on procurement data, enabling accurate identification and extraction of key information. At the same time, through matching and verification with industry standards, policies, and regulations, it is ensured that the generated procurement requirement texts are compliant and reasonable. For example, during the semantic analysis and element extraction process, the pre-trained procurement entity recognition model, fine-tuned with procurement domain data, can accurately extract key elements such as technical service requirements and industry-specific qualification requirements, avoiding omissions or errors that may occur during manual extraction. In the compliance verification process, the rule engine strictly checks the requirement content according to policies and regulations to prevent the occurrence of non-compliant clauses and ensure the legal and compliant conduct of procurement activities, greatly improving the quality and reliability of procurement requirements.

[0058] 4. By constructing a procurement knowledge graph, in-depth association and integration of procurement data are achieved, enabling the system to comprehensively understand the relationships between procurement-related information. When generating procurement requirement texts, more accurate and comprehensive content can be provided based on this associated information. For example, through the "product category - parameter - standard - regulation" association structure in the procurement knowledge graph, it is possible to clearly understand the technical parameter standards and regulation requirements corresponding to different product categories, and accurately match relevant parameters and clauses when generating requirements, avoiding problems such as parameter errors or clause omissions, and ensuring that the procurement requirement texts accurately reflect actual requirements and regulatory requirements.

[0059] 5. Automatically generating procurement requirement texts reduces the workload of procurement personnel in data collection, collation, and requirement compilation, and reduces the dependence on a large amount of manual input. Procurement personnel do not need to spend a lot of time and energy on tedious data searching, analysis, and document writing, and can devote more energy to the overall control and strategic decision-making of procurement projects. For example, previously, it might take several procurement personnel several weeks to compile the requirements for a complex procurement project, but this invention enables a small number of procurement personnel to complete it in a short time, greatly reducing labor costs and improving the utilization efficiency of human resources.

[0060] 6. Generating procurement requirement texts based on unified standards and rules helps to standardize and regularize the compilation of procurement requirement texts. Whether it is technical service requirements, commercial terms, or qualification conditions, etc., they are all generated and verified according to established industry standards and policies and regulations, avoiding problems of inconsistent and irregular standards caused by human factors. This not only facilitates the implementation and management of procurement projects but also is conducive to the comparison and evaluation between different procurement projects, enhancing the overall standardization and transparency of procurement activities.

[0061] 7. Continuously collect user revision data and system operation data through user feedback and the log system to optimize the model and the system; as new policies, regulations, industry standards, and commodity information emerge, the procurement knowledge graph can be updated in a timely manner, and the model can also be adjusted accordingly to enable the system to always adapt to the changing procurement environment; at the same time, the system architecture design is flexible, facilitating the integration of new data sources, technology modules, or functional components to achieve system expansion and upgrade to meet the diverse needs of future procurement business development.

[0062] 8. Integrate heterogeneous data sources (PDF / Word / HTML / tables, etc.) with diverse data collection tools through API interfaces, breaking through the limitations of traditional single data sources; adopt differentiated preprocessing strategies (numerical outlier detection + text semantic error correction) to significantly improve the quality of multi-modal data; standardize non-standard data through structured conversion technologies (ElementTree + deep learning NER + dependency syntax analysis) to lay a foundation for knowledge graph construction, ultimately significantly enhancing the multi-source data fusion ability.

[0063] 9. Build a dynamically evolving procurement knowledge graph based on the Neo4j graph database to achieve visual association and in-depth reasoning of entity relationships; in the knowledge matching stage, expand the retrieval dimension through semantic association, breaking through the limitations of traditional keyword matching; the procurement entity relationship network stored in the procurement knowledge graph provides an interpretable decision-making path for procurement demand generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The present invention will be further described below with reference to the accompanying drawings in conjunction with embodiments.

[0065] Figure 1 It is a flowchart of a procurement demand generation method based on multi-source data and AI according to the present invention.

[0066] Figure 2 It is a schematic structural diagram of a procurement demand generation system based on multi-source data and AI according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0067] The overall idea of the technical solution in the embodiments of the present application is as follows: Based on the AI-based procurement entity recognition model and procurement demand generation model, identify procurement entities and generate procurement demands, overcoming the limitations of traditional keyword matching and the lack of semantic understanding. By obtaining multi-source procurement data, build a procurement knowledge graph, effectively integrate multi-source procurement data and associate various procurement knowledge to make full use of procurement data. The entire process is automatically executed without manual intervention, and subsequently, the procurement knowledge graph, procurement entity recognition model, and procurement demand generation model can be continuously updated and optimized based on the latest data or requirements to adapt to current needs, thereby improving the accuracy, efficiency, adaptability, and scalability of procurement demand generation.

[0068] Please refer to Figures 1 to 2 shown in the following, a preferred embodiment of a procurement requirement generation method based on multi-source data and AI according to the present invention includes the following steps:

[0069] Step S1: Collect a large amount of multi-source procurement data through API interfaces or data collection tools, and perform preprocessing on each of the procurement data, including at least data cleaning, formatting, and structuring;

[0070] In the procurement data collection link, adopt an adapted acquisition method for different data sources; for the government procurement official website, if it provides legal ways to obtain data, such as official API interfaces or data download interfaces, preferentially obtain the required government procurement public information, such as tender announcements, result announcements, etc. through such methods; for multi-department standard libraries and manufacturer databases, if they have API interfaces, directly call the API interfaces to obtain data; if the data is only presented in the form of web pages and the data source allows data to be collected through technical means, on the premise of following relevant laws, regulations and data source usage regulations, use legal and compliant data collection tools and technologies for operation; for example, when obtaining product parameter information from a certain manufacturer's official website, with the help of legal and compliant data collection technologies, locate the label positions containing parameters according to the web page HTML structure and extract the data; during the whole process, strictly set a reasonable data acquisition frequency to avoid overloading the data source and ensure the legality and sustainability of the data acquisition behavior.

[0071] Step S2: Store each of the preprocessed procurement data, and build a procurement knowledge graph based on the stored procurement data;

[0072] Step S3: Create a procurement entity recognition model and a procurement requirement generation model, and train the procurement entity recognition model and the procurement requirement generation model;

[0073] Step S4: Obtain the input procurement requirement text, input the procurement requirement text into the procurement entity recognition model to obtain the procurement entities carried by the procurement requirement text, and match and associate the procurement knowledge from the procurement knowledge graph based on the procurement entities;

[0074] Step S5: Perform compliance verification and supplementation and improvement on each of the associated procurement knowledge;

[0075] Step S6: Input each of the procurement knowledge into the procurement requirement generation model to generate a procurement requirement text, and verify the procurement requirement text.

[0076] After generating the procurement requirement text, the rule engine and semantic analysis technology are used again to comprehensively check the generated procurement requirement text according to policies, regulations and industry standards. For example, check whether the industry qualification requirements for suppliers in the text comply with relevant laws and regulations, and whether the technical parameters meet the industry mandatory standards. By matching the key information in the procurement requirement text with the rules and standards in the procurement knowledge graph, judge its compliance. For the parts that do not meet the regulations, automatically mark and give modification suggestions.

[0077] In specific implementation, a cost-benefit analysis model is also used, combined with market price data and project budget information, to evaluate the economy of procurement requirements. For example, by collecting price and performance index data of different brands and models of commodities, a price-performance ratio model is established to judge whether the purchased commodities are cost-effective. At the same time, the constraint satisfaction algorithm (such as the backtracking algorithm) is used to check whether there are conflicts or contradictions between the elements in the procurement requirements, such as whether the delivery time matches the production cycle, and whether the technical requirements match the budget. For the parts that do not meet the requirements, the model automatically adjusts and optimizes to ensure that the generated procurement requirement text is both compliant and practical. Through multiple iterations and optimizations, the quality of the procurement requirement text is continuously improved.

[0078] In specific implementation, a manual review and feedback mechanism is also provided, which provides operation functions such as annotation, modification, and marking for the reviewers on the generated requirement content, collects the revision data, including the modification content, modification reason, usage scenario, etc. of the purchaser, and uses the feedback content as corpus to strengthen the learning of the large model, forming a feedback loop. Classify and code the feedback information of the reviewers, convert it into a format that the model can understand, use this feedback information as new training data, add it to the training set of the model, and retrain and optimize the model. By continuously repeating this process, the model gradually learns the preferences and demand characteristics of users, and the generated procurement requirements are more in line with the actual application scenarios. At the same time, a log system is established to record the key information in the requirement generation process, such as the basic requirements input, the content of the first draft generated, the feedback opinions of the reviewers, the adjustment process of the model, etc., so as to monitor and analyze the system and continuously improve the system performance.

[0079] In step S1, the data cleaning is specifically as follows: for the numerical data in the procurement data, identify outliers by calculating the interquartile range, and correct or delete the outliers; for the text data in the procurement data, use regular expressions to remove special characters and stop words, and use a pre-trained language model for semantic error correction;

[0080] For numerical data, outliers are identified by calculating the interquartile range (IQR = Q3 - Q1, where Q1 is the lower quartile and Q3 is the upper quartile), and data outside the range of Q1 - 1.5IQR or Q3 + 1.5IQR is considered an outlier and processed, such as being corrected or deleted. For text data, in addition to using regular expressions to remove special characters and stop words, a language model is also used for semantic error correction. By learning from a large number of correct texts, the language model can judge the rationality of the words and sentences in the input text and correct semantically incorrect expressions.

[0081] The formatting is specifically as follows: Based on the file format of the procurement data, the corresponding data processing library is used for formatting to convert it into a unified text format, and it is segmented according to punctuation marks and paragraph marks; specifically: use PyPDF2 to process PDF-format procurement data, use python-docx to process Word-format procurement data, use BeautifulSoup to process HTML-format procurement data, use the pandas library to process tabular-format procurement data, and use the dateutil library to process date-format procurement data;

[0082] During the formatting process, different project sub-items are also identified according to the text structure and keywords. For example, paragraphs containing the keyword "technical specifications" are classified as technical specification items.

[0083] The structuring is specifically as follows:

[0084] The semi-structured procurement data (such as standard files in XML format) is converted into structured procurement data through the ElementTree library; for unstructured procurement data, entities are identified through a named entity recognition model (NER) based on deep learning, and entity relationships are determined through dependency syntactic analysis (such as the StanfordCoreNLP tool) to convert the unstructured procurement data into structured procurement data.

[0085] The named entity recognition model can be the BERT-CRF model based on the Transformer architecture to identify entities in the text (such as supplier names, product names, technical parameters, etc.).

[0086] Step S2 is specifically as follows:

[0087] Each of the preprocessed procurement data is stored in a relational database, a procurement knowledge graph is constructed based on the Neo4j graph database, procurement knowledge including procurement entities and entity relationships is extracted from the stored procurement data, and the procurement knowledge is stored in the procurement knowledge graph.

[0088] MySQL or PostgreSQL can be used as a relational database, and the database table structure can be designed according to the category and attributes of the data; for example, a "tender requirements table" is created to store structured information related to the tender, including fields such as project name, item, technical specifications, and qualification requirements; a "scoring criteria table" is created to record evaluation methods and standard-related data; through reasonable primary key and foreign key settings, the relationship between tables is established to ensure data consistency and integrity.

[0089] In specific implementation, for semi-structured text information, such as some free text description parts in the bidding documents, document databases such as MongoDB can be used for storage; each document is treated as an independent record and stored in JSON format to facilitate flexible query and management of text content.

[0090] The procurement knowledge graph takes "items", "parameters", "standards", "regulations", etc. as nodes (procurement entities), and the relationships between them (such as "items-include-parameters", "parameters-comply-with-standards", "standards-based-regulations") as edges (entity relationships); adds detailed attributes to each node and edge, such as the classification information of the item, the specific numerical range of the parameter, the issuing agency and date of the standard, the content of the provisions of the regulation, etc.; in the construction process, the entity linking technology is used to calculate the similarity between the text entity and the knowledge base entity (using the cosine similarity algorithm based on word vectors, representing the entity in vector form, and calculating the cosine value of the angle between vectors). When the similarity exceeds the set threshold (such as 0.8), a link is established, thereby integrating data from different sources into the procurement knowledge graph.

[0091] In the specific implementation, we can deeply study the classification standards of the items and the contents of the standards of various industries, and define the core domain ontology based on this. Taking the "office equipment" item as an example, we can further subdivide the subcategories such as "printer" and "laptop computer", and clarify the key attributes and relationships under each subcategory. For example, the attributes of "printer" include printing speed, resolution, consumable type, etc., and there is an association relationship with commodities such as "ink cartridge". By establishing this hierarchical classification system and ontology structure, a clear framework is provided for the procurement knowledge graph. Natural language processing technology (NLP) can also be used to perform semantic analysis on existing knowledge items. Through lexical analysis, syntactic analysis and semantic role labeling, the meaning of each entity and relationship in the text can be understood, so as to help the system better grasp the internal connection between different technical parameters and business terms under the same item. For example, for the description of "a certain model of printer requires a specific specification of ink cartridge", the usage relationship between "printer" and "ink cartridge" is determined through semantic analysis, and the corresponding edge and attribute description is established in the procurement knowledge graph. At the same time, with the help of the reasoning mechanism of the procurement knowledge graph, reasoning is performed based on the existing entities and relationships, potential semantic associations are mined, and the content of the procurement knowledge graph is enriched.

[0092] In step S3, the procurement entity recognition model is used to recognize procurement entities from the input text, and is constructed based on a language model, a classifier, a part-of-speech tagging tool, a named entity recognition technology, a part-of-speech information assistance model, and an attention mechanism;

[0093] That is, a pre-trained language model (such as BERT, GPT, Qwen, DeepSeek, etc.) is fine-tuned, and for the characteristics of the procurement field, a large amount of labeled procurement text data (including tender documents, result announcements, etc.) is used for supervised learning; a specific classifier (such as a multi-layer perceptron MLP) is added to the output layer of the procurement entity recognition model to map the output of the procurement entity recognition model to the corresponding entity category or element type; for example, through the fine-tuned procurement entity recognition model, it can accurately judge which words in the text represent technical specifications and which represent qualification requirements, etc.; at the same time, combined with a part-of-speech tagging tool (using NLTK or spaCy tools) and a named entity recognition technology (NER), the part-of-speech information assistance model (NER) is used to more accurately identify entities. For example, by part-of-speech tagging to determine that a certain word is a noun, combined with the context and procurement field knowledge, it can more accurately judge whether it is a commodity name; and an attention mechanism (Attention Mechanism) is introduced to improve the model's ability to understand the context and accurately grasp the meaning of each entity and element in a specific context.

[0094] The procurement requirement generation model is used to generate procurement requirement texts based on the input procurement knowledge and is constructed based on the Transformer module.

[0095] Adopt a vertical industry large model (Transformer architecture) trained for the procurement field. During the generation process, use the extracted elements as input, and design appropriate prompts through Prompt engineering to guide the procurement demand generation model to generate procurement demand texts that meet the requirements. For example, the prompt can be "Generate a complete procurement demand based on the following procurement items, technical requirements, and commercial terms: [specific item], [detailed description of technical requirements], [description of commercial terms]". The model encodes, transforms, and generates the input information through the self-attention mechanism and feed-forward neural network in multiple Transformer modules. After learning the language patterns of a large number of procurement texts, the procurement demand generation model can generate natural and fluent procurement demand texts that conform to the expression habits of the procurement field based on elements such as the input commodity name and technical requirements. To make the generated procurement demand texts more in line with the needs of specific procurement projects, add project-specific datasets for fine-tuning during the training process of the procurement demand generation model, and at the same time combine the method of template fusion to combine the general tender template paragraphs with personalized content to ensure that the generated demand texts are both professional and meet the project's personalized needs. For example, for different types of procurement projects (such as goods procurement, service procurement, etc.), use the corresponding general template framework, and then fill in the personalized content such as specific technical indicators and commercial terms generated by the model into the template to improve the generation efficiency and text quality.

[0096] The specific steps of step S5 are as follows:

[0097] Perform compliance verification on each of the associated procurement knowledge through a rule engine and semantic role labeling technology, and supplement and improve each of the procurement knowledge through an association rule mining algorithm and a generative adversarial network.

[0098] That is, use semantic role labeling technology (Semantic Role Labeling, SRL) to determine the semantic roles (such as agent, patient, time, place, etc.) of each predicate (verb) in the text (procurement knowledge), and deeply understand the deep semantics of the text. For example, for the sentence "The supplier shall deliver the goods within 30 days after the contract is signed", SRL can label "the supplier" as the agent, "the goods" as the patient, and "within 30 days after the contract is signed" as the time, and then match this semantic information with the rules in the industry standards and policy and regulation knowledge bases. The rule engine adopts a rule-based reasoning system (such as Drools) to transform policies, regulations, and industry standards into a set of rules, such as the rule "If the procurement project is of type A, then the supplier must have qualification B and the product must meet standard C". Through forward or backward reasoning mechanisms, perform compliance verification on the procurement demand. When a non-compliant situation is found, the system triggers a warning and recommends more reasonable / compliant parameters or terms based on the rules.

[0099] Based on association rule mining algorithms (such as the Apriori algorithm), the association relationships between different elements are mined from the procurement knowledge graph. For example, by analyzing a large amount of historical procurement data, it is found that there is a certain association rule between "the procurement quantity of a certain office equipment" and "the corresponding after-sales service period". When a certain element in the procurement requirement is missing, according to the mined association rules, relevant information is automatically supplemented from the procurement knowledge graph. At the same time, a generative adversarial network (GAN) is used to generate possible supplementary content, and the discriminator judges its rationality, continuously optimizing the generated content to make it meet the actual requirements; the discriminator can judge the semantics and grammar of the content generated by the generator based on a pre-trained language model, and feedback to the generator for adjustment until the generated content meets the requirements.

[0100] A preferred embodiment of a procurement requirement generation system based on multi-source data and AI of the present invention includes the following modules:

[0101] The procurement data preprocessing module is used to collect a large amount of multi-source procurement data through API interfaces or data collection tools, and perform preprocessing on each of the procurement data including at least data cleaning, formatting, and structuring;

[0102] In the procurement data collection link, an adapted acquisition method is adopted for different data sources; for the government procurement official website, if it provides a legal way to obtain data, such as an official API interface or a data download interface, this type of method is preferred to obtain the required government procurement public information, such as tender announcements, result announcements, etc.; for multi-department standard libraries and manufacturer databases, if they have API interfaces, directly call the API interfaces to obtain data; if the data is only presented in the form of a web page and this data source allows data to be collected by technical means, on the premise of following relevant laws, regulations and data source usage regulations, legal and compliant data collection tools and technologies are used for operation; for example, when obtaining product parameter information from a certain manufacturer's official website, with the help of data collection technology that complies with legal norms, the position of the label containing the parameters is located according to the web page HTML structure and the data is extracted; throughout the process, a reasonable data acquisition frequency is strictly set to avoid overloading the data source and ensure the legality and sustainability of the data acquisition behavior.

[0103] The procurement knowledge graph construction module is used to store each of the preprocessed procurement data and construct a procurement knowledge graph based on the stored procurement data;

[0104] The model creation module is used to create a procurement entity recognition model and a procurement requirement generation model, and train the procurement entity recognition model and the procurement requirement generation model;

[0105] A procurement knowledge matching module, which is used to obtain the input procurement requirement text, input the procurement requirement text into a procurement entity recognition model to obtain the procurement entities carried by the procurement requirement text, and match the associated procurement knowledge from the procurement knowledge graph based on the procurement entities;

[0106] A procurement knowledge verification module, which is used to conduct compliance verification and supplementation and improvement on each piece of the associated procurement knowledge;

[0107] A procurement demand generation module, which is used to input each piece of the procurement knowledge into a procurement demand generation model to generate a procurement demand text and verify the procurement demand text.

[0108] That is, after generating the procurement demand text, the rule engine and semantic analysis technology are used again to comprehensively check the generated procurement demand text according to policies, regulations and industry standards. For example, check whether the industry qualification requirements for suppliers in the text comply with relevant laws and regulations, and whether the technical parameters meet the industry mandatory standards, etc. By matching the key information in the procurement demand text with the rules and standards in the procurement knowledge graph, judge its compliance. For the parts that do not meet the regulations, automatically mark them and give modification suggestions.

[0109] In specific implementation, a cost-benefit analysis model is also used, combined with market price data and project budget information, to evaluate the economy of the procurement demand. For example, by collecting price and performance index data of different brands and models of goods, establish a price-performance ratio model to judge whether the purchased goods are cost-effective. At the same time, use a constraint satisfaction algorithm (such as a backtracking algorithm) to check whether there are conflicts or contradictions between the elements in the procurement demand, such as whether the delivery time matches the production cycle, and whether the technical requirements match the budget. For the parts that do not meet the requirements, the model automatically adjusts and optimizes to ensure that the generated procurement demand text is both compliant and practical. Through multiple iterations and optimizations, continuously improve the quality of the procurement demand text.

[0110] In specific implementation, an artificial review and feedback mechanism is also provided, which provides functions such as annotation, modification, and marking for reviewers to operate on the generated requirement content, collects revision data, including information such as the modification content, modification reason, and usage scenario of the purchaser, and uses the feedback content as corpus to strengthen the learning of the large model, forming a feedback loop. Classify and code the feedback information of the reviewers, convert it into a format that the model can understand, use this feedback information as new training data, add it to the training set of the model, and retrain and optimize the model. By continuously repeating this process, the model gradually learns the preferences and requirement characteristics of users, and the generated procurement requirements are more in line with the actual application scenario. At the same time, a log system is established to record key information in the requirement generation process, such as the basic requirements input, the initial draft content generated, the feedback opinions of the reviewers, the adjustment process of the model, etc., so as to monitor and analyze the system and continuously improve the system performance.

[0111] In the procurement data preprocessing module, the data cleaning is specifically as follows: for numerical data in the procurement data, calculate the interquartile range to identify outliers, and correct or delete the outliers; for text data in the procurement data, use regular expressions to remove special characters and stop words, and use a pre-trained language model for semantic error correction;

[0112] For numerical data, calculate the interquartile range (IQR = Q3 - Q1, where Q1 is the lower quartile and Q3 is the upper quartile) to identify outliers, and consider the data outside the range of Q1 - 1.5IQR or Q3 + 1.5IQR as outliers and process them, such as correction or deletion. For text data, in addition to using regular expressions to remove special characters and stop words, a language model is also used for semantic error correction. The language model can judge the rationality of words and sentences in the input text by learning a large number of correct texts and correct the expressions with semantic errors.

[0113] The formatting is specifically as follows: based on the file format of the procurement data, use the corresponding data processing library for formatting to convert it into a unified text format, and segment it according to punctuation marks and paragraph marks; specifically: use PyPDF2 to process PDF-format procurement data, use python-docx to process Word-format procurement data, use BeautifulSoup to process HTML-format procurement data, use the pandas library to process tabular-format procurement data, and use the dateutil library to process date-format procurement data;

[0114] During the formatting process, different project sub-items are also identified according to the text structure and keywords, such as classifying paragraphs containing the keyword "technical specifications" as technical specification items.

[0115] The structuring is specifically as follows:

[0116] Convert the semi-structured procurement data into structured procurement data (such as a standard file in XML format) through the ElementTree library; identify the entities in the unstructured procurement data through a named entity recognition model (NER) based on deep learning, and determine the entity relationships through dependency syntactic analysis (such as the Stanford CoreNLP tool) to convert the unstructured procurement data into structured procurement data.

[0117] The named entity recognition model can optionally be a BERT-CRF model based on the Transformer architecture to identify the entities in the text (such as supplier names, product names, technical parameters, etc.).

[0118] The procurement knowledge graph construction module is specifically used for:

[0119] Store each of the preprocessed procurement data through a relational database, construct a procurement knowledge graph based on the Neo4j graph database, extract procurement knowledge including procurement entities and entity relationships from the stored procurement data, and store the procurement knowledge in the procurement knowledge graph.

[0120] The relational database can select MySQL or PostgreSQL, and design the database table structure according to the category and attributes of the data; for example, create a "tendering requirements table" to store structured information related to tenders, including fields such as project name, item number, technical specifications, and qualification requirements; a "scoring criteria table" to record data related to evaluation methods and criteria; establish the association relationship between tables through reasonable primary key and foreign key settings to ensure data consistency and integrity.

[0121] In specific implementation, for semi-structured text information, such as some free text description parts in tender documents, a document database such as MongoDB can be used for storage; each document is used as an independent record and stored in JSON format to facilitate flexible query and management of the text content.

[0122] The procurement knowledge graph takes "items", "parameters", "standards", "regulations", etc. as nodes (procurement entities), and the relationships between them (such as "items-include-parameters", "parameters-comply-with-standards", "standards-based-regulations") as edges (entity relationships); adds detailed attributes to each node and edge, such as the classification information of the item, the specific numerical range of the parameter, the issuing agency and date of the standard, the content of the provisions of the regulation, etc.; in the construction process, the entity linking technology is used to calculate the similarity between the text entity and the knowledge base entity (using the cosine similarity algorithm based on word vectors, representing the entity in vector form, and calculating the cosine value of the angle between vectors). When the similarity exceeds the set threshold (such as 0.8), a link is established, thereby integrating data from different sources into the procurement knowledge graph.

[0123] In the specific implementation, we can deeply study the classification standards of the items and the contents of the standards of various industries, and define the core domain ontology based on this. Taking the "office equipment" item as an example, we can further subdivide the subcategories such as "printer" and "laptop computer", and clarify the key attributes and relationships under each subcategory. For example, the attributes of "printer" include printing speed, resolution, consumable type, etc., and there is an association relationship with commodities such as "ink cartridge". By establishing this hierarchical classification system and ontology structure, a clear framework is provided for the procurement knowledge graph. Natural language processing technology (NLP) can also be used to perform semantic analysis on existing knowledge items. Through lexical analysis, syntactic analysis and semantic role labeling, the meaning of each entity and relationship in the text can be understood, so as to help the system better grasp the internal connection between different technical parameters and business terms under the same item. For example, for the description of "a certain model of printer requires a specific specification of ink cartridge", the usage relationship between "printer" and "ink cartridge" is determined through semantic analysis, and the corresponding edge and attribute description is established in the procurement knowledge graph. At the same time, with the help of the reasoning mechanism of the procurement knowledge graph, reasoning is performed based on the existing entities and relationships, potential semantic associations are mined, and the content of the procurement knowledge graph is enriched.

[0124] In the model creation module, the procurement entity identification model is used to identify the procurement entity from the input text, and is constructed based on a language model, a classifier, a part-of-speech tagging tool, a named entity recognition technology, a part-of-speech information auxiliary model, and an attention mechanism;

[0125] That is, fine-tune using pre-trained language models (such as BERT, GPT, Qwen, DeepSeek, etc.). According to the characteristics of the procurement field, use a large amount of labeled procurement text data (including tender documents, result announcements, etc.) for supervised learning; add a specific classifier (such as a multi-layer perceptron MLP) to the output layer of the procurement entity recognition model to map the output of the procurement entity recognition model to the corresponding entity category or element type; for example, through the fine-tuned procurement entity recognition model, it can accurately judge which words in the text represent technical specifications and which represent qualification requirements, etc.; at the same time, combine part-of-speech tagging tools (using NLTK or spaCy tools) and named entity recognition technology (NER), and use part-of-speech information to assist the model (NER) to more accurately identify entities. For example, determine that a certain word is a noun through part-of-speech tagging, and combine the context and procurement field knowledge to more accurately judge whether it is a commodity name; and introduce an attention mechanism to improve the model's understanding ability of the context and accurately grasp the meaning of each entity and element in a specific context.

[0126] The procurement requirement generation model is used to generate procurement requirement texts based on the input procurement knowledge and is constructed based on the Transformer module.

[0127] Adopt a vertical industry large model (Transformer architecture) trained for the procurement field. During the generation process, use the extracted elements as input, and design appropriate prompts through Prompt engineering to guide the procurement requirement generation model to generate procurement requirement texts that meet the requirements. For example, the prompt can be "Generate a complete procurement requirement based on the following procurement item, technical requirements, and commercial conditions: [specific item], [detailed description of technical requirements], [description of commercial conditions]". The model encodes, transforms, and generates the input information through the self-attention mechanism and the feed-forward neural network in the multi-layer Transformer module. After learning the language patterns of a large amount of procurement texts, the procurement requirement generation model can generate natural and fluent procurement requirement texts that conform to the expression habits of the procurement field based on elements such as the input commodity name and technical requirements. In order to make the generated procurement requirement texts more in line with the requirements of specific procurement projects, add project-specific data sets for fine-tuning during the training process of the procurement requirement generation model, and at the same time combine the method of template fusion to combine the general tender template paragraphs with personalized content to ensure that the generated requirement texts are both professional and meet the project's personalized needs. For example, for different types of procurement projects (such as goods procurement, service procurement, etc.), use the corresponding general template framework, and then fill in the personalized content such as specific technical indicators and commercial terms generated by the model into the template to improve the generation efficiency and text quality.

[0128] The procurement knowledge verification module is specifically used for:

[0129] Perform compliance verification on each of the associated procurement knowledge through a rule engine and semantic role labeling technology, and supplement and improve each of the procurement knowledge through an association rule mining algorithm and a generative adversarial network.

[0130] That is, using semantic role labeling technology (Semantic Role Labeling, SRL), determine the semantic roles (such as agent, patient, time, place, etc.) of each predicate (verb) in the text (procurement knowledge), and deeply understand the deep semantics of the text. For example, for the sentence "The supplier shall deliver the goods within 30 days after the contract is signed", SRL can label "the supplier" as the agent, "the goods" as the patient, and "within 30 days after the contract is signed" as the time, and then match this semantic information with the rules in the industry standards and policy and regulation knowledge bases. The rule engine adopts a rule-based reasoning system (such as Drools) to transform policies, regulations and industry standards into a set of rules, such as the rule "If the procurement project is of type A, then the supplier must have qualification B and the product must meet standard C". Through a forward reasoning or backward reasoning mechanism, perform compliance verification on procurement requirements. When a non-compliant situation is found, the system triggers a warning and recommends more reasonable / compliant parameters or terms according to the rules.

[0131] Based on an association rule mining algorithm (such as the Apriori algorithm), mine the association relationships between different elements from the procurement knowledge graph. For example, by analyzing a large amount of historical procurement data, it is found that there is a certain association rule between "the procurement quantity of a certain office equipment" and "the corresponding after-sales service period". When a certain element in the procurement requirement is missing, according to the mined association rules, automatically supplement relevant information from the procurement knowledge graph. At the same time, use a generative adversarial network (GAN) to generate possible supplementary content, and the discriminator judges its rationality, continuously optimizing the generated content to make it meet the actual requirements; the discriminator can, based on a pre-trained language model, perform semantic and syntactic judgments on the content generated by the generator and feedback it to the generator for adjustment until the generated content meets the requirements.

[0132] To facilitate the understanding of the present invention, the following examples are given for further illustration:

[0133] Input: The purchaser fills in the procurement requirement text through a visual interface, including information such as project name, budget, procurement category, usage scenario, etc.; after the system receives the input procurement requirement text, retrieve the corresponding item parameters and requirements from the procurement knowledge graph according to the procurement category. For example, if the procurement category is "printer", the system queries information such as relevant technical parameters, qualification requirements, and commercial terms of the "printer" item in the procurement knowledge graph.

[0134] Dynamic assembly: According to the matching results and the judgment of the rule engine, dynamically combine technical indicators, service requirements, qualification conditions, business terms, etc. For example, obtain the technical parameters of different brand printers from the procurement knowledge graph, combine the project budget and usage scenarios, and select appropriate technical parameters for combination; determine the qualification conditions and business terms according to policies, regulations and industry standards. At the same time, use the rule engine to conduct a preliminary verification of the combined content to ensure its compliance and rationality. For example, check whether the selected technical parameters meet the industry standards and whether the qualification conditions violate relevant policies and regulations.

[0135] Automatic polishing and structured output: Generate a draft of the procurement requirement text through the procurement requirement generation model, polish the draft for sentence coherence and compliance, adjust the sentences in the draft to make it more natural and fluent and conform to the expression habits in the procurement field, and at the same time optimize the structure again, such as automatically splitting clauses, adding or merging duplicates, etc. For example, split long clauses into multiple clear sub-clauses, merge duplicate technical parameters, and arrange them in a certain logical order to form a structured procurement requirement text.

[0136] In summary, the advantages of the present invention are as follows:

[0137] 1. Collect a large amount of multi-source procurement data through API interfaces or data collection tools, and perform preprocessing on each procurement data including at least data cleaning, formatting, and structuring; store the preprocessed procurement data, construct a procurement knowledge graph based on the stored procurement data, create and train a procurement entity recognition model and a procurement requirement generation model; then obtain the input procurement requirement text, input the procurement requirement text into the procurement entity recognition model to obtain the procurement entities carried by the procurement requirement text, match and associate the procurement knowledge related to the procurement entities from the procurement knowledge graph, conduct compliance verification and supplementary improvement on the associated procurement knowledge, and then input each procurement knowledge into the procurement requirement generation model to generate the procurement requirement text and verify the procurement requirement text; that is, identify procurement entities and generate procurement requirements based on the AI-based procurement entity recognition model and procurement requirement generation model, overcome the limitations of traditional keyword matching and the lack of semantic understanding, effectively integrate multi-source procurement data by constructing a procurement knowledge graph, associate each procurement knowledge, and make full use of the procurement data. The whole process is automatically executed without manual intervention, and subsequently, the procurement knowledge graph, procurement entity recognition model, and procurement requirement generation model can be continuously updated and optimized based on the latest data or requirements to adapt to current needs, ultimately greatly improving the accuracy, efficiency, adaptability, and scalability of procurement requirement generation.

[0138] 2. By leveraging automated procurement data collection technologies, the required procurement data can be quickly obtained from multiple channels such as government official websites, multi-department industry standards, and manufacturer data. Through an efficient data preprocessing process, massive multi-source (heterogeneous) data can be transformed into a usable form. For example, in the generation phase, based on a powerful procurement demand generation model and a procurement knowledge graph, the procurement demand text can be automatically generated quickly according to the procurement requirement text, significantly shortening the preparation time, enabling the procurement process to progress more rapidly, and greatly improving the overall procurement efficiency.

[0139] 3. By using advanced NLP technologies and a procurement knowledge graph to conduct in-depth semantic analysis and element extraction on procurement data, key information can be accurately identified and extracted. At the same time, through matching and verification with industry standards, policies, and regulations, it is ensured that the generated procurement demand text is compliant and reasonable. For example, during the semantic analysis and element extraction process, a pre-trained procurement entity recognition model, fine-tuned with procurement domain data, can accurately extract key elements such as technical service requirements and industry-specific qualification requirements, avoiding omissions or errors that may occur in manual extraction. In the compliance verification phase, the rule engine strictly checks the requirement content according to policies and regulations to prevent the occurrence of non-compliant clauses, ensuring that procurement activities are carried out legally and compliantly, and greatly enhancing the quality and reliability of procurement requirements.

[0140] 4. By constructing a procurement knowledge graph, the deep association and integration of procurement data are achieved, enabling the system to comprehensively understand the relationships between procurement-related information. When generating the procurement demand text, more accurate and comprehensive content can be provided based on these associated information. For example, through the "product category - parameters - standards - regulations" association structure in the procurement knowledge graph, the technical parameter standards and regulatory requirements corresponding to different product categories can be clearly understood, and relevant parameters and clauses can be accurately matched when generating requirements, avoiding problems such as parameter errors or clause omissions, and ensuring that the procurement demand text accurately reflects the actual needs and regulatory requirements.

[0141] 5. By automatically generating the procurement demand text, the workload of procurement personnel in data collection, collation, and requirement preparation is reduced, and the dependence on a large amount of manual input is lowered. Procurement personnel do not need to spend a lot of time and energy on tedious data searching, analysis, and document writing, and can invest more energy in the overall control and strategic decision-making of procurement projects. For example, in the past, it might take several procurement personnel several weeks to prepare the requirements for a complex procurement project, but this invention enables a small number of procurement personnel to complete it in a short time, greatly reducing labor costs and improving the utilization efficiency of human resources.

[0142] 6. Generating procurement requirement texts based on unified standards and rules helps to standardize and regulate the compilation of procurement requirement texts. Whether it is technical service requirements, business terms, or qualification conditions, etc., they are generated and verified in accordance with established industry standards and policies and regulations, avoiding problems of inconsistent and non-standard standards caused by human factors. This not only facilitates the implementation and management of procurement projects but also benefits the comparison and evaluation between different procurement projects, enhancing the overall standardization and transparency of procurement activities.

[0143] 7. Continuously collect user revision data and system operation data through user feedback and log systems to optimize the model and system. With the emergence of new policies and regulations, industry standards, and commodity information, the procurement knowledge graph can be updated in a timely manner, and the model can also be adjusted accordingly, enabling the system to always adapt to the changing procurement environment. At the same time, the system architecture design is flexible, facilitating the integration of new data sources, technology modules, or functional components to achieve system expansion and upgrade to meet the diverse needs of future procurement business development.

[0144] 8. Integrate heterogeneous data sources (PDF / Word / HTML / tables, etc.) with diverse data collection tools through API interfaces, breaking through the limitations of traditional single data sources. Adopt differential preprocessing strategies (numerical outlier detection + text semantic error correction) to significantly improve the quality of multimodal data. Achieve the standardization of non-standard data through structured conversion technologies (ElementTree + deep learning NER + dependency syntax analysis), laying a foundation for knowledge graph construction, and ultimately significantly enhancing the multi-source data fusion ability.

[0145] 9. Build a dynamically evolving procurement knowledge graph based on the Neo4j graph database to achieve visual association and in-depth reasoning of entity relationships. In the knowledge matching stage, expand the retrieval dimension through semantic association, breaking through the limitations of traditional keyword matching. The procurement entity relationship network stored in the procurement knowledge graph provides an interpretable decision-making path for procurement requirement generation.

[0146] Although the specific implementation manners of the present invention are described above, those skilled in the art of this technology should understand that the specific embodiments we described are illustrative rather than used to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be covered by the scope protected by the claims of the present invention.

Claims

1. A procurement requirement generation method based on multi-source data and AI, characterized in that: Including the following steps: Step S1: Collect a large amount of multi-source procurement data through an API interface or a data collection tool, and perform preprocessing on each of the procurement data, including at least data cleaning, formatting, and structuring; Step S2: Store each of the preprocessed procurement data, and construct a procurement knowledge graph based on the stored procurement data; Step S3: Create a procurement entity recognition model and a procurement requirement generation model, and train the procurement entity recognition model and the procurement requirement generation model; Step S4: Obtain the input procurement requirement text, input the procurement requirement text into the procurement entity recognition model to obtain the procurement entities carried by the procurement requirement text, and match the associated procurement knowledge from the procurement knowledge graph based on the procurement entities; Step S5: Perform compliance verification and supplementation and improvement on each of the associated procurement knowledge; Step S6: Input each of the procurement knowledge into the procurement requirement generation model to generate a procurement requirement text, and verify the procurement requirement text.

2. The procurement requirement generation method based on multi-source data and AI according to claim 1, characterized in that: In step S1, the data cleaning is specifically as follows: for numerical data in the procurement data, identify outliers by calculating the interquartile range, and correct or delete the outliers; for text data in the procurement data, use regular expressions to remove special characters and stop words, and use a pre-trained language model for semantic error correction; The formatting is specifically as follows: based on the file format of the procurement data, use the corresponding data processing library for formatting to convert it into a unified text format, and segment it according to punctuation marks and paragraph marks; specifically: use PyPDF2 to process PDF-format procurement data, use python-docx to process Word-format procurement data, use BeautifulSoup to process HTML-format procurement data, use the pandas library to process tabular-format procurement data, and use the dateutil library to process date-format procurement data; The structuring is specifically as follows: Convert semi-structured procurement data into structured procurement data through the ElementTree library; identify entities in unstructured procurement data through a deep learning-based named entity recognition model, and determine entity relationships through dependency syntax analysis to convert unstructured procurement data into structured procurement data.

3. The procurement requirement generation method based on multi-source data and AI according to claim 1, characterized in that: Step S2 is specifically as follows: Store each of the preprocessed procurement data through a relational database, construct a procurement knowledge graph based on the Neo4j graph database, extract procurement knowledge including procurement entities and entity relationships from the stored procurement data, and store the procurement knowledge in the procurement knowledge graph.

4. A procurement requirement generation method based on multi-source data and AI according to claim 1, characterized in that: In step S3, the procurement entity recognition model is used to identify procurement entities from the input text, and is constructed based on a language model, a classifier, a part-of-speech tagging tool, a named entity recognition technology, a part-of-speech information assistance model, and an attention mechanism; The procurement requirement generation model is used to generate a procurement requirement text based on the input procurement knowledge, and is constructed based on the Transformer module.

5. The procurement requirement generation method based on multi-source data and AI according to claim 1, characterized in that: Step S5 is specifically as follows: Perform compliance verification on each of the associated procurement knowledge through a rules engine and semantic role labeling technology, and supplement and improve each of the procurement knowledge through an association rule mining algorithm and a generative adversarial network.

6. A procurement requirement generation system based on multi-source data and AI, characterized in that: It includes the following modules: A procurement data preprocessing module, which is used to collect a large amount of multi-source procurement data through an API interface or a data collection tool, and perform preprocessing on each of the procurement data, including at least data cleaning, formatting, and structuring; A procurement knowledge graph construction module, which is used to store each of the preprocessed procurement data, and construct a procurement knowledge graph based on the stored procurement data; A model creation module, which is used to create a procurement entity recognition model and a procurement demand generation model, and train the procurement entity recognition model and the procurement demand generation model; A procurement knowledge matching module, which is used to obtain the input procurement requirement text, input the procurement requirement text into the procurement entity recognition model to obtain the procurement entities carried by the procurement requirement text, and match the associated procurement knowledge from the procurement knowledge graph based on the procurement entities; A procurement knowledge verification module, which is used to perform compliance verification, supplementation, and improvement on each of the associated procurement knowledge; A procurement demand generation module, which is used to input each of the procurement knowledge into the procurement demand generation model to generate a procurement demand text, and verify the procurement demand text.

7. The procurement requirement generation system based on multi-source data and AI according to claim 6, wherein: In the procurement data preprocessing module, the data cleaning is specifically as follows: for numerical data in the procurement data, identify outliers by calculating the interquartile range, and correct or delete the outliers; for text data in the procurement data, use regular expressions to remove special characters and stop words, and use a pre-trained language model for semantic error correction; The formatting is specifically as follows: based on the file format of the procurement data, use the corresponding data processing library for formatting to convert it into a unified text format, and segment it according to punctuation marks and paragraph marks; specifically: use PyPDF2 to process PDF-format procurement data, use python-docx to process Word-format procurement data, use BeautifulSoup to process HTML-format procurement data, use the pandas library to process table-format procurement data, and use the dateutil library to process date-format procurement data; The structuring is specifically as follows: Convert semi-structured procurement data into structured procurement data through the ElementTree library; identify entities in unstructured procurement data through a deep learning-based named entity recognition model, and determine entity relationships through dependency syntactic analysis to convert unstructured procurement data into structured procurement data.

8. The procurement requirement generation system based on multi-source data and AI according to claim 6, characterized in that: The procurement knowledge graph construction module is specifically used for: Store each of the preprocessed procurement data through a relational database, construct a procurement knowledge graph based on the Neo4j graph database, extract procurement knowledge including procurement entities and entity relationships from the stored procurement data, and store the procurement knowledge in the procurement knowledge graph.

9. The procurement requirement generation system based on multi-source data and AI according to claim 6, wherein: In the model creation module, the procurement entity recognition model is used to recognize procurement entities from the input text, and is constructed based on a language model, a classifier, a part-of-speech tagging tool, a named entity recognition technology, a part-of-speech information assistance model, and an attention mechanism; The procurement requirement generation model is used to generate procurement requirement texts according to the input procurement knowledge, and is constructed based on the Transformer module.

10. The procurement requirement generation system based on multi-source data and AI according to claim 6, wherein: The procurement knowledge verification module is specifically used for: Performing compliance verification on each of the associated procurement knowledge through a rule engine and semantic role labeling technology, and supplementing and improving each of the procurement knowledge through an association rule mining algorithm and a generative adversarial network.

Images