Chip data batch structured processing method, device and equipment and storage medium

Abstract

The application discloses a chip data batch structured processing method and device, equipment and a storage medium, and relates to the technical field of chip data management. The method comprises the following steps: a document processor is used to analyze a chip data manual into structured text content, and a target abstract document corresponding to the chip data manual is generated; an initial chip data table is constructed based on chip basic information, a target parameter is obtained by extracting and classifying the target abstract document, and a target chip data table is constructed based on the target parameter and the initial chip data table; an abstract document block is extracted from the target abstract document by a document block divider, vectorization processing is performed on the abstract document block, and a target semantic vector is generated and stored in a vector knowledge base. By constructing the chip data table and the vector database, the degree of automation of chip data processing, the query efficiency, and the retrieval flexibility and accuracy are improved.

Description

Technical Field

[0001] This application relates to the field of chip data management technology, and in particular to methods, apparatus, devices and storage media for batch structured processing of chip data. Background Technology

[0002] Chip datasheets, as core technical documents in chip R&D, selection, and application, contain crucial information such as chip performance parameters, application scenarios, and usage methods. Currently, chip datasheets are mostly published and stored independently by each chip manufacturer in unstructured document formats such as PDFs. These documents contain various types of information, including text descriptions, data tables, circuit examples, and performance curves. There is a lack of unified standards in formatting and content organization among different manufacturers' datasheets. Some chip manufacturers provide simple performance parameter lists for users to look up, while a few information platforms manually compile chip data into tables to assist users in selecting target chips.

[0003] However, the most obvious drawback of existing technologies is the low efficiency of key information extraction and retrieval: the unstructured document format makes it impossible for users to quickly locate the core content through the information system, and they have to flip through lengthy documents one by one to extract the required information; even if they rely on performance parameter lists or manually organized tables, they can only achieve simple keyword matching queries, and it is difficult to associate the semantic relationship between the retrieval needs and chip characteristics and application scenarios, which cannot meet the flexible retrieval needs in complex scenarios, and seriously affects the efficiency of chip research and development and selection.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this application is to provide a method, apparatus, device, and storage medium for batch structured processing of chip data, aiming to solve the technical problem of low retrieval efficiency of chip data.

[0006] To achieve the above objectives, this application proposes a method for batch structured processing of chip data, the method comprising:

[0007] The chip datasheet is parsed into structured text content using a document processor, and a target summary document corresponding to the chip datasheet is generated.

[0008] An initial chip data table is constructed based on the chip's basic information. The target summary document is extracted and classified to obtain target parameters. A target chip data table is constructed based on the target parameters and the initial chip data table.

[0009] The document chunker extracts summary document blocks based on the target summary document, and the summary document blocks are vectorized to generate target semantic vectors which are stored in the vector knowledge base.

[0010] In one embodiment, the step of parsing the chip datasheet into structured text content using a document processor and generating a target summary document corresponding to the chip datasheet includes:

[0011] The chip datasheet is input into the document processor, which parses it to obtain the structured content of the chip datasheet, which includes text information, table information and image information;

[0012] The structured content is input into a pre-built summary extractor to generate a target summary document of the chip datasheet. The target summary document contains the chip's main features, chip application scenarios, and chip product descriptions.

[0013] In one embodiment, the step of inputting the structured content into a pre-built digest extractor to generate a target digest document of the chip datasheet further includes:

[0014] Obtain the training summary document as a first-context learning paradigm;

[0015] Generate the first prompt word based on the main features, application scenarios and product descriptions in the chip datasheet;

[0016] Based on the first context learning example and the first prompt word, the large language model is trained in a guided manner to generate a summary extractor.

[0017] In one embodiment, the steps of constructing an initial chip data table based on basic chip information, extracting and classifying the target summary document to obtain target parameters, and constructing a target chip data table based on the target parameters and the initial chip data table include:

[0018] Based on the chip datasheet, obtain the basic chip information of the corresponding target chip, and construct an initial chip data table based on the basic chip information. The basic chip information includes chip number, chip name and company name.

[0019] The parameter information dictionary of the target chip is obtained by extracting parameters from the target summary document using a pre-built parameter extractor.

[0020] The target summary document is type-identified by a pre-built type classifier to determine the chip type of the target chip;

[0021] The initial chip data table is supplemented based on the parameter information dictionary and the chip type to obtain the target chip data table for the target chip.

[0022] In one embodiment, the step of extracting parameters from the target summary document using a pre-built parameter extractor to obtain the parameter information dictionary of the target chip further includes:

[0023] We obtain manually annotated parameter extraction examples as second context learning paradigms;

[0024] Based on the names of the key parameters of the chip to be extracted, generate a second prompt word to guide the extraction of the corresponding parameter values ​​and ranges;

[0025] Based on the second context learning paradigm and the second prompt word, the large language model is trained in a guided manner to construct the parameter extractor.

[0026] In one embodiment, the step of extracting summary document blocks based on the target summary document using a document chunker, vectorizing the summary document blocks, and generating target semantic vectors to be stored in a vector knowledge base includes:

[0027] Based on the title hierarchy of the target summary document, the document chunker is used to divide the target summary document into several summary document blocks;

[0028] The summary document blocks are vectorized using a pre-trained text embedding model to generate corresponding target semantic vectors;

[0029] Metadata is generated and associated based on the summary document block, and the metadata includes at least the chip summary document identifier to which the summary document block belongs;

[0030] The summary document block, its corresponding target semantic vector, and associated metadata are stored in the vector knowledge base.

[0031] In one embodiment, the step of using a pre-trained text embedding model to vectorize the summarized document block and generate the corresponding target semantic vector includes:

[0032] The summary document block is input into the pre-trained text embedding model and converted into a word sequence;

[0033] The text embedding model maps the word sequence into word vectors.

[0034] The word vectors obtained from the mapping are pooled to generate target semantic vectors of fixed dimensions.

[0035] Furthermore, to achieve the above objectives, this application also proposes a chip data batch structuring processing apparatus, the chip data batch structuring processing apparatus comprising:

[0036] The text parsing module is used to parse the chip datasheet into structured text content through a document processor, and generate a target summary document corresponding to the chip datasheet.

[0037] The table generation module is used to construct an initial chip data table based on basic chip information, extract and classify the target summary document to obtain target parameters, and construct a target chip data table based on the target parameters and the initial chip data table.

[0038] The vector generation module is used to extract summary document blocks based on the target summary document through a document chunker, perform vectorization processing on the summary document blocks, generate target semantic vectors, and store them in a vector knowledge base.

[0039] In addition, to achieve the above objectives, this application also proposes a chip data batch structuring processing device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the chip data batch structuring processing method described above.

[0040] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the chip data batch structured processing method described above.

[0041] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the chip data batch structured processing method described above.

[0042] One or more technical solutions proposed in this application have at least the following technical effects:

[0043] This application proposes a method, apparatus, device, and storage medium for batch structured processing of chip data. A document processor parses chip datasheets into structured text content, generating target summary documents corresponding to the chip datasheets. An initial chip data table is constructed based on basic chip information. Target parameters are extracted and classified from the target summary documents. A target chip data table is constructed based on the target parameters and the initial chip data table. A document chunking device extracts summary document blocks from the target summary documents. These summary document blocks are then vectorized to generate target semantic vectors, which are stored in a vector knowledge base. By constructing the chip data table and vector database, the automation level, query efficiency, and retrieval flexibility and accuracy of chip data processing are improved. Attached Figure Description

[0044] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart illustrating an embodiment of the chip data batch structured processing method of this application.

[0047] Figure 2 This is a schematic diagram of the module structure of the chip data batch structuring processing device according to an embodiment of this application;

[0048] Figure 3 This is a schematic diagram of the device structure of the hardware operating environment involved in the chip data batch structured processing method in the embodiments of this application.

[0049] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0050] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0051] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0052] The main solution of this application embodiment is as follows: A document processor parses the chip datasheet into structured text content, generating a target summary document corresponding to the chip datasheet; an initial chip data table is constructed based on the chip's basic information; target parameters are extracted and classified from the target summary document; a target chip data table is constructed based on the target parameters and the initial chip data table; a document chunker extracts summary document blocks from the target summary document; the summary document blocks are vectorized to generate target semantic vectors, which are then stored in a vector knowledge base.

[0053] In this embodiment, for ease of description, the following description will focus on a chip data batch structured processing system.

[0054] Chip datasheets, as core technical documents in chip R&D, selection, and application, contain crucial information such as chip performance parameters, application scenarios, and usage methods. Currently, chip datasheets are mostly published and stored independently by each chip manufacturer in unstructured document formats such as PDFs. These documents contain various types of information, including text descriptions, data tables, circuit examples, and performance curves. There is a lack of unified standards in formatting and content organization among different manufacturers' datasheets. Some chip manufacturers provide simple performance parameter lists for users to look up, while a few information platforms manually compile chip data into tables to assist users in selecting target chips.

[0055] However, the most obvious drawback of existing technologies is the low efficiency of key information extraction and retrieval: the unstructured document format makes it impossible for users to quickly locate the core content through the information system, and they have to flip through lengthy documents one by one to extract the required information; even if they rely on performance parameter lists or manually organized tables, they can only achieve simple keyword matching queries, and it is difficult to associate the semantic relationship between the retrieval needs and chip characteristics and application scenarios, which cannot meet the flexible retrieval needs in complex scenarios, and seriously affects the efficiency of chip research and development and selection.

[0056] This application provides a solution that improves the automation of chip data processing, query efficiency, and the flexibility and accuracy of retrieval by constructing chip data tables and vector databases.

[0057] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of implementing the above functions, such as a chip data batch structured processing system. The following description uses a chip data batch structured processing system as an example to illustrate this embodiment and the subsequent embodiments.

[0058] Based on this, embodiments of this application provide a method for batch structured processing of chip data, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the chip data batch structured processing method of this application.

[0059] In this embodiment, the chip data batch structured processing method includes steps S11 to S13:

[0060] Step S11: The chip datasheet is parsed into structured text content by a document processor to generate a target summary document corresponding to the chip datasheet.

[0061] It should be noted that a document processor is a tool capable of parsing unstructured chip datasheets (such as PDF documents). Its core technology includes document format parsing algorithms that can identify and separate different types of information in the document, such as headers and footers, text descriptions, images, and tables. Chip datasheets are carriers of chip-related information, containing core content such as chip characteristics, parameters, and application scenarios, and serve as the raw input for data processing.

[0062] Additionally, it should be noted that structured text content is information formatted according to specific rules after parsing, facilitating accurate data extraction by the subsequent extractor. The target summary document is a condensed document integrating the chip's main features, application scenarios, and product specifications. It adopts Markdown format, and its core technology is a summary extraction technology based on the LLM (Large Language Model) general-purpose language model, which guides the model generation through context learning examples and dedicated prompt words.

[0063] Understandably, the purpose of this step is to address the problem of existing chip datasheets existing in unstructured form, with scattered and disorganized information making extraction difficult. The principle is to first break down the format limitations of unstructured documents using a document processor, transforming the disorganized information into structured content. Then, leveraging the semantic understanding and summarization capabilities of the LLM (Large Language Model), core information is extracted to form a summary document. This provides accurate and condensed foundational data for subsequent data storage and retrieval, avoiding the tedious process of users flipping through original manuals one by one to extract information.

[0064] Specifically, firstly, chip datasheets (e.g., in PDF format) are uploaded in batches to a document processor. The document processor parses the documents using a format parsing algorithm, separating structured content such as text, tables, and images, and generating intermediate files containing this content. Then, the intermediate files are compressed into compressed files and downloaded. Next, the compressed files are decompressed in batches, and the decompressed Markdown format documents are renamed with the chip name, while irrelevant files are deleted. Finally, the renamed documents are input into a pre-built summary extractor to generate target summary documents. Selecting a document processor that supports batch processing improves parsing efficiency and is suitable for scenarios involving massive amounts of datasheets; renaming documents by chip name facilitates subsequent data association and traceability; storing summary documents in Markdown format ensures the standardization of the document structure, facilitating subsequent block processing. Combining these implementation methods enables efficient and standardized conversion from raw unstructured datasheets to structured summary documents.

[0065] For example, 100 PDF (Portable Document Format) datasheets for chips from different manufacturers are collected and uploaded to a PDF document processor. The processor parses the datasheets and extracts structured content such as text descriptions (e.g., chip performance specifications), parameter tables (e.g., frequency range, gain), and package images. Corresponding intermediate files are generated and compressed into a ZIP (Zone Improvement Plan) package. After downloading and unzipping the ZIP package, the document corresponding to the ADH424S chip manufactured by Analog Devices Inc. is renamed to "ADH424S.md," and the temporary configuration file is deleted. This Markdown document is then input into a summary extractor to generate a target summary document containing the company name, key chip features (e.g., high-frequency communication compatibility, low noise), application scenarios (e.g., 5G base station communication modules), and product specifications (e.g., package type, power supply voltage range).

[0066] Step S12: Construct an initial chip data table based on the chip's basic information, extract and classify the target summary document to obtain target parameters, and construct a target chip data table based on the target parameters and the initial chip data table.

[0067] It should be noted that basic chip information is used to identify and describe the chip's fundamental attributes, including chip serial number, chip name, company name, and summary document storage address. Its purpose is to establish a unique identifier and basic profile for the chip, facilitating data association and retrieval. The initial chip data table is a preliminary data table built upon this basic chip information. Its core technology is structured data table design, using CSV (Comma-Separated Values) format or other relational database formats for storage, providing a basic data framework.

[0068] Additionally, it should be noted that the target summary document serves as the data source for parameter extraction and type classification. Extraction and classification refer to the process of extracting parameter information from the summary document and determining the chip type using specific tools (parameter extractor, type classifier). The core technology is parameter extraction and type classification based on the LLM model. Target parameters are the chip's key performance parameters, such as frequency range, gain, and IP1dB (Input 1dB Gain Compression Point). Their function is to reflect the chip's core performance and provide a basis for chip selection and comparison. The target chip data table is a complete data table supplemented with target parameters and chip type. The core technology is data field expansion and integration technology, which enables structured storage of chip data for easy and accurate querying.

[0069] Understandably, the purpose of this step is to address the problems of lack of structured management of core chip data, low query efficiency, and high error rates caused by manual processing in existing technologies. The principle is to first construct an initial data table using the chip number as a unique identifier, laying the foundation for data structuring. Then, using LLM-based extraction and classification tools, parameter and type information is automatically obtained from the summary document and added to the initial data table, forming a complete structured data table. Its function is to achieve standardized storage of chip data, allowing users to quickly query target chips through precise field matching, reducing manual intervention and improving data accuracy and query efficiency.

[0070] Specifically, firstly, basic chip information (chip number, name, company name, and summary storage address) is extracted from chip datasheets or summary documents to construct an initial chip data table, stored in CSV format. Then, a parameter extractor extracts parameter information from the target summary document, forming a parameter information dictionary. Next, a type classifier analyzes the target summary document to determine the chip's primary and secondary types. The keys and values ​​from the parameter information dictionary, along with the chip type as a new field, are added to the initial chip data table to form the target chip data table. Storing the initial data table in CSV format ensures data universality and ease of editing; automated processing by the parameter extractor and type classifier reduces human error and improves processing efficiency; expanding the data table fields enriches data dimensions and meets multi-dimensional query needs. Combining these implementation methods enables comprehensive structured storage of chip data, from basic information to core parameters and type information.

[0071] For example, basic information is extracted from the summary document of the ADH424S chip: chip number "1", chip name "ADH424S", company name "Analog Devices Inc (ADI)", and summary document storage address " / docs / ADI / ADH424S_Summary.md", to construct an initial CSV format data table. Parameter information is extracted from the target summary document of the chip using a parameter extractor, resulting in a dictionary: {"Frequency Range": [400, 1700] MHz, "Gain": 15dB, "IP1dB": 0dBm}. The type classifier determines the chip's primary type as "amplifier" and its secondary type as "low-noise amplifier". These parameter fields and type fields are then added to the initial data table to form a target chip data table containing basic chip information, key parameters, and chip type.

[0072] Step S13: Extract summary document blocks based on the target summary document using a document chunker, vectorize the summary document blocks, and generate target semantic vectors to be stored in the vector knowledge base.

[0073] It's important to note that a document chunker is a tool capable of splitting documents according to specific rules. Its core technology is a document structure splitting algorithm, which divides a target summary document into multiple semantically independent document chunks. These summary document chunks are semantic units obtained after chunking, retaining some core information from the original document, such as key features and application scenarios. Their purpose is to facilitate subsequent vectorization processing and semantic retrieval. Vectorization processing is the process of converting text information into computer-recognizable vector forms. Its core technology is a pre-trained Embeddings text embedding model, which captures the semantic information of the text.

[0074] Additionally, it should be noted that the target semantic vector is a fixed-dimensional vector obtained after vectorization, which can represent the semantic connotation of the summary document block and is used for semantic similarity matching. The vector knowledge base is a database that stores the target semantic vector and related information. Its core technology is vector storage and indexing technology, and its role is to provide data support for semantic retrieval.

[0075] Understandably, the purpose of this step is to address the limitations of traditional retrieval methods, such as their simplistic approach and poor semantic relevance. The principle involves first using a document chunker to break down the summary document into semantically independent chunks, avoiding inaccurate semantic capture due to excessive document length. Then, a pre-trained text embedding model is used to convert these chunks into semantic vectors. Finally, these vectors are stored in a vector knowledge base, providing a data foundation for subsequent semantic retrieval. The goal is to achieve flexible semantic-based retrieval, allowing users to quickly find chip data that meets their needs through natural language descriptions, thus improving the flexibility and accuracy of retrieval.

[0076] Specifically, first, configure the document chunker and set rules for chunking according to Markdown level 1 headings; second, input the target summary document into the document chunker to split it into multiple summary document chunks, such as "Main Features" chunks, "Application Scenarios" chunks, and "Product Description" chunks; third, use a pre-trained Embeddings text embedding model to vectorize each summary document chunk, generating target semantic vectors; fourth, generate metadata for each summary document chunk (including chunk number, associated summary document number, chip name, and summary document storage address) and associate it with the corresponding target semantic vector; fifth, store the summary document chunks, target semantic vectors, and associated metadata in a vector knowledge base. The advantages of different implementation methods are: chunking according to Markdown level 1 headings ensures the semantic integrity of document chunks and avoids semantic breaks; selecting a high-performance pre-trained text embedding model improves the representation accuracy of semantic vectors; associating metadata facilitates the tracing of search results and data correlation; and using an efficient vector knowledge base improves search response speed. Combining these implementation methods enables the efficient construction of vector knowledge bases from summary documents, providing reliable support for semantic retrieval.

[0077] For example, taking the target summary document of the ADH424S chip as the processing object, this document contains three first-level heading modules: "Main Features," "Application Scenarios," and "Product Description." Inputting it into a document chunker, it is chunked according to the first-level headings to obtain three summary document blocks: the "Main Features" document block (containing a description of the chip's core performance), the "Application Scenarios" document block (containing a description of the communication scenarios the chip is adapted to), and the "Product Description" document block (containing information such as the chip's packaging and power supply). A pre-trained Embeddings model is used to process each document block. First, the "Main Features" document block is converted into a token sequence, then each token is mapped to a word vector, and a target semantic vector of dimension 768 is generated through mean pooling. Metadata is generated for this document block: block number "1-1," the corresponding summary document number "1," the chip name "ADH424S," and the storage address " / docs / ADI / ADH424S_Summary.md," and associated with the corresponding target semantic vector. The three document blocks, their semantic vectors, and metadata are all stored in the vector knowledge base.

[0078] This embodiment solves the problem of difficult information extraction by converting unstructured chip datasheets into structured summary documents; it constructs a complete structured data table to achieve standardized storage of core data and improve query efficiency; and it builds a vector knowledge base to realize semantic retrieval functions, improving retrieval flexibility and accuracy. The entire technical solution forms a complete process from data parsing and structured storage to semantic retrieval, effectively solving technical problems such as scattered information, inefficient queries, and limited retrieval methods in existing chip data processing, providing efficient data processing and retrieval support for chip R&D and selection.

[0079] Based on the above implementation scheme, in one feasible implementation, the step of parsing the chip datasheet into structured text content using a document processor and generating a target summary document corresponding to the chip datasheet includes S21~S22:

[0080] Step S21: Input the chip datasheet into the document processor and parse it to obtain the structured content of the chip datasheet. The structured content includes text information, table information and image information.

[0081] It should be noted that structured content is a collection of information that has been parsed and organized by category. The core technology is information classification and organization. Text information refers to the textual descriptions in the manual (such as chip function descriptions and usage precautions); tabular information refers to the tables recording parameters in the manual (such as performance parameter tables and pin definition tables); and image information refers to the images in the manual (such as package CAD (Computer Aided Design) drawings and performance parameter curves). The purpose of this structured content is to provide a clearly categorized and easily processed data foundation for subsequent summary extraction.

[0082] Specifically, for a single chip datasheet, it is directly input into a document processor. The processor reads the document using a PDF parsing algorithm, extracts text information using a text recognition algorithm, identifies and extracts table information using a table extraction algorithm, and separates image information using an image separation algorithm. Finally, it outputs structured content containing these three types of information. For batches of chip datasheets, multiple datasheets are uploaded to a document processor that supports batch processing. The processor parses each datasheet sequentially according to the parsing logic for a single datasheet, outputting structured content in batches. The first implementation method is suitable for processing a small number of datasheets, offering simplicity and specificity. The second implementation method is suitable for processing massive amounts of datasheets, significantly improving parsing efficiency.

[0083] For example, a PDF datasheet for a chip with the model number "ADL5565" is input into a document processor. The processor opens the datasheet using a PDF parsing algorithm, extracts the text description in the "Product Overview" section of the datasheet (such as "ADL5565 is a high linearity, low noise RF amplifier") using an OCR text recognition algorithm, identifies and extracts the "Electrical Characteristics" table (containing parameters such as frequency range, gain, IP3, and corresponding values) using a table extraction algorithm, and separates the "Package Pin Diagram" and "Frequency-Gain Characteristic Curve" images from the datasheet using an image separation algorithm. Finally, it outputs structured content containing the aforementioned text information, table information, and image information.

[0084] Step S22: Input the structured content into a pre-built summary extractor to generate a target summary document of the chip datasheet. The target summary document contains the main characteristics of the chip, the application scenarios of the chip, and the product description of the chip.

[0085] It should be noted that the abstract extractor is a tool built on the LLM (Large Language Model) general-purpose large language model. Its core technology is large language model guided training, combined with contextual learning examples and specific prompts. Its function is to extract core information from structured content and organize it into a summary document. The chip's main characteristics refer to its core performance advantages (such as high linearity and low noise); the chip's application scenarios refer to the usage environments the chip is suited for (such as 5G communication and satellite navigation); the chip's product description refers to its basic attributes (such as package type and power supply voltage). The purpose of the target summary document is to provide condensed core information for subsequent parameter extraction, type classification, and semantic retrieval.

[0086] Specifically, first, ensure the summary extractor has been built (including context learning examples and specific prompts); input the obtained structured content (text, tables, image-related descriptive information) into the summary extractor. Guided by the prompts and combined with the context examples, the extractor extracts the chip's main features, application scenarios, and product specifications from the structured content; organize the extracted information in Markdown format to generate the target summary document, with the company name in the first line and each part separated by "---". Choosing an LLM model with strong semantic understanding can improve extraction accuracy; optimizing prompts makes the extracted information more relevant to the needs; using Markdown format ensures the standardization and readability of the summary document. Combining these implementation methods enables efficient, accurate extraction and standardized presentation of core information.

[0087] For example, the structured information of the ADL5565 chip (including a text description of "high linearity, low noise," a table of information on "frequency range 100MHz-6GHz," and an image-related description of "QFN package") is input into the summary extractor. Guided by the prompt "Extract the main characteristics, application scenarios, and product description of the chip, output in Markdown format, with the company name in the first line and each part separated by '---'", and combining the context example, the summary extractor extracts the main characteristics as "high linearity, low noise, frequency range 100MHz-6GHz", the application scenarios as "5G communication base station, satellite navigation receiver module", and the product description as "QFN package, power supply voltage 3.3V". The target summary document is then generated as required, with "Analog Devices Inc (ADI)" in the first line and each part separated by "---".

[0088] This embodiment solves the problems of scattered and disorganized information in the original manual by parsing the unstructured chip datasheet into clearly categorized structured content. It then utilizes a pre-built abstract extractor to extract core information from the structured content, generating a target abstract document containing key features, application scenarios, and product descriptions, thus addressing the difficulty for users to quickly obtain core abstract information. This achieves the transformation from unstructured manuals to structured core abstracts, providing a high-quality data foundation for subsequent parameter extraction, structured storage, and semantic retrieval, thereby improving the overall efficiency and convenience of chip data processing.

[0089] Based on the above implementation scheme, in one feasible implementation, the step of inputting the structured content into a pre-built summary extractor to generate the target summary document of the chip datasheet further includes S31~S33:

[0090] Step S31: Obtain the training summary document as the first context learning example.

[0091] It's important to note that the training summary document is a high-quality summary written by the user based on the chip datasheet. It contains core information such as the chip's main features, application scenarios, and product specifications, and is formatted correctly and contains precise content. In this step, the training summary document serves as the first context learning example. Its purpose is to provide learning samples for the large language model, allowing the model to understand the writing logic, information dimensions, and format requirements of chip summaries, laying the foundation for accurate summary extraction in subsequent steps. The first context learning example refers to the reference sample used to guide the large language model in learning the summary extraction task. Its core technology is example-driven learning, which helps the model quickly grasp the task objectives and execution standards by providing specific input and output examples.

[0092] Specifically, on the one hand, chip datasheets covering different chip types (such as amplifiers, mixers, and filters) are selected, and human experts with expertise in the chip field write training summary documents for each one. This ensures that each type of chip has a corresponding example. The purpose of this approach is to allow the model to learn the commonalities and characteristics of summaries from different chip types, improving the model's adaptability to various chips. On the other hand, multiple training summary documents are written for multiple datasheets of the same type of chip, highlighting the descriptions of the core parameters and application scenarios of that type of chip. The purpose of this approach is to enhance the accuracy of the model's summary extraction for specific types of chips. Combining these two approaches allows the model to have both broad adaptability and accurate extraction capabilities for specific types of chips, meeting the summary extraction needs in different scenarios.

[0093] Step S32: Generate the first prompt word based on the main features, application scenarios and product descriptions in the chip datasheet.

[0094] It should be noted that the main characteristics of a chip datasheet refer to its core performance advantages that distinguish it from other chips, such as key parameters related to frequency range, gain, and noise figure. The core technology is chip performance parameter identification technology, which clarifies the core performance information that needs to be highlighted in the abstract. Chip application scenarios refer to the suitable usage environments and fields for the chip, such as 5G communication and automotive electronics. The core technology is application scenario association technology, which allows abstract users to quickly understand the chip's applicable scope. Chip product descriptions refer to the chip's basic attribute information, such as package type, power supply voltage, and operating temperature. The core technology is product basic information extraction technology, which provides basic usage references for the chip. The first prompt word is guiding text designed based on the above three information dimensions. The core technology is prompt word engineering technology, which clearly informs the large language model of the task objectives, output format, and information dimension requirements for abstract extraction, guiding the model to output the abstract according to the specifications.

[0095] Specifically, in one embodiment, basic prompts are generated based on the general information dimensions of the chip datasheet, explicitly requiring the model to extract key features, application scenarios, and product descriptions, outputting them in Markdown format, with the company name in the first line and each part separated by "---". This approach ensures the basic standardization and universality of the extracted summary. In another embodiment, for the characteristics of a specific chip field (such as RF chips), the basic prompts are supplemented with the extraction requirements for core parameters of that field, such as "the key features section needs to specify parameters such as frequency range, gain, and IP1dB". This approach improves the accuracy and relevance of the chip summary extraction for that specific field. In yet another embodiment, based on the format requirements of subsequent data processing, details such as field separators and parameter unit formats are specified in the prompts, such as "frequency range is expressed in [min,max]MHz (megahertz), and gain is in dB (decibels)". This approach makes the summary output more aligned with the needs of subsequent structured processing. Combining these implementation methods can generate first prompts that are both general and specific, and meet the requirements of subsequent processing.

[0096] Step S33: Based on the first context learning example and the first prompt word, guide the training of the large language model to generate a summary extractor.

[0097] It's important to note that the large language model is an AI model with powerful semantic understanding, learning, and generation capabilities. Its core technologies are the Transformer architecture and large-scale corpus pre-training. Its role is as the core processing unit of the summarization extractor, receiving examples and prompts to guide the summarization task. Guided training refers to the process of inputting examples and prompts into the large language model, allowing the model to learn the summarization task through their combined action. Its core technologies are context learning and instruction following techniques, which enable the model to adapt to specific summarization requirements, improving the accuracy and standardization of the extraction.

[0098] Specifically, a suitable large language model (such as GPT-4, Llama 2, etc., which have strong context learning capabilities) is selected; the first context learning example and the first prompt word are combined in a specific format (e.g., first input the example "Input: [Chip Datasheet Structured Content 1], Output: [Training Summary Document 1]; Input: [Chip Datasheet Structured Content 2], Output: [Training Summary Document 2]", then input the first prompt word); the combined content is input into the large language model, allowing the model to learn the context and understand the task logic of summary extraction; through multiple rounds of testing and adjustment (e.g., inputting new chip datasheet structured content and observing whether the model's output summary meets the requirements; if not, optimizing the example or prompt word), a stable and accurate summary extractor is finally generated. Selecting a large language model with strong context learning capabilities can improve training efficiency and extraction accuracy; a reasonable combination format of example and prompt word can help the model better understand the task; multiple rounds of testing and adjustment can optimize the performance of the summary extractor and ensure that it is suitable for practical application needs.

[0099] This embodiment, through the above-described scheme, obtains high-quality training summary documents as learning examples, providing specific learning references for the model; generates targeted first prompt words to clarify the model's task objectives and output requirements; and trains a large language model under the joint guidance of examples and prompt words, generating a dedicated summary extractor. An automated, high-precision summary extraction tool has been constructed, effectively solving the problems of low efficiency, high error rate, and non-standard information in manual summary extraction. It achieves efficient and batch extraction of core chip information, laying a solid foundation for subsequent structured processing and semantic retrieval of chip data, and improving the overall automation level and reliability of chip data processing.

[0100] Based on the above implementation scheme, in one feasible implementation, the steps of constructing an initial chip data table based on chip basic information, extracting and classifying the target summary document to obtain target parameters, and constructing a target chip data table based on the target parameters and the initial chip data table include S41~S43:

[0101] Step S41: Extract parameters from the target summary document using a pre-built parameter extractor to obtain a parameter information dictionary for the target chip.

[0102] It should be noted that the target chip is the specific chip being processed, the object of data processing. The core technology is chip identification technology, which clarifies the specific direction of data processing. Basic chip information consists of core attributes used to identify and distinguish the chip. The core technology is chip information standardization technology. The chip number is a unique identifier for the chip, enabling unique chip identification and data association. The chip name is the specific model name, facilitating intuitive chip identification for users. The company name is the name of the company that manufactures the chip, providing information about the chip's origin. The initial chip data table is a preliminary data table built based on the basic chip information. The core technology is structured data table design, stored in CSV format or other relational database formats, providing a basic data framework for subsequent supplementation of parameter information and chip type.

[0103] Specifically, in one embodiment, basic chip information is extracted from the cover, first page, or product overview section of the chip datasheet. This information is then organized according to preset fields (chip number, chip name, company name) and an initial chip data table is constructed using Excel. This method is suitable for scenarios involving a small number of chips, offering flexibility and high targeting. In another embodiment, basic chip information is automatically extracted from specified areas (such as headers or specific chapters) of the chip datasheet using a document parsing tool. An initial chip data table in CSV format is then automatically generated using a programming tool (such as Python). This method is suitable for batch chip processing scenarios and significantly improves construction efficiency. Chip numbers are generated using a rule of "manufacturer code + chip type code + serial number" to ensure uniqueness and prevent data association errors caused by duplicate numbers. Combining these implementation methods can meet the needs of scenarios with different processing scales and data accuracy requirements, achieving efficient and standardized construction of initial chip data tables.

[0104] Step S42: Extract parameters from the target summary document using a pre-built parameter extractor to obtain a parameter information dictionary for the target chip.

[0105] It should be noted that the parameter extractor is a dedicated tool pre-built based on the LLM general-purpose large language model. Its core technologies are large language model-guided training and parameter recognition, combined with contextual learning examples and specific prompts. Its function is to accurately extract key chip parameters from the target abstract document. Parameter extraction refers to the process of identifying and extracting key chip performance parameters from the abstract document. Its core technologies are semantic understanding and data extraction, and its function is to obtain core data reflecting chip performance. The parameter information dictionary is a dictionary-structured data that stores the extracted parameter names and their corresponding values ​​(or ranges). Its core technology is structured data storage. The keys are parameter names (such as frequency range, gain), and the values ​​are parameter values ​​(or ranges) and units. Its function is to standardize the storage of chip parameters, facilitating subsequent addition to the data table.

[0106] Specifically, ensure the parameter extractor has been built, including context-learning examples and specific prompts for chip parameter extraction; input the target summary document into the parameter extractor; the parameter extractor uses semantic understanding to identify parameter names and corresponding values ​​(or ranges) in the document, and generates a parameter information dictionary according to a preset format (e.g., parameter name as key, value + unit as value); validate the extracted parameter information dictionary, and if there are missing parameters or incorrect formats, return to adjust the prompts or supplement examples before re-extracting. Choosing an LLM model with high semantic recognition accuracy to build the parameter extractor can improve extraction accuracy; pre-setting a unified parameter name and unit format can ensure the standardization of the parameter information dictionary; adding a validation step can further improve the reliability of parameter data. Combining these implementation methods can achieve efficient, accurate, and standardized extraction of chip parameters.

[0107] Step S43: The target summary document is type-identified by a pre-built type classifier to determine the chip type of the target chip.

[0108] It should be noted that the type classifier is a dedicated tool pre-built based on the LLM general-purpose large language model. Its core technologies are large language model-guided training and type recognition, combined with contextual learning examples and specific prompts. Its function is to determine the type of the target chip. Type recognition refers to the process of determining the chip's type by analyzing relevant information in the summary document. Its core technologies are semantic association and classification matching, which clarify the chip's functional category. Chip types are classified into functional categories using a two-level classification system. The core technology is chip classification system construction. The first-level categories include mixers, inverters, filters, amplifiers, etc., while the second-level categories expand on amplifier types (such as power amplifiers, low-noise amplifiers, etc.). This facilitates users' chip retrieval by type, meeting different granularity of classification query needs.

[0109] Specifically, ensure the type classifier is built, including a description of the two-level chip classification system, corresponding context learning examples, and specific prompts; input the target summary document into the type classifier; the type classifier analyzes information such as the chip's function (e.g., "low-noise amplification") and application scenario (e.g., "RF signal amplification") in the document and matches it with the preset classification system; output the chip's primary and secondary types (if it is an amplifier type). A clear two-level classification system can meet different granularity retrieval needs; rich context learning examples can improve the classifier's adaptability to different chip type descriptions; precise prompts can guide the classifier to focus on core classification criteria, improving classification accuracy. Combining these implementation methods enables efficient, accurate, and standardized identification of chip types.

[0110] Step S44: Supplement the initial chip data table based on the parameter information dictionary and the chip type to obtain the target chip data table for the target chip.

[0111] It should be noted that the target chip data table is a complete data table supplemented with parameter information and chip type. The core technology is data field expansion and integration technology, which enables the integrated structured storage of basic chip information, key parameters, and type information, facilitating multi-dimensional queries and data analysis for users.

[0112] Specifically, open the constructed initial chip data table (such as a CSV file); add each key from the parameter information dictionary as a new field name to the field list of the initial data table; fill in the corresponding values ​​from the parameter information dictionary into the new fields, corresponding to the basic information of the chip; add two new fields, "Primary Chip Type" and "Secondary Chip Type," and fill in the corresponding fields with the determined chip types; save the supplemented data table to form the target chip data table. Using CSV format ensures the data table's universality and ease of editing; adding fields in parameter name order improves the data table's regularity; setting primary and secondary type fields separately meets different granularity query needs. Combining these implementation methods enables the standardized and efficient construction of the target chip data table.

[0113] This embodiment, through the above-described scheme, establishes a standardized storage framework for basic chip information by constructing an initial chip data table; automatically extracts key chip parameters using a parameter extractor to form a parameter information dictionary; automatically determines the two-level chip type using a type classifier; and supplements the parameter information and chip type into the initial data table to form a complete target chip data table. This achieves automated and structured integrated storage of core chip data (basic information, key parameters, and type), effectively solving the problems of scattered chip data, low query efficiency, and high manual processing errors in existing technologies. It provides users with a standardized and comprehensive foundation for chip data querying, improving the efficiency and accuracy of chip selection, R&D, and other work, and has strong practicality and application value.

[0114] Based on the above implementation scheme, in one feasible implementation, before the step of extracting parameters from the target summary document using a pre-built parameter extractor to obtain the parameter information dictionary of the target chip, the method further includes steps S51 to S53:

[0115] Step S51: Obtain manually annotated parameter extraction examples as second context learning examples.

[0116] It should be noted that the manually annotated parameter extraction examples are created by human experts after extracting parameters from microarray digest documents. The core technologies are expert-based annotation and data structuring. Each example contains a mapping relationship between a "microarray digest document fragment" and its corresponding parameter information dictionary. The "microarray digest document fragment" is the text content containing parameter information, and the "parameter information dictionary" is the standard parameter data extracted and organized by experts (the key is the parameter name, and the value is the parameter value / range + unit). In this step, the manually annotated parameter extraction examples serve as the second context learning paradigm. The core technology is paradigm-driven learning, which provides specific parameter extraction references for the large language model, allowing the model to understand the different parameter expressions, extraction logic, and parameter dictionary construction specifications, thus laying the foundation for accurate parameter extraction by the subsequent model.

[0117] Specifically, chip summary document fragments covering various chip types (such as amplifiers, mixers, and filters) and multiple core parameters (such as frequency range, gain, IP3, and insertion loss) are selected. Parameters are manually extracted and labeled one by one, forming multiple sets of parameter extraction examples. This method allows the model to learn the extraction logic for different chip types and parameters, improving the model's generalization ability. For different expressions of the same parameter (e.g., frequency range can be expressed as "400MHz-1700MHz" or "frequency adapted to 400 to 1700 MHz"), multiple sets of examples are constructed. This method strengthens the model's ability to recognize different expressions of the same parameter, improving extraction accuracy. The labeled examples are validated to ensure the accuracy of parameter extraction and the standardization of dictionary format, eliminating erroneous examples. This method ensures the quality of the second context learning examples and avoids erroneous examples misleading the model's learning. Combining these implementation methods allows for the construction of a high-quality, highly adaptable set of second context learning examples, providing reliable support for model training.

[0118] Step S52: Based on the name of the key chip parameter to be extracted, generate a second prompt word to guide the extraction of the corresponding parameter value and value range.

[0119] It should be noted that the names of the key chip parameters to be extracted refer to a list of parameters reflecting the core performance of the chip. The core technology is chip parameter screening technology, including frequency range, gain, IP1dB, IP3 (Third-Order Intercept Point), insertion loss, input resistance, and output resistance. Its purpose is to clearly define the specific objects of model parameter extraction and avoid extracting irrelevant parameters. Parameter values ​​refer to the specific values ​​of the key chip parameters (e.g., gain 15dB). The core technology is numerical recognition technology, which quantifies the chip's performance indicators. Numerical ranges refer to the value intervals of some parameters (e.g., frequency range 400MHz-1700MHz). The core technology is range recognition and formatting technology, which clarifies the applicable range of the parameters. The second prompt is guiding text designed based on the names of the parameters to be extracted. The core technology is prompt engineering technology. Its purpose is to clearly inform the large language model of the task objective of parameter extraction, the list of parameters to be extracted, the output format (e.g., dictionary format), and the expression specifications for values / ranges, guiding the model to accurately extract parameters as required.

[0120] Specifically, based on a list of general core parameters for chips, basic second prompt words are generated, clearly specifying the names of the parameters to be extracted and the output format as a dictionary (keys are parameter names, values ​​are numerical values / ranges + units). This method ensures the universality and basic standardization of parameter extraction. For domain-specific chips (such as RF chips), the domain-specific parameter names (such as noise figure, phase noise) are added to the basic prompt words. This method enhances the specificity of parameter extraction for domain-specific chips. The formatting requirements for numerical values ​​and ranges are clearly specified in the prompt words (e.g., ranges are represented by "[min,max]", and units are uniformly MHz, dB, Ω (ohm), etc.). This method ensures the consistency of parameter output format and facilitates subsequent data processing and comparison. The prompt words include the explanation that "if a parameter is not mentioned in the document, the corresponding value is marked as 'none'". This method avoids missing parameter fields in the model and ensures the completeness of the parameter information dictionary. Combining these implementation methods generates second prompt words that are both universal and specific, with standardized formats.

[0121] Step S53: Based on the second context learning paradigm and the second prompt word, guide the training of the large language model to construct the parameter extractor.

[0122] It should be noted that guided training refers to the process of inputting a second context learning paradigm and a second cue word into a large language model, allowing the model to learn the parameter extraction task through the combined effect of both. The core technologies are context learning and instruction following techniques, which enable the model to adapt to the specific requirements of chip parameter extraction, improving the accuracy and standardization of the extraction. The parameter extractor is a specialized tool generated after guided training, with core technologies of model fine-tuning and task adaptation. Its function is to automatically extract key parameters from the chip target summary document and generate a parameter information dictionary that meets the requirements.

[0123] Specifically, a suitable large language model (such as GPT-3.5 Turbo, Llama 2 70B, etc., which have strong semantic understanding and instruction following capabilities) is selected; the acquired second context learning examples and generated second prompt words are combined in the format of "example + prompt word", and the example part is presented in the form of "input: [chip digest document fragment], output: [parameter information dictionary]" with multiple sets of examples; the combined content is input into the large language model to allow the model to learn the context and understand the task logic and format requirements of parameter extraction; through multiple rounds of testing and optimization (inputting a new chip digest document and observing whether the parameter information dictionary output by the model meets the requirements; if there are problems such as missing parameters, numerical errors, or non-standard formats, the examples or prompt words are optimized and retrained); when the parameter information dictionary output by the model meets the requirements of accuracy, completeness, and standardization, the parameter extractor is completed. Choosing a large language model with strong instruction compliance can improve training efficiency and extraction accuracy; combining multiple sets of examples can allow the model to learn more diverse parameter representations; multiple rounds of testing and optimization can continuously improve the performance of the parameter extractor and ensure that it is suitable for real-world application scenarios.

[0124] This embodiment, through the above-described scheme, uses manually annotated high-quality parameter extraction examples as learning models to provide specific parameter extraction references for the model; it generates targeted second prompt words to clarify the goals, scope, and format requirements of parameter extraction; and it trains a large language model under the joint guidance of examples and prompt words to construct a dedicated parameter extractor. The entire technical solution achieves automated, accurate, and standardized extraction of key chip parameters, effectively solving the problems of low efficiency, high error, and chaotic formatting associated with manual parameter extraction. It provides high-quality parameter data for the structured storage of chip data, improves the automation level and data reliability of chip data processing, and provides strong support for subsequent chip selection, performance comparison, and other work.

[0125] Based on the above implementation scheme, in a feasible implementation, the steps of extracting summary document blocks from the target summary document using a document chunker, vectorizing the summary document blocks, and generating target semantic vectors to be stored in a vector knowledge base include S61~S64:

[0126] Step S61: Based on the title hierarchy of the target summary document, use a document chunker to divide the target summary document into several summary document blocks.

[0127] It should be noted that the heading hierarchy refers to the hierarchical form of headings in the target summary document. The core technology is document structured typesetting technology. In this step, it specifically refers to the Markdown first-level heading hierarchy, which provides the basis for dividing the document into chunks. The document chunker is a tool that can split documents according to heading hierarchy. Its core technology is a document structured splitting algorithm. Its function is to split the target summary document into multiple semantically independent document chunks according to a preset heading hierarchy. The summary document chunk is the semantic unit obtained after chunking. The core technology is semantic unit extraction technology. Each document chunk corresponds to the content of a first-level heading module. Its function is to preserve the semantic integrity of each core information module, so as to accurately capture semantic information during subsequent vectorization processing, while avoiding semantic confusion caused by excessive document length.

[0128] Specifically, configure the document chunker and set the chunking rule to "split by Markdown Level 1 headings" to directly chunk the target summary document. This method is suitable for scenarios where the summary document has a standardized format and clear Level 1 headings, and its purpose is to quickly and accurately split the document. If the target summary document has inconsistent Level 1 heading descriptions, first use a document preprocessing tool to unify the Level 1 heading format (e.g., unify "Core Features" and "Chip Features" into "Main Features") before chunking. This method ensures the consistency of the chunking rules and avoids omissions or errors in chunking due to differences in heading descriptions. After chunking, name each summary document chunk according to the naming rule "Chip Name - Level 1 Heading Name" (e.g., "ADH424S - Main Features"). This method facilitates the identification and association of subsequent document chunks.

[0129] Step S62: The summary document block is vectorized using a pre-trained text embedding model to generate the corresponding target semantic vector.

[0130] It's important to note that pre-trained text embedding models refer to models trained on large-scale corpora that possess the ability to represent text semantics. Their core technologies are the Transformer architecture and unsupervised pre-training techniques (such as BERT, Sentence-BERT, and Word2Vec). Their function is to transform textual information into vectors that represent semantic meaning. Vectorization is the process of converting textual summary document blocks into computer-recognizable vector forms. Its core technology is text semantic mapping, which quantifies the semantic information of the text into vectors. Target semantic vectors are fixed-dimensional vectors obtained after vectorization. Their core technology is semantic vector generation, which accurately represents the semantic meaning of summary document blocks. Their purpose is for subsequent semantic similarity matching, enabling semantic-based retrieval.

[0131] Specifically, a lightweight pre-trained text embedding model (such as a small version of Sentence-BERT) is selected to vectorize the summarized document blocks. This approach is suitable for scenarios with high processing speed requirements and limited hardware resources, and its purpose is to quickly generate target semantic vectors. A high-precision pre-trained text embedding model (such as BERT-base or GPT-4Embeddings) is selected. This approach is suitable for scenarios with high requirements for semantic representation accuracy, and its purpose is to improve the semantic capture capability of the target semantic vectors and ensure retrieval accuracy. The dimension of the target semantic vectors is unified (e.g., 768-dimensional or 1024-dimensional), achieved through model parameter configuration or vector dimension conversion tools. This approach ensures that the dimension of the target semantic vectors corresponding to all summarized document blocks is consistent, facilitating subsequent semantic similarity calculations. Based on the needs of the actual application scenario, a balance can be struck between processing speed and semantic accuracy to generate high-quality target semantic vectors.

[0132] Step S63: Generate and associate metadata based on the summary document block, wherein the metadata includes at least the chip summary document identifier to which the summary document block belongs.

[0133] It should be noted that associated metadata describes the attributes and associated information of summary document blocks. The core technology is metadata construction technology, which provides identification and association clues for summary document blocks and their target semantic vectors, facilitating the tracing of search results and data association. The chip summary document identifier is information used to uniquely identify the original chip summary document to which the summary document block belongs. It is usually the chip number or the unique filename of the summary document. Its function is to establish the association between the summary document block and the original chip summary document, ensuring that after retrieving the document block, it can be traced back to the complete chip summary document. In addition to the chip summary document identifier, metadata may also include block numbers, chip names, summary document storage addresses, etc., to enrich the association information and improve the convenience of data management.

[0134] Specifically, the metadata includes the chip digest document identifier (chip number), block number, and chip name. The block number is generated according to the rule of "chip number - block sequence number" (e.g., "1-1", "1-2"). This method provides multi-dimensional association information, facilitating accurate traceability and management. In addition to the above information, the metadata also includes the digest document storage address. This method directly provides the access path to the original digest document, improving the convenience of obtaining complete information after retrieval. Relevant information is automatically extracted from the naming or association data tables of the digest document blocks using programming tools (such as Python) to generate metadata and bind it to the document blocks and semantic vectors. This method is suitable for batch processing scenarios and improves the efficiency of metadata generation and association. Combining these implementation methods enables comprehensive and efficient construction and association of metadata, providing strong support for subsequent data management and retrieval traceability.

[0135] Step S64: Store the summary document block, its corresponding target semantic vector, and associated metadata into the vector knowledge base.

[0136] It should be noted that the vector knowledge base is a database specifically designed to store semantic vectors and related information. Its core technology is vector storage and indexing technology, which supports efficient vector similarity calculation and retrieval. Its role is to provide data storage and query support for semantic retrieval, enabling rapid matching of semantic vectors and return of results.

[0137] Specifically, open-source vector databases (such as Milvus and FAISS) are selected as the vector knowledge base. The associated summary document blocks, target semantic vectors, and metadata are imported according to the database's specified format. This method is suitable for scenarios with high cost control requirements and certain technical deployment capabilities, achieving efficient storage and retrieval of vector data. Cloud-native vector databases (such as Alibaba Cloud Vector Database and AWS OpenSearch Service) are selected, and data is imported in batches via API interfaces. This method is suitable for large-scale data storage and high-concurrency retrieval scenarios, improving data storage stability and retrieval response speed. During data import, appropriate index types are configured for the vector knowledge base (such as IVF_FLAT (Inverted File Flat) and HNSW (Hierarchical Navigable Small World)). This optimizes vector similarity query efficiency and improves retrieval response speed. The data in the vector knowledge base is regularly updated and maintained, deleting invalid data and supplementing relevant data for newly added chips. This ensures the accuracy and timeliness of the vector knowledge base data.

[0138] This embodiment, through the above-described scheme, ensures the accuracy of semantic capture by splitting the target summary document into semantically independent summary document blocks; transforms the document blocks into target semantic vectors to achieve quantitative semantic representation; generates and associates metadata to provide retrieval and traceability clues; and stores the document blocks, semantic vectors, and metadata in a vector knowledge base to construct a complete semantic retrieval data foundation. This constructs a vector knowledge base supporting semantic retrieval, effectively solving the problems of poor semantic relevance and limited retrieval methods in traditional keyword retrieval. It achieves flexible and accurate semantic-based retrieval, allowing users to quickly find chip data that meets their needs through natural language descriptions, improving the retrieval efficiency and utilization value of chip data, and providing strong support for chip R&D, selection, and other related work.

[0139] Based on the above implementation scheme, in one feasible implementation, the step of using a pre-trained text embedding model to vectorize the summary document block and generate the corresponding target semantic vector includes S71~S73:

[0140] Step S71: Input the summary document block into the pre-trained text embedding model and convert it into a word sequence.

[0141] It should be noted that the lexical sequence is a sequence of basic language units obtained by splitting the text of the summary document block according to the model's preset rules. The core technology is lexical segmentation technology. A lexical can be a single Chinese character, word, or sub-word (such as the root of an English word). Its function is to split continuous text into discrete units that the model can process, laying the foundation for subsequent word vector mapping.

[0142] Specifically, the raw text of the summary document block can be directly input into a pre-trained text embedding model (such as Sentence-BERT). The model's built-in tokenizer automatically performs lexical segmentation and generates a lexical sequence. This method is suitable for scenarios with standardized text formatting and no special characters, and its purpose is to quickly and conveniently generate lexical sequences. If the summary document block contains special characters (such as formula symbols, special punctuation) or redundant information (such as meaningless spaces and line breaks), special characters and redundant information are first removed using a text preprocessing tool (such as Python's `re` library) before being input into the model for lexical segmentation. This method aims to avoid interference from special characters and redundant information in the lexical segmentation results, ensuring the accuracy and effectiveness of the lexical sequence. The two implementation methods can be flexibly selected based on the text quality of the summary document block. When the text quality is high, the first method is used to improve efficiency; when the text contains interfering information, the second method is used to ensure the quality of segmentation.

[0143] Step S72: Map the word sequence into word vectors using the text embedding model.

[0144] It should be noted that the lexical sequence is a generated sequence of discrete language units, with the core technology being lexical segmentation, which provides the basic units that the model can process for word vector mapping. Word vectors are low-dimensional dense vectors obtained after mapping lexical units, with the core technology being word embedding, which can represent the semantic connotation of lexical units. Furthermore, word vectors corresponding to semantically similar lexical units are relatively close, and their function is to achieve a quantitative representation of lexical semantics.

[0145] Specifically, a Transformer-based text embedding model (such as BERT-base) is used for word vector mapping. This model can fully capture long-distance contextual semantic relationships through a multi-head attention mechanism, making it suitable for semantically complex and context-dependent summary document blocks, thus improving the semantic representation accuracy of word vectors. A lightweight text embedding model (such as DistilBERT) is also employed. This model, while maintaining a certain semantic representation capability, offers faster processing speed and lower resource consumption, making it suitable for batch processing of large numbers of summary document blocks, thus balancing processing efficiency and semantic accuracy. During the mapping process, the positional encoding information of the words is preserved. Positional encoding reflects the relative position of words in the sequence, helping the model distinguish the same words in different positions (such as "characteristics" in different contexts), further enhancing the semantic discriminativeness of word vectors.

[0146] Step S73: Perform pooling operation on the word vectors obtained by mapping to generate a target semantic vector of fixed dimension.

[0147] It's important to note that pooling is a technique for aggregating word vector sequences. The core technology is vector aggregation algorithms; in this step, mean pooling is used. Its function is to transform variable-length word vector sequences into fixed-length vectors while integrating the semantic information of all words. Fixed dimension refers to the uniformity of the target semantic vector's dimension (e.g., 768-dimensional, 1024-dimensional). The core technology is vector dimension standardization, which ensures that the target semantic vectors corresponding to all summary document blocks have consistent dimensions, facilitating subsequent semantic similarity calculation and storage. The target semantic vector is the final semantic vector generated after the pooling operation. Its core technology is semantic integration, which comprehensively represents the overall semantic connotation of the summary document block. Its function is to serve as the core matching basis for semantic retrieval, enabling semantic similarity comparison between different texts.

[0148] Specifically, one approach uses mean pooling to calculate the mean of each dimension in the word vector sequence, generating the target semantic vector. This method balances the semantic contribution of all word elements and is suitable for most scenarios, ensuring the comprehensiveness and stability of semantic representation. Another approach uses weighted mean pooling, assigning different weights to word elements based on their importance in the text (e.g., keywords have higher weight than ordinary words), and then calculating the weighted mean. This method is suitable for scenarios where the semantics of core keywords need to be highlighted, improving the target semantic vector's ability to represent the core semantics. A third approach removes word vectors corresponding to special word elements (such as punctuation marks and meaningless auxiliary words) before pooling, avoiding interference from meaningless word elements and improving the purity of the target semantic vector.

[0149] This embodiment addresses the problem that continuous text cannot be directly embedded into a model by converting summary document blocks into a sequence of tokens that the model can process. A pre-trained text embedding model maps tokens to word vectors containing contextual semantics, achieving quantitative representation of token semantics and capture of contextual associations. Pooling operations integrate variable-length word vector sequences into fixed-dimensional target semantic vectors, ensuring the vectors' standardization and comparability. This implementation achieves the semantic quantification process for summary document blocks, and the generated target semantic vectors accurately represent the overall semantic connotation of the document blocks. This provides core data support for semantic retrieval in vector knowledge bases, effectively solving the problems of difficulty in quantifying text semantics and the inability to directly compare semantic similarity. It lays a key technical foundation for flexible and accurate semantic-based retrieval, improving the semantic relevance and efficiency of chip data retrieval.

[0150] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the chip data batch structured processing method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0151] This application also provides a device for batch structuring chip data; please refer to... Figure 2 The chip data batch structuring processing device includes:

[0152] The text parsing module 201 is used to parse the chip datasheet into structured text content through a document processor and generate a target summary document corresponding to the chip datasheet.

[0153] The table generation module 202 is used to construct an initial chip data table based on chip basic information, extract and classify the target summary document to obtain target parameters, and construct a target chip data table based on the target parameters and the initial chip data table.

[0154] The vector generation module 203 is used to extract summary document blocks based on the target summary document through a document chunker, perform vectorization processing on the summary document blocks, generate target semantic vectors, and store them in a vector knowledge base.

[0155] The chip data batch structuring processing apparatus provided in this application, employing the chip data batch structuring processing method in the above embodiments, can solve the technical problem of low retrieval efficiency of chip data. Compared with the prior art, the beneficial effects of the chip data batch structuring processing apparatus provided in this application are the same as those of the chip data batch structuring processing method provided in the above embodiments, and other technical features in the chip data batch structuring processing apparatus are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0156] This application provides a chip data batch structuring processing device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the chip data batch structuring processing method in the above embodiment 1.

[0157] The following is for reference. Figure 3 This document illustrates a structural diagram of a chip data batch structuring processing device suitable for implementing embodiments of this application. The chip data batch structuring processing device in this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 3 The illustrated chip data batch structuring device is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0158] like Figure 3As shown, the chip data batch structuring processing device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the chip data batch structuring processing device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the chip data batch structuring processing device to communicate wirelessly or wiredly with other devices to exchange data. While the figure shows chip data batch structuring processing devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0159] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0160] The chip data batch structuring processing device provided in this application, employing the chip data batch structuring processing method in the above embodiments, can solve the technical problem of low retrieval efficiency of chip data. Compared with the prior art, the beneficial effects of the chip data batch structuring processing device provided in this application are the same as those of the chip data batch structuring processing method provided in the above embodiments, and other technical features in this chip data batch structuring processing device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0161] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0162] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0163] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the chip data batch structuring processing method in the above embodiments.

[0164] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0165] The aforementioned computer-readable storage medium may be included in a chip data batch structuring device; or it may exist independently and not be assembled into a chip data batch structuring device.

[0166] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the chip data batch structuring device, the chip data batch structuring device performs the following actions: 1) Parses the chip datasheet into structured text content using a document processor, generating a target summary document corresponding to the chip datasheet; 2) Constructs an initial chip data table based on basic chip information, extracts and classifies the target summary document to obtain target parameters, and constructs a target chip data table based on the target parameters and the initial chip data table; 3) Extracts summary document blocks from the target summary document using a document chunker, performs vectorization processing on the summary document blocks, and generates target semantic vectors which are stored in a vector knowledge base.

[0167] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0168] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0169] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0170] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described chip data batch structured processing method, thereby solving the technical problem of low retrieval efficiency of chip data. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the chip data batch structured processing method provided in the above embodiments, and will not be repeated here.

[0171] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the chip data batch structuring processing method described above.

[0172] The computer program product provided in this application can solve the technical problem of low retrieval efficiency of chip data. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the chip data batch structured processing method provided in the above embodiments, and will not be repeated here.

[0173] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for batch structured processing of chip data, characterized in that, The method for batch structuring of chip data includes: The chip datasheet is parsed into structured text content using a document processor, and a target summary document corresponding to the chip datasheet is generated. An initial chip data table is constructed based on the chip's basic information. The target summary document is extracted and classified to obtain target parameters. A target chip data table is constructed based on the target parameters and the initial chip data table. The document chunker extracts summary document chunks based on the target summary document, and the summary document chunks are vectorized to generate target semantic vectors which are stored in the vector knowledge base. The step of parsing the chip datasheet into structured text content using a document processor and generating a target summary document corresponding to the chip datasheet includes: The chip datasheet is input into the document processor, which parses it to obtain the structured content of the chip datasheet, which includes text information, table information and image information; The structured content is input into a pre-built summary extractor to generate a target summary document of the chip datasheet. The target summary document contains the main characteristics of the chip, chip application scenarios, and chip product descriptions. The steps of constructing an initial chip data table based on basic chip information, extracting and classifying target parameters from the target summary document, and constructing a target chip data table based on the target parameters and the initial chip data table include: Based on the chip datasheet, obtain the basic chip information of the corresponding target chip, and construct an initial chip data table based on the basic chip information. The basic chip information includes chip number, chip name and company name. The parameter information dictionary of the target chip is obtained by extracting parameters from the target summary document using a pre-built parameter extractor. The target summary document is type-identified by a pre-built type classifier to determine the chip type of the target chip, wherein the chip type includes primary chip type and secondary chip type; The initial chip data table is supplemented based on the parameter information dictionary and the chip type to obtain the target chip data table for the target chip; The steps of extracting summary document blocks from the target summary document using a document chunker, vectorizing the summary document blocks, and generating target semantic vectors to be stored in a vector knowledge base include: Based on the title hierarchy of the target summary document, the document chunker is used to divide the target summary document into several summary document blocks; The summary document blocks are vectorized using a pre-trained text embedding model to generate corresponding target semantic vectors; Metadata is generated and associated based on the digest document block. The metadata includes at least the chip digest document identifier to which the digest document block belongs. The metadata is data describing the attributes and associated information of the digest document block. The summary document block, its corresponding target semantic vector, and associated metadata are stored in the vector knowledge base.

2. The chip data batch structured processing method as described in claim 1, characterized in that, The step of inputting the structured content into a pre-built summary extractor to generate the target summary document of the chip datasheet also includes the following before: Obtain the training summary document as a first-context learning paradigm; Generate the first prompt word based on the main features, application scenarios and product descriptions in the chip datasheet; Based on the first context learning example and the first prompt word, the large language model is trained in a guided manner to generate a summary extractor.

3. The chip data batch structured processing method as described in claim 1, characterized in that, Before the step of extracting parameters from the target summary document using a pre-built parameter extractor to obtain the parameter information dictionary of the target chip, the method further includes: We obtain manually annotated parameter extraction examples as second context learning paradigms; Based on the names of the key parameters of the chip to be extracted, generate a second prompt word to guide the extraction of the corresponding parameter values ​​and ranges; Based on the second context learning paradigm and the second prompt word, the large language model is trained in a guided manner to construct the parameter extractor.

4. The chip data batch structured processing method as described in claim 1, characterized in that, The step of vectorizing the summary document blocks using a pre-trained text embedding model to generate corresponding target semantic vectors includes: The summary document block is input into the pre-trained text embedding model and converted into a word sequence; The text embedding model maps the word sequence into word vectors. The word vectors obtained from the mapping are pooled to generate target semantic vectors of fixed dimensions.

5. A device for batch structuring chip data, characterized in that, The chip data batch structuring processing device includes: The text parsing module is used to parse the chip datasheet into structured text content through a document processor, and generate a target summary document corresponding to the chip datasheet. The table generation module is used to construct an initial chip data table based on basic chip information, extract and classify the target summary document to obtain target parameters, and construct a target chip data table based on the target parameters and the initial chip data table. The vector generation module is used to extract summary document blocks based on the target summary document through the document chunker, perform vectorization processing on the summary document blocks, and generate target semantic vectors to be stored in the vector knowledge base. The text parsing module is further configured to input the chip datasheet into the document processor and parse it to obtain the structured content of the chip datasheet, wherein the structured content includes text information, table information and image information; The structured content is input into a pre-built summary extractor to generate a target summary document of the chip datasheet. The target summary document contains the main characteristics of the chip, chip application scenarios, and chip product descriptions. The table generation module is further configured to obtain basic chip information of the corresponding target chip based on the chip datasheet, and construct an initial chip data table based on the basic chip information, wherein the basic chip information includes chip number, chip name and company name; The parameter information dictionary of the target chip is obtained by extracting parameters from the target summary document using a pre-built parameter extractor. The target summary document is type-identified by a pre-built type classifier to determine the chip type of the target chip, wherein the chip type includes primary chip type and secondary chip type; The initial chip data table is supplemented based on the parameter information dictionary and the chip type to obtain the target chip data table for the target chip; The vector generation module is further configured to divide the target summary document into several summary document blocks using a document blocker based on the title hierarchy structure of the target summary document. The summary document blocks are vectorized using a pre-trained text embedding model to generate corresponding target semantic vectors; Metadata is generated and associated based on the digest document block. The metadata includes at least the chip digest document identifier to which the digest document block belongs. The metadata is data describing the attributes and associated information of the digest document block. The summary document block, its corresponding target semantic vector, and associated metadata are stored in the vector knowledge base.

6. A chip data batch structured processing device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the chip data batch structuring processing method as described in any one of claims 1 to 4.

7. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the chip data batch structuring processing method as described in any one of claims 1 to 4.

Images