Method of data processing, electronic device

Abstract

The present disclosure provides a data processing method and electronic equipment, and relates to the technical field of data processing. The present disclosure receives a storage path of a target document and a processing range, uses a large model to analyze the document to generate structured data containing information such as a to-be-developed hardware module code, a version number, a project to which the to-be-developed hardware module belongs, and multi-level classification, stores and associates each type of information after verification, and combines permission control and version management to achieve a technical scheme for systematic control. The technical scheme can solve the problems of low data accuracy, repeated entry, difficult version tracing, and missing multi-dimensional classification statistics in the existing data management of the to-be-developed hardware module due to the dependence on the mixed mode of documents and tables, thereby achieving the technical effects of reducing manual operation errors, improving data management security, supporting full version tracing, and providing data support for server research and development decisions through intelligent classification and statistical analysis.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of data processing, and in particular, to a data processing method and an electronic device. BACKGROUND

[0002] In the field of server hardware research and development, Common Building Block (CBB) is widely used in key links such as motherboard design, storage architecture, IO module and GPU platform as a core element to improve research and development efficiency, reduce development cost, enhance system flexibility and product consistency. With the continuous evolution of server product diversification and customer demand, the number and version of CBB components show exponential growth, and the management and analysis requirements are increasingly complex. In related technologies, a preliminary CBB data management process is constructed by recording CBB design information through a PDF document, assisting data organization through an Excel table, and managing file paths through an SVN version control system. However, in the existing CBB data management method, a mixed mode of PDF and Excel is directly used, and automatic parsing and structured storage of data are not realized, which may lead to problems such as data duplication, version information confusion, and classification and statistical difficulties. The existing method cannot effectively support multi-version tracing and multi-dimensional statistical analysis, limiting the application value of CBB data in research and development decision-making, and it is difficult to meet the deep-seated needs of modern server hardware development for data governance and intelligent management. SUMMARY

[0003] The present disclosure provides a data processing method and an electronic device. The main purpose is to solve the problem of CBB data management confusion in related technologies.

[0004] According to a first aspect of the present disclosure, a data processing method is provided, comprising:

[0005] receiving storage path information of a target document in a file version library and a to-be-processed data range input through a human-computer interaction interface, and obtaining the target document based on the storage path information;

[0006] calling a data extraction interface to perform data extraction on document content in the to-be-processed data range in the target document, and inputting the extracted data to a pre-trained model, so as to perform format processing on the extracted data based on a preset prompt word, to obtain preset format data;

[0007] integrating the preset format data into target format data of a preset format based on a predefined field mapping relationship, the target format data at least including coding, version number, belonging project and multi-level classification information of a to-be-developed hardware module;

[0008] outputting and displaying the target format data for verification;

[0009] In response to the target format data passing the verification, the encoding, the version number, the project information to which the hardware module belongs, and the multi-level classification information of the hardware module to be developed included in the verified target format data are stored respectively and are associated.

[0010] According to a second aspect of the present disclosure, an electronic device is provided, comprising:

[0011] at least one processor; and

[0012] a memory connected to the at least one processor in communication; wherein,

[0013] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of data processing of the first aspect.

[0014] According to a third aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to perform the method of data processing of the first aspect.

[0015] According to a fourth aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method of data processing of the first aspect.

[0016] The present disclosure provides a method of data processing and an electronic device. The present disclosure receives a storage path of a target PDF document and a range to be processed, uses a large model to parse the document to generate structured data containing CBB encoding, version number, project to which the CBB belongs, and multi-level classification, etc., stores and associates each type of information after verification, and realizes systematic control by combining permission control and version management. The technical solution can solve the problems of low data accuracy, repeated input, difficult version traceability, and missing multi-dimensional classification statistics in existing CBB data management due to the dependence on the mixed mode of PDF and Excel, thereby reducing manual operation errors, improving data management security, supporting full version traceability, and providing data support for server research and decision-making through intelligent classification and statistical analysis.

[0017] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings are used to better understand the present scheme and do not limit the present disclosure. Among them:

[0019] Figure 1A flowchart of a data processing method provided by an embodiment of the present disclosure is shown in FIG. 1.

[0020] Figure 2 A structural diagram of a data processing apparatus provided by an embodiment of the present disclosure is shown in FIG. 2. DETAILED DESCRIPTION

[0021] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to help the understanding of the present disclosure. These should be considered in the context of the overall description and should not be considered to limit the scope of the present disclosure. Thus, those ordinarily skilled in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and structures are omitted in the following description for the sake of clarity and conciseness.

[0022] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] In combination with a specific application environment architecture or a specific hardware architecture on which the data processing method is executed, the specific application environment architecture or the specific hardware architecture is described herein.

[0024] The data processing method and the electronic device of the embodiments of the present disclosure are described below with reference to the accompanying drawings.

[0025] Figure 1 A flowchart of a data processing method provided by an embodiment of the present disclosure is shown in FIG. 1.

[0026] As shown in FIG. 1, the method comprises the following steps: Figure 1

[0027] In step 101, the storage path information of a target document in a file version library and a data range to be processed input through a human-computer interaction interface are received, and the target document is acquired based on the storage path information.

[0028] ​In the embodiments of the present disclosure, in the CBB data management process developed by the server, when new CBB data needs to be processed, first, relevant information is received through a man-machine interaction interface. The man-machine interaction interface here is a visual interface for CBB responsible persons to operate, and the interface is provided with an "add" operation portal. The responsible person enters the information input link by triggering the "add" operation. In this link, the information that the responsible person needs to input includes the storage path information of the target document in the file version library and the data range to be processed. In some possible embodiments, the target document specifically refers to a PDF document recording CBB development related information. The document is generated after the CBB discussion result is consistent, and contains key contents such as basic information required for CBB development, customer requirement detail description, discussion detail record, default save location of CBB final development file, unique CBB code, version number, etc., and is saved together with the final development file of CBB in the file version library. The file version library adopts a Subversion (SVN) version control system, the core function of which is to specify the location of the version library to ensure that the development team can collaboratively manage version changes of codes and related files, so the storage path information refers to the network address of the PDF document in the SVN, and the target document can be accurately located through the address. The data range to be processed refers to the page number list of the table in the target PDF document from which the data needs to be extracted. Specifying the range can effectively reduce the amount of text data to be processed in the subsequent data parsing process, and improve the processing efficiency. After the responsible person completes the above information input and confirms, the background module of the system receives and recognizes these input information, and then automatically connects with the SVN version library based on the obtained storage path information, locates the target PDF document according to the path, and performs a download operation, thereby completing the acquisition of the target document. Through the explicit information input and the automatic document acquisition mechanism, the accuracy and pertinence of the target document based on the subsequent data processing are ensured, and a reliable foundation is laid for the intelligent filling and statistical classification process of the entire CBB data. Its beneficial effects lie in that the target information is accurately received and acquired, the tedious operation of manually searching and acquiring the document is reduced, and the efficiency and accuracy of the whole process are improved by specifying the data range to be processed.

[0029] In step 102, a data extraction interface is called to extract the document content in the data range to be processed in the target document, and the extracted data is input into a pre-trained model to format the extracted data based on a preset prompt word, to obtain preset format data.

[0030] In an embodiment of the present disclosure, after obtaining the target document, the system enters the data extraction and format processing phase of the document content. The target document here is a PDF file containing CBB related information, and the data range to be processed has been clearly defined as the page number list of the table whose data needs to be extracted. The system will call the data extraction interface, which can be but is not limited to the Tabula library based on Python to realize the function. The core function of the interface is to accurately extract the table data in the specified page range of the target PDF document. The Tabula library can recognize the table structure in the PDF and convert the originally unstructured table content into structured data, such as text or table objects arranged by rows and columns, ensuring that the extracted data retains the original row-column correspondence and content integrity of the table, and providing standardized basic data for subsequent processing.

[0031] After extracting the structured data, the system inputs it into the pre-trained model, which can be a pre-set large language model. The large language model adopts a local deployment mode, which can effectively ensure the security of the company's CBB data and avoid the risk of data leakage. At the same time of inputting the data, the system will input a pre-set prompt word to the large language model. The prompt word is specially designed to clearly require the model to format the input structured table data, for example, the prompt word can be set as "You are a data processing expert, please convert the input table data into standard JSON format. Requirements: 1. Process each table separately 2. Keep the original column name 3. Automatically identify numerical types 4. Fill in null for missing values". After receiving the structured data and the pre-set prompt word, the large language model will process it according to the requirements of the prompt word based on its understanding ability of natural language and data format, and convert the structured data into pre-set JSON format data, ensuring that the output data fields are clear and the format is uniform, which is convenient for subsequent system integration and use of data.

[0032] This step realizes the efficient conversion from PDF table to structured data and then to pre-set format data through the cooperation of automatic data extraction interface and large language model. Its beneficial effects are to reduce the manual extraction and format conversion operations, reduce the risk of data errors caused by human operation, and at the same time, the unified data format provides convenience for subsequent data processing links, improving the efficiency and accuracy of the whole process.

[0033] In step 103, based on the pre-defined field mapping relationship, the pre-set format data is integrated into target format data in a pre-set format, and the target format data includes at least the code, version number, project to which the hardware module belongs, and multi-level classification information of the hardware module to be developed.

[0034] In the embodiments of the present disclosure, in the process of integrating the preset format data to obtain the target format data, firstly, the predefined field mapping relationship is relied on. The mapping relationship is predefined by the system based on the CBB data management requirements in the design stage, specifically, the Chinese-English correspondence of the CBB data fields stored in the database and the internal field definition of the system. For example, for the original field of “CBB code”, the mapping relationship corresponds to the “code” field defined in the system; “first classification” corresponds to “firstCat”, “second classification” corresponds to “secondCat”, “third classification” corresponds to “thirdCat”, “version number” corresponds to “version”, “project” corresponds to “project”, etc., and also includes the mapping of the custom fields under each classification, such as “CPU platform” corresponds to “platform”, “package” corresponds to “package”, etc. These mapping relationships ensure the accurate conversion from the original data to the system recognizable data.

[0035] When the system background receives the preset format data (usually in JSON format, containing “CBB code”, “version number”, “project”, “first classification”, “second classification”, “third classification” and various custom fields, etc.) obtained after being processed by the large language model, the preset integration logic is called to convert and reorganize the data according to the above-mentioned predefined field mapping relationship. In the integration process, not only will the original field name be replaced by the internal field name of the system, but also structured data containing “label” and “value” will be constructed for each field, where “label” retains the Chinese name of the original field, and “value” is the specific data value corresponding to the field. For example, “CBB code”: “CBB_mainboard_CPU_AMD_001_V1.0” in the original JSON is integrated into “code”: {“label”: “CBB code”, “value”: “CBB_mainboard_CPU_XXX_001_V1.0”} (XXX is the manufacturer's name), and so on. The final target format data contains the code, version number, project (such as “project 1”, “project 2”, etc.) and multi-level classification information (first classification such as “mainboard”, second classification such as “CPU”, third classification such as “XXX”, etc.) of the hardware module to be developed (i.e. CBB), and also covers the custom field data corresponding to each classification.

[0036] Regarding the accuracy and efficiency of large language model processing data, in terms of accuracy, on the one hand, clear prompt words are preset (such as requiring table data to be converted into standard JSON format, retaining original column names, automatically identifying numerical types, and filling missing values with null, etc.) to guide the model to process data according to specifications; on the other hand, the model processing results will be manually checked by CBB responsible persons, and the data with parsing errors will be manually modified to ensure the accuracy of the final data. In terms of efficiency, by limiting the range of data to be processed (i.e. specifying the page list of the table), the amount of text data that the model needs to process is reduced; by using a locally deployed large language model, data transmission delay is reduced; at the same time, by using the Tabula library of Python to efficiently extract table data, the processing speed before the data is input into the model is improved, thereby improving the overall processing efficiency.

[0037] This step realizes the standardized integration of data through pre-defined field mapping relationship, ensuring the unified recognition and management of CBB data by the system, which has the beneficial effect of reducing the processing obstacles caused by non-uniform data formats, laying a standard foundation for the storage, statistics and classification of subsequent data, and improving the consistency and reliability of data processing.

[0038] Step 104, output and display the target format data for verification.

[0039] In the embodiments of the present disclosure, after the target format data generated in step 103 is re-integrated, it is output to the front-end web interface developed based on the Spring+Vue front-end and back-end separation mode for display, so as to be checked by the CBB person in charge. In the integration process, the original JSON format data is converted into a structured display form containing "label" and "value" by combining the pre-stored English and Chinese corresponding relationship of each field in the database, that is, "label" corresponds to the Chinese name of the field, and "value" corresponds to the specific data value obtained by parsing, for example, the display form of the "code" field is { "label": "CBB code", "value": "CBB_mainboard_CPU_xxx (xxx is the manufacturer's name)_001_V1.0"}, the display form of the "firstCat" field is { "label": "first-level classification", "value": "mainboard"}, and so on. This form can make the CBB person in charge more intuitively understand the meaning of each field and the corresponding data. As the core carrier of human-computer interaction, the front-end web page clearly presents all the contents contained in the target format data of the hardware module to be developed, such as code, version number, project to which it belongs, and multi-level classification information, so that the person in charge can comprehensively view the parsing result. The reason for display and verification is that although the intelligent filling of PDF table data through a large model can be realized, due to factors such as complexity of document format and ambiguity of table content, the parsing result may be biased. Therefore, the CBB person in charge needs to check the accuracy of the data field by field on the front-end web page, and if it is found that the "value" of a certain field does not match the actual content of "Table 1: CBB attribute table" in the PDF document (for example, "CPU number" is parsed as "2S" while it should be "1S"), manual modification can be directly performed on the web interface until it is confirmed that all data is accurate. This verification process is a key link to ensure the reliability of subsequent data storage and management, and through manual intervention, it ensures that the target format data is consistent with the original document information, and lays a foundation for the standardized management of CBB data.

[0040] In step 105, in response to the target format data passing the verification, the code, version number, project information and multi-level classification information of the hardware module to be developed included in the verified target format data are stored and associated respectively.

[0041] In the embodiments of the present disclosure, when the target format data passes the verification of the CBB supervisor, the system will respond to the confirmation operation and store the code, version number, project information and multi-level classification information of the to-be-developed hardware module (i.e. CBB) contained in the verified target format data, and realize the association between the information through identifiers (IDs). Specifically, the system background will store different types of information in corresponding database tables: the code (such as "CBB_mainboard_CPU_xxx (xxx is the manufacturer's name)_001_V1.0", which is used to uniquely identify the CBB) and version number (such as "V1.0", which records the version iteration state) of the CBB will be stored in the CBB main data table as core information to support the basic information management of a single CBB; the project information (such as "project 1", "project 2", etc., which identifies the customer's customized demand project corresponding to the CBB) will be stored in the project information table; and the multi-level classification information (including first-level classification such as "mainboard", second-level classification such as "CPU", and third-level classification such as "xxx (xxx is the manufacturer's name)", which reflects the technical attributes and hierarchical attribution of the CBB) will be stored in the classification information table.

[0042] During the storage process, the system will automatically verify whether these information already exists in the corresponding table: if the project information table already contains the project "general", the system will directly obtain the corresponding project ID; if not, the system will add the project in the project information table and generate a new project ID. The storage logic of the classification information is consistent with this, for example, when the third-level classification "xxx (xxx is the manufacturer's name)" already exists in the classification information table, the classification ID is directly obtained, otherwise the ID is added and generated. Finally, these project IDs and classification IDs generated through verification or addition will be filled into the CBB main data table, associated with the code and version number of the CBB, forming an association relationship of "code-version number-project ID-classification ID", ensuring the logical connection between the information, and providing a data basis for subsequent version tracing, classification statistics and other functions. At the same time, this way of storing and associating in separate tables not only ensures the standardization and independence of the data, but also realizes the integration of the information through ID association, avoids data redundancy, and improves the efficiency and accuracy of CBB data management.

[0043] The present disclosure provides a method for data processing, which comprises the following steps: receiving a storage path of a target PDF document and a processing range, using a large model to analyze the document to generate structured data containing CBB code, version number, project information and multi-level classification information, and after verification, storing and associating each type of information, and combining permission control and version management to realize systematic control. The technical scheme can solve the problems of low data accuracy, repeated input, difficult version traceability and multi-dimensional classification statistics in existing CBB data management due to the dependence on the mixed mode of PDF and Excel, thereby reducing manual operation errors, improving data management security, supporting full version traceability, and providing data support for server research and development decisions through intelligent classification and statistical analysis.

[0044] In the embodiments of the present disclosure, for the operation of "storing and associating the code, version number, project information and multi-level classification information of the to-be-developed hardware module included in the target format data after verification", the specific implementation modes are various. For clear illustration, the following enumerated implementation modes include but are not limited to some implementation modes: storing the code and the version number of the to-be-developed hardware module as a version record in a first data table; querying a second data table to determine whether the project information exists, if not, inserting a new record and obtaining a unique identifier of the project information, if yes, directly obtaining the unique identifier of the project information; querying a third data table to determine whether the multi-level classification information exists, if not, inserting a new record and obtaining a unique identifier of the multi-level classification information, if yes, directly obtaining the unique identifier of the multi-level classification information; and storing the unique identifier of the version record, the unique identifier of the project information and the unique identifier of the multi-level classification information in a relationship table to complete the association.

[0045] Specifically, the to-be-developed hardware module is a CBB (common building block) in the research and development of a server, the code (such as "CBB_mainboard_CPU_xxx (xxx is the name of a manufacturer)_001_V1.0", used to uniquely identify the CBB) and version number (such as "V1.0", recording the version iteration state) of which are stored in a first data table (namely, a CBB master data table), which is mainly used to record the core identification and version information of a single CBB and is a basic carrier for associating other information. The information of the project to which the CBB belongs is stored in a second data table (namely, a project information table), which is used to uniformly manage all project names and related attributes, and to ensure the standardization and uniqueness of the project information. Multi-level classification information (including first-level classification such as "mainboard" and "infrastructure", second-level classification such as "CPU" and "storage", and third-level classification such as "xxx (xxx is the name of a manufacturer)" and "SAS", reflecting the technical attributes and hierarchical attribution of the CBB) is stored in a third data table (namely, a classification information table), which records the classification names and corresponding relationships according to the hierarchy and supports hierarchical classification management of the CBB.

[0046] The association of the three tables is achieved through predetermined identification information (namely, unique identifiers ID of each table): when storing data, the system will first check whether the project information to which the CBB belongs already exists in the second data table. If it exists, the unique project ID corresponding to the project is directly obtained; if it does not exist, the project information is added to the second data table and a unique project ID is automatically generated. For the multi-level classification information in the third data table, the system will also check whether the corresponding classification information already exists, and if it exists, the classification ID is obtained, and if it does not exist, the classification ID is added and generated. Then, the system fills the obtained project ID and classification ID into the first data table (CBB master data table), binds them with the code and version number of the CBB, and forms an association relationship of "code-version number-project ID-classification ID". For example, for a CBB data with a code of "CBB_mainboard_CPU_xxx (xxx is the name of a manufacturer)_001_V1.0", a version number of "V1.0", a project of "general", and a multi-level classification of "mainboard (first level)-CPU (second level)-xxx (xxx is the name of a manufacturer) (third level)", after the code and version number are stored in the first data table, the project ID corresponding to the "general" project in the second data table and the classification ID corresponding to the "mainboard-CPU-xxx (xxx is the name of a manufacturer)" classification in the third data table are recorded in the corresponding fields of the first data table, thereby realizing the accurate association of the three tables and providing structured data support for subsequent version tracing, classification statistics and other functions.

[0047] The first data table is a CBB master data table for storing the code and version number of a hardware module to be developed (CBB), the second data table is a project information table for storing project information, the third data table is a classification information table for storing multi-level classification information, the predetermined identification information is a unique identifier (ID) recorded in each table, and the target data table is the first data table (CBB master data table). When the target format data after inspection enters the storage link, the system first checks the second data table (project information table): if the project information (such as "general") to be stored already exists in the project information table, the system will directly obtain the unique project ID corresponding to the project; if it does not exist, the system will add the project information in the project information table and automatically generate a unique project ID. For the third data table (classification information table), the system will also check whether the multi-level classification information to be stored already exists, and if it exists, the corresponding classification ID will be obtained, and if it does not exist, the classification ID will be added and generated. Subsequently, the system stores the project ID and classification ID (i.e., the predetermined identification information) obtained above into the corresponding fields in the first data table (CBB master data table). By storing the project ID and classification ID in the CBB master data table (target data table) in this way, the CBB master data table can be directly associated with the project information table and the classification information table through these IDs. When the details of the project to which a CBB belongs or the classification level of the CBB needs to be queried, the system can quickly locate the corresponding records in the project information table and the classification information table through the project ID and classification ID stored in the CBB master data table, realize the linkage query and management of multi-table data, and ensure the accuracy and efficiency of the CBB data association.

[0048] In the embodiments of the present disclosure, the method for data processing further includes other specific implementation steps in addition to the foregoing. To clearly present these components, the following detailed description of the related specific implementation is expanded: before storing the code, version number, project information and multi-level classification information of the hardware module to be developed included in the target format data after inspection, the identity information of the currently logged-in user is obtained; the information of the person in charge of the module is queried from the permission configuration table according to the code of the hardware module to be developed; it is checked whether the current user is the person in charge of the module or has administrator permission; if the check fails, the storage process is terminated and a prompt information of insufficient permission is returned.

[0049] Specifically, before storing the target format data after inspection, strict permission verification procedures need to be performed to ensure data security and operation standardization. First, the system automatically obtains the identity information of the currently logged-in user, which usually includes the user's unique identifier (such as user ID, username, etc.) for accurate identification of the operation subject, which is the basis for permission verification. The CBB, a developed hardware module, has a unique code, such as "CBB_mainboard_CPU_AMD_001_V1.0". This code not only identifies the type and version of the CBB, but also serves as a key index for querying permission information. The system has a pre-configured permission configuration table that stores the association information between each CBB code and the corresponding responsible person, including the user identifier of the responsible person. Through the CBB code, the responsible person information can be directly located. After obtaining the current user's identity information and querying the responsible person information of the CBB, the system starts the permission verification mechanism: compares the current user's identity information with the queried responsible person information to determine whether the current user is the designated responsible person of the CBB; at the same time, the system also checks whether the current user has administrator privileges, which is a special privilege that allows users to operate all CBB data, usually held by system administrators. If the current user is neither the responsible person of the CBB nor has administrator privileges, the verification fails, and the system will immediately terminate the subsequent data storage process and return a prompt message to the user through the human-computer interaction interface that the user lacks permission and cannot complete the data storage. The beneficial effect of this permission verification step is that through strict identity recognition and permission control, only authorized personnel (responsible person or super user) can perform storage operations on CBB data, effectively preventing unauthorized users from modifying or storing data at will, ensuring the accuracy, security, and integrity of CBB data, and avoiding the risk of data errors or leaks due to chaotic permissions.

[0050] In the embodiments of the present disclosure, the method of data processing further includes other specific implementation steps in addition to the foregoing. To clearly present these components, the following detailed description of the related specific implementation is expanded: in response to the query instruction of the target to-be-developed hardware module, the first data table is queried according to the code of the target to-be-developed hardware module, and all version records of the target to-be-developed hardware module are obtained; the version records are sorted according to the size of the version number or the creation time sequence; and the sorted version record list is returned, and each record in the list contains a version number, a creation time, and a unique identifier of the version record.

[0051] Specifically, when querying the version record of the target hardware module to be developed (i.e., the target CBB), the system first responds to the query instruction. The query instruction is usually triggered by the user through the man-machine interaction interface. Specifically, the user can input the code of the target CBB (such as "CBB_mainboard_CPU_AMD_001") in the query function area of the interface and click the query button to initiate the query. The uniqueness of the code ensures the accurate positioning of the query object. After receiving the query instruction, the system will access the first data table according to the code of the target CBB. The first data table is a database table specially used to store the basic information of each version of the CBB. Its structure includes the code of the target CBB, the version number (such as "V1.0" and "V2.0", using semantic version number rules, composed of major version number and minor version number, and major version number change indicating major function change), creation time (time stamp accurate to seconds, recording the time when the version data is first stored), version record unique identifier (a system-generated string or numeric ID for uniquely distinguishing different version records of the same CBB), and other fields. By matching the code in the query instruction with the "code" field in the table, the system can filter out all version records of the target CBB, covering all historical data from the initial version to the latest version. After obtaining all version records, the system needs to sort them. There are two sorting rules: one is to sort by version number size, based on the semantic version number rule, comparing the values of the major version number and the minor version number, for example, "V2.1" is greater than "V2.0", and "V3.0" is greater than "V2.5"; the other is to sort by creation time in chronological order, usually in descending order, that is, the version record with newer creation time is placed in front, which is convenient for the user to view the latest version first. In actual application, the sorting method can be selected according to the user's settings when querying, and if no special settings are made, the default is to sort in descending order of creation time. After sorting, the system returns the sorted version record list to the man-machine interaction interface. Each record in the list clearly shows the version number, creation time, and version record unique identifier, so that the user can intuitively view the version evolution history of the target CBB.

[0052] In the embodiments of the present disclosure, the method of data processing further includes other specific implementation steps in addition to the foregoing. To clearly present these components, the following detailed description of the related specific implementation is expanded: receiving a statistical query request, the statistical query request containing a statistical dimension, the statistical dimension being a certain level in the project or multi-level classification information; according to the statistical dimension, aggregating querying the second data table or the third data table, calculating the number of modules under each dimension value and the percentage of the total number.

[0053] Specifically, in processing the statistical query request, first, the system receives the statistical query request initiated by the user. The request is usually triggered through a human-computer interaction interface, and the user can select the required statistical dimension in the statistical function area of the interface, which specifically includes a certain level of multi-level classification information or a project to which the user belongs. Among them, the project to which the user belongs refers to the project name set by the company according to the customer's customized needs; the multi-level classification information is the classification of the CBB by the system, including the first-level classification (such as "mainboard" "infrastructure"), the second-level classification (such as "CPU" "security" under "mainboard", "storage" "IO" under "infrastructure"), the third-level classification, etc., and the user can select any level of classification as the statistical dimension according to the needs. After receiving the statistical query request and identifying the statistical dimension therein, the system will determine the corresponding data source table for aggregation query according to the dimension type. Specifically, if the statistical dimension is the project to which the user belongs, the second data table is queried - this table is specially used to store the project information of the CBB, and records the association between the project name and the corresponding CBB data; if the statistical dimension is a certain level of multi-level classification (such as first-level classification, second-level classification or third-level classification), the third data table is queried - this table is used to store the multi-level classification information of the CBB, including the classification name of each level and its associated ID with the CBB data. In the aggregation query process, the system will extract all associated CBB data records from the corresponding table based on the selected statistical dimension, and count the number of CBB modules under each dimension value (i.e. the total number of CBBs belonging to the same project or the same classification level) through counting operation. For example, if the statistical dimension is "project to which the user belongs", the number of CBBs under "project 1", "project 2", etc. is counted; if the statistical dimension is "first-level classification", the number of CBBs under "mainboard", "infrastructure", etc. is counted. Then, the system calculates the percentage of the number of each dimension value in the total number of CBBs, and the calculation method is (the number under a certain dimension value ÷ the total number of CBBs) x 100%, and the result is kept to a certain number of decimal places (such as integer percentage). Finally, the system arranges the statistical results into structured data containing dimension value label (label), corresponding number (value) and percentage (percent), and returns to the human-computer interaction interface for the user to view.

[0054] In the embodiments of the present disclosure, for the operation of "integrating the preset format data into the target format data of the preset format based on the pre-defined field mapping relationship", the specific implementation manners are various. For clear illustration, the following enumerated implementation manners include but are not limited to part of the implementation manners: converting the fields in the preset format data based on the pre-defined field mapping relationship to generate the target format data containing the field label and the field value. The preset format data and the target format data are data exchange formats.

[0055] Specifically, the database pre-stores the Chinese-English corresponding relationship of the CBB data field (i.e., the pre-defined field mapping relationship), which is set according to the system design requirements. For example, the "CBB code" corresponds to the English field "code", the "version number" corresponds to "version", the "project" corresponds to "project", the "first classification" corresponds to "firstCat", the "second classification" corresponds to "secondCat", the "third classification" corresponds to "thirdCat", and the mapping of the self-defined field under each third classification (such as "CPU platform" of manufacturer 1 platform corresponding to "platform", "package method" corresponding to "package", etc.). The preset format data is the standard JSON format data (one of the data exchange formats) returned after the large model parses the PDF table, and the fields are presented in Chinese column names, such as containing "CBB code", "version number", "project", "first classification", etc. Key names and corresponding values. When integrating, the system converts the Chinese fields in the preset format data into corresponding English fields one by one based on the pre-defined field mapping relationship in the database, and constructs a structure containing "label" and "value" for each converted field - "label" retains the original Chinese column name (used for field description when displayed on the front end), and "value" is the specific data value corresponding to the field. For example, the value corresponding to "CBB code" in the preset format data is "CBB_mainboard_CPU_AMD_001_V1.0", and after conversion based on the mapping relationship, the "code" field will be generated in the target format data, and its structure is {"label": "CBB code", "value": "CBB_mainboard_CPU_xxx (xxx is the manufacturer's name)_001_V1.0"}; for the "first classification" field, the value in the preset format data is "mainboard", and after conversion, the structure of the "firstCat" field in the target format data is {"label": "first classification", "value": "mainboard"}. In this way, the integration of the preset format data is completed, and the final generated target format data is still in JSON format (data exchange format), which not only meets the database's specification for field naming, but also meets the needs of front-end page display through the structure of "label" and "value", providing a unified and standardized data basis for subsequent manual verification and data storage.

[0056] In the embodiments of the present disclosure, the method for data processing further includes other specific implementation steps in addition to the foregoing. To clearly present these components, the following detailed description of the related specific implementation is expanded: a text extraction interface is called to perform text extraction on the target document to obtain text data; a summary prompt word is constructed, which instructs a pre-trained model to generate a summary for the input text data; the text data and the summary prompt word are spliced and input into a preset large language model to obtain a summary text; and based on the summary text, retrieval information of the target document is generated.

[0057] Specifically, the target document is a PDF document (such as “CBB development design final draft.pdf”) recording CBB (common building block) related information, which contains basic information of CBB development, customer demand description, discussion details, etc. When the target document is text extracted, the system calls the Slate library of Python (a PDF extension library based on PDF, which provides a simple API for extracting PDF text) to read all the text content of the PDF document and obtain complete text data, ensuring that all descriptive information in the form of non-tables in the document is covered, providing a basis for subsequent semantic analysis.

[0058] When the extracted text data is subjected to semantic analysis to generate summary information, the system passes the text data to a locally deployed large model and guides the large model to perform semantic understanding and extraction of the text data through a preset prompt word (such as “Please output the summary information of the above text content, with a summary word limit of 500 words or less”) to finally generate summary information that can summarize the core content of the document. The summary can briefly reflect the development background, key features and discussion points of CBB, etc., facilitating quick grasp of the core of the document.

[0059] When generating retrieval information based on the target data table and the summary information in the target file, the target data table refers to a table storing CBB related structured data (such as a CBB master data table storing CBB codes, version numbers, project affiliations, multi-level classifications, etc., as well as associated project information tables, classification information tables, etc.), and the system saves these table data (including structured fields such as CBB codes, versions, classifications, projects, etc.) and the above generated summary text to an ElasticSearch retrieval engine. As an engine supporting full-text retrieval, ElasticSearch can combine the structured information of the table data and the unstructured information of the summary to form retrieval information. When a user needs to find existing CBB data through a fuzzy query (such as retrieving the development background or features of a related CBB based on a keyword), the system can quickly match the retrieval information in ElasticSearch and return the corresponding CBB data according to the matching degree, improving the retrieval efficiency and accuracy and providing support for quick query of CBB data.

[0060] In the embodiments of the present disclosure, the target document is a portable document format file, and the hardware module to be developed is a common building module.

[0061] Specifically, the target document is specifically a PDF format document, which is a "CBB development design final draft.pdf" generated after CBB (common building module) related discussion results are confirmed, and records key contents such as basic information required for CBB development (such as CBB code ensuring uniqueness, version number), customer requirement detail description, discussion detail record, and default storage location of the final CBB development file, and is stored in the same SVN directory as the final CBB development file, and is the core data source for system data analysis and intelligent filling. The hardware module to be developed, namely the common building module (CBB), is an important common component in the field of server hardware research and development, and plays a key role in improving research and development efficiency, reducing cost, improving product quality, and enhancing system flexibility. It is divided into multiple categories according to different application platforms such as CPU type, storage, IO module, GPU, and the number and version are increasing with the change of demand and technological development, which is the core object of data management, intelligent filling and statistical classification in the present scheme.

[0062] It should be noted that the embodiments of the present disclosure can include multiple steps, which are numbered for ease of description, but these numbers are not a limitation on the execution time slot and execution order between the steps; these steps can be implemented in any order, and the embodiments of the present disclosure do not limit this.

[0063] Corresponding to the above-mentioned data processing method, the present disclosure also proposes a data processing device. Since the device embodiments of the present disclosure correspond to the above-mentioned method embodiments, for the details not disclosed in the device embodiments, reference can be made to the above-mentioned method embodiments, which will not be described in detail in the present disclosure.

[0064] Figure 2 The structure diagram of a data processing device provided by the embodiments of the present disclosure is shown in Figure 2 as shown, comprising:

[0065] The receiving unit 21 is configured to receive the storage path information of the target document in the file version library and the data range to be processed input through the human-computer interaction interface, and acquire the target document based on the storage path information.

[0066] The extraction unit 22 is configured to call a data extraction interface to perform data extraction on the document content in the data range to be processed in the target document, and input the extracted data to a pre-trained model, so as to perform format processing on the extracted data based on a preset prompt word, and obtain preset format data.

[0067] The integration unit 23 is configured to integrate the preset format data into target format data of a preset format based on a predefined field mapping relationship, and the target format data includes at least the code of the hardware module to be developed, the version number, the project to which the hardware module to be developed belongs, and multi-level classification information.

[0068] The inspection unit 24 is configured to output and display the target format data for inspection.

[0069] The storage unit 25 is configured to store the code of the hardware module to be developed, the version number, the project information, and the multi-level classification information included in the target format data after inspection, and associate them.

[0070] The present disclosure provides a device for data processing, which receives a storage path of a target PDF document and a range to be processed, uses a large model to analyze the document to generate structured data containing CBB code, version number, project to which the hardware module to be developed belongs, and multi-level classification information, stores and associates each type of information after inspection, and realizes systematic control by combining permission control and version management. The technical scheme can solve the problems of low data accuracy, repeated input, difficult version traceability, and missing multi-dimensional classification statistics in existing CBB data management due to reliance on a mixed mode of PDF and Excel, thereby reducing manual operation errors, improving data management security, supporting full version traceability, and providing data support for server research and development decisions through intelligent classification and statistical analysis.

[0071] It should be noted that the above explanation and description of the method embodiment also apply to the device of the present embodiment, and the principles are the same. In the present embodiment, no further limitation is made.

[0072] The description of the features in the device corresponding to the embodiment of the data processing method can be referred to the related description of the embodiment corresponding to the data processing method, which will not be repeated here.

[0073] Embodiments of the present application also provide an electronic device comprising a memory and a processor, the memory storing a computer program, and the processor being configured to run the computer program to perform the steps in any of the above data processing method embodiments.

[0074] Embodiments of the present application also provide a computer readable storage medium storing a computer program, wherein the computer program is configured to perform the steps in any of the above data processing method embodiments when running.

[0075] In an example embodiment, the computer readable storage medium described above can include, but is not limited to, a U disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0076] Embodiments of the present application also provide a computer program product, which includes a computer program, and the computer program, when executed by a processor, implements the steps in any of the method embodiments of data processing described above.

[0077] Embodiments of the present application also provide another computer program product, which includes a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the steps in any of the method embodiments of data processing described above.

[0078] The skilled in the art can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in the above description in general terms. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. The skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0079] The above describes in detail the method of data processing and the electronic device provided by the present application. The principle and implementation of the present application are described by applying specific examples. The above description of the examples is only to help understand the method of the present application and its core idea. It should be pointed out that for ordinary skilled in the art, without departing from the principle of the present application, some improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A data processing method, characterized in that, include: Receive the storage path information of the target document in the file version library and the range of data to be processed, which are input through the human-computer interaction interface, and obtain the target document based on the storage path information; The data extraction interface is called to extract data from the document content within the range of data to be processed in the target document, and the extracted data is input into the pre-trained model so that the extracted data is formatted based on preset prompt words to obtain preset format data. Based on predefined field mapping relationships, the preset format data is integrated into a target format data in a preset format. The target format data includes at least the code, version number, project, and multi-level classification information of the hardware module to be developed. The target format data is output and displayed for verification. In response to the target format data passing verification, the code, version number, project information, and multi-level classification information of the hardware module to be developed included in the verified target format data are stored and associated, including: The code and version number of the hardware module to be developed are stored as a version record in a first data table; the second data table is queried to determine if the project information exists. If it does not exist, a new record is inserted and the unique identifier of the project information is obtained; if it exists, the unique identifier of the project information is directly obtained; the third data table is queried to determine if the multi-level classification information exists. If it does not exist, a new record is inserted and the unique identifier of the multi-level classification information is obtained; if it exists, the unique identifier of the multi-level classification information is directly obtained; the unique identifier of the version record, the unique identifier of the project information, and the unique identifier of the multi-level classification information are associated and stored in a relationship table to complete the association.

2. The data processing method according to claim 1, characterized in that, Before storing the verified target format data, including the code, version number, project information, and multi-level classification information of the hardware module to be developed, the method further includes: Get the identity information of the currently logged-in user; Based on the code of the hardware module to be developed, query the person in charge information of the module from the permission configuration table; Verify whether the current user is the person in charge of this module or has administrator privileges; If the verification fails, the storage process will be terminated and an insufficient permissions message will be returned.

3. The data processing method according to claim 1, characterized in that, The method further includes: In response to a query command for a target hardware module to be developed, the first data table is queried according to the code of the target hardware module to be developed to obtain all version records of the target hardware module to be developed; The version records are sorted according to their version number or creation time. Returns a sorted list of version records. Each record in the list contains a version number, creation time, and a unique identifier for the version record.

4. The data processing method according to claim 1, characterized in that, The method further includes: Receive a statistical query request, the statistical query request containing statistical dimensions, the statistical dimension being a certain level in the project or multi-level classification information; Based on the statistical dimensions, aggregate queries are performed on the second data table or the third data table to calculate the number of modules under each dimension value and their percentage of the total.

5. The data processing method according to claim 1, characterized in that, The preset format data and the target format data are a data exchange format; The process of integrating the preset format data into a target format data based on predefined field mapping relationships includes: Based on the predefined field mapping relationship, the fields in the preset format data are transformed to generate the target format data containing field labels and field values.

6. The data processing method according to claim 1, characterized in that, The method further includes: Call the text extraction interface to extract text from the target document and obtain text data; Construct summary prompts, which instruct a pre-trained model to generate a summary of the input text data; The text data is concatenated with the summary prompts and input into a preset large language model to obtain the summary text. Based on the abstract text, retrieval information for the target document is generated.

7. The data processing method according to claim 1, characterized in that, The process of calling the data extraction interface to extract data from the document content within the range of data to be processed in the target document includes: Analyze the range of data to be processed to determine the target page number where the document content is located; Call the data extraction interface to read the table data from the target page; The table data is converted into a structured data array containing column names and cell values ​​to complete the data extraction.

8. The data processing method according to any one of claims 1-7, characterized in that, The hardware module to be developed is a shared building block.

9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the data processing method according to any one of claims 1-8.

Images