Project declaration data integration and analysis method and system based on mapping relation

Abstract

The invention discloses a project declaration data integration and analysis method and system based on a mapping relation, and the method comprises the following steps: S1, obtaining project declaration data, and carrying out the preprocessing of the data; s2, storing the preprocessed data to a data server, and forwarding the preprocessed data to a feature server for feature extraction; s3, the feature server performs similarity retrieval in a feature database based on the feature vectors of the data items to be audited; and S4, extracting related data according to the audit record, and generating an approval document. According to the method, each piece of data is converted into a feature vector through feature extraction and convolution operation, the relevance between the data is judged by calculating the similarity, the traditional manual comparison and rule-based comparison modes are broken through, the data comparison efficiency is improved, and the comparison time can be greatly shortened; after data comparison and relevance judgment are completed, related approval files are automatically generated, manual intervention is not needed, the risk of manual filling errors is reduced, and the efficiency of the approval process is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a method for integrating and analyzing project application data based on feature mapping relationships. Background Art

[0002] In recent years, project application systems have been widely adopted in various industries, and online transmission and filling of application data has become the mainstream project application method. In order to ensure the accuracy and compliance of project application data, the higher-level competent authorities usually need to conduct strict review and approval of the application data.

[0003] At present, most units still use manual sorting to review the declared data, that is, manually comparing and judging the data in the Excel table, and manually filling in the relevant approval documents and technical specifications. The traditional data review method requires manual operation. As the amount of declared data increases, manual review becomes inefficient and time-consuming. The manual sorting and review process is prone to errors, which may affect the approval effect. In large-scale data processing, the traditional comparison method is inefficient and difficult to meet the needs of fast and accurate comparison. Summary of the invention

[0004] In view of the above problems, the present invention proposes a method for integrating and analyzing project application data based on mapping relationships. This method realizes efficient review and classification integration of project application data through the process of data uploading, feature extraction and comparison, data retrieval, and automatic generation of approval documents.

[0005] In order to achieve the above-mentioned purpose, the method of the present invention is a method for integrating and analyzing project application data based on mapping relationships, comprising the following steps:

[0006] S1. Obtain project application data and pre-process the data;

[0007] S2, storing the pre-processed data in the data server, and then forwarding it to the feature server for feature extraction;

[0008] S3, performing similarity search in a feature database based on the feature vector of the data item to be reviewed;

[0009] S4. Extract relevant data based on the audit records and generate approval documents.

[0010] As a preferred embodiment of the present invention, the specific steps of preprocessing the data in step S1 are:

[0011] S11. Conduct format review on uploaded declaration data, including field integrity check and value range verification;

[0012] S12. After the format is reviewed and approved, the data is converted into a database field format suitable for storage;

[0013] S13. The transformed data is stored in the database of the data server, and the transformed data fields are sent to the feature server.

[0014] As a preferred embodiment of the present invention, the specific steps of converting the data into a database field format suitable for storage in step S12 are:

[0015] According to the predefined database table structure, each field in the data is mapped to the corresponding database table field type;

[0016] For specific fields, format them to ensure they meet storage standards;

[0017] Perform data cleansing on uploaded data, including removing invalid fields, removing null values, and filling missing values ​​to ensure data integrity and consistency.

[0018] As a preferred embodiment of the present invention, the specific steps of step S2 storing the pre-processed data to the data server and then forwarding it to the feature server for feature extraction are as follows:

[0019] S21, feature matrix construction: extract 400 characters from the uploaded data and arrange them to form a 20×20 feature matrix;

[0020] S22, convolution dimensionality reduction processing: The feature matrix is ​​reduced in dimension through multiple convolution operations, and the 20×20 matrix is ​​generated into a 6×6×3 matrix, so as to extract the main feature information of the data;

[0021] S23, feature vectorization: Flatten the 6×6×3 matrix into a feature vector and store it in the feature database.

[0022] As a preferred embodiment of the present invention, the specific method of performing dimensionality reduction processing on the feature matrix by multiple convolution operations in step S22 is: the 20×20 matrix is ​​operated by a unit convolution kernel, an inverse unit convolution kernel, and a cross convolution kernel, and the operation is performed according to the formula

[0023]

[0024] Where: M′ i,j,k is the value corresponding to position (i, j) in the matrix after convolution, M i+p,j+q is the value at position (i+p,j+q) in the input matrix, K p,q,k is the value at position (p,q) in the kth convolution kernel.

[0025] As a preferred embodiment of the present invention, the specific method of step S3 is:

[0026] S31. Use full associativity to calculate the similarity score. Assume that the two compared feature vectors are v 1 and v 2 ; ; Calculate the similarity of two features through the similarity calculation formula

[0027]

[0028] Where and are the th element of the two eigenvectors, and represent the modulus of the vector;

[0029] S32. When the similarity score exceeds the threshold, the system identifies it as a related entry, extracts the complete data of the related entry from the data server, and presents it to the client for further review by the reviewer; if the similarity score does not reach the threshold, the system determines that there is no related data for the current entry, and will not present the entry matching result to the reviewer.

[0030] As a preferred embodiment of the present invention, the process of automatically generating the approval document described in step S4 is as follows:

[0031] S41, after obtaining the relevance of the entry confirmed by the client, extract relevant data based on the audit record;

[0032] S42 calls the preset approval document template, fills the data into the template in Word format, and generates an approval document.

[0033] The present invention also discloses a system for integrating and analyzing project application data based on mapping relationships, comprising a data upload and format review module, a data storage and feature generation module, a related data retrieval module and an automatic generation of approval documents module; the data upload and format review module is used to obtain project application data and pre-process the data; the data storage and feature generation module is used to forward the pre-processed data to a feature server for feature extraction; the related data retrieval module is used to perform similarity retrieval in a feature database based on feature vectors of data items to be reviewed; the automatic generation of approval documents module is used to extract relevant data based on review records and generate approval documents.

[0034] Compared with the existing solutions, the present invention has the following beneficial effects:

[0035] 1. Efficient data comparison and correlation judgment: The present invention converts each piece of data into a feature vector through feature extraction and convolution operation, and judges the correlation between data by calculating similarity. This method breaks through the traditional manual comparison and rule-based comparison methods, improves the efficiency of data comparison, and can significantly shorten the comparison time, especially when processing large amounts of data.

[0036] 2. Automatically generate approval documents. After data comparison and relevance determination are completed, the present invention can automatically generate relevant approval documents without manual intervention, thus reducing the risk of manual filling errors and improving the efficiency of the approval process;

[0037] 3. Intelligent feature extraction: The present invention extracts features from data and converts complex data into feature vectors that are easy to calculate and compare. It also takes into account the particularity of key fields such as the amount and performs dimensionality reduction on the data matrix through convolution operations, thus avoiding the subjectivity and inconsistency of manual feature selection in traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flow chart of a method for integrating and analyzing project application data based on mapping relationships according to the present invention; Figure 2 This is a diagram of the data uploading and feature extraction architecture of steps S1 and S2 of the present invention;

[0039] Figure 3 This is a structural diagram of data relevance retrieval and similarity comparison in steps S3 and S4 of the present invention;

[0040] Figure 4 This is the overall architecture diagram of the system of the present invention. DETAILED DESCRIPTION

[0041] In order to enable ordinary technicians in this field to better understand the technical solution of the present invention, the following Figure 1-4 The technical solution of the present invention is further described with embodiments.

[0042] Embodiment 1:

[0043] See attached Figure 1-4 This embodiment discloses a method for integrating and analyzing project application data based on a mapping relationship, comprising the following steps:

[0044] Step S1: Data upload and format review

[0045] The project applicant uploads the project application data to the server through the client. After the system receives the data, it first conducts a format review. The format review includes the following steps:

[0046] S11. Field integrity check: The system detects whether there are any unfilled fields in the data to ensure data integrity.

[0047] S12. Numerical range verification: Verify key numerical fields to ensure that the data is within the valid range set by the system, such as performing validity checks on amount fields.

[0048] S13, Data formatting: If the data passes the format review, the system formats the data field into a database-compatible format and maps the data field to a predefined database table field. For example, if the amount entry has fields such as "80,000", "80,000 yuan", and "80,000.00", there is an automatic processing flow to normalize all fields into a floating point number of 80,000.00, with a fixed two digits after the decimal point, and then store them in the data server to meet the storage standard.

[0049] S14. Data cleaning: Clean unnecessary empty values ​​or invalid fields, fill missing fields, and remove redundant information to ensure that the data meets the storage requirements. The final formatted data will be stored in the database of the data server.

[0050] Step S2: Data storage and feature generation

[0051] After the formatted data is stored in the data server, the system forwards the data to the feature server for feature extraction. The specific steps of feature extraction are as follows:

[0052] S21. Feature matrix construction: The system extracts 400 characters from the uploaded data and arranges them into a 20×20 feature matrix.

[0053] S22, convolution dimensionality reduction processing: The feature matrix is ​​reduced in dimensionality through multiple convolution operations. Specifically, the 20×20 matrix is ​​operated through three types of convolution kernels (unit convolution kernel, inverse unit convolution kernel, cross convolution kernel), and the operation is performed according to the formula

[0054]

[0055] Where: M′ i,j,k is the value corresponding to position (i, j) in the matrix after convolution, M i+p,j+q is the value at position (i+p,j+q) in the input matrix, K p,q,k is the value of position (p,q) in the kth convolution kernel; a 6×6×3 matrix is ​​generated to extract the main feature information of the data.

[0056] S23, Feature vectorization: Flatten the 6×6×3 matrix into a feature vector for storage in the feature database. The feature vector is used for subsequent data retrieval and comparison. If the declared data contains a specific amount field, its value is added to the end of the feature vector to form an extended feature vector of 6×6×3+1.

[0057] Step S3: Related data retrieval

[0058] The reviewer selects the data item to be reviewed on the client, and the feature server performs a similarity search in the feature database based on the feature vector of the item. The specific search steps are:

[0059] S31. Similarity score calculation: The system uses the vector similarity calculation formula to calculate the similarity between the selected entry and other entries in the database. Assume that the comparison vector is v 1 and v 2 , the calculation formula is as follows:

[0060]

[0061] Where: (v 1,i ) and (v 2,i ) are the i-th elements of the two eigenvectors, respectively, ||v 1 || and ||v 2 || represents the magnitude of a vector.

[0062] S32. Data association confirmation: When the similarity score is higher than the preset threshold, the system determines that the entry is associated data and extracts the complete data from the data server; if the similarity score does not reach the threshold, the system determines that there is no associated data for the current entry and will not present the entry matching result to the reviewer. The reviewer can choose to terminate the association review of the entry according to the specific situation.

[0063] Step S4: Automatically generate approval documents

[0064] When the reviewer confirms the item association, the system automatically generates an approval document based on the review record. The specific operations are as follows:

[0065] S41. Audit data extraction: The system extracts relevant data fields from the audit records.

[0066] S42. Approval document generation: The system calls the preset Word document template, fills the audit data into the corresponding field position, and generates an approval document for use in further processes.

[0067] Embodiment 2:

[0068] This embodiment discloses a system for integrating and analyzing project application data based on mapping relationships, including a data uploading and format review module, a data storage and feature generation module, a related data retrieval module, and an automatic approval document generation module.

[0069] The data upload and format review module is used to obtain project application data and pre-process the data; the data storage and feature generation module is used to forward the pre-processed data to the feature server for feature extraction; the associated data retrieval module is used to perform similarity retrieval in the feature database based on the feature vectors of the data items to be reviewed; the automatic generation of approval documents module is used to extract relevant data based on the audit records and generate approval documents.

[0070] The present invention realizes the efficient integration, analysis and review functions of project application data, greatly improving the data processing efficiency and the automation level of the approval process. Compared with the traditional manual sorting and review methods, the present invention realizes automatic correlation judgment of data through feature vector similarity calculation, which not only improves the accuracy of data comparison, but also avoids the omissions that may occur in manual review, especially in the processing of massive data. On the basis of automatic comparison and generation of approval documents, the system reduces the operating steps of auditors, ensures the seamless transmission of data and the automatic generation of results, thereby significantly shortening the approval cycle. The present invention also realizes standardized management of data through formatting and other steps, ensuring the integrity and consistency of the data.

[0071] The basic principle, main features and advantages of the present invention are shown and described above. It should be understood by those skilled in the art that the present invention is not limited to the specific contents of the above embodiments, and the above embodiments and description are only used to illustrate the basic principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various modifications and optimizations, all of which are within the scope of protection of the present invention. The scope of protection of the present invention is determined by the attached claims and their equivalents.

Claims

1. A method for integrating and analyzing project application data based on mapping relationships, characterized in that: The steps include: S1. Obtain project application data and pre-process the data; S2, storing the pre-processed data in the data server, and then forwarding it to the feature server for feature extraction; S3, performing similarity search in a feature database based on the feature vector of the data item to be reviewed; S4. Extract relevant data based on the audit records and generate approval documents.

2. The project application data integration and analysis method based on mapping relationship according to claim 1 is characterized in that: The specific steps of preprocessing the data in step S1 are: S11. Conduct format review on uploaded declaration data, including field integrity check and value range verification; S12. After the format is reviewed and approved, the data is converted into a database field format suitable for storage; S13. The transformed data is stored in the database of the data server, and the transformed data fields are sent to the feature server.

3. The project application data integration and analysis method based on mapping relationship according to claim 2 is characterized in that: The specific steps of converting the data into a database field format suitable for storage in step S12 are: According to the predefined database table structure, each field in the data is mapped to the corresponding database table field type; For specific fields, format them to ensure they meet storage standards; Perform data cleansing on uploaded data, including removing invalid fields, removing null values, and filling missing values ​​to ensure data integrity and consistency.

4. The project application data integration and analysis method based on mapping relationship according to claim 1 is characterized in that: The specific steps of step S2 are as follows: after storing the pre-processed data in the data server, forwarding it to the feature server for feature extraction; S21, feature matrix construction: extract 400 characters from the uploaded data and arrange them to form a 20×20 feature matrix; S22, convolution dimensionality reduction processing: The feature matrix is ​​reduced in dimension through multiple convolution operations, and the 20×20 matrix is ​​generated into a 6×6×3 matrix, so as to extract the main feature information of the data; S23, feature vectorization: Flatten the 6×6×3 matrix into a feature vector and store it in the feature database.

5. The project application data integration and analysis method based on mapping relationship according to claim 4 is characterized by: The specific method of performing dimensionality reduction processing on the feature matrix by multiple convolution operations in step S22 is: the 20×20 matrix is ​​operated by the unit convolution kernel, the inverse unit convolution kernel, and the cross convolution kernel, and the operation is performed according to the formula Where: M′ i,j,k is the value corresponding to position (i, j) in the matrix after convolution, M i+p,j+q is the value at position (i+p,j+q) in the input matrix, K p,q,k is the value at position (p,q) in the kth convolution kernel.

6. The method for integrating and analyzing project application data based on mapping relationships according to claim 4 is characterized in that: The specific method of step S3 is: S31. Use full associativity to calculate the similarity score. Assume that the two compared feature vectors are v1 and v2. Calculate the similarity of the two features using the similarity calculation formula. Where and are the th element of the two eigenvectors, and represent the modulus of the vector; S32. When the similarity score exceeds the threshold, the system identifies it as a related entry, extracts the complete data of the related entry from the data server, and presents it to the client for further review by the reviewer; if the similarity score does not reach the threshold, the system determines that there is no related data for the current entry, and will not present the entry matching result to the reviewer.

7. The project application data integration and analysis method based on mapping relationship according to claim 1 is characterized by: The process of automatically generating the approval document described in step S4 is as follows: S41, after obtaining the relevance of the entry confirmed by the client, extract relevant data based on the audit record; S42 calls the preset approval document template, fills the data into the template in Word format, and generates an approval document.

8. A system for integrating and analyzing project application data based on mapping relationships based on the method according to any one of claims 1 to 7, characterized in that: It includes a data upload and format review module, a data storage and feature generation module, a related data retrieval module and an automatic approval document generation module; the data upload and format review module is used to obtain project application data and pre-process the data; the data storage and feature generation module is used to forward the pre-processed data to the feature server for feature extraction; the related data retrieval module is used to perform similarity retrieval in the feature database based on the feature vector of the data item to be reviewed; the automatic approval document generation module is used to extract relevant data according to the review record and generate approval documents.

Images