Electronic document processing method and device, equipment, storage medium and program product

Abstract

The invention provides an electronic document processing method and device, equipment, a storage medium and a program product, and can be applied to the technical field of big data processing. The method comprises: in response to an electronic document storage request for a target service, storing file meta-information and service meta-information of an electronic document to a target area node, the target area node being associated with a transaction attribute of the target service; processing the file meta-information corresponding to the plurality of sub-documents in the target area node by using a classification model to obtain business types corresponding to the plurality of sub-documents; based on a service quality rule corresponding to the service type, performing quality detection on the sub-document to obtain a target sub-document matched with the service quality rule; and according to the service type and the service meta-information of the target sub-document, constructing a distributed index structure for the target sub-document so as to query the target sub-document according to the distributed index structure in a document query process.

Description

Technical Field

[0001] This disclosure relates to the field of big data processing technology, and more specifically, to an electronic document processing method, apparatus, device, storage medium, and program product. Background Technology

[0002] In financial business systems, as business grows and scenarios expand, the required documentation becomes increasingly diverse and digitized. When new business scenarios are launched, the documents used are universal to those generated in different previous stages; therefore, these documents can be reused to improve processing efficiency.

[0003] In the process of realizing the present invention, the inventors discovered at least the following problems in the related technology: In the current process of electronic document management, the file object storage system for storing electronic documents has a single function, resulting in a low document reuse rate. Summary of the Invention

[0004] In view of the above, this disclosure provides an electronic document processing method, apparatus, device, storage medium, and program product.

[0005] One aspect of this disclosure provides an electronic document processing method, comprising: in response to an electronic document storage request for a target business, storing file metadata and business metadata of the electronic document in a target region node, wherein the target region node is associated with the transaction attributes of the target business; processing the file metadata corresponding to multiple sub-documents in the target region node using a classification model to obtain business types corresponding to each of the multiple sub-documents; performing quality detection on the sub-documents based on business quality rules corresponding to the business types to obtain target sub-documents that match the business quality rules; and constructing a distributed index structure for the target sub-documents according to their business types and business metadata, so as to enable querying the target sub-documents according to the distributed index structure during document querying.

[0006] According to embodiments of this disclosure, a sub-document is subjected to quality inspection based on a business quality rule corresponding to a business type to obtain a target sub-document that matches the business quality rule. This includes: determining inspection items corresponding to the sub-document based on the business quality rule and the business type; performing quality inspection on the business metadata related to the inspection items in the sub-document to obtain a quality inspection result; and determining the target sub-document based on the quality inspection result.

[0007] According to embodiments of this disclosure, the business quality rules include at least one of the following: a first rule characterizing whether the certificates in the sub-document meet the validity period; and a second rule characterizing whether the contract documents in the sub-document meet the document validity requirements.

[0008] According to embodiments of this disclosure, constructing a distributed index structure for a target sub-document based on classification results and business element information of the target sub-document includes: processing the electronic document using statistical algorithms to obtain word frequency ranking results; extracting keywords from the business element information to obtain keywords; and processing the word frequency ranking results, classification results, and keywords using a probability ranking function to obtain a distributed index structure.

[0009] According to embodiments of this disclosure, the electronic document processing method further includes: in response to a document retrieval request for a target object, determining at least one sub-document related to the target keyword information indicated in the document retrieval request based on a distributed index structure; using a relevance algorithm to score the relevance of the at least one sub-document to obtain a score ranking result, the score ranking result including scores in descending order; and sending the sub-document corresponding to the score result located in the first position in the score ranking result to the target object.

[0010] According to embodiments of this disclosure, the target area node includes multiple primary nodes and one backup node, the number of primary nodes being determined based on the business volume related to the target area; wherein, storing the file metadata and business metadata of electronic documents to the target area node includes: determining the target primary node in the target area based on the transaction subject information of the target business; storing the file metadata and business metadata of electronic documents to the target primary node; and synchronizing the file metadata and business metadata of the target primary node to the backup node.

[0011] According to embodiments of this disclosure, the electronic document processing method further includes: dividing the electronic document into multiple data blocks based on a preset document space threshold; storing the multiple data blocks into multiple main storage devices of the target master node, and determining the main storage location of each of the multiple data blocks; and storing the main storage location of each of the multiple data blocks into the target master node.

[0012] According to embodiments of this disclosure, the electronic document processing method further includes: a target master node sending operation logs generated by storing multiple data blocks to a backup node based on the generation sequence relationship; the backup node reading the logical path information of the primary storage location in the multiple operation logs using a reading thread based on the generation sequence relationship; and storing the multiple data blocks into multiple backup storage devices of the backup node based on the multiple logical path information, and determining the backup storage location of each of the multiple data blocks, wherein the logical path information of the backup storage location of the data block is consistent with the logical path information of the primary storage location.

[0013] According to embodiments of this disclosure, the electronic document processing method further includes: responding to a document subscription request from a target object, identifying a business identifier carried in the document subscription request or a transaction scenario of the target object; determining a target business type based on the business identifier or transaction scenario; performing asymmetric encryption on a sub-document corresponding to the target business type to obtain an encrypted document; and sending the encrypted document to the target object.

[0014] Another aspect of this disclosure provides an electronic document processing apparatus, comprising: a response module, configured to respond to an electronic document storage request for a target business, and store file metadata and business metadata of the electronic document in a target region node, wherein the target region node is associated with the transaction attributes of the target business, and the electronic document includes multiple sub-documents; a classification module, configured to process the file metadata corresponding to each of the multiple sub-documents in the target region node using a classification model to obtain the business type corresponding to each of the multiple sub-documents; a quality inspection module, configured to perform quality inspection on the sub-documents based on business quality rules corresponding to the business types to obtain target sub-documents that match the business quality rules; and an indexing module, configured to construct a distributed index structure for the target sub-documents based on the business type and business metadata of the target sub-documents, so as to enable the querying of target sub-documents based on the distributed index structure during document querying.

[0015] Another aspect of this disclosure provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the electronic document processing method described above.

[0016] Another aspect of this disclosure provides a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the electronic document processing method described above.

[0017] Another aspect of this disclosure provides a computer program product including computer-executable instructions that, when executed, implement the electronic document processing method described above.

[0018] According to the electronic document processing method, apparatus, device, storage medium, and program products provided in this disclosure, after storing the file metadata and business metadata of the electronic document, the business type identification of multiple sub-documents is performed, thereby realizing the business type grouping and marking function of the document; based on the business quality rules corresponding to the business type, the sub-documents are quality checked, and after filtering out the sub-documents that fail the quality check, the waste of invalid resource space is reduced, and the reliability guarantee of the document quality subscribed to by the subsequent document subscription function is provided. On this basis, an index structure is constructed for the target sub-documents that match the business quality rules. When users need to download relevant sub-documents, they can quickly obtain documents through the distributed index structure. In addition to the electronic document storage and download function, the system realizes multiple functions such as business type grouping and marking, quality inspection, and classification indexing of sub-documents, effectively supporting the reuse of document materials in various new business scenarios. Attached Figure Description

[0019] The above and other objects, features and advantages of this disclosure will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0020] Figure 1 This illustrates an application scenario of the electronic document processing method according to embodiments of the present disclosure;

[0021] Figure 2 A flowchart of an electronic document processing method according to an embodiment of the present disclosure is shown;

[0022] Figure 3 A flowchart illustrating the determination of a target sub-document according to an embodiment of this disclosure is shown;

[0023] Figure 4 A structural block diagram of an electronic document processing apparatus according to an embodiment of the present disclosure is shown;

[0024] Figure 5 A block diagram of an electronic device suitable for implementing an electronic document processing method according to an embodiment of the present disclosure is shown. Detailed Implementation

[0025] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0026] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0027] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0028] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0029] In the process of developing this disclosure, it was discovered that in the early stages of application system construction, due to fewer regulatory rules, lower business complexity, and simpler inspection methods, fewer electronic documents were generated at each stage. As business grows and scenarios expand, the required materials become increasingly diverse, and the degree of digitization increases. However, when new business scenarios are implemented, the documents used are universal to those generated in previous stages, providing conditions for document reuse. However, the current file object storage system for storing documents has limited functionality, only offering upload and download capabilities. It lacks functions such as collecting, quality checking, and retrieving all generated electronic documents. This results in the need to retrieve electronic documents from different data sources each time a new business scenario is added, significantly increasing future development costs, operational complexity for business personnel, and low document reuse rates across different scenarios.

[0030] In view of the above, embodiments of this disclosure provide an electronic document processing method, apparatus, device, storage medium, and program product. The method includes: in response to an electronic document storage request for a target business, storing file metadata and business metadata of the electronic document in a target region node, wherein the target region node is associated with the transaction attributes of the target business; processing the file metadata corresponding to multiple sub-documents in the target region node using a classification model to obtain business types corresponding to each of the multiple sub-documents; performing quality detection on the sub-documents based on business quality rules corresponding to the business types to obtain target sub-documents that match the business quality rules; and constructing a distributed index structure for the target sub-documents based on their business types and business metadata, so as to enable querying of target sub-documents based on the distributed index structure during document querying.

[0031] It should be noted that the electronic document processing method and device specified in this disclosure can be used in the financial field, or in any field other than the financial field. The application fields of the electronic document processing method and device specified in this disclosure are not limited.

[0032] In the technical solution disclosed herein, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0033] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, they do not mean that the applicant has used or necessarily used the solution.

[0034] Figure 1 An application scenario of the electronic document processing method according to an embodiment of the present disclosure is shown.

[0035] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0036] Users can interact with server 105 via network 104 using at least one of the first terminal device 101, second terminal device 102, and third terminal device 103 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, second terminal device 102, and third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0037] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0038] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0039] It should be noted that the electronic document processing method provided in this embodiment can generally be executed by server 105. Correspondingly, the electronic document processing device provided in this embodiment can generally be located in server 105. The electronic document processing method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the electronic document processing device provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0040] Alternatively, the electronic document processing method provided in this embodiment of the present disclosure can also be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103, or by other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103. Correspondingly, the electronic document processing apparatus provided in this embodiment of the present disclosure can also be disposed in the first terminal device 101, the second terminal device 102, or the third terminal device 103, or in other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103.

[0041] It should be understood that Figure 1 The number of terminal devices, networks, and servers in the system is only a limited number. Depending on implementation needs, there can be any number of terminal devices, networks, and servers.

[0042] Figure 2 A flowchart of an electronic document processing method according to an embodiment of the present disclosure is shown.

[0043] like Figure 2 As shown, the method 200 includes operations S210 to S240.

[0044] In operation S210, in response to an electronic document storage request for a target business, the file metadata and business metadata of the electronic document are stored in the target area node.

[0045] The electronic document contains multiple sub-documents.

[0046] The target business is the transaction business that the target user has completed. Completing the target business may require the collaboration of multiple business systems. For example, for resource borrowing business A, it may require the collaboration of the vehicle transaction system and the bank transaction system.

[0047] The client layer uploads and aggregates sub-documents from various business systems associated with the target business, resulting in electronic documents. For example, sub-documents could be target user identification documents, mortgage contracts, or transaction vouchers. The client layer includes a console, third-party applications, and terminal tools.

[0048] Electronic document storage requests are initiated through the application interface. In response to electronic document storage requests for the target business, the server performs integrity verification on the electronic document. After obtaining the integrity verification result, the server stores the file metadata and business metadata of the electronic document in the target area node.

[0049] File metadata refers to the system attribute characteristics of electronic documents, such as file size, file type, and creation time.

[0050] Business metadata refers to the application attribute characteristics related to the target business recorded in electronic documents, such as resource borrowing account application number, target user name, identification information, handler information, handling agency information, and material type.

[0051] The target region node is associated with the transaction attributes of the target business. Transaction attributes can include the region where the business processing agency is located. The target region node is a node set up for the region where the business processing agency is located. The file metadata and business metadata of the electronic document are stored in the target region node.

[0052] In operation S220, the classification model is used to process the file meta information corresponding to each of the multiple sub-documents in the target area node, so as to obtain the business type corresponding to each of the multiple sub-documents.

[0053] Classification models can be built based on convolutional neural networks (CNNs).

[0054] The file metadata corresponding to each of the multiple sub-documents is input into the classification model, and the business type corresponding to each of the multiple sub-documents is output to achieve automated labeling.

[0055] For example, the business type can be document type, contract type, authorization type, voucher type, etc.

[0056] In operation S230, based on the business quality rules corresponding to the business type, the sub-documents are subjected to quality checks to obtain target sub-documents that match the business quality rules.

[0057] Business quality rules are rules for verifying whether the content in a sub-document meets valid quality standards.

[0058] Set corresponding business quality rules for different business types, perform quality checks on sub-documents based on the business quality rules, and obtain target sub-documents that match the business quality rules.

[0059] The target sub-document that matches the business quality rules represents the sub-document that satisfies the business quality rules.

[0060] For example, if sub-document A is a certificate, sub-document B is a contract, and sub-document C is a contract, then sub-document A is subjected to quality checks according to the business quality rules corresponding to the certificate category. Sub-document B and sub-document C are subjected to quality checks according to the business quality rules corresponding to the contract category. It is found that sub-document A meets the business quality rules, while sub-document B and sub-document C do not. Therefore, sub-document A is identified as the target sub-document, while sub-document B and sub-document C are marked as unqualified and sent back to the external business system for re-upload.

[0061] In operation S240, a distributed index structure is constructed for the target sub-document based on its business type and business metadata, so that the target sub-document can be queried based on the distributed index structure during the document query process.

[0062] Keywords are extracted based on the business type and business metadata of the target sub-documents. The Elasticsearch algorithm is then used to build a full-text index on the target sub-documents based on the keywords, resulting in a distributed index structure.

[0063] When a target user needs to download a sub-document, they can enter keywords, and the system will quickly locate and retrieve the sub-document based on a distributed index structure.

[0064] For example, sales personnel can search for a target user's electronic mortgage contract document by entering the customer's name and license plate number.

[0065] According to embodiments of this disclosure, after storing the file metadata and business metadata of electronic documents, the business types of multiple sub-documents are identified, thereby realizing the business type grouping and marking function of documents. Based on the business quality rules corresponding to the business type, the sub-documents are quality checked. After filtering out sub-documents that fail the quality check, the waste of invalid resource space is reduced, and the reliability guarantee of the document quality subscribed to by the subsequent document subscription function is provided. On this basis, an index structure is constructed for the target sub-documents that match the business quality rules. When users need to download relevant sub-documents, they can quickly obtain documents through the distributed index structure. In addition to the storage and download function of electronic documents, the system realizes multiple functions such as business type grouping and marking, quality inspection and classification indexing of sub-documents, effectively supporting the reuse of document materials in various new business scenarios.

[0066] Figure 3 A flowchart illustrating the determination of a target sub-document according to an embodiment of this disclosure is shown.

[0067] like Figure 3 As shown, the process of determining the target sub-document includes operations S310~S330.

[0068] In S310, based on the business quality rules and business type, determine the detection items corresponding to the sub-document.

[0069] In operation S320, quality inspection is performed on the business metadata related to the inspection items in the sub-document to obtain the quality inspection results.

[0070] When operating S330, the target sub-document is determined based on the quality inspection results.

[0071] The business quality rules can include detection rules corresponding to multiple business types. Detection items are extracted from the detection rules, and each detection item is the name of the item to be detected.

[0072] For example, the testing item for certificates could be the expiration date, and the testing item for contracts could be the validity.

[0073] The business metadata related to the detection items is extracted from the sub-documents for quality detection, and the quality detection results are obtained.

[0074] For example, if the test item is validity period, the validity period needs to be extracted, and the public security system needs to be contacted to check whether the document is valid; if the test item is validity, the official seal information or signature information needs to be extracted to check for forgery.

[0075] Quality inspection results include whether the inspection passed or failed.

[0076] The sub-document whose quality inspection result is "passed" is identified as the target sub-document.

[0077] Sub-documents that fail the quality inspection are labeled as unqualified and sent back to the external business system for re-upload.

[0078] According to the embodiments of this disclosure, the functionality of the file object storage system is expanded. Based on the automatic business type marking of sub-documents, the sub-documents are quality inspected according to business quality rules and business types. Target sub-documents that pass the quality inspection are selected, and sub-documents that fail the quality inspection are labeled as unqualified. This effectively filters out a large number of unusable documents, reduces space waste, and provides a reliable guarantee for the quality of documents subscribed to by the subsequent document subscription function.

[0079] According to embodiments of this disclosure, the business quality rules include at least one of the following: a first rule characterizing whether the certificates in the sub-document meet the validity period; and a second rule characterizing whether the contract documents in the sub-document meet the document validity requirements.

[0080] For the first rule, the "validity date" recorded in the document-type sub-document is compared with the date recorded in the public security system. If the validity date matches the recorded date, the quality inspection result of "inspection passed" is output; otherwise, the quality inspection result of "inspection failed" is output.

[0081] For the second rule, the "official seal information" or signature information recorded in the contract sub-document is compared with the target user name or the target entrusted agency name. If the two match, the quality inspection result of "inspection passed" is output; otherwise, the quality inspection result of "inspection failed" is output.

[0082] According to embodiments of this disclosure, constructing a distributed index structure for a target sub-document based on classification results and business element information of the target sub-document includes: processing the electronic document using statistical algorithms to obtain word frequency ranking results; extracting keywords from the business element information to obtain keywords; and processing the word frequency ranking results, classification results, and keywords using a probability ranking function to obtain a distributed index structure.

[0083] Statistical algorithms are used to segment electronic documents into words, and the word frequencies of keywords are statistically analyzed to obtain the word frequency statistics and position information for each keyword. Based on descending order and the word frequency statistics, multiple keywords are sorted to obtain the word frequency ranking results.

[0084] Keyword extraction is performed on the business metadata to obtain multiple keywords and location information.

[0085] The probability sorting function is used to process the word frequency sorting results, classification results, and keyword and position information, and index allocation is performed to obtain a distributed index structure. The distributed index structure is a full-text index structure, which can realize cross-node index merging query.

[0086] According to embodiments of this disclosure, the electronic document processing method further includes: in response to a document retrieval request for a target object, determining at least one sub-document related to the target keyword information indicated in the document retrieval request based on a distributed index structure; using a relevance algorithm to score the relevance of the at least one sub-document to obtain a score ranking result, the score ranking result including scores in descending order; and sending the sub-document corresponding to the score result located in the first position in the score ranking result to the target object.

[0087] In a distributed index structure, electronic documents are segmented and stored on multiple nodes. Each node maintains a local index. When a server responds to a document retrieval request for a target object, each node simultaneously queries its local inverted index to find sub-documents related to the target keywords indicated in the document retrieval request and collects the set of sub-documents returned by all nodes.

[0088] Relevance algorithms can be machine learning algorithms, spatial algorithms, or inverse document frequency algorithms, etc.

[0089] The relevance algorithm is used to score the relevance of multiple sub-documents, and the scores are ranked. The sub-documents returned by each node are then sorted according to the scores.

[0090] The score that is in first place is the highest score.

[0091] The sub-document and its metadata corresponding to the score that ranks first in the score ranking results are formatted, encapsulated, and sent to the target object.

[0092] According to embodiments of this disclosure, the distributed index structure is a data structure for efficiently searching the full-text content of electronic documents. Combined with the accuracy of relevance algorithms, it can efficiently handle the retrieval needs of massive amounts of electronic documents while ensuring that the most relevant results are returned.

[0093] According to embodiments of this disclosure, the target area node includes multiple primary nodes and one backup node, the number of primary nodes being determined based on the business volume related to the target area; wherein, storing the file metadata and business metadata of electronic documents to the target area node includes: determining the target primary node in the target area based on the transaction subject information of the target business; storing the file metadata and business metadata of electronic documents to the target primary node; and synchronizing the file metadata and business metadata of the target primary node to the backup node.

[0094] The system architecture includes an access gateway layer, which serves as the request entry point, with one set deployed on each of the primary and backup nodes. Core functions include request authentication, load balancing (distributing requests to backend nodes using consistent hashing), fault detection (real-time detection of node health status and removal of abnormal nodes), and automatic updating of routing rules during primary / backup node switching to ensure that requests are routed to the new primary node.

[0095] The electronic documents are stored in a regional distributed manner, with multiple master nodes and one backup node set up in each region according to the division of regions.

[0096] The number of master nodes in each region can be dynamically expanded according to the business volume. For example, 2 to 3 master nodes can be set up in areas with low business volume, while 7 to 8 master nodes can be set up in areas with high business volume.

[0097] For example, if the transaction entity information of the target business is Bank A, the region where Bank A is located is designated as the target region, and the preset matching node in the region where Bank A is located is determined as the target master node.

[0098] Asymmetric cryptography algorithms can be used to encrypt key information in the file metadata and business metadata of electronic documents before storing them to the target master node.

[0099] The target master node synchronizes file metadata and business metadata to the backup node.

[0100] The system architecture includes a data storage layer, which includes metadata storage: each primary and backup node deploys a distributed database, and the primary node synchronizes file metadata to the backup node in real time, recording information such as the object's key, size, storage location, and permissions.

[0101] According to the embodiments of this disclosure, a multi-master-one-slave structure is adopted, that is, a backup node is set up in each region to improve backup services for all nodes in that region. When a node fails, the data can be recovered from the backup node, improving the security and reliability of the data. In addition, in order to isolate business securely, the electronic documents of multiple businesses of the same transaction entity are generally stored in the same master node, which improves resource utilization.

[0102] According to embodiments of this disclosure, the electronic document processing method further includes: dividing the electronic document into multiple data blocks based on a preset document space threshold; storing the multiple data blocks into multiple main storage devices of the target master node, and determining the main storage location of each of the multiple data blocks; and storing the main storage location of each of the multiple data blocks into the target master node.

[0103] The preset document space threshold represents the storage space threshold occupied by a single data block. For example, the preset document space threshold is 128MB.

[0104] The data storage layer includes content storage: electronic documents are stored by dividing them into multiple data blocks based on a preset document space threshold, and then storing the multiple data blocks into multiple main storage devices of the target master node.

[0105] The primary storage device can be a disk.

[0106] The target master node stores two electronic document copies, and the backup area synchronously stores one electronic document copy. The master storage location of each data block is associated with the file metadata, and the master storage location of each data block is stored in the target master node to ensure fast location during reading.

[0107] According to embodiments of this disclosure, the electronic document processing method further includes: a target master node sending operation logs generated by storing multiple data blocks to a backup node based on the generation sequence relationship; the backup node reading the logical path information of the primary storage location in the multiple operation logs using a reading thread based on the generation sequence relationship; and storing the multiple data blocks into multiple backup storage devices of the backup node based on the multiple logical path information, and determining the backup storage location of each of the multiple data blocks, wherein the logical path information of the backup storage location of the data block is consistent with the logical path information of the primary storage location.

[0108] The system architecture includes a business logic layer, where primary and backup nodes are deployed. The primary node handles all read and write logic, while the backup node handles some read requests and data synchronization. The business logic layer comprises three core modules: object management (creating / deleting / querying objects), metadata operations (updating file names / permissions, etc.), and lifecycle management (executing expired data deletion / archiving policies). It also includes a built-in deduplication and compression engine for optimization before data storage.

[0109] The system architecture includes a primary / backup synchronization layer, an underlying synchronization module independent of the business logic layer, deployed between the core nodes of the primary and backup nodes.

[0110] The target master node generates operation logs during the writing of multiple data blocks and streams these logs to the backup nodes in real time. The generation time sequence is the same as the time sequence of operation log generation.

[0111] The backup node synchronizes data by reading the logical path information of the primary storage location from multiple operation logs based on the generation time sequence relationship and using a read thread.

[0112] The operation log records the primary storage location of data blocks written to the target master node. For example, the primary storage location of data block A is / disk1 / data / chunk_A_001.bin, the physical location is / disk1, and the logical path information is / data / chunk_A_001.bin.

[0113] Based on the logical path information of each data block, the data block is stored in the backup storage device corresponding to the logical path information in the backup node, and the backup storage location is recorded.

[0114] For example, the backup storage location of data block A is / storage_poolA / data / chunk_A_001.bin, the physical location is / storage_poolA, and the logical path information is data / chunk_A_001.bin, so the logical path information is consistent.

[0115] The primary and backup synchronization layer has a built-in heartbeat detection function (e.g., every 1 second). When the primary node fails, the backup node will trigger a primary / backup node switch within 10 seconds.

[0116] According to the embodiments of this disclosure, the primary node generates a corresponding operation log each time it completes a data block write. The backup node obtains and replays the operation log in real time through a read thread. The primary and backup data and logical path information are consistent, thereby ensuring consistent storage mapping. When the primary node fails, the application can continue to access the backup node using the same path without modifying the configuration, which improves access efficiency and reliability.

[0117] According to embodiments of this disclosure, the electronic document processing method further includes: responding to a document subscription request from a target object, identifying a business identifier carried in the document subscription request or a transaction scenario of the target object; determining a target business type based on the business identifier or transaction scenario; performing asymmetric encryption on a sub-document corresponding to the target business type to obtain an encrypted document; and sending the encrypted document to the target object.

[0118] In addition to manually entering keywords to search for target documents, users or other external systems can subscribe to electronic documents by tagging them according to business type.

[0119] The sub-documents of the electronic document are grouped and stored according to business type tags. When the server responds to the document subscription request of the target object, it identifies the business identifier carried in the document subscription request. The business identifier is a unique identifier of the business type, and the target business type is directly determined based on the business identifier.

[0120] For example, if the business identifier is A001, then the target business type is determined to be a certificate type.

[0121] The server identifies the transaction scenario of the target object and determines the target business type based on the transaction scenario and the correspondence between scenario type and business type.

[0122] For example, for the direct connection scenario of car purchase mortgage, the target business type can be determined as document type or mortgage contract type, and then the customer's ID card photo and mortgage contract can be packaged and pushed.

[0123] The national cryptographic algorithm is used to perform asymmetric encryption on the sub-documents corresponding to the target business type to obtain the encrypted document, which is then sent to the target object.

[0124] According to embodiments of this disclosure, based on the subscription requirements corresponding to the target, a certain business type or scenario is identified to determine sub-documents, and the sub-documents are pushed to the file exchange service of the designated institution on a regular basis, saving the target object time in organizing information; in addition, during the push process, the national cryptographic algorithm is used to perform asymmetric encryption on the pushed documents and data to ensure the integrity, reliability and immutability of the data during transmission.

[0125] Based on the above-described electronic document processing method, this disclosure also provides an electronic document processing apparatus. The following will be combined with... Figure 4 The device is described in detail.

[0126] Figure 4 A structural block diagram of an electronic document processing apparatus according to an embodiment of the present disclosure is shown.

[0127] like Figure 4 As shown, the electronic document processing device 400 of this embodiment includes a response module 410, a classification module 420, a quality inspection module 430, and an indexing module 440.

[0128] The response module 410 is used to respond to an electronic document storage request for a target business by storing the file metadata and business metadata of the electronic document in a target area node. The target area node is associated with the transaction attributes of the target business, and the electronic document includes multiple sub-documents. In one embodiment, the response module 410 can be used to perform the operation S210 described above, which will not be repeated here.

[0129] The classification module 420 is used to process the file metadata corresponding to each of the multiple sub-documents in the target region node using a classification model, and obtain the business type corresponding to each of the multiple sub-documents. In one embodiment, the classification module 420 can be used to perform the operation S220 described above, which will not be repeated here.

[0130] The quality inspection module 430 is used to perform quality inspection on sub-documents based on business quality rules corresponding to the business type, and obtain target sub-documents that match the business quality rules. In one embodiment, the quality inspection module 430 can be used to perform the operation S230 described above, which will not be repeated here.

[0131] The index module 440 is used to construct a distributed index structure for the target sub-document based on its business type and business metadata, so as to enable the querying of the target sub-document based on the distributed index structure during the document query process. In one embodiment, the index module 440 can be used to perform the operation S240 described above, which will not be repeated here.

[0132] According to embodiments of this disclosure, the quality inspection module 430 includes a first detection submodule, a second detection submodule, and a third detection submodule.

[0133] The first detection submodule is used to determine the detection items corresponding to the sub-document based on the business quality rules and business type.

[0134] The second detection submodule is used to perform quality detection on the business metadata related to the detection items in the sub-document and obtain the quality detection results.

[0135] The third detection submodule is used to determine the target sub-document based on the quality detection results.

[0136] According to embodiments of this disclosure, the index module 440 includes a first index submodule, a second index submodule, and a third index submodule.

[0137] The first indexing submodule is used to process electronic documents using statistical algorithms to obtain word frequency ranking results.

[0138] The second index submodule is used to extract keywords from business metadata to obtain keywords.

[0139] The third index submodule is used to process the word frequency ranking results, classification results, and keywords using a probability ranking function to obtain a distributed index structure.

[0140] According to embodiments of this disclosure, the electronic document processing apparatus 400 further includes a first determining module, a scoring module, and a first sending module.

[0141] The first determining module is used to determine, in response to a document retrieval request for a target object, at least one sub-document related to the target keyword information indicated in the document retrieval request, based on a distributed index structure.

[0142] The scoring module is used to score the relevance of at least one sub-document using a relevance algorithm and obtain a score ranking result, which includes scores in descending order.

[0143] The second sending module is used to send the sub-document corresponding to the score result that is in the first position in the score sorting results to the target object.

[0144] According to embodiments of this disclosure, the response module includes a first response submodule, a second response submodule, and a third response submodule.

[0145] The first response submodule is used to determine the target master node in the target area based on the transaction entity information of the target business.

[0146] The second response submodule is used to store the file metadata and business metadata of the electronic document to the target master node.

[0147] The third response submodule is used by the target master node to synchronize file metadata and business metadata to the backup node.

[0148] According to embodiments of this disclosure, the electronic document processing apparatus 400 further includes a partitioning module, a first storage module, and a second storage module.

[0149] The partitioning module is used to partition electronic documents into multiple data blocks based on a preset document space threshold.

[0150] The first storage module is used to store multiple data blocks into multiple main storage devices of the target master node, and to determine the main storage location of each data block.

[0151] The second storage module is used to store the main storage locations of multiple data blocks to the target master node.

[0152] According to embodiments of this disclosure, the electronic document processing apparatus 400 further includes a third storage module, a reading module, and a fourth storage module.

[0153] The third storage module is used by the target master node to store the operation logs generated by each of the multiple data blocks and send them to the backup node based on the generation time sequence.

[0154] The read module is used by backup nodes to read the logical path information of the main storage location from multiple operation logs based on the generation time sequence relationship.

[0155] The fourth storage module is used to store multiple data blocks into multiple backup storage devices of the backup node based on multiple logical path information, and to determine the backup storage location of each data block, wherein the logical path information of the backup storage location of the data block is consistent with the logical path information of the primary storage location.

[0156] According to embodiments of this disclosure, the electronic document processing apparatus 400 further includes an identification module, a second determination module, an encryption module, and a second sending module.

[0157] The identification module is used to identify the business identifier carried in the document subscription request or the transaction scenario of the target object in response to the document subscription request of the target object.

[0158] The second determination module is used to determine the target business type based on the business identifier or transaction scenario.

[0159] The encryption module is used to perform asymmetric encryption on sub-documents corresponding to the target business type to obtain encrypted documents.

[0160] The second sending module is used to send the encrypted document to the target object.

[0161] For example, any plurality of the response module 410, classification module 420, quality inspection module 430, and indexing module 440 may be combined into one module / unit / subunit, or any one of these modules / units / subunits may be split into multiple modules / units / subunits. Alternatively, at least part of the functionality of one or more of these modules / units / subunits may be combined with at least part of the functionality of other modules / units / subunits and implemented in one module / unit / subunit. According to embodiments of the present disclosure, at least one of the response module 410, classification module 420, quality inspection module 430, and indexing module 440 may be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging the circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the response module 410, classification module 420, quality inspection module 430 and index module 440 may be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0162] Figure 5 A block diagram of an electronic device suitable for implementing an electronic document processing method according to an embodiment of the present invention is shown.

[0163] Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0164] like Figure 5As shown, a computer electronic device 500 according to an embodiment of the present invention includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage portion 508 into a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0165] RAM 503 stores various programs and data required for the operation of electronic device 500. Processor 501, ROM 502, and RAM 503 are interconnected via bus 504. Processor 501 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 502 and / or RAM 503. It should be noted that programs may also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in one or more memories.

[0166] Optionally, the electronic device 500 may also include an input / output (I / O) interface 505, which is also connected to the bus 504. The electronic device 500 may also include one or more of the following components connected to the input / output (I / O) interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output (I / O) interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 510 as needed so that computer programs read from it can be installed into the storage section 508 as needed.

[0167] Optionally, the method flow according to embodiments of the present invention can be implemented as a computer software program. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by processor 501, it performs the functions defined in the system of embodiments of the present invention. Optionally, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0168] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the electronic document processing method according to embodiments of the present invention.

[0169] Optionally, the computer-readable storage medium can be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0170] For example, optionally, the computer-readable storage medium may include the ROM 502 and / or RAM 503 described above and / or one or more memories other than ROM 502 and RAM 503.

[0171] Embodiments of the present invention also include a computer program product comprising a computer program containing program code for performing the methods provided in the embodiments of the present invention. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the electronic document processing method provided in the embodiments of the present invention.

[0172] When the computer program is executed by the processor 501, it performs the functions defined in the system / apparatus of this embodiment of the invention. Optionally, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0173] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 509, and / or installed from a removable medium 511. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0174] Optionally, program code for executing the computer programs provided in the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0175] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features described in the various embodiments of the present disclosure can be combined and / or combined in various ways, even if such combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments of this disclosure may be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0176] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. An electronic document processing method, characterized in that, The method includes: In response to an electronic document storage request for a target business, the file metadata and business metadata of the electronic document are stored in a target region node, wherein the target region node is associated with the transaction attributes of the target business, and the electronic document includes multiple sub-documents; The file metadata corresponding to each of the multiple sub-documents in the target region node is processed using a classification model to obtain the business type corresponding to each of the multiple sub-documents. Based on the business quality rules corresponding to the business type, the sub-document is subjected to quality detection to obtain the target sub-document that matches the business quality rules. Based on the business type and business metadata of the target sub-document, a distributed index structure is constructed for the target sub-document so that the target sub-document can be queried according to the distributed index structure during the document query process.

2. The method according to claim 1, characterized in that, The step of performing quality detection on the sub-document based on the business quality rules corresponding to the business type to obtain target sub-documents that match the business quality rules includes: Based on the business quality rules and the business type, determine the detection items corresponding to the sub-document; Perform quality inspection on the business metadata related to the detection item in the sub-document to obtain the quality inspection result; The target sub-document is determined based on the quality inspection results.

3. The method according to claim 2, characterized in that, The business quality rules include at least one of the following: The first rule for determining whether the documents in the sub-document meet the validity period requirement; The second rule characterizes whether the contract documents in the sub-documents meet the document validity criteria.

4. The method according to claim 1, characterized in that, The step of constructing a distributed index structure for the target sub-document based on the classification result and the business metadata of the target sub-document includes: The electronic document was processed using statistical algorithms to obtain the word frequency ranking results; Keyword extraction is performed on the business metadata to obtain keywords; The word frequency ranking result, the classification result, and the keyword are processed using a probability ranking function to obtain a distributed index structure.

5. The method according to claim 4, characterized in that, The method further includes: In response to a document retrieval request for a target object, at least one sub-document is determined based on the distributed index structure that is related to the target keyword information indicated in the document retrieval request; A relevance score is assigned to at least one of the sub-documents using a relevance algorithm to obtain a score ranking result, wherein the score ranking result includes scores in descending order. Send the sub-document corresponding to the score result that is in the first position in the score sorting results to the target object.

6. The method according to claim 1, characterized in that, The target area nodes include multiple primary nodes and one backup node, and the number of primary nodes is determined based on the traffic volume related to the target area. The step of storing the file metadata and business metadata of the electronic document to the target area node includes: Based on the transaction entity information of the target business, determine the target master node in the target region; The file metadata and business metadata of the electronic document are stored in the target master node; The target master node synchronizes file metadata and business metadata to the backup node.

7. The method according to claim 6, characterized in that, The method further includes: The electronic document is divided into multiple data blocks based on a preset document space threshold. The multiple data blocks are stored in multiple main storage devices of the target master node, and the main storage location of each of the multiple data blocks is determined. The primary storage locations of the multiple data blocks are stored in the target master node.

8. The method according to claim 7, characterized in that, The method further includes: The target master node will store the operation logs generated by each of the multiple data blocks and send them to the backup node based on the generation time sequence. Based on the generation timing relationship, the backup node uses a reading thread to read the logical path information of the main storage location in multiple operation logs; Based on the multiple logical path information, the multiple data blocks are respectively stored in the multiple backup storage devices of the backup node, and the backup storage location of each of the multiple data blocks is determined, wherein the logical path information of the backup storage location of the data block is consistent with the logical path information of the primary storage location.

9. The method according to claim 1, characterized in that, The method further includes: In response to a document subscription request from a target object, identify the business identifier carried in the document subscription request or the transaction scenario of the target object; The target business type is determined based on the business identifier or the transaction scenario; Asymmetric encryption is performed on the sub-document corresponding to the target business type to obtain an encrypted document; The encrypted document is sent to the target object.

10. An electronic document processing device, characterized in that, The device includes: The response module is used to respond to an electronic document storage request for a target business and store the file metadata and business metadata of the electronic document to a target area node, wherein the target area node is associated with the transaction attributes of the target business, and the electronic document includes multiple sub-documents; The classification module is used to process the file meta information corresponding to each of the multiple sub-documents in the target area node using a classification model, and obtain the business type corresponding to each of the multiple sub-documents. The quality inspection module is used to perform quality inspection on the sub-document based on the business quality rules corresponding to the business type, and obtain the target sub-document that matches the business quality rules; The indexing module is used to construct a distributed index structure for the target sub-document based on its business type and business metadata, so as to enable the target sub-document to be queried based on the distributed index structure during the document query process.

11. An electronic device, comprising: One or more processors; Memory, used to store one or more programs. Wherein, when one or more programs are executed by one or more processors, the one or more processors implement the method of any one of claims 1 to 9.

12. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 9.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images