Folder sharing auditing system and method

Abstract

The invention relates to a folder sharing auditing system and method, and belongs to the technical field of computers, and the system comprises an intelligent gateway which is used for analyzing a folder carried by a received user request and sending an analyzed file processing request to a fingerprint generation cluster; the fingerprint generation cluster receives the file processing request, generates a unique identifier of a file based on the size of the file, and sends the unique identifier to the cache module; the cache module receives the unique identifier of the file, determines the cache hierarchy of the file, and sends the target file to the clustering engine module under the condition of determining that the target file does not exist in the cache hierarchy; the clustering engine module receives the target file, and sends the target file to the grading auditing module under the condition that it is determined that the target file does not have a similar group; the grading auditing module receives the target file and determines an auditing channel according to the size of the target file. According to the system, the mass file processing efficiency is improved, and the system resource utilization rate is optimized.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, and in particular relates to a folder sharing approval system and method. Background Technology

[0002] In related technologies, common folder sharing approval schemes often employ a direct and linear processing flow. That is, upon receiving a folder shared by a user, the system simply submits all files within the folder to an approval queue one by one. However, when the same file is shared multiple times by different users, the system performs the exact same duplicate approval, consuming significant computing and manual review resources. Therefore, how to better implement folder sharing approval has become an urgent problem to be solved. Summary of the Invention

[0003] In view of the shortcomings of the prior art, the purpose of the invention is to provide a folder sharing review system and method.

[0004] In a first aspect, the present invention provides a folder sharing review system, comprising: a smart gateway, configured to, upon receiving a user request, parse the folder carried in the user request and send the parsed file processing request to a fingerprint generation cluster; the fingerprint generation cluster, configured to receive the file processing request, generate a unique identifier for the file based on the file size, and send the unique identifier to a caching module, wherein the file processing request carries the parsed folder; the caching module, configured to receive the unique identifier of the file, calculate a popularity score for each file, determine the caching level of the file based on the popularity score and a threshold range, and, if it is determined that no target file exists in the caching level, send the target file to a clustering engine module; the clustering engine module, configured to receive the target file, and, if it is determined that no similar group exists for the target file, send the target file to a hierarchical review module; the hierarchical review module, configured to receive the target file and determine a review channel based on the size of the target file.

[0005] Further, generating a unique identifier for the file based on its size includes: determining whether the file size is less than a first threshold; if the file size is less than the first threshold, calculating the SHA-512 hash value of the file and using the SHA-512 hash value and the file's metadata features as the file's unique identifier; if the file size is not less than the first threshold, dividing the file into multiple file blocks based on a preset size and calculating the hash value of each file block; calculating the sum of the hash values ​​of the multiple file blocks to obtain a target value, and using the target value and the file's metadata features as the file's unique identifier.

[0006] Further, calculating the popularity score for each file includes: determining the access frequency factor for each file, where the access frequency factor = log(number of accesses to the file + 1) × 0.6, and 0.6 represents the access frequency weight; and determining the time decay factor for each file, where... Based on the size of each file, a file size factor is determined for each file; the product of the access frequency factor, the time decay factor, and the file size factor for each file is used as the popularity score for each file.

[0007] Further, determining the cache level of the file based on the file's popularity score and threshold range includes: determining the file type based on the file's popularity score and threshold range; and determining the cache level of the file based on the file type.

[0008] Further, determining whether the target file exists in the similarity group includes: converting the target file into a target feature vector; finding similar vectors of the target feature vector based on Locality Sensitive Hash (LSH), and determining similar files based on the similar vectors; constructing a target graph based on a Graph Neural Network (GNN), where the nodes of the target graph are the target file and the similar files, and the edges of the target graph are the cosine similarity between the target file and the similar files; wherein, calculating the cosine similarity between the target file and the similar files includes: calculating the cosine similarity between the target feature vector and the similar vector; wherein, if the cosine similarity is greater than the cosine similarity threshold, it is determined that the target file exists in the similarity group; if the cosine similarity is not greater than the cosine similarity threshold, it is determined that the target file does not exist in the similarity group.

[0009] Furthermore, the clustering engine module is also used to obtain the shared audit results of the similar groups when it is determined that the target file has similar groups, and to update the cache module.

[0010] Further, determining the review channel based on the size of the target file includes: determining the size of the target file; if the size of the target file is less than a second threshold, determining the review channel as real-time micro-review; if the size of the target file is greater than a third threshold but less than a fourth threshold, determining the review channel as standard review; and if the size of the target file is greater than a fifth threshold, determining the review channel as deep review.

[0011] A second aspect of the present invention provides a folder sharing approval method, comprising: upon receiving a user request, a smart gateway parses the folder carried in the user request and sends the parsed file processing request to a fingerprint generation cluster; the fingerprint generation cluster receives the file processing request, generates a unique identifier for the file based on the file size, and sends the unique identifier to a caching module, wherein the file processing request carries the parsed folder; the caching module receives the unique identifier of the file, calculates a popularity score for each file, determines the caching level of the file based on the popularity score and a threshold range, and if no target file exists in the caching level, sends the target file to a clustering engine module; the clustering engine module receives the target file, and if no similar group exists for the target file, sends the target file to a hierarchical approval module; the hierarchical approval module receives the target file and determines an approval channel based on the size of the target file.

[0012] A third aspect of the present invention provides an electronic device comprising: at least one processor; and a memory communicatively connected to said at least one processor; wherein the memory stores instructions executable by said at least one processor, said instructions being executed by said at least one processor to enable said at least one processor to perform the method described in any one of the second aspects of the present invention.

[0013] A fourth aspect of the present invention provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described in any one of the second aspects of the present invention.

[0014] The beneficial effects of this invention are as follows:

[0015] The folder sharing review system and method of this invention includes: an intelligent gateway, used to parse the folder carried in a user request upon receiving the request, and send the parsed file processing request to a fingerprint generation cluster; a fingerprint generation cluster, used to receive the file processing request, generate a unique identifier for the file based on its size, and send the unique identifier to a caching module, wherein the file processing request carries the parsed folder; a caching module, used to receive the unique identifier of the file, calculate the popularity score of each file, determine the caching level of the file based on the popularity score and a threshold range, and send the target file to a clustering engine module if no target file exists in the caching level; a clustering engine module, used to receive the target file, and send the target file to a hierarchical review module if no similar group exists; and a hierarchical review module, used to receive the target file and determine the review channel based on the size of the target file. This system, by establishing an intelligent pipeline for modular processing, significantly reduces repetitive work and resource consumption while ensuring review effectiveness, thus improving the efficiency of processing massive amounts of files and optimizing system resource utilization. Attached Figure Description

[0016] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. It is obvious that the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings.

[0017] Figure 1 This is a schematic diagram of a folder sharing approval system according to an embodiment of the present invention;

[0018] Figure 2 This is a schematic diagram of a folder sharing approval system according to a specific embodiment of the present invention;

[0019] Figure 3 This is a flowchart of a folder sharing approval method according to an embodiment of the present invention;

[0020] Figure 4 This is a structural block diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts disclosed in this invention.

[0023] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The terms "installed," "connected," and "linked" should be interpreted broadly; for example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0024] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of methods and systems consistent with some aspects of the invention as detailed in the appended claims.

[0025] Current large folder sharing systems suffer from the following technical problems: Duplicate review issues: The same file needs to be reviewed repeatedly in different sharing links, with a duplicate review rate exceeding 95% in scenarios with tens of millions of files; Database performance bottlenecks: High-frequency queries cause peak database QPS to exceed 15,000, significantly increasing response latency; Severe response latency: The average response time for sharing tens of millions of files exceeds 30 seconds, resulting in a poor user experience; Inability to share results: Review results across users and sessions are isolated and cannot be reused.

[0026] To this end, the present invention proposes a folder sharing approval system, method and related equipment. Specifically, the folder sharing approval system, method and related equipment of the present invention are described below with reference to the accompanying drawings.

[0027] Figure 1 This is a schematic diagram of a folder sharing approval system according to an embodiment of the present invention. It should be noted that the folder sharing approval system of this embodiment can be applied to the folder sharing approval method of this embodiment. This folder sharing approval system can be configured on an electronic device or in a server. This application does not limit the scope of the application.

[0028] like Figure 1 As shown, the folder sharing approval system includes:

[0029] The smart gateway 110 is used to parse the folder carried in the user request when it receives a user request, and send the parsed file processing request to the fingerprint generation cluster 120.

[0030] The fingerprint generation cluster 120 is used to receive file processing requests, generate a unique identifier for the file based on the file size, and send the unique identifier to the cache module 130. The file processing request carries the parsed folder.

[0031] The caching module 130 is used to receive the unique identifier of the file, calculate the popularity score of each file, determine the caching level of the file based on the popularity score and threshold range, and send the target file to the clustering engine module 140 if it is determined that the target file does not exist in the caching level.

[0032] Clustering engine module 140 is used to receive target files and, if it is determined that no similar groups exist in the target files, send the target files to hierarchical review module 150.

[0033] The tiered review module 150 is used to receive target files and determine the review channel based on the size of the target files.

[0034] In other words, the intelligent gateway 110 breaks down user-shared folders into independent file processing tasks. The fingerprint generation cluster 120 generates a unique "fingerprint" for each file, enabling the system to instantly identify duplicate files. The caching module 130 dynamically determines the storage location of files based on their access frequency. The clustering engine module 140 groups non-duplicate files with similar content into the same group. The hierarchical review module 150 distributes files to different review channels based on factors such as file size. By establishing an intelligent pipeline for modular processing, this system significantly reduces repetitive work and resource consumption while ensuring review effectiveness. In other words, the system improves the efficiency of processing massive amounts of files and optimizes system resource utilization.

[0035] In an embodiment of the present invention, the folder requested by the user is parsed, that is, the file list, structure, and metadata features of each file in the folder are parsed. The metadata features include, but are not limited to, file name, size, type, etc.

[0036] In embodiments of the present invention, the file size can be determined by checking if it is less than a first threshold. If the file size is less than the first threshold, the SHA-512 hash value of the file is calculated, and the SHA-512 hash value and the file's metadata features are used as the file's unique identifier. If the file size is not less than the first threshold, the file is divided into multiple file blocks based on a preset size, and the hash value of each file block is calculated. The sum of the hash values ​​of the multiple file blocks is calculated to obtain a target value, and the target value and the file's metadata features are used as the file's unique identifier. For example, the first threshold can be 10MB.

[0037] In other words, for files smaller than 10MB, the SHA-512 hash value is calculated directly. This means the file is quickly read into memory, and the hash calculation takes little time. However, the overhead of parallel processing (thread scheduling) may exceed the computation itself. Files larger than or equal to 10MB are divided into blocks, for example, each block is 5MB. This avoids reading the entire large file into memory, reducing memory pressure. Parallel computing can fully utilize multi-core CPUs, significantly improving the processing speed of large files.

[0038] In embodiments of the present invention, an access frequency factor is determined for each file, wherein the access frequency factor = log(number of file accesses + 1) × 0.6, and 0.6 represents the access frequency weight; a time decay factor is determined for each file, wherein... Based on the size of each file, a file size factor is determined for each file; the product of the access frequency factor, the time decay factor, and the file size factor for each file is used as the popularity score for each file.

[0039] The implementation of determining the access frequency factor for each file can be understood as follows: Maintain an `access_count` field for each file's unique identifier, recording the historical access count. Each time a file is accessed, the `access_count` is incremented. Then, the access frequency factor for each file is calculated based on log(file access count + 1) × 0.6.

[0040] The implementation of determining the time decay factor for each file can be understood as follows: Maintain a `last_access_time` field for each file's unique identifier, recording the timestamp (milliseconds) of the last access. And obtain the current timestamp `current_time`. Then, based on... Determine the time decay factor for each file.

[0041] The implementation of the file size factor for each file can be understood as follows: This factor is weighted based on file size, with smaller files being cached first (weighted at 1.2), and larger files potentially consuming more resources (weighted at 0.8). Specifically, small files (files less than 1MB) are weighted at 1.2 (i.e., the size of the small file is multiplied by 1.2): caching small files is encouraged because they have extremely high transmission efficiency on CDNs, maximizing the reduction of user latency. Large files (files greater than 100MB) are weighted at 0.8 (i.e., the size of the large file is multiplied by 0.8): placing very large files on CDNs or SSDs is discouraged because caching them is cost-effective.

[0042] In embodiments of the present invention, the file type is determined based on the file's popularity score and threshold range; the file's cache level is determined based on the file type.

[0043] For example, if a file's popularity score is greater than 8, the file type can be determined as a hot file, and the corresponding caching level is an edge CDN node; if the file's popularity score is between 5 and 8, the file type can be determined as a warm file, and the corresponding caching level is a local SSD cache; if the file's popularity score is between 2 and 5, the file type can be determined as a cold file, and the corresponding caching level is a Redis cluster; if the file's popularity score is less than 2, the file type can be determined as a historical file, and the corresponding caching level is a distributed file system.

[0044] Among them, edge CDN nodes are distributed nodes, ensuring proximity to users and the lowest latency; local SSD cache is a high-speed solid-state drive on the server, with access speed faster than the network but slower than CDN; Redis cluster is an in-memory database, which is very fast but has high capacity costs, suitable for storing "cold" data that is only occasionally accessed; distributed file system is the lowest-cost large-capacity storage, used for archiving, with slow but acceptable access speed.

[0045] In an embodiment of the present invention, when determining the cache level of each file, the target file can be identified, and then it can be determined whether the target file exists in the cache level, i.e., whether the cache is hit. If the target file exists in the cache level, the review result of the target file is returned immediately. If the target file does not exist in the cache level, the target file is sent to the clustering engine module 140 so that the clustering engine module 140 receives the target file. If it is determined that the target file does not have a similar group, the target file is sent to the hierarchical review module 150 so that the hierarchical review module 150 receives the target file and determines the review channel according to the size of the target file.

[0046] The method for determining whether there are similar groups in the target files is as follows: the target files are converted into target feature vectors; similar vectors of the target feature vectors are found based on Locality Sensitive Hashing (LSH), and similar files are determined based on the similar vectors; a target graph is constructed based on a Graph Neural Network (GNN), where the nodes of the target graph are the target files and similar files, and the edges of the target graph are the cosine similarity between the target files and similar files.

[0047] The calculation of the cosine similarity between the target file and similar files includes: calculating the cosine similarity between the target feature vector and the similar vector; wherein, if the cosine similarity is greater than the cosine similarity threshold, it is determined that the target file has a similar group; if the cosine similarity is not greater than the cosine similarity threshold, it is determined that the target file does not have a similar group.

[0048] In an embodiment of the present invention, the clustering engine module 140 is further configured to obtain the shared audit results of the similar groups when it is determined that there are similar groups in the target files, and update the cache module.

[0049] In an embodiment of the present invention, the method for determining the review channel based on the size of the target file is as follows: determine the size of the target file; if the size of the target file is less than a second threshold, determine the review channel as real-time micro-review; if the size of the target file is greater than a third threshold and less than a fourth threshold, determine the review channel as standard review; if the size of the target file is greater than a fifth threshold, determine the review channel as deep review.

[0050] For example, when the target file size is less than 100KB, real-time micro-review is performed, which involves direct memory processing for text, small icons, etc., with a response time of less than 50ms. When the target file size is greater than 100KB but less than 10MB, standard review is performed, which involves multi-threaded processing for ordinary images, short audio files, etc., supporting 500 concurrent requests per node. When the target file size is greater than 10MB, deep review is performed, which involves high-definition images, long videos, compressed files, etc. GPU-equipped computing nodes are used to leverage the parallel computing capabilities of GPUs for frame-level and pixel-level complex analysis (such as OCR, object recognition, speech-to-text, and recognition of pornographic and violent content).

[0051] In an embodiment of the present invention, once the review of a file is completed, the review result can be immediately written to the file's cache level (such as a Redis cluster, local SSD, etc.). That is, when the next user requests the same or similar file, the system can directly return the result from the cache in milliseconds without having to go through the time-consuming review process again, which greatly improves the user experience and system throughput.

[0052] In an embodiment of the present invention, once a file has been reviewed, the review result can be stored as a permanent record in a distributed file system such as HDFS or S3, which ensures that the review result is never lost. Even if the entire caching system restarts or crashes, the data can be recovered from the distributed file system. Furthermore, when it is necessary to expand or rebuild the caching cluster, all data can be reloaded from here.

[0053] In an embodiment of the present invention, after a file has been reviewed, the fingerprint and review result of this new file are associated with an existing fingerprint clustering engine module. In the future, if similar but not identical files need to be reviewed, the clustering engine module can find this group and directly share the review results, avoiding duplicate reviews and saving significant computing resources.

[0054] In one specific embodiment of the present invention, such as Figure 2As shown, the client initiates a sharing request containing a folder ID. The smart gateway receives this request, parses the folder, and sends the parsed folder to the fingerprint generation cluster based on the file processing request. The fingerprint generation cluster generates a unique identifier (i.e., fingerprint) for each file, resulting in a fingerprint set (i.e., a list of fingerprints for multiple files), which is then sent to the caching module. The caching module checks if these fingerprints already exist in the cache. The caching system stores previously processed file fingerprints and their corresponding sharing links. If all fingerprints are found in the cache, the caching module directly returns the corresponding sharing link. The process jumps to the "return sharing link" step. If only some fingerprints are found in the cache, the caching module generates a "missing fingerprint list," containing fingerprints that do not exist in the cache. This missing fingerprint list is submitted to the clustering engine module, which performs similarity analysis on these missing fingerprints to determine if they belong to an existing file group (similar group). Specifically, the clustering engine module searches for similar groups based on the missing fingerprints. If a missing fingerprint matches an existing file group (similar group), the clustering engine module returns a "sharing result." This means these files can be categorized into existing groups without needing re-approval. The cache module is then updated (associating the new fingerprint with the existing share link) and proceeds to the "Return Link" step. If no fingerprint is found and no similar group is identified, the clustering engine module marks it as a "New File List." These files require further approval; the new file list is sent to the hierarchical approval module, which assigns an approval channel and executes the approval task. After approval, the results are stored, and the clustering engine module is updated. Regardless of whether the approval is achieved through cache hit, similar group sharing, or approval completion, a share link is ultimately generated and returned to the client.

[0055] According to an embodiment of the present invention, the folder sharing approval system dynamically selects a hash generation strategy based on file size (with 10MB as a threshold). Small files are directly hashed using SHA-512, reducing memory usage and computational overhead; large files are processed in blocks in parallel, fully utilizing multi-core CPUs, avoiding memory pressure, and improving processing speed; file metadata and hash values ​​are combined to ensure the uniqueness and consistency of file identifiers; file popularity (e.g., hot, warm, cold, historical files) is dynamically calculated by considering access frequency, time decay, and file size factors, achieving precise cache hierarchy allocation; weighting factors (e.g., prioritizing small files for caching) prevent large files from occupying expensive storage, improving cache cost-effectiveness; locality-sensitive hashing (LSH) is used to quickly find similar files, and a file relationship graph is constructed using graph neural networks (GNNs), with cosine hashing used to further refine the hashing. Similarity thresholds are used to determine similarity groups; if similar groups exist in the target file, historical review results are directly shared to avoid duplicate calculations and improve review efficiency; similarity group results are synchronously updated in the cache module, further accelerating subsequent request responses and achieving dynamic cache updates; review resources are allocated according to file size to avoid excessive GPU resource consumption by small files, while ensuring that large files are fully analyzed, thus improving overall review throughput; high-end storage usage is reduced and maintenance costs are lowered through popularity scoring and caching tier strategies; tiered review channels avoid wasting computing power and optimize hardware investment; the system achieves high performance, low latency, and high accuracy, and reduces the rate of duplicate reviews when millions of files are shared multiple times in a short period of time.

[0056] To enable those skilled in the art to more readily understand the present invention, Figure 3 This is a folder sharing approval method according to an embodiment of the present invention, such as... Figure 3 As shown, the folder sharing approval method includes:

[0057] When the S310 smart gateway receives a user request, it parses the folder carried in the user request and sends the parsed file processing request to the fingerprint generation cluster.

[0058] The S320 fingerprint generation cluster receives file processing requests, generates a unique identifier for the file based on its size, and sends the unique identifier to the cache module. The file processing request carries the parsed folder.

[0059] In an embodiment of the present invention, it is determined whether the size of the file is less than a first threshold; if the size of the file is determined to be less than the first threshold, the SHA-512 hash value of the file is calculated, and the SHA-512 hash value and the metadata features of the file are used as the unique identifier of the file; if the size of the file is determined to be not less than the first threshold, the file is divided into multiple file blocks based on a preset size, and the hash value of each file block is calculated; the sum of the hash values ​​of the multiple file blocks is calculated to obtain the target value, and the target value and the metadata features of the file are used as the unique identifier of the file.

[0060] S330: The caching module receives the unique identifier of the file, calculates the popularity score of each file, and determines the caching level of the file based on the popularity score and threshold range. If the target file does not exist in the caching level, the target file is sent to the clustering engine module.

[0061] In embodiments of the present invention, an access frequency factor is determined for each file, wherein the access frequency factor = log(number of file accesses + 1) × 0.6, and 0.6 represents the access frequency weight; a time decay factor is determined for each file, wherein... Based on the size of each file, a file size factor is determined for each file; the product of the access frequency factor, the time decay factor, and the file size factor for each file is used as the popularity score for each file.

[0062] In embodiments of the present invention, the file type is determined based on the file's popularity score and threshold range; the file's cache level is determined based on the file type.

[0063] S340, the clustering engine module receives the target file and, if it determines that there are no similar groups in the target file, sends the target file to the hierarchical review module.

[0064] In embodiments of the present invention, the target file is converted into a target feature vector; similar vectors of the target feature vector are found based on Locality Sensitive Hash (LSH), and similar files are determined based on the similar vectors; a target graph is constructed based on a Graph Neural Network (GNN), where the nodes of the target graph are the target file and similar files, and the edges of the target graph are the cosine similarity between the target file and similar files; wherein, calculating the cosine similarity between the target file and similar files includes: calculating the cosine similarity between the target feature vector and the similar vector; wherein, if the cosine similarity is greater than a cosine similarity threshold, it is determined that the target file has a similar group; if the cosine similarity is not greater than the cosine similarity threshold, it is determined that the target file does not have a similar group.

[0065] In an embodiment of the present invention, if it is determined that there are similar groups for the target file, the shared audit results of the similar groups are obtained and the cache module is updated.

[0066] S350, the hierarchical review module receives the target file and determines the review channel based on the size of the target file.

[0067] In an embodiment of the present invention, the size of the target file is determined; if the size of the target file is less than a second threshold, the review channel is determined to be real-time micro-review; if the size of the target file is greater than a third threshold and less than a fourth threshold, the review channel is determined to be standard review; if the size of the target file is greater than a fifth threshold, the review channel is determined to be deep review.

[0068] According to the folder sharing approval method of this invention, the intelligent gateway breaks down user-shared folders into independent file processing tasks. A fingerprint generation cluster generates a unique "fingerprint" for each file, enabling the system to instantly identify duplicate files. The caching module dynamically determines the storage location of files based on their access frequency. The clustering engine module groups non-duplicate files with similar content into the same group. The hierarchical approval module distributes files to different approval channels based on factors such as file size. Thus, by establishing an intelligent pipeline for modular processing, the method significantly reduces repetitive work and resource consumption while ensuring approval effectiveness. In other words, this method improves the efficiency of processing massive amounts of files and optimizes system resource utilization.

[0069] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0070] According to one aspect of the present invention, an electronic device is provided.

[0071] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Figure 4 As shown, an electronic device may include one or more ( Figure 4Only one is shown in the image. A processor 102 (which may include, but is not limited to, a microprocessor unit (MPU) or a programmable logic device (PLD)) and a memory 104 for storing data are also shown. In one exemplary embodiment, the electronic device may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 4 The structure shown is for illustrative purposes only and does not limit the structure of the terminal device described above. For example, the terminal device may also include components that are more... Figure 4 The more or fewer components shown, or having the same Figure 4 Equivalent functions or ratios shown Figure 4 The functions shown have more different configurations.

[0072] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the folder sharing and auditing method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to terminal devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0073] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the switching device. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0074] This invention proposes a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute a folder sharing auditing method.

[0075] The applicant of this invention has provided a detailed description of the embodiments of the invention in conjunction with the accompanying drawings. However, those skilled in the art should understand that the above embodiments are merely preferred embodiments of the invention. The detailed description is only intended to help readers better understand the spirit of the invention and is not intended to limit the scope of protection of the invention. On the contrary, any improvements or modifications made based on the inventive spirit of the invention should fall within the scope of protection of the invention.

[0076] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0077] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A folder sharing review system, characterized by, The system comprises: an intelligent gateway, configured to parse a folder carried by a user request and send a file processing request of the parsed folder to a fingerprint generation cluster when the user request is received; the fingerprint generation cluster, configured to receive the file processing request, generate a unique identifier of the file based on a size of the file, and send the unique identifier of the file to a cache module, wherein the file processing request carries the parsed folder; the cache module, configured to receive the unique identifier of the file, calculate a heat score of each file, and determine a cache level of the file based on the heat score of the file and a threshold range, and send a target file to a clustering engine module when the target file does not exist in the cache level; the clustering engine module, configured to receive the target file and send the target file to a hierarchical review module when the target file does not exist in a similar group; the hierarchical review module, configured to receive the target file and determine a review channel according to a size of the target file.

2. The folder sharing audit system of claim 1, wherein, The unique identifier of the file is generated based on the size of the file, comprising: determining whether the size of the file is less than a first threshold value; calculating a SHA-512 hash value of the file and taking the SHA-512 hash value and metadata features of the file as the unique identifier of the file when it is determined that the size of the file is less than the first threshold value; splitting the file into a plurality of file blocks based on a preset size when it is determined that the size of the file is not less than the first threshold value, calculating a hash value of each file block; calculating a sum of the hash values of the plurality of file blocks to obtain a target value, and taking the target value and the metadata features of the file as the unique identifier of the file.

3. The file share review system of claim 1, wherein, The heat score of each file is calculated, comprising: determining an access frequency factor of each file, wherein the access frequency factor = log (access times of the file + 1) x 0.6, and 0.6 represents an access frequency weight; determining a time decay factor for each of the files, wherein determining a file size factor of each file based on the size of each file; taking a product of the access frequency factor of each file, a time decay factor of each file, and the file size factor of each file as the heat score of each file.

4. The file share review system of claim 1, wherein, The cache level of the file is determined based on the heat score of the file and the threshold range, comprising: determining a type of the file based on the heat score of the file and the threshold range; determining the cache level of the file based on the type of the file.

5. The file share review system of claim 1, wherein, Determining whether the target file exists in the similar group, comprising: converting the target file into a target feature vector; finding a similar vector of the target feature vector based on local sensitive hashing (LSH) and determining a similar file based on the similar vector; constructing a target graph based on graph neural network (GNN), wherein nodes of the target graph are the target file and the similar file, and edges of the target graph are cosine similarities of the target file and the similar file; wherein the cosine similarities of the target file and the similar file are calculated, comprising: calculating the cosine similarity of the target feature vector and the similar vector; wherein, in a case that the cosine similarity is greater than the cosine similarity threshold, it is determined that the target file exists in the similar group; in a case that the cosine similarity is not greater than the cosine similarity threshold, it is determined that the target file does not exist in the similar group.

6. The folder sharing audit system of claim 1, wherein, In a case that it is determined that the target file exists in the similar group, the clustering engine module is further configured to acquire a shared review result of the similar group and update the cache module.

7. The folder sharing audit system of claim 1, wherein, The method further comprises: determining the size of the target file; in a case that the size of the target file is less than a second threshold, determining that the review channel is real-time micro review; in a case that the size of the target file is greater than a third threshold and less than a fourth threshold, determining that the review channel is standard review; in a case that the size of the target file is greater than a fifth threshold, determining that the review channel is deep review.

8. A folder sharing review method characterized by, The method further comprises: In a case that the intelligent gateway receives a user request, the intelligent gateway parses a folder carried by the user request and sends a file processing request of the parsed folder to a fingerprint generation cluster; The fingerprint generation cluster receives the file processing request, generates a unique identifier of the file based on the size of the file, and sends the unique identifier of the file to a cache module, wherein the file processing request carries the parsed folder; The cache module receives the unique identifier of the file, calculates a heat score of each file, and determines a cache level of the file based on the heat score of the file and a threshold range, and in a case that the target file does not exist in the cache level, the target file is sent to a clustering engine module; The clustering engine module receives the target file, and in a case that it is determined that the target file does not exist in a similar group, the target file is sent to a hierarchical review module; The hierarchical review module receives the target file, and determines a review channel according to the size of the target file.

9. An electronic device, comprising: The method further comprises: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of claim 8.

10. A non-transitory computer-readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method of claim 8. The computer instructions are used to enable the computer to perform the method of claim 8.

Images