Dores-based file tag design method and system

Abstract

The invention provides a Dores-based file tag design method and system. The method comprises the following steps: acquiring an input to-be-marked file and file information of the file; screening file source information, file service information and file affiliation information from the file information; constructing a file label based on the file information, wherein the file label comprises a file source label corresponding to the file source information and a file service label corresponding to the file service information; a service request of a user is obtained, a corresponding label combination is screened based on the service request of the user, and the label combination comprises at least one file label; and constructing a file set by the files with the file tags in the tag combination, obtaining home addresses of the files in the file set, and screening in each home address. According to the scheme, the tag combination can be constructed through the service request of the user to obtain the corresponding file set, the file search range can be preliminarily narrowed, and the processing efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a Doris-based file label design method and system. Background Art

[0002] The full name of Doris is Apache Doris, which is a high-performance, real-time analytical database based on the MPP (Massively Parallel Processing) large-scale parallel processing architecture. It can not only support high-concurrency point query scenarios, but also support high-throughput complex analysis scenarios.

[0003] MPP (Massively Parallel Processing) means large-scale parallel processing. In a non-shared database cluster, each node has an independent disk storage system and memory system. Business data is divided into various nodes according to the database model and application characteristics. Each data node is connected to each other through a dedicated network or a commercial general network, and they work together to provide database services as a whole. Non-shared database clusters have the advantages of full scalability, high availability, high performance, excellent cost-effectiveness, and resource sharing. In simple terms, MPP is to distribute tasks in parallel to multiple servers and nodes. After the calculation is completed on each node, the results of each part are aggregated together to obtain the final result.

[0004] However, the existing Doris data processing system often directly connects the interface layer to the application layer to perform corresponding data processing work. Since MPP distributes tasks in parallel to multiple servers and nodes, direct processing is difficult to quickly locate data files and has low efficiency. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a file tag design method based on Doris to eliminate or improve one or more defects existing in the prior art.

[0006] One aspect of the present invention provides a Doris-based file label design method, the method comprising the following steps:

[0007] Obtain the entered files to be marked and the file information of the files;

[0008] Filtering file source information, file business information and file ownership information from the file information;

[0009] Building a file tag based on the file information, the file tag including a file source tag corresponding to the file source information and a file service tag corresponding to the file service information;

[0010] Obtaining a business request from a user, and filtering a corresponding tag combination based on the business request from the user, wherein the tag combination includes at least one file tag;

[0011] A file set is constructed from files having file tags in the tag combination, the attribution addresses of the files in the file set are obtained, and screening is performed in each attribution address.

[0012] By adopting the above scheme, this scheme constructs a file tag of the file through the file information of the file while entering the file. This scheme constructs a tag layer based on the file tags of each file. The tag layer is arranged between the interface layer and the application layer. When performing data screening, a tag combination can be constructed through the user's business request to obtain a corresponding file set, which can preliminarily narrow the search scope of the file and improve processing efficiency.

[0013] In some implementations of the present invention, the step of constructing a file tag based on the file information further includes:

[0014] Construct a file number for each file;

[0015] The files having the file tag are constructed into a tag file set, and the file numbers of the corresponding files are stored in the tag file set.

[0016] In some embodiments of the present invention, in the step of constructing a file collection from files having file tags in the tag combination:

[0017] Get the tag file set corresponding to each tag in the tag combination;

[0018] The intersection of the file numbers in each tag file set is calculated to obtain the file set.

[0019] By adopting the above scheme, this scheme numbers the files and constructs a label file set for each file label. In the process of actually determining the file set, after obtaining the target label combination, the file set can be efficiently determined by constructing the intersection of the label file set in the label combination, thereby improving processing efficiency and processing accuracy.

[0020] In some embodiments of the present invention, in the step of obtaining the attribution address of the file in the file collection and screening each attribution address, the file attribution information corresponding to each file number in the file collection is extracted, and the attribution address of the file is determined based on the file attribution information.

[0021] In some embodiments of the present invention, the file source information includes an uploading IP and an uploading user. In the step of constructing a file tag based on the file information, the uploading IP tag in the file source tag is constructed based on the uploading IP, and the uploading user tag in the file source tag is constructed based on the uploading user.

[0022] In some embodiments of the present invention, the file business information includes file scene information and time information. In the step of constructing a file tag based on the file information, the file scene tag in the file business tag is constructed based on the file scene information, and the file time tag in the file business tag is constructed based on the time information.

[0023] In some implementations of the present invention, in the step of screening the corresponding tag combination based on the user's service request:

[0024] Performing word segmentation processing on the text of the user's service request;

[0025] Encode each word segment and calculate the similarity between the business request and each file label based on the obtained encoding;

[0026] The file tags are filtered based on the similarity to obtain a tag combination corresponding to the user's business request.

[0027] By adopting the above scheme, this scheme analyzes the user's business request, determines the degree of correlation between the business request and each file tag, and then determines the relevant file tags to obtain a tag combination. The tag combination represents the target range of the user's business request, which can quickly narrow the target file range and ensure processing efficiency.

[0028] In some embodiments of the present invention, in the step of encoding each word segment and calculating the similarity between the service request and each file tag based on the obtained encoding:

[0029] Combine multiple participles and encode each participle combination;

[0030] Compute the similarity between each encoding and the encoding corresponding to each file tag.

[0031] By adopting the above scheme, this scheme segments the business request and combines the segmented words, which can ensure that the information in the business request is fully released, ensure the final matching range of the file label, and prevent the corresponding files from being missed; and although the combination of this scheme will lead to the generation of some meaningless phrases, the meaningless phrases often have a low similarity with each file label and will not affect the processing accuracy.

[0032] In some implementations of the present invention, the method further includes modifying the file tag of the file based on a preset file tag modification channel.

[0033] The second aspect of the present invention also provides a Doris-based file label design system, which includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, the processor is used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the system implements the steps implemented by the method described above.

[0034] The third aspect of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps implemented by the aforementioned Doris-based file label design method.

[0035] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention can be specifically pointed out and obtained in the specification and the accompanying drawings.

[0036] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0038] Figure 1 A schematic diagram of an implementation of the Doris-based file label design method of the present invention;

[0039] Figure 2 This is a schematic diagram of the architecture of this solution;

[0040] Figure 3 This is a schematic diagram of the file label of this scheme;

[0041] Figure 4 This is a schematic diagram of data processing for this scheme. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0043] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0044] Apache Doris is a high-performance, real-time analytical database based on the MPP architecture. It is well-known for its extremely fast and easy-to-use features. It can return query results under massive data in sub-second response time. It can not only support high-concurrency point query scenarios, but also support high-throughput complex analysis scenarios. Based on this, Apache Doris can better meet the use scenarios such as report analysis, ad hoc query, unified data warehouse construction, data lake federated query acceleration, etc. Users can build user behavior analysis, AB experiment platform, log retrieval analysis, user portrait analysis, order analysis and other applications on it.

[0045] Introduction to existing technology:

[0046] ClickHouse: The data warehouse adopts a flat table design, and ClickHouse performs well in flat table reading. It is also known for its high performance, claiming to be 100 to 1000 times faster than transactional databases. ClickHouse has a high-performance vectorized execution engine with rich functions and high reliability. ClickHouse may be better in terms of import performance and single-table query performance. Although it is rich in functions, the threshold for use is relatively high. Its configuration files and permission management may be more complex, requiring more customization and deeper technical strength. Although cluster expansion is also supported, the expansion and reduction process may be relatively complex and cumbersome. It requires self-developed tool support or manual configuration balance, which increases the operation and maintenance costs.

[0047] To support cross-table joins and point queries, users need a star schema, but it is difficult to implement in ClickHouse

[0048] Changes in financial data need to be updated to the data warehouse in real time. In ClickHouse, this is done by recreating the flattened table overwriting the old one, but it is not fast enough.

[0049] MySQL: A typical C / S (Client / Server) architecture, divided into two parts: client and server. Usually, it is necessary to vertically expand the server, that is, to increase the server's hardware resources (such as CPU and memory, etc.) to meet higher concurrency requirements. Applicable scenarios: Suitable for processing real-time transactions and business operations, such as online shopping, banking transactions, etc. Limited to processing large-scale data and complex queries, it may face performance bottlenecks and scalability limitations.

[0050] Apache HBase: An open source, distributed NoSQL database that runs on the Hadoop ecosystem and is primarily designed to handle massive amounts of structured data. Application scenarios: Suitable for scenarios that require real-time updates and random read and write of large amounts of data, such as social media, log data, etc. At the same time, HBase is also commonly used to build real-time analysis systems, but the query performance may not be as good as that of specialized OLAP databases. HBase has high-speed read and write performance and is particularly suitable for real-time analysis and high-concurrency read and write scenarios. However, its query performance is relatively low, especially in complex query scenarios. Ease of use: The configuration and management of HBase is relatively complex and requires certain expertise and experience. At the same time, its data model and query language may also cause certain learning costs for beginners. Compatibility: HBase is tightly integrated with the Hadoop ecosystem, but its compatibility with other database systems may be limited.

[0051] These components must work together to meet all requirements, making the data warehouse overloaded. It is not easy to get started because engineers must be trained on all these components. In addition, the complexity of the architecture increases the risk of delays.

[0052] Therefore, users try to find a tool that can better meet their needs. First, real-time capabilities are needed, including real-time writing, real-time updating, and real-time response to data queries. Secondly, more flexible data analysis capabilities are needed to support customer-oriented self-service queries, such as multi-dimensional analysis, large table connection queries, primary key indexes, summarization, and drilling. Then, in terms of batch processing, data writing is also expected to have high throughput.

[0053] Disadvantages of existing technology:

[0054] ClickHouse: The data warehouse adopts a flat table design, and ClickHouse performs well in flat table reading. However, as the business develops, ClickHouse faces two challenges:

[0055] To support cross-table joins and point queries, users need a star schema, but it is difficult to implement in ClickHouse

[0056] Changes in financial data need to be updated to the data warehouse in real time. In ClickHouse, this is done by recreating the flattened table overwriting the old one, but it is not fast enough.

[0057] Mysql: The calculated data indicators are stored in MySQL, but as the data size grows, MySQL problems include extended execution time and errors.

[0058] Apache HBase: HBase is responsible for primary key queries. It reads customer status from MySQL and Hive, including customer credit, insurance period, and total insured amount.

[0059] Since HBase does not support secondary indexes, it is limited in its ability to read non-primary key columns.

[0060] As a NoSQL database, HBase does not support SQL statements.

[0061] like Figure 1 and 4 As shown, the present invention proposes a file label design method based on Doris, and the steps of the method include:

[0062] In the specific implementation process, Doris adopts a distributed architecture design to store data in multiple nodes to achieve parallel processing and analysis of data. This design enables Doris to process massive amounts of data and has high concurrency and high throughput capabilities.

[0063] The Doris architecture mainly includes two types of processes: Frontend (FE) and Backend (BE).

[0064] Frontend (FE): Mainly responsible for user request access, query parsing planning, metadata storage, node management, etc. FE is divided into three roles: Leader, Follower, and Observer. Leader and Follower form a Paxos selection group to ensure high availability of metadata and data writing. Observer is used to expand query nodes, synchronize Leader metadata for backup, and improve the query capability of the cluster.

[0065] Backend (BE): Mainly responsible for data storage and query plan execution. BE stores Doristable table data in a distributed manner. Table data is partitioned and bucketed to form tablets. Tablets are stored in a columnar manner and have three copies by default. BE receives FE commands to create, query, and delete tables, and receives execution plans from FE and executes them in a distributed manner.

[0066] Doris uses column storage to store a column of data together, which can significantly improve query performance and compression ratio. By compressing the storage column and creating zone map information for the column, Doris can filter by zone map when querying without reading file data, thereby greatly improving query efficiency.

[0067] Doris supports standard SQL syntax, including aggregation, association, sorting, window functions, custom functions and other functions. Users can use standard SQL to perform flexible analysis and calculations on data. The query processing flow of Doris is as follows:

[0068] 1. FE receives the query request from the client and generates an execution plan.

[0069] 2. FE distributes the execution plan to BEs that need the execution plan.

[0070] 3.BE executes the corresponding execution plans respectively.

[0071] 4. BE aggregates the query results to the BE node specified by FE.

[0072] 5. Specify BE to return the summary results to FE.

[0073] 6. FE returns the result to the client.

[0074] Doris supports multiple indexing technologies, such as BitMap indexing and BloomFilter indexing. Users can use different indexing technologies to improve query efficiency according to specific business scenarios. At the same time, Doris can also greatly improve query efficiency through a distributed vectorized execution engine.

[0075] Doris is highly available and fault-tolerant, with multiple copies of metadata and data stored, and all modules in the cluster can run multiple instances simultaneously, without a single point of failure. This design ensures the high availability of the entire Doris service, and any node downtime, offline or abnormality will not affect the overall stability of the Doris cluster service.

[0076] like Figure 2 As shown in the figure, the overall structure of this solution is divided into five main parts from top to bottom: "business layer", "interface layer", "label layer", "application layer" and "basic layer". The "label layer" depends on the "application layer" and "basic layer". Through the encapsulation of the "interface layer", it provides the "business layer" with the functions of querying, storing and changing the label parameters.

[0077] The newly added "label layer" provides several key functions: "File source", "File business identification", "File business type", "File attribution", "File custom parameters" and "File label modification".

[0078] Step S100, obtaining the input file to be marked and the file information of the file;

[0079] Step S200, filtering file source information, file service information and file ownership information from the file information;

[0080] In the specific implementation process, the file source information may be the IP address of the file upload or the device number of the file upload, etc.; the file ownership information may be the distributed node location where the file is stored in this system.

[0081] Step S300, constructing a file tag based on the file information, wherein the file tag includes a file source tag corresponding to the file source information and a file service tag corresponding to the file service information;

[0082] Step S400, obtaining a user's service request, and filtering a corresponding tag combination based on the user's service request, wherein the tag combination includes at least one file tag;

[0083] Step S500: construct a file set from the files having the file tags in the tag combination, obtain the attribution addresses of the files in the file set, and perform screening in each attribution address.

[0084] By adopting the above scheme, this scheme constructs a file tag of the file through the file information of the file while entering the file. This scheme constructs a tag layer based on the file tags of each file. The tag layer is arranged between the interface layer and the application layer. When performing data screening, a tag combination can be constructed through the user's business request to obtain a corresponding file set, which can preliminarily narrow the search scope of the file and improve processing efficiency.

[0085] In some implementations of the present invention, the step of constructing a file tag based on the file information further includes:

[0086] Construct a file number for each file;

[0087] The files having the file tag are constructed into a tag file set, and the file numbers of the corresponding files are stored in the tag file set.

[0088] In some embodiments of the present invention, in the step of constructing a file collection from files having file tags in the tag combination:

[0089] Get the tag file set corresponding to each tag in the tag combination;

[0090] The intersection of the file numbers in each tag file set is calculated to obtain the file set.

[0091] By adopting the above scheme, this scheme numbers the files and constructs a label file set for each file label. In the process of actually determining the file set, after obtaining the target label combination, the file set can be efficiently determined by constructing the intersection of the label file set in the label combination, thereby improving processing efficiency and processing accuracy.

[0092] In some embodiments of the present invention, in the step of obtaining the attribution address of the file in the file collection and screening each attribution address, the file attribution information corresponding to each file number in the file collection is extracted, and the attribution address of the file is determined based on the file attribution information.

[0093] like Figure 3 As shown, in some embodiments of the present invention, the file source information includes an uploading IP and an uploading user. In the step of constructing a file tag based on the file information, the uploading IP tag in the file source tag is constructed based on the uploading IP, and the uploading user tag in the file source tag is constructed based on the uploading user.

[0094] In some embodiments of the present invention, the file business information includes file scene information and time information. In the step of constructing a file tag based on the file information, the file scene tag in the file business tag is constructed based on the file scene information, and the file time tag in the file business tag is constructed based on the time information.

[0095] In some implementations of the present invention, in the step of screening the corresponding tag combination based on the user's service request:

[0096] Performing word segmentation processing on the text of the user's service request;

[0097] Encode each word segment and calculate the similarity between the business request and each file label based on the obtained encoding;

[0098] The file tags are filtered based on the similarity to obtain a tag combination corresponding to the user's business request.

[0099] By adopting the above scheme, this scheme analyzes the user's business request, determines the degree of correlation between the business request and each file tag, and then determines the relevant file tags to obtain a tag combination. The tag combination represents the target range of the user's business request, which can quickly narrow the target file range and ensure processing efficiency.

[0100] In some embodiments of the present invention, in the step of encoding each word segment and calculating the similarity between the service request and each file tag based on the obtained encoding:

[0101] Combine multiple participles and encode each participle combination;

[0102] Compute the similarity between each encoding and the encoding corresponding to each file tag.

[0103] In some embodiments of the present invention, in the step of encoding each word segmentation combination, a recurrent neural network encoder, a long short-term memory network encoder or a word embedding encoder is used.

[0104] The recurrent neural network (RNN) encoder is one of the earliest encoders used for NLP tasks. It is able to process sequence data and capture the temporal dependencies in the sequence. The RNN encoder passes each element (usually a word or character) in the input sequence to the network in turn and updates its internal state. Ultimately, the output of the RNN encoder can be regarded as a fixed-length representation of the input sequence;

[0105] The Long Short-Term Memory Network (LSTM) encoder is a variant of RNN. It solves the gradient vanishing and gradient exploding problems during long sequence training by introducing a gating mechanism (input gate, forget gate, and output gate). The LSTM encoder can more effectively capture long-term dependencies in a sequence, and therefore performs well in many NLP tasks.

[0106] In natural language processing, word embedding is a technique for mapping vocabulary to a high-dimensional vector space. Word embedding encoders convert words or phrases into fixed-length vector representations that capture the semantic relationship between words. Common word embedding methods include Word2Vec, GloVe, and BERT. These word embedding encoders play an important role in tasks such as text classification, sentiment analysis, and named entity recognition. .

[0107] By adopting the above scheme, this scheme segments the business request and combines the segmented words, which can ensure that the information in the business request is fully released, ensure the final matching range of the file label, and prevent the corresponding files from being missed; and although the combination of this scheme will lead to the generation of some meaningless phrases, the meaningless phrases often have a low similarity with each file label and will not affect the processing accuracy.

[0108] In some implementations of the present invention, the method further includes modifying the file tag of the file based on a preset file tag modification channel.

[0109] In the specific implementation process, Apache Doris, as a high-performance analytical database, can be easily integrated into such a system, and through the RestfulApi and gateway provided by the interface layer, it can achieve seamless connection with other modules. In this way, the file tagging function can be efficiently integrated into the entire file management system to improve overall efficiency.

[0110] Apache Doris, with its powerful data processing and analysis capabilities, can efficiently manage and query tags. Whether it is the source of the file, business identification, business type or custom parameters, etc., they can all be stored as tags in Doris, and through efficient indexing and query mechanisms, fast and accurate tag retrieval and modification can be achieved.

[0111] Apache Doris adopts a distributed architecture and supports horizontal expansion, which can easily cope with the processing needs of large-scale data sets. In the scenario of file tagging, you may need to process massive amounts of file data, and Doris's high performance and scalability can ensure that the system can still run stably under high concurrency and large data volumes. In addition, Doris also supports online table structure changes and data updates, and can update file tags in real time to ensure the timeliness and accuracy of tag information.

[0112] Apache Doris is not only a high-performance database, but also has powerful data analysis and visualization capabilities. By connecting with other BI tools and data sources, users can easily conduct in-depth analysis and visualization of file tags. This helps users better understand file data, discover potential business rules, and provide support for decision-making.

[0113] Apache Doris has the characteristics of high availability and fault tolerance. Its data and metadata are stored in multiple copies to ensure high availability and fault tolerance of data. Even if a node fails, Doris can ensure service continuity through automatic failover mechanism and ensure the stable operation of file tagging function.

[0114] In summary, this solution solves the problem of large amounts of data in the file center, including real-time writing, real-time updating, and real-time response to data queries. Secondly, this solution supports customer-oriented self-service queries, such as multi-dimensional analysis, large table connection queries, primary key indexes, summarization, and drilling. Then, in terms of batch processing, it has high throughput in data writing.

[0115] An embodiment of the present invention also provides a Doris-based file label design system, which includes a computer device, wherein the computer device includes a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps implemented by the method described above.

[0116] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps implemented by the above-mentioned file label design method based on Doris are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0117] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0118] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0119] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or replace features of other embodiments.

[0120] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A file label design method based on Doris, characterized in that: The steps of the method include: Obtain the entered files to be marked and the file information of the files; Filtering file source information, file business information and file ownership information from the file information; Building a file tag based on the file information, the file tag including a file source tag corresponding to the file source information and a file service tag corresponding to the file service information; Obtaining a business request from a user, and filtering a corresponding tag combination based on the business request from the user, wherein the tag combination includes at least one file tag; A file set is constructed from files having file tags in the tag combination, the attribution addresses of the files in the file set are obtained, and screening is performed in each attribution address.

2. The Doris-based file label design method according to claim 1, characterized in that: The step of constructing a file tag based on the file information also includes: Construct a file number for each file; The files having the file tag are constructed into a tag file set, and the file numbers of the corresponding files are stored in the tag file set.

3. The file label design method based on Doris according to claim 2, characterized in that: In the step of building a file collection from files with file tags in a tag combination: Get the tag file set corresponding to each tag in the tag combination; The intersection of the file numbers in each tag file set is calculated to obtain the file set.

4. The file label design method based on Doris according to claim 1, characterized in that: In the step of obtaining the attribution addresses of the files in the file set and screening the attribution addresses, the file attribution information corresponding to each file number in the file set is extracted, and the attribution address of the file is determined based on the file attribution information.

5. The Doris-based file label design method according to claim 1, characterized in that: The file source information includes the uploading IP and the uploading user. In the step of constructing a file tag based on the file information, the uploading IP tag in the file source tag is constructed based on the uploading IP, and the uploading user tag in the file source tag is constructed based on the uploading user.

6. The file label design method based on Doris according to claim 1, characterized in that: The file service information includes file scene information and time information. In the step of constructing a file tag based on the file information, a file scene tag in the file service tag is constructed based on the file scene information, and a file time tag in the file service tag is constructed based on the time information.

7. The Doris-based file label design method according to any one of claims 1 to 6, characterized in that: In the step of filtering the corresponding tag combination based on the user's business request: Performing word segmentation processing on the text of the user's service request; Encode each word segment and calculate the similarity between the business request and each file label based on the obtained encoding; The file tags are filtered based on the similarity to obtain a tag combination corresponding to the user's business request.

8. The Doris-based file label design method according to claim 7, characterized in that: In the step of encoding each word segment and calculating the similarity between the business request and each file tag based on the obtained encoding: Combine multiple participles and encode each participle combination; Compute the similarity between each encoding and the encoding corresponding to each file tag.

9. The Doris-based file label design method according to claim 1, characterized in that: The method further includes modifying the file tag of the file based on a preset file tag modification channel.

10. A file label design system based on Doris, characterized in that: The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions. The processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps implemented by the method as described in any one of claims 1 to 9.

Images