A method and system for fast file lookup

Abstract

The application discloses a kind of implementation method and system for quickly finding file, method includes: command line parameter analysis and configuration loading;Multi-thread parallel directory traversal, according to the CPU core number of current system Dynamic thread pool is created, and the top directory set under root search path is obtained;Matching rule precompilation and efficient matching;Result filtering, caching and sorting;Cross-platform adaptation processing;Result output, according to the differentiated color highlight output of file type to the final result set after sorting, and support pure text output mode.Through intelligent default configuration and simple parameter design, the problem of traditional command syntax cumbersome, parameter coupling degree high is solved, and learning and use cost is greatly reduced;Built-in glob and regular dual-mode matching can complete complex search in one station;Through generating native executable file, realize zero-cost cross-platform adaptation, solve the problem of poor compatibility of traditional tools across platforms.

Description

Technical Field

[0001] This invention belongs to the field of computer software technology and relates to a method and system for quickly finding files. Background Technology

[0002] Currently, mainstream file search technologies are centered around the `find` command in traditional Unix / Linux systems, while also including scripts that wrap this command and simple search functions built into various operating systems. Among these, the `find` command, as a native standard tool in Unix-like systems, is relatively comprehensive, supporting specifying search paths and filename matching via parameters (supporting simple wildcards). Filtering by attributes such as file type, size, and modification time can meet basic file retrieval needs.

[0003] However, the aforementioned existing technologies still have significant drawbacks in practical use: their syntax design is redundant and complex, with high coupling between options and parameters, making it easy for beginners to make mistakes due to issues such as parameter order, quotation mark usage, and wildcard escaping, resulting in a high learning and usage threshold; their traversal efficiency is low, employing a single-threaded synchronous traversal strategy, which leads to severe I / O blocking in large directories with more than 100,000 files, making it impossible to utilize the parallel capabilities of multi-core CPUs; their matching capabilities are limited, with no native support for regular expressions, requiring external tools such as grep for complex matching, making the operation cumbersome; and they lack intelligent default configuration, defaulting to traversing all hidden files, version control directories, package management directories, and other invalid paths. Users need to manually write complex exclusion rules, resulting in a high percentage of invalid traversals; cross-platform compatibility is poor, as the find command is exclusive to Unix-like systems, and Windows systems require third-party tools such as WSL and Cygwin to simulate its operation, and it does not mask differences in path separators, permission mechanisms, filename encodings, etc., between different operating systems; resource consumption is unreasonable, with no targeted I / O optimization and caching mechanisms, and when repeatedly searching the same directory, the entire directory is traversed repeatedly, resulting in high CPU and memory usage; the result display is simplistic, lacking file type color highlighting and flexible sorting and filtering functions, and the plain text path output has poor readability when dealing with massive amounts of results.

[0004] In summary, existing file search tools have significant shortcomings in terms of ease of use of syntax, traversal efficiency, matching capabilities, intelligence level, cross-platform compatibility, resource utilization efficiency, and result display. Therefore, there is an urgent need for a fast file search method that is syntactically concise, efficient in retrieval, intelligent in function, cross-platform compatible, resource-efficient, and provides user-friendly output to meet the file search needs of daily development and system management across different operating systems. Summary of the Invention

[0005] The purpose of this invention is to solve the problem that existing technologies cannot meet the file retrieval needs of daily development and system management under different operating systems, and to provide a method and system for quickly finding files.

[0006] To achieve the above objectives, the present invention employs the following technical solution: A method for quickly finding files includes the following steps: S1, command line parameter parsing and configuration loading, parses the command line parameters entered by the user, generates a standardized parameter object containing search keywords, root search path, case sensitivity configuration, hidden file filtering configuration, and directory exclusion configuration, loads the preset default configuration, and merges the user-defined parameters with the default configuration to generate the final search configuration. S2, multi-threaded parallel directory traversal: A thread pool is dynamically created based on the current number of CPU cores in the system. The top-level directory set under the root search path is obtained, and each entry in the top-level directory set is assigned to a different thread. Each thread independently and recursively traverses the directory it is responsible for. During the traversal, invalid paths are skipped according to the final search configuration, and file / directory metadata is collected to generate a file metadata set. Among them, a synchronous traversal method is used for directories with a number of files ≤ the first threshold, and a block traversal method is used for directories with a number of files > the first threshold, splitting the directory into multiple sub-blocks for parallel traversal. S3, Matching rule pre-compilation and efficient matching: Based on the user-input search keywords and case sensitivity configuration, the search keywords are pre-compiled into a reusable regular expression matcher. For each file metadata in the file metadata set, the regular expression matcher is called to perform a matching operation on the file name or full path, filter out the successfully matched file metadata, and generate a matching result set. S4, Result Filtering, Caching, and Sorting: Receive the matching result set, perform secondary filtering based on user-specified file type, file size, or modification time filtering conditions to obtain the final result set; store the root search path, search keywords, and final result set in a memory cache, with the root search path and search keywords as keys and the final result set and storage timestamp as values, and set an expiration period and capacity limit; sort the final result set according to user-specified sorting rules. S5, cross-platform adaptation processing, during traversal and output, uniformly converts file paths to forward slash delimiters, uniformly converts file names to UTF-8 encoded strings, and generates native executable files for the corresponding operating system through conditional compilation; S6, Result Output: The sorted final result set is output with differentiated color highlighting based on file type, and plain text output mode is also supported.

[0007] The block traversal in S2 specifically includes: obtaining a list of all entries in the target directory, determining the block size based on the ratio of the number of entries to the number of threads, allocating the list of entries to different blocks according to the file name hash value modulo the number of blocks, and recursively traversing each block by an independent thread.

[0008] The process of precompiling the regular expression matcher in S3 includes: determining whether the search keyword contains wildcards; if it contains wildcards, it is determined to be in glob mode, and the glob_to_regex function is called to convert the glob mode into a regular expression; if it does not contain wildcards, it is determined to be in ordinary string mode, and the string is directly escaped using regular expressions; the regular expression matcher is based on the Regex::builder method of the regex library, and whether to enable the case-insensitive flag according to the case sensitivity configuration, and completes the precompiling.

[0009] Skipping invalid paths in S2 includes: skipping the recursive traversal of a directory whose name belongs to the exclusion list when the directory is traversed according to the default configuration or a user-defined exclusion directory list; skipping files or directories whose filenames begin with "." when the hidden file filtering configuration is set to filter hidden files and the user has not specified to include hidden files; determining whether a file is hidden by reading its file attribute flags under the Windows operating system; and silently skipping files or directories that do not have access permissions when traversing and optionally recording a warning log.

[0010] The memory cache in S4 adopts an LRU eviction policy. The capacity limit is calculated based on the memory size after the cache result set is serialized. When the total cache size exceeds the capacity limit, the least used cache entry is evicted. The validity period is a configurable time threshold. When reading the cache, the difference between the stored timestamp and the current time is checked. If it exceeds the time threshold, the cache is determined to be invalid and the entry is deleted. The cache key also includes at least one parameter from case sensitivity configuration, matching range configuration, file type filtering configuration, and sorting rule configuration.

[0011] The cross-platform adaptation processing in S5 also includes a symbolic link processing strategy: by default, symbolic links are not followed. When the user specifies to follow symbolic links through command-line parameters, the directory pointed to by the symbolic link is recursively traversed during the traversal process, and a circular link detection mechanism is adopted to record the visited inodes or paths. When a circular link is detected, the recursion stops.

[0012] The formula for calculating the number of threads in the dynamically created thread pool in S2 is: number of threads = number of CPU cores × 2, and the minimum number of threads is not less than 2 and the maximum number is not more than 32; the thread pool is created based on the scope function of the crossbeam library to ensure that all threads complete traversal and summarize metadata before the scope ends.

[0013] The default configurations preset in S1 include: the default excluded directories are at least one or more of .git, .svn, node_modules, and target; the default setting for filtering hidden files is true; the default setting for case-insensitive matching is true; the default root search path is the current directory; and the command-line parameter parsing is based on the derive feature of the clap library to implement parameter structure definition and automatic validation.

[0014] The differentiated color highlighting in S6 specifically includes: directories are output in blue, executable files in green, compressed files in red, and ordinary files in white. Color highlighting is implemented based on the colored library and detects whether the current output terminal supports color. If it does not support color or the user specifies the --no-color parameter, it will automatically degrade to plain text output.

[0015] A system for quickly finding files, used to execute the method described in any of the preceding items, wherein the system adopts a modular layered architecture, comprising: The command-line parsing layer is used to parse the command-line arguments entered by the user and generate standardized parameter objects; The configuration layer is used to load the default configuration and merge it with user parameters to generate the final lookup configuration; The file traversal layer is used to implement multi-threaded parallel directory traversal, employing directory-level parallelism and adaptive IO optimization, skipping invalid paths and collecting file metadata. The matching engine layer is used to precompile matching rules and perform batch matching filtering; The results processing layer is used for secondary filtering, memory caching, and sorting of the matching results; The output layer is used to output the final result with color highlighting or plain text output; The layers communicate with each other through a standardized data structure, which includes at least a parameter object, a file metadata object, and a result set object.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The fast file search method in this invention solves the problems of cumbersome syntax and high parameter coupling of the traditional find command by using intelligent default configuration and concise parameter design, greatly reducing the learning and usage costs. It has built-in glob and regular expression dual-mode matching, supporting path matching, filename matching, and case-sensitive / insensitive matching, and can complete complex searches in one stop without the need to cooperate with external tools such as grep. By uniformly converting path separators to forward slashes, uniformly converting filenames to UTF-8 encoding, and conditional compilation, it generates native executable files for Windows / Linux / macOS, without the need for third-party tools such as WSL and Cygwin, achieving zero-cost cross-platform adaptation and solving the problem of poor cross-platform compatibility of traditional tools.

[0017] This invention utilizes multi-threaded parallel directory traversal and adaptive I / O strategies, combined with pre-compiled regular expression matching in step S3, to achieve a 3-5 times faster search speed compared to the `find` command in large directories with over 100,000 files, resolving single-threaded I / O blocking. Dynamic thread scheduling and adaptive strategies avoid excessive context switching, and lightweight memory caching reduces redundant traversal and matching. Actual measurements show a 20%-40% reduction in CPU usage and a more than 30% reduction in memory usage, with no resource contention issues. For secondary searches using the same root path and keywords, the caching mechanism improves speed by over 80%, making it particularly suitable for frequent search needs during development and solving the problems of redundant traversal and unreasonable resource consumption in traditional tools.

[0018] This invention significantly improves the readability of results by using file type-differentiated color highlighting and multi-dimensional sorting. It also supports --no-color plain text output and pipelined output modes, adapting to various scenarios such as interactive terminals, background scripts, and tool linkage, thus solving the problem of the single result display in traditional tools. In addition, this invention adopts a modular layered architecture, with each layer communicating through standardized data structures, completely decoupled, allowing for flexible expansion of functions according to needs. It has low maintenance and upgrade costs and has good practical value and promotion prospects. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the method for quickly finding files according to the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0022] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0023] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0024] The present invention will now be described in further detail with reference to the accompanying drawings: See Figure 1 The flowchart below shows a method for quickly finding files according to the present invention, which includes the following steps: S1, Command Line Argument Parsing and Configuration Loading: Parses the user-input command line arguments, generates a standardized parameter object containing search keywords, root search path, case sensitivity configuration, hidden file filtering configuration, and excluded directory configuration, and loads the preset default configuration. The user-defined parameters and default configuration are merged to generate the final search configuration. The preset default configuration includes: default excluded directories are at least one or more of .git, .svn, node_modules, and target; default hidden file filtering is true; default case insensitivity matching is true; default root search path is the current directory. The command line argument parsing utilizes the derive feature of the clap library to define and automatically validate the parameter structure.

[0025] S2, Multi-threaded parallel directory traversal: A thread pool is dynamically created based on the current number of CPU cores in the system. The top-level directory set under the root search path is obtained, and each entry in the top-level directory set is assigned to a different thread. Each thread independently recursively traverses the directory it is responsible for. During the traversal, invalid paths are skipped according to the final search configuration, and file / directory metadata is collected to generate a file metadata set. For directories with a file count ≤ a first threshold, a synchronous traversal method is used; for directories with a file count > the first threshold, a block traversal method is used, splitting the directory into multiple sub-blocks for parallel traversal. The block traversal specifically includes: obtaining a list of all entries under the target directory; determining the block size based on the ratio of the number of entries to the number of threads; allocating the entry list to different blocks according to the filename hash value modulo the block number; and recursively traversing each block by an independent thread. Skipping invalid paths includes: skipping recursive traversal of directories directly when the directory name belongs to the excluded directory list, based on default configuration or a user-defined exclusion directory list; skipping files or directories whose filenames begin with "." when the hidden file filtering configuration is set and the user has not specified including hidden files; determining whether a file is hidden by reading file attribute flags under Windows operating systems; and silently skipping files or directories with inaccessible permissions during traversal, optionally logging warnings. The formula for calculating the number of threads in the dynamically created thread pool is: number of threads = number of CPU cores × 2, with a minimum of 2 and a maximum of 32 threads; the thread pool is created based on the `scope` function of the `crossbeam` library, ensuring that all threads complete traversal and aggregate metadata before the scope ends.

[0026] S3, Matching rule pre-compilation and efficient matching: Based on the user-input search keywords and case-sensitive configuration, the search keywords are pre-compiled into a reusable regular expression matcher. For each file metadata in the file metadata set, the regular expression matcher is called to perform a filename or full path matching operation, filtering out successfully matched file metadata and generating a matching result set. The process of pre-compiling the regular expression matcher includes: determining whether the search keyword contains wildcards; if it contains wildcards, it is determined to be in glob mode, and the glob_to_regex function is called to convert the glob mode into a regular expression; if it does not contain wildcards, it is determined to be in ordinary string mode, and the string is directly escaped using regular expressions. The regular expression matcher is based on the Regex::builder method of the regex library, and the case-insensitive flag is set to enable or disable according to the case-sensitive configuration to complete the pre-compilation.

[0027] S4, Result Filtering, Caching, and Sorting: The system receives the matching result set and performs secondary filtering based on user-specified file type, file size, or modification time filtering conditions to obtain the final result set. The root search path, search keywords, and final result set are stored in a memory cache. The memory cache uses the root search path and search keywords as keys and the final result set and storage timestamp as values, and sets an expiration period and capacity limit. The final result set is sorted according to user-specified sorting rules. The memory cache employs an LRU eviction policy. The capacity limit is calculated based on the memory size after serializing the cached result set. When the total cache size exceeds the capacity limit, the least recently used cache entry is evicted. The expiration period is a configurable time threshold. When reading the cache, the difference between the storage timestamp and the current time is checked. If the difference exceeds the time threshold, the cache is deemed invalid and the entry is deleted. The cache key also includes at least one parameter from case sensitivity configuration, matching range configuration, file type filtering configuration, and sorting rule configuration.

[0028] S5, a cross-platform adaptation tool, converts file paths to forward slash separators and filenames to UTF-8 encoded strings during traversal and output. It also generates native executables for the corresponding operating system through conditional compilation. The cross-platform adaptation tool also includes a symbolic link handling strategy: by default, symbolic links are not followed. When the user specifies to follow symbolic links via command-line parameters, the directories pointed to by the symbolic links are recursively traversed during the traversal process. A circular link detection mechanism is used to record the visited inodes or paths. When a circular link is detected, the recursion stops.

[0029] S6, Result Output: The sorted final result set is output with differentiated color highlighting based on file type, and plain text output mode is also supported. Specifically, directory names are output in blue, executable files in green, compressed files in red, and regular files in white. Color highlighting is implemented using the `colored` library and checks if the current output terminal supports color; if not, or if the user specifies the `--no-color` parameter, it automatically downgrades to plain text output.

[0030] One embodiment of this invention is a system for quickly finding files. The system adopts a modular layered architecture and includes: The command-line parsing layer is used to parse the command-line arguments entered by the user and generate standardized parameter objects; The configuration layer is used to load the default configuration and merge it with user parameters to generate the final lookup configuration; The file traversal layer is used to implement multi-threaded parallel directory traversal, employing directory-level parallelism and adaptive IO optimization, skipping invalid paths and collecting file metadata. The matching engine layer is used to precompile matching rules and perform batch matching filtering; The results processing layer is used for secondary filtering, memory caching, and sorting of the matching results; The output layer is used to output the final result with color highlighting or plain text output; The layers communicate with each other through a standardized data structure, which includes at least a parameter object, a file metadata object, and a result set object.

[0031] This invention discloses a fast file search method implemented in Rust. Its core principles include multi-threaded parallelism to eliminate I / O blocking, pre-compiled matching rules to reduce redundant computation, intelligent default configuration to filter invalid traversals, and cross-platform abstraction to shield system differences. Combined with lightweight caching and modular architecture design, it constructs an efficient, easy-to-use, and compatible file search system. Leveraging Rust's memory safety, zero-cost abstraction, and high concurrency, as well as mature open-source libraries such as walkdir, regex, and clap, this invention achieves end-to-end optimization of the "parsing-traversal-matching-processing-output" process, completely solving the problems of low efficiency, cumbersome syntax, and poor cross-platform compatibility in traditional file search tools.

[0032] The core principle of this invention for fast file searching addresses four major pain points of traditional file searching: "single-threaded traversal I / O blocking, repeated compilation of matching rules, full traversal of invalid paths, and complex adaptation to system differences." It employs four core design principles to create a synergistic optimization effect: Multi-threaded parallel I / O principle: The core bottleneck of file search is the blocking nature of disk I / O and directory traversal. This invention adopts a "directory-level parallel" strategy, which allocates the top-level directory to different threads and uses the advantages of CPU multi-core to distribute I / O blocking time. At the same time, the number of threads is dynamically adjusted according to the number of CPU cores to avoid the context switching overhead caused by too many threads, thereby maximizing traversal efficiency.

[0033] The principle of pre-compiled matching rules: The search keywords (glob pattern / ordinary string / regular expression) input by the user are pre-compiled only once to generate a reusable matcher. All subsequent file / directory matching directly calls the pre-compiled result, avoiding the invalid calculation of repeated parsing and compilation of matching rules, and greatly improving matching efficiency.

[0034] Intelligent filtering principle: Based on high-frequency scenarios in daily development and system management, it presets filtering rules for invalid paths (version control directory, package management directory, hidden files), and skips the traversal and matching of such invalid paths by default, reducing invalid IO and calculation operations and shortening search time; at the same time, it supports user-defined filtering rules, taking into account both versatility and flexibility.

[0035] Cross-platform abstraction principle: Based on the std::fs standard library and walkdir cross-platform traversal library of the Rust language, it abstracts and encapsulates the file systems (path separators, permission mechanisms, filename encoding) of different operating systems, provides a unified interface and processing logic, and enables a single code library to run natively across Windows / Linux / macOS without the need for third-party simulation tools.

[0036] The four core principles work together to cover the entire file search process. From input parsing to result output, each step incorporates targeted optimizations, forming a complete technical system of "efficient traversal + fast matching + intelligent filtering + cross-platform compatibility".

[0037] This invention adopts a modular layered architecture, with each layer having a single responsibility, strong decoupling, and flexible expansion capabilities, while ensuring efficient linkage between layers. The overall architecture is divided into 6 core layers, forming a complete search chain from top to bottom. The core functions, dependencies, and application principles of each layer are as follows: Table 1. Architecture Layering Details

[0038] Architecture linkage process The interconnected workflow of each layer is as follows: User inputs command-line parameters → Command-line parsing layer parses parameters → Configuration layer loads and merges configurations → File traversal layer performs multi-threaded parallel traversal, generating file metadata → Matching engine layer pre-compiles matching rules and performs efficient matching → Result processing layer filters, caches, and sorts results → Output layer beautifies the output results. The entire process operates in a closed loop, with each layer communicating through standardized data structures (parameter objects, file metadata, and result sets) to ensure decoupling and scalability.

[0039] Specific implementation steps Based on the above core principles and overall architecture, the file search method of this invention follows the core steps of "parsing-traversal-matching-processing-output". Each step includes specific operation logic, parameter settings, and principle application. The steps are closely connected and can be directly implemented. The specific steps are as follows: Step 1: Command line argument parsing and configuration loading (corresponding to command line parsing layer + configuration layer) The core purpose of this step is to convert the user's intuitive command-based input into a standardized configuration that the system can recognize, while loading the default optimized configuration to guide the subsequent search process. The specific steps are as follows: Parameter definition and reception: Based on the derive feature of the clap library, a command-line parameter structure (Cli) is defined, which includes core parameters and optional parameters. The core parameters are "search keyword" (supports glob mode, ordinary string, and regular expression) and "root search path" (default is the current directory "."). Optional parameters include case-sensitive matching (-s / --case-sensitive), including hidden files (-H / --hidden), matching full paths (-p / --path-match), matching only ordinary files (-f / --file), matching only directories (-d / --dir), sorting rules (--sort), reversing order (--reverse), and disabling color highlighting (--no-color).

[0040] Parameter parsing and validation: The CLI::parse() method parses the command-line arguments entered by the user, automatically performs parameter validation (such as path validity validation), and generates a standardized CLI parameter object; if the user enters invalid parameters (such as a non-existent path), it automatically outputs help documentation and exits, improving usability.

[0041] Default configuration loading: The lazy_static library is used to implement lazy loading of global default configurations. The default configurations include: default excluded directories (.git, .svn, node_modules, target, which are invalid directories that do not need to be searched frequently in daily development), default filtering of hidden files (true), and default case-insensitive matching (true). The lazy loading mechanism can reduce the resource consumption when the program starts up and loads the configuration only when it is used for the first time.

[0042] Configuration merging: Merges user-defined parameters with the default configuration to generate the final search configuration, following the rule that "user parameters have higher priority than default configuration". If the user specifies -s (case sensitive), the default case-insensitive configuration will be overridden; if the user does not specify a search path, the default current directory will be used.

[0043] The principle behind this step is to solve the problem of cumbersome syntax in the traditional find command by using concise parameter definitions and automatic validation; and to reduce subsequent invalid traversal operations by using default configurations for frequently invalid paths and applying intelligent filtering principles.

[0044] Step 2: Multi-threaded parallel directory traversal (corresponding to file traversal layer) This step is crucial for improving search efficiency. Its core purpose is to use multi-threaded parallel traversal to distribute I / O blocking time, skip invalid paths, and quickly obtain file / directory metadata. The specific operations are as follows: Thread pool initialization: The thread pool is created based on the scope function of the crossbeam library. The number of threads is dynamically adjusted according to the current number of CPU cores in the system. The default number of threads is "CPU cores × 2". The reason for this setting is that file traversal is an I / O intensive operation. Appropriately increasing the number of threads can make full use of the advantages of the multi-core CPU, distribute I / O blocking time, and avoid the context switching overhead caused by too many threads.

[0045] Top-level directory retrieval: Starting from the root path specified in the configuration layer, the WalkDir::new(root) method of the walkdir library is used to traverse the top-level directory of the root path (max_depth=1), retrieve all entries of the top-level subdirectories / files, filter invalid entries (such as entries without access permissions), and collect them into a set of top-level entries.

[0046] Directory-level parallel traversal: Each entry in the top-level entry set (skipping the root path itself) is assigned to a different thread, and each thread independently and recursively traverses the entries (subdirectories / files) it is responsible for; during the traversal, the following filtering operations are performed, applying the principle of intelligent filtering: Hidden file filtering: If the default configuration is to filter hidden files (and the user does not specify -H), then files / directories whose filenames begin with "." (a cross-platform identifier for hidden files); Excluded directory filtering: If the directory being traversed belongs to the default excluded directory (.git, etc.) or the user-defined excluded directory, the recursive traversal of the directory is skipped directly to reduce invalid I / O; Permission filtering: Skip files / directories without access permissions to avoid program crashes caused by traversal errors.

[0047] IO optimization strategy: Adaptive IO optimization is adopted during traversal, and the traversal method is dynamically adjusted according to the directory size: For small directories with ≤1000 files, synchronous traversal is adopted to reduce thread scheduling overhead; for large directories with >1000 files, block traversal is adopted, splitting the directory into multiple sub-blocks for parallel traversal, further reducing the blocking time of a single IO operation.

[0048] Metadata collection: During the traversal of each thread, for each valid file / directory, its metadata (path, name, type, size, modification time) is obtained, encapsulated in the FileMeta structure (with a unified data format), and collected into the local thread's metadata set; after the traversal is completed, the local metadata sets of all threads are summarized to generate a global file metadata set, which is then passed to the matching engine layer.

[0049] The principles applied in this step are: multi-threaded parallel I / O principle (distributed I / O blocking), cross-platform abstraction principle (walkdir library shields system path differences), and intelligent filtering principle (skipping invalid paths). The core solution is to address the problem of low efficiency in traditional single-threaded traversal.

[0050] Step 3: The efficient dual-mode matching engine performs the matching (corresponding to the matching engine layer). The core purpose of this step is to achieve fast matching between keywords and file metadata through pre-compiled matching rules, reducing redundant calculations. The specific operations are as follows: Matcher initialization: Create a Matcher structure (the core of the matching engine), receive user input of search keywords and case sensitivity configuration, and complete the pre-compilation of matching rules: Pattern detection: Determine the type of keywords entered by the user; if it contains... Wildcards such as , ?, and [] are used in glob mode; if the string is a plain string or a regular expression, it is used in regular expression mode. glob pattern conversion: Call the glob_to_regex utility function to convert the glob pattern into a regular expression (conversion rules: →. 、?→. 、Escape special characters, such as [→ [), to avoid users manually writing regular expressions, thus lowering the barrier to entry; Pre-compiled regular expressions: Based on the Regex::builder() method of the regex library, regular expressions are pre-compiled according to the case sensitivity configuration (user-specified -s enables case sensitivity, otherwise it is disabled), generating reusable Regex objects; pre-compilation is executed only once, and all subsequent matches directly call this object, reducing the invalid calculations of repeated compilation.

[0051] Batch matching execution: Receive the global file metadata set passed in step 2, iterate through each FileMeta object, and perform matching operations: Match range selection: Select the match range based on user configuration (-p / --path-match) - path matching (match the complete file path) or filename matching (default, only match filenames); Quick Match: Call the Matcher's is_match method, use the pre-compiled Regex object to perform a match between the keyword and the matching range (path / filename), and return the matching result (true / false). Result filtering: Filter out FileMeta objects with a matching result of true, form a matching result set, and pass it to the result processing layer; objects that fail to match are discarded directly to reduce subsequent processing overhead.

[0052] The principle behind this step is based on pre-compiled matching rules, which solves the problems of repetitive compilation and low matching efficiency in traditional matching rules. At the same time, it automatically converts to regular expressions through glob mode, solving the problem of traditional tools having limited matching capabilities and requiring additional tools.

[0053] Step 4: Result filtering, caching, and sorting (corresponding to the result processing layer) The core objective of this step is to optimize the quality of matching results and improve the efficiency of secondary searches. The specific steps are as follows: Secondary filtering: Receive the matching result set from step 3, and perform secondary filtering based on user-selectable parameters to further filter the results that meet the requirements. File type filtering: If the user specifies -f (match only regular files), FileMeta objects with is_file set to true are retained; if -d (match only directories) is specified, FileMeta objects with is_dir set to true are retained; if not specified, all types are retained. Extended filtering (reserved interface): Supports extended file size filtering (--size +1M / -100k) and modification time filtering (--mtime -1d / +7d). File size is obtained through meta.len() and modification time is obtained through meta.modified(). The filtering logic is completed in conjunction with the humansize and chrono libraries to adapt to more scenario requirements.

[0054] Lightweight memory cache: Implements memory caching of recently searched results, optimizing the efficiency of secondary searches. Cache design: The cache storage structure is "root path → search keyword → matching result set", the cache validity period is 5 minutes (to avoid inaccurate results due to cache expiration), and the cache capacity limit is 100MB (to avoid excessive memory consumption); Caching logic: After the current search is completed, the root path, search keywords, and final result set are stored in the cache. During the next search, it is first determined whether there is a valid cache with the same root path and keywords. If it exists, the cached result is returned directly without repeating the traversal and matching steps. If the cache expires or does not exist, the complete search process is executed.

[0055] Result sorting: Sort the filtered result set according to the user-configured sorting rules (--sort, default is sort by name), supporting ascending / descending order (--reverse): Sorting rules are adapted as follows: Sort by name (sort=name), using the name field of FileMeta as the sort key; Sort by size (sort=size), using the size field as the sort key; Sort by modification time (sort=mtime), using the mtime field as the sort key. Sorting implementation: The FileMeta sort_key method returns comparable key values ​​for the corresponding sorting rules. The sorting is then performed using the Rust standard library's sort_by method. For reverse sorting, the comparison results are reversed.

[0056] Result standardization: Convert the sorted result set into a standardized format to ensure that the output layer can directly call it, while retaining the complete information of file metadata (path, name, type, etc.) to support output beautification.

[0057] The principles applied in this step are: intelligent filtering (secondary filtering to optimize results) and pre-compiled matching rules (caching to reduce redundant calculations), which ultimately improves result quality and secondary search efficiency.

[0058] Step 5: Cross-platform adaptation (through all layers, with a focus on the traversal / output layer) The core purpose of this step is to mask the differences in file systems across different operating systems and achieve native cross-platform operation. The specific operations are carried out throughout steps 2-4, and the key points are as follows: Path separator adaptation: Windows systems use backslashes ( As a path separator, Linux / macOS uses forward slashes ( / ). This invention uses Rust's PathBuf structure and the to_string_lossy() method to uniformly convert all paths to forward slash separators, ensuring consistency in path parsing and avoiding search failures caused by differences in path separators.

[0059] Permission mechanism adaptation: Different operating systems have different file permission mechanisms (Linux / macOS uses rwx permissions, Windows uses ACL permissions). When retrieving file metadata through e.metadata().unwrap() during traversal, the permission verification of different systems is automatically adapted to avoid program crashes by skipping files / directories without read permissions. At the same time, the differences in permission details are hidden to provide a unified permission filtering interface for the upper layer.

[0060] Filename encoding adaptation: Windows systems use GBK encoding for filenames by default, while Linux / macOS uses UTF-8 encoding. This invention uses the to_string_lossy() method to convert filenames into UTF-8 encoded strings, avoiding filename garbled characters and matching failures caused by encoding differences, and ensuring cross-platform matching accuracy.

[0061] Executable file adaptation: Through Rust's conditional compilation features, corresponding native executable files (.exe for Windows, executable binary for Linux / macOS) are compiled for different operating systems (Windows / Linux / macOS), and can be run directly without the need for third-party emulation tools (such as WSL, Cygwin).

[0062] The principle behind this step is cross-platform abstraction, which solves the problem of poor cross-platform compatibility of the traditional find command and achieves multi-platform compatibility with a single codebase.

[0063] Step 6: Enhance output and interaction (corresponding to the output layer) The core purpose of this step is to improve the readability of the results and adapt them to different use cases. The specific steps are as follows: Color highlighting configuration: Based on the colored library, implement differentiated color highlighting for different file types to improve the recognizability of results. Table of Contents: Highlighted in blue; Executable files (.sh, .rs, .exe, etc.): highlighted in green; Compressed files (.zip, .rar, .tar, .gz, etc.): highlighted in red; Other regular files: highlighted in white.

[0064] Multi-mode output: Supports two output modes to adapt to different usage scenarios: Interactive output (default): Enables color highlighting and outputs file paths according to sorting rules, suitable for interactive use on user terminals; Plain text output: When the user specifies --no-color, color highlighting is turned off, and the plain text path is output. This is suitable for terminal-free scenarios such as pipe redirection and script calls (e.g., when used with the xargs tool to batch process search results).

[0065] Output results: Iterate through the standardized result set generated in step 4, call the print method of FileMeta, and output the path of each file (including color highlighting or plain text) according to the output mode; if the search result is empty, output the prompt message ("No matching file found") to improve the user experience.

[0066] The core purpose of this step is to improve the readability and scenario adaptability of the results, solving the problems of traditional tools having a single result display and poor readability.

[0067] The core difference between this invention and existing traditional file search technologies lies in its targeted optimization and intelligent design of the entire file search process. Specifically, this involves the following six core technical features, which work together to create an overall advantage: 1. Multi-threaded parallel traversal strategy based on dynamic adjustment of CPU core count: adopts directory-level parallel processing, combined with crossbeam to achieve safe multi-threaded scheduling, distributes the IO blocking of large directory traversal to multiple threads, solves the efficiency bottleneck of traditional single-threaded traversal, and dynamically adapts the number of threads to the hardware to avoid resource waste. 2. High-efficiency matching engine that integrates glob and regular expression modes: It realizes automatic conversion from glob mode to regular expression, and the pre-compiled regular expression is executed only once, reducing the overhead of repeated calculations. At the same time, it supports configurable case-sensitive / insensitive matching, balancing matching flexibility and efficiency, without the need for additional tools. 3. Intelligent default configuration system for practical use scenarios: It ignores version control directories, package management directories and hidden files by default, eliminating the need for users to manually write exclusion rules, greatly reducing invalid traversal and matching, simplifying command line input, and solving the problem of cumbersome syntax in traditional tools; 4. Cross-platform seamless implementation based on the Rust language: By using Rust's std::fs and walkdir libraries to shield the differences in file systems (path separators, permissions, encoding) between Windows / Linux / macOS, a single code library can be natively compiled and run across platforms without any third-party simulation dependencies, resulting in zero adaptation costs; 5. A resource management strategy that combines lightweight memory caching with IO optimization: The recently traversed directory structure and matching results are cached in memory with a limited duration and limited capacity to improve the efficiency of secondary queries. At the same time, an adaptive IO strategy of block traversal is adopted for large directories and synchronous traversal is adopted for small directories to reduce IO blocking time. 6. Modular syntax parsing and beautified output design: Based on clap, it implements concise command line parameter parsing, supports intuitive imperative syntax (such as fast-find txt to find all txt files), and implements file type differentiation color highlighting, multi-dimensional sorting, taking into account ease of use and result readability, and is compatible with traditional pipeline processing habits.

[0068] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for quickly finding files, characterized in that, Includes the following steps: S1, command line parameter parsing and configuration loading, parses the command line parameters entered by the user, generates a standardized parameter object containing search keywords, root search path, case sensitivity configuration, hidden file filtering configuration, and directory exclusion configuration, loads the preset default configuration, and merges the user-defined parameters with the default configuration to generate the final search configuration. S2, multi-threaded parallel directory traversal: A thread pool is dynamically created based on the current number of CPU cores in the system. The top-level directory set under the root search path is obtained, and each entry in the top-level directory set is assigned to a different thread. Each thread independently and recursively traverses the directory it is responsible for. During the traversal, invalid paths are skipped according to the final search configuration, and file / directory metadata is collected to generate a file metadata set. Among them, a synchronous traversal method is used for directories with a number of files ≤ the first threshold, and a block traversal method is used for directories with a number of files > the first threshold, splitting the directory into multiple sub-blocks for parallel traversal. S3, Matching rule pre-compilation and efficient matching: Based on the user-input search keywords and case sensitivity configuration, the search keywords are pre-compiled into a reusable regular expression matcher. For each file metadata in the file metadata set, the regular expression matcher is called to perform a matching operation on the file name or full path, filter out the successfully matched file metadata, and generate a matching result set. S4, Result Filtering, Caching, and Sorting: Receive the matching result set, perform secondary filtering based on user-specified file type, file size, or modification time filtering conditions to obtain the final result set; store the root search path, search keywords, and final result set in a memory cache, with the root search path and search keywords as keys and the final result set and storage timestamp as values, and set an expiration period and capacity limit; sort the final result set according to user-specified sorting rules. S5, cross-platform adaptation processing, during traversal and output, uniformly converts file paths to forward slash delimiters, uniformly converts file names to UTF-8 encoded strings, and generates native executable files for the corresponding operating system through conditional compilation; S6, Result Output: The sorted final result set is output with differentiated color highlighting based on file type, and plain text output mode is also supported.

2. The method for quickly finding files as described in claim 1, characterized in that, The block traversal in S2 specifically includes: obtaining a list of all entries in the target directory, determining the block size based on the ratio of the number of entries to the number of threads, allocating the list of entries to different blocks according to the file name hash value modulo the number of blocks, and recursively traversing each block by an independent thread.

3. The method for quickly finding files as described in claim 1, characterized in that, The process of precompiling the regular expression matcher in S3 includes: determining whether the search keyword contains wildcards; if it contains wildcards, it is determined to be in glob mode, and the glob_to_regex function is called to convert the glob mode into a regular expression; if it does not contain wildcards, it is determined to be in ordinary string mode, and the string is directly escaped using regular expressions; the regular expression matcher is based on the Regex::builder method of the regex library, and whether to enable the case-insensitive flag according to the case sensitivity configuration, and completes the precompiling.

4. The method for quickly finding files as described in claim 1, characterized in that, Skipping invalid paths in S2 includes: skipping the recursive traversal of a directory whose name belongs to the exclusion list when traversed, based on the default configuration or a user-defined exclusion directory list; skipping files or directories whose filenames begin with "." when the hidden file filtering configuration is set to filter hidden files and the user has not specified to include hidden files; determining whether a file is hidden by reading its file attribute flags under the Windows operating system; and silently skipping files or directories with inaccessible access permissions during traversal, with optional warning logs recorded.

5. The method for quickly finding files as described in claim 1, characterized in that, The memory cache in S4 adopts an LRU eviction policy. The capacity limit is calculated based on the memory size after the cache result set is serialized. When the total cache size exceeds the capacity limit, the least used cache entry is evicted. The validity period is a configurable time threshold. When reading the cache, the difference between the stored timestamp and the current time is checked. If it exceeds the time threshold, the cache is determined to be invalid and the entry is deleted. The cache key also includes at least one parameter from case sensitivity configuration, matching range configuration, file type filtering configuration, and sorting rule configuration.

6. The method for quickly finding files as described in claim 1, characterized in that, The cross-platform adaptation processing in S5 also includes a symbolic link processing strategy: by default, symbolic links are not followed. When the user specifies to follow symbolic links through command-line parameters, the directory pointed to by the symbolic link is recursively traversed during the traversal process, and a circular link detection mechanism is adopted to record the visited inodes or paths. When a circular link is detected, the recursion stops.

7. The method for quickly finding files as described in claim 1, characterized in that, The formula for calculating the number of threads in the dynamically created thread pool in S2 is: number of threads = number of CPU cores × 2, and the minimum number of threads is not less than 2 and the maximum number is not more than 32; the thread pool is created based on the scope function of the crossbeam library to ensure that all threads complete traversal and summarize metadata before the scope ends.

8. The method for quickly finding files as described in claim 1, characterized in that, The default configurations preset in S1 include: the default excluded directories are at least one or more of .git, .svn, node_modules, and target; the default setting for filtering hidden files is true; the default setting for case-insensitive matching is true; the default root search path is the current directory; and the command-line parameter parsing is based on the derive feature of the clap library to implement parameter structure definition and automatic validation.

9. The method for quickly finding files as described in claim 1, characterized in that, The differentiated color highlighting in S6 specifically includes: directories are output in blue, executable files in green, compressed files in red, and ordinary files in white. Color highlighting is implemented based on the colored library and detects whether the current output terminal supports color. If it does not support color or the user specifies the --no-color parameter, it will automatically degrade to plain text output.

10. A system for quickly finding files, characterized in that, For performing the method of any one of claims 1 to 9, the system employs a modular layered architecture, comprising: The command-line parsing layer is used to parse the command-line arguments entered by the user and generate standardized parameter objects; The configuration layer is used to load the default configuration and merge it with user parameters to generate the final lookup configuration; The file traversal layer is used to implement multi-threaded parallel directory traversal, employing directory-level parallelism and adaptive IO optimization, skipping invalid paths and collecting file metadata. The matching engine layer is used to precompile matching rules and perform batch matching filtering; The results processing layer is used for secondary filtering, memory caching, and sorting of the matching results; The output layer is used to output the final result with color highlighting or plain text output; The layers communicate with each other through a standardized data structure, which includes at least a parameter object, a file metadata object, and a result set object.

Images