Index processing method and device, computer equipment and storage medium

Abstract

The invention relates to an index processing method. The method comprises the steps of judging whether a target event is monitored or not; if yes, determining a target directory corresponding to the target event, and performing division processing on the target directory based on a preset target division strategy to obtain a corresponding high-priority directory and a low-priority directory; reading all target files under the high-priority directory based on a preset scanning thread, and calculating stable file identifiers of the target files; performing index construction processing on a preset construction index based on the stable file identifier of the target file to obtain a corresponding target index; performing consistency check on the target index; if the target index passes the consistency check, switching a preset active index into the target index based on an uninterruptible operation; and pushing the update information corresponding to the target index to a preset playing module. The invention further provides an index processing device, computer equipment and a storage medium. According to the method and the device, the index reconstruction processing efficiency is effectively improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to indexing methods, apparatus, computer equipment and storage media. Background Technology

[0002] In media file management scenarios of embedded devices, existing technologies generally suffer from insufficient index reconstruction efficiency for playback of large-scale media files stored on SD cards, eMMC, and other storage media. Traditional solutions mainly rely on two implementation methods: one is to traverse the file system in real time (such as through the readdir interface), matching the target file file by file each time playback or retrieval occurs; the other is to build a simple index structure in memory (such as a hash table or linear list), reducing the overhead of real-time traversal by caching file metadata.

[0003] However, the aforementioned methods reveal the following technical problems in scenarios with a large number of files or significant differences in storage media performance: Existing technologies require a full disk scan to rebuild the index during device startup, storage media insertion / removal, or file system changes. This time consumption increases linearly with the number of files. Especially in embedded devices with limited computing power, the index rebuilding process can significantly prolong device response time, resulting in low efficiency and negatively impacting user experience. Therefore, an efficient index rebuilding method is urgently needed to address the technical problem of long full disk scan times in existing technologies. Summary of the Invention

[0004] The purpose of this application is to provide an index processing method, apparatus, computer device, and storage medium to solve the technical problem of low processing efficiency in existing methods of index reconstruction by full disk scanning.

[0005] Firstly, an index processing method is provided, including: Determine whether the target event has been detected. If so, determine the target directory corresponding to the target event, and divide the target directory according to the preset target division strategy to obtain the corresponding high-priority directory and low-priority directory; Based on a preset scanning thread, all target files in the high-priority directory are read, and the stable file identifier of the target files is calculated. Based on the stable file identifier of the target file, the preset construction index is indexed to obtain the corresponding target index; Perform a consistency check on the target index; If the target index passes the consistency check, the preset active index will be switched to the target index based on an uninterruptible operation. The update information corresponding to the target index is pushed to the preset playback module.

[0006] Secondly, an index processing apparatus is provided, comprising: The first judgment module is used to determine whether the target event has been detected. The partitioning module is used to determine the target directory corresponding to the target event if the target event is true, and to partition the target directory based on a preset target partitioning strategy to obtain the corresponding high-priority directory and low-priority directory. The processing module is used to read all target files in the high-priority directory based on a preset scanning thread, and calculate the stable file identifier of the target files; The first construction module is used to perform index construction processing on the preset construction index based on the stable file identifier of the target file to obtain the corresponding target index; The first checking module is used to perform a consistency check on the target index; The switching module is used to switch the preset active index to the target index based on an uninterruptible operation if the target index passes the consistency check. The first push module is used to push the update information corresponding to the target index to the preset playback module.

[0007] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described indexing method.

[0008] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described indexing method.

[0009] In the above-described index processing method, apparatus, computer device, and storage medium, the solution first determines whether a target event has been detected. If so, the target directory corresponding to the target event is determined, and the target directory is divided into high-priority and low-priority directories based on a preset target partitioning strategy. Then, all target files in the high-priority directory are read using a preset scanning thread, and the stable file identifier of the target file is calculated. Subsequently, an index building process is performed on a preset index based on the stable file identifier of the target file to obtain the corresponding target index. A consistency check is then performed on the target index. If the target index passes the consistency check, the preset active index is switched to the target index based on an uninterruptible operation. Finally, the update information corresponding to the target index is pushed to a preset playback module. Based on the above automated processing flow, after detecting a target event, this application divides the target directory corresponding to the target event into high-priority and low-priority directories using a target partitioning strategy. Then, using a scanning thread, it reads all target files in the high-priority directory and calculates the stable file identifier of each target file. Next, based on the stable file identifier, it performs index building on a preset index to obtain the target index. Furthermore, it performs a consistency check on the target index. If the target index passes the consistency check, it switches the preset active index to the target index using an uninterruptible operation. Finally, it pushes the update information corresponding to the target index to the preset playback module. Thus, unlike existing methods that require a full scan to rebuild the index, this application improves the index availability speed and effectively enhances the processing efficiency of index rebuilding by combining priority scanning and fast index switching. Attached Figure Description

[0010] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is an exemplary system architecture diagram to which this application can be applied; Figure 2 This is a flowchart of an embodiment of the index processing method according to this application; Figure 3 This is a schematic diagram of the structure of an embodiment of the index processing apparatus according to this application; Figure 4 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0012] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0013] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0014] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0015] like Figure 1 As shown, system architecture 100 may include terminal device 101, network 102, and server 103. Terminal device 101 may be a laptop 1011, tablet 1012, or mobile phone 1013. Network 102 is used as a medium to provide a communication link between terminal device 101 and server 103. Network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0016] Users can use terminal device 101 to interact with server 103 via network 102 to receive or send messages, etc. Various communication client applications can be installed on terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0017] Terminal device 101 can be various electronic devices with a display screen and support web browsing. In addition to laptops 1011, tablets 1012, or mobile phones 1013, terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer, and a desktop computer, etc.

[0018] Server 103 can be a server that provides various services, such as a backend server that provides support for the pages displayed on terminal device 101.

[0019] It should be noted that the index processing method provided in this application embodiment is generally executed by a server / terminal device, and correspondingly, the index processing device is generally located in the server / terminal device.

[0020] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0021] Continue to refer to Figure 2 The flowchart illustrates an embodiment of the indexing method according to this application. Depending on different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted. The indexing method provided by this application embodiment can be applied to any scenario requiring indexing, and therefore can be applied to products in these scenarios. The indexing method includes the following steps: Step S201: Determine whether the target event has been detected.

[0022] In this embodiment, the indexing processing method runs on an electronic device (e.g., Figure 1The server / terminal device shown can monitor target events via wired or wireless connections. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra-wideband) connections, and other currently known or future-developed wireless connection methods. The executing entity of this application is specifically an index processing system, which can be simply referred to as the system. A storage device monitoring module is pre-built, which is responsible for real-time monitoring of changes in the storage device's status, such as insertion and removal, and changes in the file system, such as the addition, deletion, and modification of files. Changes in the storage device's status or the file system are the direct cause of triggering index building; only after these changes are detected will the subsequent index building process be initiated.

[0023] Specifically, when the storage device monitoring module detects that a target event has been triggered, such as a storage device insertion event, a system startup event, a file change event (such as adding, deleting, or modifying files), or a scheduled scan time event has been reached, the index building process will be initiated.

[0024] Step S202: If yes, determine the target directory corresponding to the target event, and divide the target directory according to the preset target division strategy to obtain the corresponding high-priority directory and low-priority directory.

[0025] In this embodiment, the target directory refers to the set of file system paths on the storage device that need to be indexed and scanned. It typically includes: the root directory of the storage device or a specified partition (e.g., / media / external_drive); specific directories manually configured by the user (e.g., folders added through the settings interface); and directories that the system focuses on by default (e.g., media library, document library, etc.). Example: If the monitoring module detects a USB drive being inserted, the target directory might be the root directory of the USB drive, / mnt / usb, and all its subdirectories; if a file change event is detected, the target directory might be the specific path where the change occurred (e.g., / home / user / Videos).

[0026] Directory partitioning is based on rules such as user behavior, system requirements, and file access frequency to dynamically divide these directories into two categories: "high-priority directories" and "low-priority directories" to optimize resource allocation and response speed for index building. The purpose of directory partitioning is to prioritize the processing of files accessed frequently by users, while avoiding low-frequency access directories from occupying too much system resources. The processing of target partitioning rules (target partitioning strategies) is usually based on the following factors: (1) User behavior data. Recent access records: Identify directories recently opened by users (such as recently played videos or edited documents) through system logs or playback records. Example: If a user played / home / user / Videos / Movie1.mp4 in the past 24 hours, then / home / user / Videos will be marked as high priority. Thumbnail / metadata directory: The directory that stores file preview images or metadata (such as the image thumbnail cache directory .thumbnails). These files need to be loaded quickly to support the interface display. (2) File type and access frequency. High-frequency access types: Media files (videos, audios), documents (PDFs, Office files), etc. are usually processed first. Low-frequency access types: System logs, temporary files, backup files, etc. can be downgraded. (3) System default rules. Fixed high-priority directories: such as the media library path ( / var / lib / media) and desktop directory ( / home / user / Desktop) preset in the system configuration. Dynamic adjustment: according to the device usage scenario (such as the navigation map directory being processed first in the vehicle system). (4) Resource consumption and performance trade-off. Directory size: directories containing a large number of files (such as a photo album with tens of thousands of pictures) may be split into multiple batches for processing to avoid blocking the system. I / O load: if the directory is located on a mechanical hard drive, its priority may be reduced to reduce seek time overhead.

[0027] Specifically, the target directories can be partitioned based on the aforementioned target partitioning strategy, dividing them into high-priority and low-priority directories. Furthermore, directory partitioning is a key mechanism for the indexing system to achieve differentiated services. By combining user behavior, file characteristics, and system status, resources are concentrated in high-value directories while ensuring eventual consistency in low-priority directories. This design can significantly improve user experience and system stability in scenarios such as media playback and file management.

[0028] Step S203: Read all target files in the high-priority directory based on a preset scanning thread, and calculate the stable file identifier of the target files.

[0029] In this embodiment, the scanning thread is a pre-built processing thread that can be used to perform a full and fast scan of the partitioned high-priority directory and read the file list containing all files in the high-priority directory, i.e., the target file.

[0030] The specific implementation process for calculating the stable file identifier of the target file will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0031] Step S204: Based on the stable file identifier of the target file, perform index building processing on the preset construction index to obtain the corresponding target index.

[0032] In this embodiment, the system maintains two index buffers upon startup: ActiveIndex, which stores currently active index data for direct use by the query module; it is initially empty and updated via atomic operations; and BuildingIndex, which serves as a temporary buffer for incremental building of new indexes; it is initially empty and replaces the ActiveIndex after construction. Data structure selection: A composite structure of hash table and doubly linked list is used. The hash table is used for fast file location (key is SFID), and the linked list is used to maintain file access order or priority.

[0033] The specific implementation process of indexing the preset construction index based on the stable file identifier of the target file to obtain the corresponding target index will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0034] Step S205: Perform a consistency check on the target index.

[0035] In this embodiment, the specific implementation process of performing consistency checks on the target index will be described in more detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0036] Step S206: If the target index passes the consistency check, the preset active index is switched to the target index based on an uninterruptible operation.

[0037] In this embodiment, after the target index passes the consistency check, the system executes an uninterruptible complete operation (the non-interruptible operation) to switch the new target index to the "active index," that is, to switch the preset active index to the target index. This operation ensures the atomicity of the switching process, avoiding inconsistencies in the indexes during the switching process. The selection of the non-interruptible operation is not limited and can be determined according to actual business needs. For example, it can be achieved through locking mechanisms, atomic pointer swaps, or memory barriers to completely replace the hash table and linked list of the new target index with the active index.

[0038] Step S207: Push the update information corresponding to the target index to the preset playback module.

[0039] In this embodiment, the updated information can be "Active index updated". The playback module can be a module corresponding to the target directory and dependent on index queries (such as browsing services or on-demand services). Furthermore, the system can proactively push new target index references or "Active index updated" signals to the playback module through event notification mechanisms (such as message queues or callback functions). After receiving the notification, the playback module can asynchronously load the new index to avoid blocking the main thread.

[0040] This application first determines whether a target event has been detected. If so, it identifies the target directory corresponding to the target event and divides the target directory into high-priority and low-priority directories based on a preset target partitioning strategy. Then, it reads all target files in the high-priority directory using a preset scanning thread and calculates the stable file identifier of the target files. Next, it performs index building processing on a preset index based on the stable file identifier of the target files to obtain the corresponding target index. Subsequently, it performs a consistency check on the target index. If the target index passes the consistency check, it switches the preset active index to the target index based on an uninterruptible operation. Finally, it pushes the update information corresponding to the target index to a preset playback module. Based on the above automated processing flow, after detecting a target event, this application divides the target directory corresponding to the target event into high-priority and low-priority directories using a target partitioning strategy. Then, using a scanning thread, it reads all target files in the high-priority directory and calculates the stable file identifier of each target file. Next, based on the stable file identifier, it performs index building on a preset index to obtain the target index. Furthermore, it performs a consistency check on the target index. If the target index passes the consistency check, it switches the preset active index to the target index using an uninterruptible operation. Finally, it pushes the update information corresponding to the target index to the preset playback module. Thus, unlike existing methods that require a full scan to rebuild the index, this application improves the index availability speed and effectively enhances the processing efficiency of index rebuilding by combining priority scanning and fast index switching.

[0041] In some optional implementations, calculating the stable file identifier of the target file in step S203 includes the following steps: Calculate the checksum corresponding to the header data of the target file.

[0042] In this embodiment, the header data of the target file (e.g., the first N bytes (e.g., 512 bytes)) can be read for check value calculation, and then the CRC32 algorithm can be performed on the header data to generate the corresponding check value.

[0043] Obtain the file size and last modified timestamp of the target file.

[0044] In this embodiment, the required file size (number of bytes, left-shifted by 7 bits to increase weight) and the last modified timestamp (accurate to seconds or milliseconds, taking the lower 32 bits) can be extracted by acquiring information from the target file.

[0045] Call the preset identifier generation formula.

[0046] In this embodiment, the above identification generation formula includes: SFID = CRC32(file header data) XOR (file size << 7) XOR (last modification timestamp & 0xFFFFFFFF).

[0047] The verification value, the file size, and the last modification timestamp are calculated based on the identifier generation formula to obtain the corresponding calculation results.

[0048] In this embodiment, the above-mentioned check value, file size and last modification timestamp can be substituted into the above-mentioned identifier generation formula for calculation, and the generated calculation result can be used as the stable file identifier of the target file.

[0049] The calculation result is used as the stable file identifier of the target file.

[0050] This application calculates a checksum corresponding to the header data of the target file; obtains the file size and last modified timestamp of the target file; then calls a preset identifier generation formula; subsequently, it calculates the checksum, file size, and last modified timestamp based on the identifier generation formula to obtain the corresponding calculation result; and finally, it uses the calculation result as the stable file identifier of the target file. Based on the above processing flow, this application calculates the checksum corresponding to the header data of the target file, obtains the file size and last modified timestamp of the target file, and then calculates the checksum, file size, and last modified timestamp based on the identifier generation formula, and uses the obtained calculation result as the stable file identifier of the target file. This allows for efficient and accurate calculation of the stable file identifier of the target file, ensuring the accuracy of the obtained stable file identifier data.

[0051] In some optional implementations of this embodiment, step S204 includes the following steps: Obtain the hash table corresponding to the constructed index.

[0052] In this embodiment, the system maintains two index buffers upon startup: ActiveIndex, which stores currently active index data for direct use by the query module; it is initially empty and updated via atomic operations; and BuildingIndex, which serves as a temporary buffer for incremental building of new indexes; it is initially empty and replaces the ActiveIndex after construction. Data structure selection: A composite structure of hash table and doubly linked list is used. The hash table is used for fast file location (key is SFID), and the linked list is used to maintain file access order or priority.

[0053] The hash table is queried based on the target file and the stable file identifier to obtain the corresponding query results.

[0054] In this embodiment, by using the target file and its stable file identifier to query the index being built, i.e. the hash table corresponding to the index being built, the following query results are obtained: there is no corresponding record for the target file in the hash table, i.e. the target file is a new file; there is a target file in the hash table but the stable file identifier is different; there is a target file in the hash table and the stable file identifier is the same.

[0055] Based on the query results, the index information of the constructed index is updated to obtain the corresponding specified index.

[0056] In this embodiment, updating the index information based on the query results includes: 1) If the target file is a new file, meaning there is no corresponding record in the hash table, a new index node is created. The index node contains information such as the filename, stable file identifier, file size, timestamp, and pointers to the previous and next nodes in the linked list and the next node in the hash table. The new node is then inserted into the hash table and the temporary linked list. 2) If the target file already exists in the hash table but the identifier (stable file identifier) ​​is different, it indicates that the file content has actually changed. In this case, the information of the target file in the index is updated, including the file size, timestamp, and other related data. 3) If the target file exists in the hash table and the identifier is the same, it indicates that the target file has not actually changed its content, and there is no need to update the index information. The target file is skipped to save system resources.

[0057] Use the specified index as the target index.

[0058] This application obtains a hash table corresponding to the constructed index; then, based on the target file and the stable file identifier, it queries the hash table to obtain the corresponding query result; subsequently, based on the query result, it updates the index information of the constructed index to obtain the corresponding specified index; and finally, it uses the specified index as the target index. Based on the above processing flow, this application obtains a hash table corresponding to the constructed index, queries the hash table based on the target file and the stable file identifier to obtain the query result, updates the index information of the constructed index based on the query result, and uses the obtained specified index as the required target index. This allows for efficient and accurate completion of the index construction process, improving the efficiency and intelligence of index construction.

[0059] In some alternative implementations, step S205 includes the following steps: Perform hash table integrity verification on the target index.

[0060] In this embodiment, the hash table integrity verification includes traversing the hash table of the target index, checking whether the linked lists of all buckets are correctly linked (without broken links or circular references), and verifying whether the pointers of each node (predecessor, successor, and hash collision chain) point to a valid memory address or are NULL. If it is detected that the linked lists of all buckets are correctly linked and the pointers of each node point to a valid memory address, the target index is determined to have passed the hash table integrity verification; otherwise, the target index is determined to have failed the hash table integrity verification, and thus the target index is directly determined to have failed the consistency check.

[0061] If the target index passes the hash table integrity verification, then the data accuracy of the target index is verified.

[0062] In this embodiment, the data accuracy verification includes: randomly sampling inodes in the target index, recalculating the SFID (using file header data, size, and timestamp), comparing it with the SFID stored in the inode to ensure consistency, and checking if the file path exists to avoid dangling pointers (e.g., files are deleted but the index is not cleaned up). If the inodes are found to be consistent and the file path exists, the target index is determined to have passed the data accuracy verification; otherwise, the target index is determined to have failed the data accuracy verification, and thus directly fails the consistency check.

[0063] If the target index passes the data accuracy verification, then the target index is subjected to statistical information verification.

[0064] In this embodiment, the statistical information verification includes checking whether the total number of files and the total size recorded in the target index match the actual scan results. If the total number of files and the total size recorded in the target index match the actual scan results, the target index is determined to have passed the statistical information verification; otherwise, the target index is determined to have failed the statistical information verification, and thus the target index is directly determined to have failed the consistency check.

[0065] If the target index passes the statistics verification, it is determined that the target index has passed the consistency check; otherwise, it is determined that the target index has failed the consistency check.

[0066] In this embodiment, the target index is determined to have passed the consistency check only if it has simultaneously passed the hash table integrity verification, data accuracy verification, and statistical information verification; otherwise, it is determined to have failed the consistency check.

[0067] This application verifies the target index by performing hash table integrity verification; if the target index passes the hash table integrity verification, it then verifies the data accuracy of the target index; if the target index passes the data accuracy verification, it then verifies the statistical information of the target index; if the target index passes the statistical information verification, it is determined that the target index has passed the consistency check; otherwise, it is determined that the target index has failed the consistency check. Based on the above processing flow, this application automatically and accurately completes the consistency check of the target index by performing hash table integrity verification, data accuracy verification, and statistical information verification, ensuring the accuracy of the obtained consistency check results and effectively ensuring that the new target index can correctly reflect the file status on the storage medium, avoiding query anomalies due to construction errors.

[0068] In some alternative implementations, after step S207, the electronic device may further perform the following steps: Monitor system load information and determine whether the system load information is greater than a preset load threshold.

[0069] In this embodiment, system input / output (I / O) load or CPU utilization can be monitored in real time to obtain corresponding system load information. The selection of the aforementioned load threshold values ​​is not specifically limited and can be set according to actual business needs. For example, the threshold corresponding to I / O load can be set to 30%, and the threshold corresponding to CPU utilization can be set to 80%. Furthermore, I / O load can be obtained through system APIs to obtain disk read / write latency or queue length. CPU utilization can be obtained by statistically analyzing the percentage of CPU used in user mode and kernel mode.

[0070] If so, the low-priority directory is scanned based on a preset sharding scanning strategy to obtain the corresponding scanned files.

[0071] In this embodiment, the aforementioned segmented scanning strategy refers to a strategy of scanning gradually in batches and intermittently. This is to avoid occupying system resources for extended periods, thus affecting the normal playback and recording functions of the device. The scanning thread monitors the system load information in real time. When the load is high, it automatically reduces the scanning speed or pauses scanning, resuming scanning only after the system load decreases. For example, if a high system load is detected, such as during video recording, the scanning thread automatically reduces the scanning speed, pauses for a period of time, and then resumes scanning to avoid affecting the recording function. Specifically, based on the aforementioned segmented scanning strategy, low-priority directories can be scanned to read all files contained within those directories, i.e., the scanned files. Furthermore, if an exception such as a read failure is encountered during the scanning process, the system will immediately suspend the current operation and retry it in the background later.

[0072] Based on the scanned file, an index building process is performed to obtain the corresponding incremental index fragment.

[0073] In this embodiment, a stable file identifier can be calculated on the scanned file, and an index building process can be performed based on the obtained stable file identifier and the scanned file to generate an incremental index fragment containing only the new / change records of low-priority files (scanned files). Specifically, the corresponding index data can be generated by repeating the index update steps (calculating SFID, querying the hash table, updating or skipping nodes) of the high-priority directory for each batch of files in the low-priority directory.

[0074] Based on the incremental index fragment, the target index is merged to obtain the corresponding target active index.

[0075] In this embodiment, the process of merging data in the target index based on incremental index fragments includes: locking or pausing writes: To ensure consistency, the playback module's write operations to the target index are briefly paused (or copy-on-write technology is used), but reading is allowed (to avoid completely freezing the service). Merging data: Adding files: File records that do not exist in the target index from the low-priority directory are inserted into the hash table and linked list. Changing files: If a file already exists in the active index but has a different stable file identifier, its metadata (such as size, timestamp) is updated, and the linked list pointers are adjusted. Unchanged files: If the specified stable file identifier is the same, it is skipped directly. Releasing locks / resuming writes: After the merge is completed, the normal operation of the playback module is immediately resumed.

[0076] Perform a consistency check on the target active index.

[0077] In this embodiment, the process of performing consistency checks on the target active index described above can be referred to the specific implementation process of performing consistency checks on the target index as described above, and will not be elaborated further here.

[0078] If the target active index passes the consistency check, the target update information corresponding to the target active index will be pushed to the playback module.

[0079] In this embodiment, index update information (i.e., target update information) that matches the aforementioned target active index can be generated and pushed to the playback module.

[0080] This application monitors system load information and determines whether the system load exceeds a preset load threshold. If so, it scans the low-priority directory using a preset sharding scanning strategy to obtain corresponding scanned files. Then, it performs index building based on the scanned files to obtain corresponding incremental index fragments. Next, it merges data from the target index using the incremental index fragments to obtain the corresponding target active index. Subsequently, it performs a consistency check on the target active index. If the target active index passes the consistency check, it pushes the target update information corresponding to the target active index to the playback module. Based on this processing flow, when the system load is detected to be greater than the load threshold, this application scans the low-priority directory using a sharding scanning strategy to obtain scanned files, performs index building based on the scanned files to obtain incremental index fragments, and then merges data from the target index using the incremental index fragments to obtain the target active index. This achieves index updates for low-priority directories through incremental merging into the target index, avoiding the overhead of full reconstruction and improving the construction efficiency and intelligent generation of the target active index. When the target active index passes the consistency check, the target update information corresponding to the target active index is pushed to the playback module, ensuring that the playback module can always query the latest and consistent file status as the target active index is gradually improved.

[0081] In some optional implementations of this embodiment, after step S206, the electronic device may further perform the following steps: After completing the index switching corresponding to the target index, the active index is marked.

[0082] In this embodiment, old active indexes can be marked as "to be cleaned up" but retained for a certain period of time (e.g., 72 hours) to support rollback.

[0083] Save the reference to the active index to a preset temporary variable.

[0084] In this embodiment, the aforementioned temporary variable is a pre-constructed variable used to store data. By saving references to the original active index, such as index objects pointed to by global variables, into this temporary variable, the ability to temporarily store old indexes can be preserved. This allows for a quick rollback to the previous stable state when problems occur, improving fault tolerance.

[0085] Determine whether the target index is in a stable operating state.

[0086] In this embodiment, the stability of the target index can be determined by detecting whether it operates stably within a preset time period, such as whether there are any rollback operations within that period. The value of the preset time period is not specifically limited and can be set according to actual business needs; for example, it could be set to 48 hours. Furthermore, if no rollback operations are detected within the preset time period, the target index is determined to be in a stable operating state; otherwise, it is determined that the target index is not in a stable operating state.

[0087] Alternatively, you can determine whether the target index is in a stable operating state by checking whether there are no error logs and whether the query timeout rate is normal.

[0088] If so, clean up the temporary variables.

[0089] In this embodiment, the cleanup process refers to releasing the memory or storage resources occupied by the old data (i.e., the original active index) in the temporary variable.

[0090] After completing the index switch corresponding to the target index, this application marks the active index; then, it saves the reference of the active index to a preset temporary variable; subsequently, it determines whether the target index is in a stable running state; if so, it cleans up the temporary variable. Based on the above processing flow, after completing the index switch corresponding to the target index, this application intelligently marks the active index and saves the reference of the active index to a preset temporary variable, thereby realizing intelligent temporary storage processing of the original active index for rollback or merging. Furthermore, when the target index is detected to be in a stable running state, the temporary variable is automatically cleaned up, thereby releasing system storage resources and achieving balanced processing of storage resource usage.

[0091] In some optional implementations of this embodiment, after step S203, the electronic device may further perform the following steps: Obtain performance statistics corresponding to the file scanning process of the scanning thread.

[0092] In this embodiment, the collection and processing of the aforementioned performance statistics includes: Metric collection: recording the file scan processing time, CPU utilization, peak memory usage, and disk I / O throughput of the current scan thread. Statistics on scan coverage (e.g., the proportion of successfully indexed files) and error rate (e.g., the number of permission denials) of high-priority directories. Contextual analysis: combining system-wide load (e.g., obtained via / proc / loadavg or API) and user behavior (e.g., playback request frequency) to analyze performance impact. The collected data should be timestamped for subsequent trend analysis. Statistics on high-priority directories should distinguish between "active scans" and "event-triggered scans."

[0093] Obtain the preset strategy adjustment rules.

[0094] In this embodiment, the policy adjustment rules include: 1) Load-sensitive adjustment: If the system load exceeds a threshold during the current scan (e.g., CPU > 80% for 5 minutes), the following rules will be implemented for the next scan: reduce the number of concurrent threads (e.g., from 8 to 4), increase the scan interval (e.g., from 1 hour to 2 hours), and enable "low priority mode" (e.g., skip non-high priority directories). 2) Priority optimization: If the high priority directory scan is effective (e.g., coverage > 95% and error rate < 1%), the following rules will be implemented for the next scan: expand the scope of high priority directories (e.g., add / media / new to the whitelist), and shorten the scan interval for high priority directories (e.g., from 24 hours to 12 hours). 3) Resource limitation: If memory usage is too high (e.g., increases > 30% during the scan), limit the number of files scanned per scan (e.g., process in batches). The policy adjustment rules can be configured as needed (e.g., through policy files or management interfaces) to avoid hard coding. An upper limit for the adjustment range (e.g., an interval of no less than 5 minutes) will be set to prevent over-correction.

[0095] Based on the performance statistics, the scanning strategy is optimized using the strategy adjustment rules to obtain the corresponding optimized scanning strategy.

[0096] In this embodiment, the aforementioned scanning strategy refers to the strategy adopted by the scanning thread during file scanning. The original scanning strategy can be optimized and adjusted based on the rule content of the aforementioned strategy adjustment rules to obtain a corresponding optimized scanning strategy.

[0097] The optimized scanning strategy is stored.

[0098] In this embodiment, the storage method for the above-mentioned optimized scanning strategy is not specifically limited and can be determined according to actual storage needs. For example, local database storage, cloud server storage, blockchain storage, etc., can be used. Furthermore, the generated optimized scanning strategy can be written to a configuration file to ensure it takes effect after a restart. Additionally, the new optimized scanning strategy can be applied first to some nodes or directories to observe performance changes (e.g., comparing scan times through A / B testing). If the new optimized scanning strategy causes a performance decrease (e.g., a 30% increase in scan timeout), it can automatically revert to the previous configuration version.

[0099] This application obtains performance statistics corresponding to the file scanning process of the scanning thread; then obtains preset strategy adjustment rules; subsequently, based on the performance statistics, it optimizes the scanning strategy using the strategy adjustment rules to obtain an optimized scanning strategy; and finally, it stores the optimized scanning strategy. Based on the above processing flow, this application obtains an optimized scanning strategy by acquiring performance statistics corresponding to the file scanning process of the scanning thread and optimizing the scanning strategy based on the use of strategy adjustment rules. This enables intelligent optimization of the scanning strategy, ensuring efficient execution of the scanning task and improved processing performance when the optimized scanning strategy is used subsequently. Furthermore, the optimized scanning strategy is intelligently stored to ensure the data security of the generated optimized scanning strategy.

[0100] In some alternative implementations, the user information obtained is subject to user consent and complies with relevant laws and policies.

[0101] Furthermore, any software tools or components not belonging to our company that appear in the embodiments of this application are merely illustrative examples and do not represent actual use.

[0102] Furthermore, this application is applicable to scenarios involving "playback file index management and rapid reconstruction" in embedded cameras, outdoor security cameras, hunting / action cameras, and other devices equipped with SD cards / EMMC storage. It is particularly useful for situations involving the insertion and removal of storage media, storage media malfunctions, or a large number of media files, enabling rapid reconstruction of the playback index, low-latency retrieval of the previous / next playback file, and reduced full-disk rescan time and CPU / IO usage. This application relates to software technology for playback file management, with main functions including: scanning media files (images / videos) on the storage medium (SD card / EMMC), establishing a playback index (doubly linked list + hash table), enabling rapid traversal (previous / next switching), deletion, index reconstruction, and index reconstruction and recovery when storage status changes (removal / insertion).

[0103] This application aims to solve the problems of slow index reconstruction, high response latency, and high system resource consumption when embedded devices manage a large number of media files. Its core lies in designing an efficient, stable, and low-latency playback file index management and reconstruction mechanism.

[0104] The core mechanisms and steps include: 1. Dual Index Buffer Mechanism: Description: The system maintains two index buffers simultaneously: an "active index" currently in use and a "building index" being built in the background. How it works: The scanning thread builds the new "building index" in the background, while the playback module continues to use the current "active index." Once the "building index" is completed and passes inspection, the system switches the "building index" to the new "active index" through a single, uninterrupted operation, temporarily storing the old "active index" for rollback or merging. This switch has minimal impact on the playback module, and is virtually imperceptible to the user.

[0105] 2. Incremental Scanning and Event Triggering: Description: A full scan of all files is not required every time. Operation: The system listens for events such as storage device insertion / removal, file system changes, or at fixed time intervals, scanning and updating the index only for directories and files that have changed. A full scan is only triggered when necessary, such as during system startup or when the storage media recovers from an anomaly.

[0106] 3. Stable File Identifier: Description: A unique and stable identifier is generated for each media file to accurately determine whether the file's content has undergone actual changes, avoiding misjudgments caused by file renaming or moving. Generation Method: This identifier is derived by comprehensively calculating the checksum of a specific byte in the file header, the file size, and the last modified timestamp, among other information.

[0107] 4. Priority Scanning and Preloading: Description: Based on user habits, prioritize files that are more likely to be accessed. How it works: The system will first scan directories that the user has recently played or directories that store thumbnails, and preload some data for files that may be played soon, thus significantly reducing the waiting time when the user switches files.

[0108] 5. Hybrid Index Data Structure: Description: This structure combines a hash table and a doubly linked list to manage the index. How it works: The hash table enables fast lookups based on information such as filename. The doubly linked list maintains the playback order of files, allowing for quick forward / backward switching. Each index node contains the filename, stable file identifier, file size, timestamp, and pointers to the previous and next nodes in the linked list and the next node in the hash table.

[0109] 6. Adaptive Scanning and Anomaly Handling: Description: The system can intelligently adjust its scanning behavior based on the current operating status. Operating Mode: The scanning thread monitors the system's input / output load and CPU utilization in real time. When the load is high, it automatically reduces the scanning speed or adopts a batch, intermittent scanning method to avoid affecting normal playback and recording functions. If an anomaly such as a read failure is encountered during scanning, the system will immediately suspend the current operation and retry it in the background later.

[0110] Furthermore, compared to existing technologies that require a full disk scan to rebuild the index, this application offers the following significant advantages: 1. Significantly improved index availability speed: Through priority scanning and fast index switching, users can begin browsing within seconds of high-priority files being ready, without waiting for a full disk scan to complete. 2. Effectively reduced system resource consumption: Incremental scanning and adaptive rate control smooth out peak input / output and CPU usage, reducing interference with core functions such as device recording and playback. 3. Enhanced index accuracy and system stability: The stable file identification mechanism effectively avoids misjudgments caused by operations such as file renaming, and combined with atomic switching and rollback mechanisms, improves the overall system robustness. 4. Support for seamless recovery: The ability to retain old indexes allows for rapid rollback to the previous stable state when problems occur, improving fault tolerance.

[0111] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0112] It should be emphasized that, to further ensure the privacy and security of the aforementioned target index, the target index can also be stored in a node of a blockchain.

[0113] The blockchain referred to in this application is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0114] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0115] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0116] Further reference Figure 3 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of an index processing apparatus, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0117] like Figure 3 As shown, the index processing device 300 described in this embodiment includes: a first judgment module 301, a partitioning module 302, a processing module 303, a first construction module 304, a first checking module 305, a switching module 306, and a first push module 307. Wherein: The first judgment module 301 is used to determine whether the target event has been detected. The partitioning module 302 is used to determine the target directory corresponding to the target event if the target event is true, and to partition the target directory based on a preset target partitioning strategy to obtain the corresponding high-priority directory and low-priority directory. The processing module 303 is used to read all target files in the high-priority directory based on a preset scanning thread, and calculate the stable file identifier of the target files; The first construction module 304 is used to perform index construction processing on the preset construction index based on the stable file identifier of the target file to obtain the corresponding target index; The first inspection module 305 is used to perform a consistency check on the target index; The switching module 306 is used to switch the preset active index to the target index based on an uninterruptible operation if the target index passes the consistency check. The first push module 307 is used to push the update information corresponding to the target index to the preset playback module.

[0118] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the index processing method in the aforementioned implementation method, and will not be repeated here.

[0119] In some optional implementations of this embodiment, the processing module 303 includes: The first calculation submodule is used to calculate the check value corresponding to the file header data of the target file; The first acquisition submodule is used to acquire the file size and last modification timestamp of the target file; Call the submodule to invoke the preset identifier generation formula; The second calculation submodule is used to calculate the verification value, the file size and the last modification timestamp based on the identifier generation formula to obtain the corresponding calculation result; The first determining submodule is used to use the calculation result as a stable file identifier for the target file.

[0120] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the index processing method in the aforementioned implementation method, and will not be repeated here.

[0121] In some optional implementations of this embodiment, the first construction module 304 includes: The second acquisition submodule is used to acquire the hash table corresponding to the constructed index; The query submodule is used to query the hash table based on the target file and the stable file identifier to obtain the corresponding query results; The update submodule is used to update the index information of the constructed index based on the query results, so as to obtain the corresponding specified index; The second determining submodule is used to use the specified index as the target index.

[0122] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the index processing method in the aforementioned implementation method, and will not be repeated here.

[0123] In some optional implementations of this embodiment, the first inspection module 305 includes: The first verification submodule is used to perform hash table integrity verification on the target index; The second verification submodule is used to verify the data accuracy of the target index if the target index passes the hash table integrity verification. The third verification submodule is used to perform statistical information verification on the target index if the target index passes the data accuracy verification. The determination submodule is used to determine that the target index passes the consistency check if the target index passes the statistical information verification, and otherwise determine that the target index fails the consistency check.

[0124] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the index processing method in the aforementioned implementation method, and will not be repeated here.

[0125] In some optional implementations of this embodiment, the index processing device further includes: The second judgment module is used to monitor system load information and determine whether the system load information is greater than a preset load threshold. The scanning module is used to scan the low-priority directory based on a preset sharding scanning strategy if the condition is met, to obtain the corresponding scanned files. The second construction module is used to perform index construction processing based on the scanned file to obtain the corresponding incremental index fragments; The merging module is used to perform data merging processing on the target index based on the incremental index fragment to obtain the corresponding target active index; The second checking module is used to perform a consistency check on the target active index; The second push module is used to push the target update information corresponding to the target active index to the playback module if the target active index passes the consistency check.

[0126] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the index processing method in the aforementioned implementation method, and will not be repeated here. In some optional implementations of this embodiment, the index processing device further includes: The marking module is used to mark the active index after the index switching corresponding to the target index is completed; A save module is used to save the reference of the active index to a preset temporary variable; The third judgment module is used to determine whether the target index is in a stable operating state; The cleanup module is used to clean up the temporary variables if necessary.

[0127] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the index processing method in the aforementioned implementation method, and will not be repeated here.

[0128] In some optional implementations of this embodiment, the index processing device further includes: The acquisition module is used to acquire performance statistics corresponding to the file scanning processing of the scanning thread; The analysis module is used to obtain preset strategy adjustment rules; The optimization module is used to optimize the scanning strategy based on the performance statistics and the strategy adjustment rules to obtain the corresponding optimized scanning strategy. The storage module is used to store the optimized scanning strategy.

[0129] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the index processing method in the aforementioned implementation method, and will not be repeated here. To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0130] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected via a system bus. It should be noted that only the computer device 4 with components 41-43 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0131] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0132] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 4. Of course, the memory 41 may include both the internal storage unit and its external storage device of the computer device 4. In this embodiment, the memory 41 is typically used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions for indexing processing methods. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or will be output.

[0133] In some embodiments, the processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions stored in the memory 41 or to process data, for example, to execute computer-readable instructions of the indexing method.

[0134] The network interface 43 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 4 and other electronic devices.

[0135] Compared with the prior art, the embodiments of this application have the following beneficial effects: In this embodiment, after detecting a target event, the application divides the target directory corresponding to the target event into high-priority and low-priority directories based on a target partitioning strategy. Then, using a scanning thread, it reads all target files in the high-priority directory and calculates the stable file identifiers of the target files. Next, based on the stable file identifiers, it performs index building on a preset index to obtain the target index. Furthermore, it performs a consistency check on the target index. If the target index passes the consistency check, it switches the preset active index to the target index using an uninterruptible operation. Finally, it pushes the update information corresponding to the target index to the preset playback module. Thus, unlike existing methods that require a full scan to rebuild the index, this application improves the index availability speed and effectively enhances the processing efficiency of index rebuilding by combining priority scanning and fast index switching.

[0136] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the indexing method described above.

[0137] Compared with the prior art, the embodiments of this application have the following main advantages: In this embodiment, after detecting a target event, the application divides the target directory corresponding to the target event into high-priority and low-priority directories based on a target partitioning strategy. Then, using a scanning thread, it reads all target files in the high-priority directory and calculates the stable file identifiers of the target files. Next, based on the stable file identifiers, it performs index building on a preset index to obtain the target index. Furthermore, it performs a consistency check on the target index. If the target index passes the consistency check, it switches the preset active index to the target index using an uninterruptible operation. Finally, it pushes the update information corresponding to the target index to the preset playback module. Thus, unlike existing methods that require a full scan to rebuild the index, this application improves the index availability speed and effectively enhances the processing efficiency of index rebuilding by combining priority scanning and fast index switching.

[0138] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0139] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. An index processing method, characterized in that, Includes the following steps: Determine whether the target event has been detected. If so, determine the target directory corresponding to the target event, and divide the target directory according to the preset target division strategy to obtain the corresponding high-priority directory and low-priority directory; Based on a preset scanning thread, all target files in the high-priority directory are read, and the stable file identifier of the target files is calculated. Based on the stable file identifier of the target file, the preset construction index is indexed to obtain the corresponding target index; Perform a consistency check on the target index; If the target index passes the consistency check, the preset active index will be switched to the target index based on an uninterruptible operation. The update information corresponding to the target index is pushed to the preset playback module.

2. The index processing method according to claim 1, characterized in that, The step of calculating the stable file identifier of the target file specifically includes: Calculate the checksum corresponding to the header data of the target file; Obtain the file size and last modified timestamp of the target file; Call the preset identifier generation formula; Based on the identifier generation formula, the verification value, the file size, and the last modification timestamp are calculated to obtain the corresponding calculation results; The calculation result is used as the stable file identifier of the target file.

3. The index processing method according to claim 1, characterized in that, The step of performing index building processing on the preset construction index based on the stable file identifier of the target file to obtain the corresponding target index specifically includes: Obtain the hash table corresponding to the constructed index; The hash table is queried based on the target file and the stable file identifier to obtain the corresponding query result; Based on the query results, the constructed index is updated to obtain the corresponding specified index. Use the specified index as the target index.

4. The index processing method according to claim 1, characterized in that, The step of performing a consistency check on the target index specifically includes: Perform hash table integrity verification on the target index; If the target index passes the hash table integrity verification, then the data accuracy of the target index is verified. If the target index passes the data accuracy verification, then the target index is subjected to statistical information verification. If the target index passes the statistics verification, it is determined that the target index has passed the consistency check; otherwise, it is determined that the target index has failed the consistency check.

5. The index processing method according to claim 1, characterized in that, After the step of pushing the update information corresponding to the target index to the preset playback module, the method further includes: Monitor system load information and determine whether the system load information is greater than a preset load threshold; If so, the low-priority directory is scanned based on a preset sharding scanning strategy to obtain the corresponding scanned files; Based on the scanned file, an index building process is performed to obtain the corresponding incremental index fragment; Based on the incremental index fragment, the target index is merged to obtain the corresponding target active index; Perform a consistency check on the target active index; If the target active index passes the consistency check, the target update information corresponding to the target active index will be pushed to the playback module.

6. The index processing method according to claim 1, characterized in that, After the step of switching the preset active index to the target index based on the non-interruptible operation, the method further includes: After completing the index switching corresponding to the target index, the active index is marked. Save the reference to the active index to a preset temporary variable; Determine whether the target index is in a stable operating state; If so, clean up the temporary variables.

7. The index processing method according to claim 1, characterized in that, After the steps of reading all target files in the high-priority directory based on a preset scanning thread and calculating the stable file identifier of the target files, the method further includes: Obtain performance statistics corresponding to the file scanning processing of the scanning thread; Obtain the preset strategy adjustment rules; Based on the performance statistics, the scanning strategy is optimized using the strategy adjustment rules to obtain the corresponding optimized scanning strategy. The optimized scanning strategy is stored.

8. An index processing apparatus, characterized in that, include: The first judgment module is used to determine whether the target event has been detected. The partitioning module is used to determine the target directory corresponding to the target event if the target event is true, and to partition the target directory based on a preset target partitioning strategy to obtain the corresponding high-priority directory and low-priority directory. The processing module is used to read all target files in the high-priority directory based on a preset scanning thread, and calculate the stable file identifier of the target files; The first construction module is used to perform index construction processing on the preset construction index based on the stable file identifier of the target file to obtain the corresponding target index; The first checking module is used to perform a consistency check on the target index; The switching module is used to switch the preset active index to the target index based on an uninterruptible operation if the target index passes the consistency check. The first push module is used to push the update information corresponding to the target index to the preset playback module.

9. A computer device, characterized in that, The system includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor, when executing the computer-readable instructions, implements the steps of the indexing processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the indexing processing method as described in any one of claims 1 to 7.

Images