Queue-based voiceprint data processing method and device and electronic equipment
By using a shared queue mechanism in voiceprint data processing, the conflict problem caused by multiple Python processes writing to the voiceprint file simultaneously is resolved, achieving balanced and efficient voiceprint file writing and improving the utilization of computing resources.
Patent Information
- Application Number
- CN202510377590.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-28
- Publication Date
- 2025-11-25
- Estimated Expiration
- 2045-03-28
AI Technical Summary
In existing technologies, multiple Python processes writing to the same voiceprint file simultaneously can easily lead to data conflicts and make it difficult to guarantee the balance of voiceprint file writing.
A shared queue mechanism is adopted to create a first-in-first-out queue with blocking function to store voiceprint file identifiers. During the initialization phase, the voiceprint data writing process opens the voiceprint file in append mode and connects it to the shared queue. The file identifier is obtained and written in first-in-first-out order, and the identifier is put back into the queue after completion.
This avoids conflicts caused by multiple processes writing to the same voiceprint file simultaneously, achieving balanced and efficient voiceprint file writing and making full use of computing resources.
Smart Images

Figure CN120148522B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to a queue-based voiceprint data processing method, apparatus, and electronic device. Background Technology
[0002] Voiceprint recognition technology is a type of biometric technology that converts sound signals into electrical signals for computer identification. In voiceprint recognition, voiceprint data extracted from audio files is typically compared with a stored voiceprint model for authentication, thus achieving the voiceprint recognition function.
[0003] In related technologies, voiceprint extraction programs can employ multiple processes to extract voiceprint data. These processes are typically Python processes. Since Python threads cannot fully utilize computing resources, multiple processes are necessary. Each process continuously processes voiceprint extraction requests and saves the extracted results to one of multiple voiceprint files. However, multiple processes simultaneously writing to the same voiceprint file can easily lead to data conflicts, and some voiceprint files may be written frequently while others remain unwritten for extended periods, making it difficult to guarantee the balance of voiceprint file writing.
[0004] Therefore, how to avoid write conflicts caused by multiple threads writing to the same voiceprint file at the same time, while ensuring the balance of voiceprint file writing, is an urgent technical problem to be solved. Summary of the Invention
[0005] In view of this, embodiments of the present invention provide a queue-based voiceprint data processing method, apparatus, and electronic device to avoid write conflicts caused by multiple threads writing to the same voiceprint file simultaneously, while ensuring the balance and efficiency of voiceprint file writing.
[0006] The technical solution of this invention is implemented as follows:
[0007] Firstly, a queue-based voiceprint data processing method is provided, the method comprising:
[0008] Create and maintain a shared queue, wherein the shared queue is a first-in-first-out queue with blocking functionality, used to store the voiceprint file identifier queue;
[0009] During the initialization phase, the voiceprint data writing process opens the voiceprint file to be written in append mode at once, caches the handle of the voiceprint file, and connects it to the shared queue.
[0010] When there is a write requirement, the voiceprint data writing process obtains the first file identifier from the shared queue and uses the handle of the first file represented by the first file identifier to write the first voiceprint data corresponding to the write requirement into the first file.
[0011] After the first voiceprint data is written, the first file identifier is returned to the shared queue.
[0012] Secondly, a queue-based voiceprint data processing device is provided, applied to a system with voiceprint data writing functionality, comprising:
[0013] The queue management module is used to create and maintain a shared queue, wherein the shared queue is a first-in-first-out queue with blocking function, and is used to store the voiceprint file identifier queue;
[0014] The process initialization module is used to open the voiceprint file to be written in append mode at once during the initialization phase of the voiceprint data writing process, cache the handle of the voiceprint file, and connect it to the shared queue.
[0015] Thirdly, an electronic device is provided, comprising: a processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the queue-based voiceprint data processing method provided in any of the above embodiments.
[0016] The technical solutions provided by the embodiments of the present invention have at least the following beneficial effects:
[0017] Since any voiceprint data writing process obtains the first file identifier from the shared queue in a first-in-first-out order when it needs to write voiceprint data, write conflicts caused by multiple processes writing to the same voiceprint file at the same time can be avoided. Attached Figure Description
[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0019] Figure 1 A flowchart illustrating a queue-based voiceprint data processing method provided in an embodiment of the present invention;
[0020] Figure 2 This is a schematic diagram of the structure of a voiceprint data writing device provided in an embodiment of the present invention;
[0021] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0022] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0024] It should also be noted that the information and data collected by this invention (e.g., voiceprint data) are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of this data all comply with the relevant laws, regulations, and standards of the relevant regions, and necessary confidentiality measures have been taken. This process does not violate public order and good morals, and corresponding operation entry points are provided for users to choose to authorize or refuse. For example, this system has an interface with relevant users or organizations. Before obtaining relevant information, a request to obtain the information needs to be sent to the aforementioned user or organization through the interface, and the relevant information is obtained only after receiving consent from the aforementioned user or organization.
[0025] In related technologies, voiceprint extraction programs can employ multiple processes to extract voiceprint data, typically Python processes. However, due to the Global Interpreter Lock (GIL) in Python, standard Python threads cannot truly utilize the computing resources of multi-core CPUs in parallel when performing computationally intensive tasks. This is because the GIL restricts only one thread to executing Python bytecode at a time. Therefore, for computationally intensive tasks such as voiceprint extraction, using multithreading does not significantly improve performance. Conversely, using multiple processes can bypass the GIL restriction, as each process has its own Python interpreter and memory space, allowing these processes to execute computational tasks in parallel, thus fully utilizing the computing resources of multi-core CPUs. However, multiple processes simultaneously writing to the same voiceprint file can easily lead to data conflicts, and some voiceprint files may be written frequently while others remain unwritten for extended periods, making it difficult to guarantee the balance of voiceprint file writing.
[0026] According to an embodiment of the present invention, an embodiment of a queue-based voiceprint data processing method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.
[0027] It should be noted that a voiceprint data writing device can serve as the execution subject of the queue-based voiceprint data processing method in this embodiment of the invention. This voiceprint data writing device can be deployed on an electronic device with voiceprint data processing capabilities, such as an electronic device in a voiceprint recognition system or a server.
[0028] The queue-based voiceprint data processing method provided in this invention can be applied to scenarios such as identity authentication based on voiceprint recognition or speech recognition.
[0029] Figure 1 This is a flowchart of a queue-based voiceprint data processing method according to an embodiment of the present invention. (Refer to...) Figure 1 As shown, this queue-based voiceprint data processing method can be used for data writing in Python threads. The method includes the following steps:
[0030] Step 101: Create and maintain a shared queue, wherein the shared queue is a first-in-first-out queue with blocking functionality, used to store the voiceprint file identifier queue.
[0031] The queue-based voiceprint data processing method provided in this invention is applied to an electronic device that supports multiple processes, such as multiple Python processes. For example, the electronic device supports a queue process and multiple voiceprint data writing processes.
[0032] Depending on the entity creating and maintaining the shared queue, step 101 may include: the operating system's thread scheduler creating a queue process and using the queue process to create and maintain the shared queue; or, the operating system's thread scheduler creating and maintaining the shared queue.
[0033] For example, (i) a shared queue is created and maintained based on a queue process. In a specific voiceprint data processing system, when the operating system's thread scheduler starts, it creates a dedicated queue process. For instance, on a voiceprint recognition server based on a Linux system, the thread scheduler creates a new process as a queue process through a system call. This queue process is tasked with creating and maintaining the shared queue.
[0034] The queue process utilizes system-provided inter-process communication mechanisms, such as shared memory and semaphores, to create a first-in, first-out (FIFO) shared queue with blocking capabilities. Shared memory stores the voiceprint file identifier queue, while semaphores ensure thread-safe access to the shared queue. The queue process continuously monitors for new voiceprint file identifiers to be added to the queue and for processes requesting identifiers from the queue. When a new voiceprint file is created, the queue process adds its identifier to the shared queue; when a process writing voiceprint data requests a file identifier, the queue process provides the identifier according to the FIFO principle.
[0035] In some cases, the queue process is a separate process, which may be referred to as the "queue process". The queue process also provides an interface for cross-process calls so that the voiceprint data writing process can access the shared queue.
[0036] (ii) The operating system's thread scheduler directly creates and maintains the shared queue. Assuming it's a Windows system's voiceprint data acquisition device, the operating system's thread scheduler is directly responsible for creating and maintaining the shared queue. The thread scheduler uses kernel objects provided by the Windows system, such as events and critical sections, to implement the first-in-first-out (FIFO) and blocking functions of the shared queue.
[0037] After creating the shared queue, the thread scheduler internally maintains a queue data structure to store voiceprint file identifiers. When a voiceprint data writing process starts and connects to the shared queue, the thread scheduler notifies the writing process of changes in the queue state via events. For example, when a new file identifier is added to the shared queue, the thread scheduler triggers a corresponding event, informing waiting writing processes that they can acquire the identifier; when the queue becomes empty, the thread scheduler triggers the event again, causing processes attempting to acquire identifiers to enter a blocked state until a new identifier is added.
[0038] In some embodiments, the queue process or the operating system thread scheduler loads the shared queue and initializes the voiceprint files. The number of voiceprint files can be specified by the process's startup parameters. These startup parameters include the number of voiceprint files, and may also include at least one of the following: filename pattern, type of data to be written, upper limit of file size, maximum number of writes to the file, etc.
[0039] In this embodiment, the shared queue, also known as the "voiceprint file queue," is used to store the file identifiers of each voiceprint file. The shared queue operates on a first-in, first-out (FIFO) basis. The file identifiers are arranged in the shared queue according to the order in which they were written, and the file identifier that entered the queue earliest will be processed first.
[0040] In this embodiment, the voiceprint data writing process can be referred to as the "voiceprint extraction process." There are multiple voiceprint data writing processes, which can concurrently process different audio files, generate voiceprint data, and write the voiceprint data into the voiceprint file. It is understood that the time required for each voiceprint data writing process to process one audio file is related to the size of the audio file, the complexity of the audio, and the system resources available at the time.
[0041] In some examples, the voiceprint data writing process can use voiceprint extraction algorithms (such as Mel-frequency cepstral coefficients, MFCC) to extract voiceprint feature data from audio files and write the extracted voiceprint feature data into the voiceprint file in an appropriate format. Multiple voiceprint data writing processes can process multiple audio files in parallel to extract voiceprint feature data.
[0042] When the voiceprint data writing process starts, it connects to the shared queue at the same time, so that the voiceprint data writing process can read the file identifiers written by other processes (such as the queue process) from the shared queue.
[0043] Here, there is no direct correspondence between multiple voiceprint data writing processes and multiple voiceprint files. The number of voiceprint files can be the same as or different from the number of voiceprint data writing processes. For example, the number of voiceprint files can be more or less than the number of voiceprint data writing processes.
[0044] In this embodiment, the amount of data extracted by each voiceprint data writing process is the same.
[0045] In some examples, before performing step 101, the queue process can pre-create multiple voiceprint files to be written with voiceprint data based on the number of files, and obtain the file identifiers of all voiceprint files.
[0046] Here, the file identifier can be used to uniquely identify a voiceprint file. This file identifier can be the file name or the file's storage path, which indicates a specific storage location for each voiceprint file in the storage system. It is understood that the file identifier can be any information that can uniquely identify a voiceprint file; no specific limitations are imposed here.
[0047] In some examples, the size difference between multiple voiceprint files is less than a preset difference value. Here, the sizes of different voiceprint files are balanced, which facilitates the subsequent allocation of voiceprint recognition tasks according to the voiceprint files.
[0048] In step 101 above, the queue process can sequentially write the file identifiers of multiple voiceprint files into a shared queue.
[0049] Step 102: During the initialization phase, the voiceprint data writing process opens the voiceprint file to be written in append mode at once, caches the handle of the voiceprint file, and connects it to the shared queue.
[0050] Step 102 can be the preparation stage or initialization stage for writing voiceprint data.
[0051] Step 103: When there is a write requirement, the voiceprint data writing process obtains the first file identifier from the shared queue, and uses the handle of the first file represented by the first file identifier to write the first voiceprint data corresponding to the write requirement into the first file.
[0052] In this embodiment, when any voiceprint data writing process needs to write voiceprint data, it will take the first file identifier at the head of the shared queue as the first file identifier.
[0053] After any voiceprint data writing process retrieves a file identifier from the voiceprint file queue, it finds the corresponding voiceprint file handle based on the file identifier, and then writes the voiceprint data into the corresponding voiceprint file through the voiceprint file handle.
[0054] Here, the file handle is used as an index value to point to the record table of open files for each process in the kernel.
[0055] The removal of any file identifier from the shared queue indicates that the file identifier has been removed from the shared queue.
[0056] When the first file identifier is rewritten into the shared queue by the voiceprint data writing process, the first file identifier will be written to the end of the shared queue, that is, it will become the last file identifier in the shared queue.
[0057] In this embodiment, only one process can retrieve or write a file identifier from the shared queue at any given time.
[0058] For example, in Python's Multiprocessing module, the Queue class can ensure that only one process can successfully operate on the queue when multiple processes attempt to access it simultaneously, by using locks maintained internally. The Queue class internally uses locks to ensure that every operation on the queue (such as put, get, etc.) is atomic, meaning it cannot be interrupted.
[0059] When a process attempts to perform a `put` (adding a file identifier to the queue) or `get` (removing a file identifier from the queue) operation on a queue, it first attempts to acquire the lock within the queue. If the lock is available (i.e., not held by another process), the process successfully acquires the lock and continues its queue operation. At this point, other processes attempting to acquire the same lock are blocked until the lock is released. Once the queue operation is complete (e.g., a file identifier has been successfully added to or removed from the queue), the process holding the lock releases it. Then, one of the blocked processes can attempt to acquire the lock again and continue its operation. In other words, while one process is operating on a queue, other processes must wait until that operation is complete.
[0060] This invention provides a queue-based voiceprint data processing method. Since any voiceprint data writing process obtains the first file identifier from the shared queue in a first-in-first-out order when it needs to write voiceprint data, it can avoid writing conflicts caused by multiple processes writing to the same voiceprint file at the same time.
[0061] Step 104: After completing the writing of the first voiceprint data, put the first file identifier back into the shared queue.
[0062] After writing the voiceprint data to the first file, the voiceprint data writing process rewrites the first file identifier into the shared queue. In this way, the voiceprint data writing process or other voiceprint data writing processes can continue to write voiceprint data to the voiceprint file corresponding to the file identifier when needed, thus realizing the sharing of voiceprint file writing.
[0063] Since the amount of voiceprint data written each time in each voiceprint data writing process is a preset amount, this combination of voiceprint file writing sharing and by controlling the amount of data written to each voiceprint file can effectively avoid the situation where some voiceprint files are written frequently while others are not written for a long time, thereby achieving balanced voiceprint file writing.
[0064] Multiple voiceprint data writing processes share access to a shared queue. After retrieving different file identifiers from the shared queue, voiceprint data can be written in parallel to the voiceprint files corresponding to each file identifier. This can make full use of computing resources and improve the efficiency of voiceprint data writing.
[0065] For example, the voiceprint data writing process and the queue process can communicate with each other through inter-process communication to inform the queue process that the first file identifier has been written. This allows the queue process to write the first file identifier back into the shared queue.
[0066] For example, when the voiceprint data writing process informs the thread scheduler that the first file identifier has been written, the thread scheduler can write the first file identifier back into the shared queue.
[0067] In one embodiment, the shared queue has a blocking function, and step 102a above further includes:
[0068] When the shared queue is empty, the operation of the voiceprint data writing process to obtain the file identifier from the shared queue will be blocked until a file identifier is added to the shared queue, at which point the first file identifier will be retrieved from the shared queue.
[0069] For example, when a shared queue is empty, if a process tries to retrieve an element from the queue, it will be placed in a blocked state (also known as a waiting state) because there is no file identifier in the queue. In this state, the process will not continue to perform subsequent operations, but will wait until there is a file identifier in the shared queue, at which point it can continue to perform the operation of retrieving the first file identifier from the shared queue.
[0070] In some embodiments, the method may further include: setting a queue buffer.
[0071] When a queue buffer is configured, the voiceprint data writing process prioritizes reading the first file identifier from the queue buffer. The voiceprint data then reads the first file identifier from the queue buffer and deletes the first file identifier that has been read from the queue buffer.
[0072] Correspondingly, maintaining the shared queue includes: reading voiceprint file identifiers from the shared queue and writing them sequentially into the queue buffer. A queue buffer is set up to temporarily store voiceprint file identifiers. When a voiceprint data writing process requests an identifier, it is preferentially retrieved from the queue buffer. If the queue buffer is empty, it is retrieved from the queue and used to replenish the shared buffer.
[0073] Compared to the voiceprint data writing process directly reading the first file identifier from the shared queue, the voiceprint data writing process reads the first file identifier from the queue buffer faster, thus further improving the voiceprint data writing speed.
[0074] In some embodiments, the method further includes: managing the queue buffer according to a preset mechanism, such that the order in which the voiceprint data writing process reads the voiceprint file identifier from the queue buffer is the same as the order in which it reads the voiceprint file identifier from the shared queue. That is, if one or more voiceprint data writing processes read the voiceprint file identifier from the shared queue in a first order, then the order in which the one or more voiceprint data writing processes read the voiceprint file identifier from the queue buffer is also the first order. In other words, the order in which the voiceprint data file identifier is written to the shared queue is the first order.
[0075] Specifically, the voiceprint data writing process can also employ a first-in, first-out (FIFO) approach to read the voiceprint file identifier from the queue buffer. In some embodiments, to ensure that each file is written sequentially even when the voiceprint data writing process reads the file identifier from the queue buffer, a preset mechanism is set. This preset mechanism includes, but is not limited to, at least one of the following:
[0076] 1. Ordered Filling Strategy for the Queue Buffer: Batch Ordered Filling: When filling the queue buffer with voiceprint file identifiers from the shared queue, a batch ordered approach is adopted. For example, a fixed number (e.g., 3 or 10) of file identifiers are read continuously from the shared queue and placed into the queue buffer in the order they appear in the shared queue. This ensures that the file identifiers in the queue buffer are ordered within a certain range. The number of voiceprint file identifiers that can be written to the queue buffer can be fixed, for example, 3 or 5. In some embodiments, the number of voiceprint file identifiers that can be written to the queue buffer is related to the number of voiceprint data writing processes. For example, the number of voiceprint file identifiers that can be written to the queue buffer is positively correlated with the number of voiceprint data writing processes. The number m of voiceprint file identifiers that can be written to the queue buffer is equal to the number n + S of voiceprint data writing processes. S is a positive integer less than or equal to 3.
[0077] 2. Reading Rules: Sequential Reading. When the voiceprint data writing process reads file identifiers from the queue buffer, it strictly follows the order of file identifiers in the queue buffer. A queue (such as collections.deque in Python) can be used to implement the queue buffer, ensuring a first-in, first-out reading order.
[0078] 3. Add a sequential number to each file identifier. When the write process reads a file identifier, it also records the highest number that has been read. When the queue buffer needs to be refilled, it finds a file identifier with a higher number than the highest number in the shared queue and starts filling from there, ensuring the continuity of the order.
[0079] 4. Establish a synchronization mechanism between the shared queue and the queue buffer, including state synchronization: Establish a state synchronization mechanism between the shared queue and the queue buffer. When the order of file identifiers in the shared queue changes (e.g., an identifier is added or deleted), the queue buffer is promptly notified to make corresponding adjustments. Synchronization can be achieved through message passing or event triggering.
[0080] 5. Data consistency checks: Regularly or irregularly perform consistency checks on file identifiers in the shared queue and queue buffer. If inconsistencies are found, such as the presence of an identifier in the queue buffer that does not exist in the shared queue, or mismatched order, correct them promptly. A hash table can be used to record the status of each file identifier for easy and rapid checking.
[0081] 6. File Identifier Return: If an exception occurs during the write process, causing a file identifier to become unusable, the write process needs to return the identifier to its correct position in the queue buffer to ensure correct order. The file identifier can be inserted into the appropriate position in the queue buffer based on its sequential number.
[0082] In one embodiment, prior to the step of obtaining the first file identifier from the shared queue in step 102a above, the method further includes:
[0083] When the voiceprint data writing process is initialized, the file name of each voiceprint file is obtained, each voiceprint file is opened in append mode according to its file name, and the file handle of each opened voiceprint file is saved.
[0084] In this embodiment, each voiceprint data writing process can perform the following operations during initialization: obtain the number of voiceprint files according to the process startup parameters, generate a list containing all voiceprint filenames based on the number of voiceprint files and predetermined rules, and iterate through the filename list. For each filename encountered, open the corresponding voiceprint file in append write mode and save the file handle of the opened voiceprint file to a data structure (such as a dictionary). The predetermined rules may include information such as filename prefixes and directories.
[0085] For each voiceprint file, the operating system allocates a file handle to it and stores the file handle as an index in a record table. The record table contains various information about the voiceprint file, such as the file's location.
[0086] During the voiceprint data writing process, when there is voiceprint data to be written, the data is appended to the corresponding voiceprint file through the file handle.
[0087] In this embodiment, by opening the voiceprint file in append mode and saving the file handle, the voiceprint data writing process can quickly locate the voiceprint file when voiceprint data needs to be written, without having to reopen the file each time. This also avoids the voiceprint file being frequently closed or opened in various processes, thus further improving the writing efficiency of the voiceprint file.
[0088] In some embodiments, the method further includes: setting a data buffer for each voiceprint data writing process, the data buffer being used to temporarily store voiceprint data to be written;
[0089] Writing the first voiceprint data corresponding to the writing requirement to the first file includes: writing the first voiceprint data to the data buffer; when the amount of data in the data buffer reaches a set threshold, writing the first voiceprint data to the first file in batches.
[0090] For example, the threshold setting can be determined during the initialization of the voiceprint data writing process, can be determined automatically based on historical processing, or can be determined based on user instructions.
[0091] First, write the first voiceprint data to the data buffer, and then write it to the first file in batches, which can further improve the writing speed.
[0092] In other embodiments, the data cache includes a first cache and a second cache; writing the first voiceprint data into the data cache includes: writing the first voiceprint data into the first cache; when the data volume in the first cache reaches a set threshold, writing the first voiceprint data into the second cache; when the data volume in the data cache reaches the set threshold, batch writing the first voiceprint data into the first file includes: when the data volume in the second cache reaches a preset threshold, batch writing the first voiceprint data in the second cache into the first file.
[0093] The first and second buffers constitute a dual-buffering mechanism for data writing. In this way, one buffer is used for receiving external voiceprint data, and the other buffer is used for writing voiceprint data to the voiceprint file, achieving seamless connection of voiceprint data writing.
[0094] In some embodiments, the method further includes:
[0095] The size of the data buffer is dynamically adjusted according to the operation rate of each voiceprint data writing process.
[0096] Dynamically adjusting the size of the data buffer based on the operation scaling ratio can reduce cache waste caused by setting the data buffer too large, and can also reduce the reduction in voiceprint data write speed caused by setting the data buffer too small.
[0097] Method 1: Dynamic adjustment based on fixed time intervals
[0098] Monitoring frequency setting: The system is set to a fixed time interval, such as every 10 or 20 seconds, to monitor the operation rate of the voiceprint data writing process. Within this time interval, the number of times the voiceprint data is acquired and written is recorded, which serves as a measure of the operation rate.
[0099] Adjustment Strategy: If, within a fixed time interval, the amount of data acquired and written by a certain voiceprint data writing process exceeds the first threshold, such as 100 records (this threshold can be adjusted according to actual conditions), then its operation rate is determined to be relatively fast. The size of the data buffer is increased. This increase can be proportional, such as increasing the capacity by 20%, or a fixed amount, or a positive correlation between the monitored write rate and the actual write rate. If the amount of data acquired and written is less than the second threshold, such as 50 records (this threshold can be adjusted according to actual conditions), then its operation rate is determined to be relatively slow. The size of the data buffer is decreased. This decrease can be proportional, such as reducing the capacity by 10%, or a fixed amount, or a positive correlation between the monitored write rate and the actual write rate.
[0100] Method 2: Dynamic adjustment based on data processing volume
[0101] Data processing volume statistics: The system provides real-time statistics on the amount of voiceprint data processed during the voiceprint data writing process. For example, the processing volume is recorded once every 100KB of data processed (this threshold can be adjusted according to actual conditions).
[0102] Timing adjustment for determining the operation rate: When the cumulative processing volume reaches a specified value (e.g., 1MB), the operation rate is determined based on the average time taken to process data during this period. If the average processing time for 100KB of data is less than 5 seconds, the operation rate is considered fast; if it is greater than 10 seconds, the operation rate is considered slow.
[0103] Buffer size adjustment: If the operation speed is fast, double the capacity of the data buffer; if the operation speed is slow, halve the capacity of the data buffer, but never less than the initial minimum value.
[0104] Method 3: Dynamic adjustment based on system load
[0105] Operating system load monitoring: The system continuously monitors the usage of system resources such as CPU and memory, using this as an indicator of system load. For example, when CPU utilization exceeds 80% and memory utilization exceeds 70%, the system load is considered high; when CPU utilization is below 30% and memory utilization is below 40%, the system load is considered low.
[0106] Adjusting based on operation rate: For the voiceprint data writing process, under high system load, if the operation rate is fast (measured by the amount of data processed per unit time), appropriately reduce the size of the data buffer to avoid excessive consumption of system resources. For example, reduce the data buffer size by 30% to free up memory for other processes. Under low system load, if the operation rate is slow, appropriately increase the size of the data buffer to improve the efficiency of the writing process. For example, increase the data buffer size by 40%.
[0107] In one embodiment, prior to the step 102a above, where the voiceprint data is written into the first file corresponding to the first file identifier, the method further includes:
[0108] The voiceprint data writing process determines whether to write the voiceprint data into the first file based on whether the current state of the first file meets the first threshold condition.
[0109] For example, the current state of the first file may include: the total number of times data has been written to the first file (i.e., the total number of times it has been written); the first threshold condition may include: a write count threshold, which limits the number of times the voiceprint file is written to prevent an excessive number of writes from causing the amount of data written to the voiceprint file to be too large.
[0110] For example, suppose the voiceprint data writing process determines whether to write voiceprint data to the first file file_A based on whether the value obtained by adding one to the number of times the first file file_A has been written is less than or equal to the preset maximum number of writes (e.g., 200 times). Alternatively, the voiceprint data writing process determines whether to write voiceprint data to the first file based on whether the value obtained by adding a preset data size (e.g., 5MB) to the amount of data already written to the first file file_A is less than or equal to the preset maximum write data size (e.g., 150MB).
[0111] Before writing voiceprint data to a voiceprint file, the voiceprint data writing process determines the current state of the voiceprint file and compares it with a first threshold condition. For example, if the first threshold condition is met, the write operation to write voiceprint data to the voiceprint file is performed; if the first threshold condition is not met, the write operation to write voiceprint data to the voiceprint file is not performed, and the file identifiers of other voiceprint files are retrieved from the shared queue.
[0112] In this embodiment, before the voiceprint data writing process writes the voiceprint data to the voiceprint file, it first determines whether to write the voiceprint data based on whether the current state of the voiceprint file meets the first threshold condition. This can limit the number of times the voiceprint file is written, so as to prevent the amount of data written to the voiceprint file from being too large due to excessive number of writes, which helps to improve the writing balance of the voiceprint file.
[0113] In one embodiment, the voiceprint data writing process, in the above steps, determines whether to write the voiceprint data into the first file based on whether the current state of the first file meets the first threshold condition, including:
[0114] The voiceprint data writing process determines whether to write the voiceprint data to the first file based on whether the value obtained by adding one to the number of times the first file has been written is less than or equal to the preset maximum number of writes.
[0115] Alternatively, the voiceprint data writing process determines whether to write voiceprint data into the first file based on whether the value obtained by adding the amount of data already written to the first file to the preset amount of data is less than or equal to the preset maximum amount of data to be written.
[0116] For example, when the value obtained by adding one to the number of times the voiceprint data is written to the first file is less than or equal to the preset maximum number of writes, the voiceprint data writing process performs the operation of writing voiceprint data to the first file.
[0117] If the value obtained by adding one to the number of times the voiceprint data has been written to the first file is greater than the preset maximum number of writes, the operation of writing the voiceprint data to the first file will not be performed.
[0118] For example, when the value obtained by adding the amount of data already written to the first file to the preset amount of data is less than or equal to the preset maximum amount of data to be written, the voiceprint data writing process performs the operation of writing voiceprint data to the first file.
[0119] If the value obtained by adding the amount of data already written to the first file to the preset amount of data exceeds the preset maximum amount of data to be written, the operation of writing voiceprint data to the first file will not be performed.
[0120] In this embodiment, when determining whether to write voiceprint data to the first file, the voiceprint data writing process will consider the number of times the file has been written, or the amount of data already written to the file and the preset amount of data. This can effectively alleviate the problem of the voiceprint file being frequently written to and the amount of data already written being too large, thereby ensuring the balance of voiceprint file data writing.
[0121] In one embodiment, prior to the step of rewriting the first file identifier into the shared queue in step 102b above, the method further includes:
[0122] The voiceprint data writing process determines whether to delay writing the first file identifier to the shared queue based on whether the current state of the first file meets the second threshold condition.
[0123] For example, the current state of the first file may include the difference between the number of times the first file has been written and the average number of times all voiceprint files have been written; the second threshold condition may include the difference threshold.
[0124] For example, suppose the voiceprint data writing process determines whether to delay writing the file identifier of the first file file_A to the shared queue based on whether the difference between the number of times the first file file_A has been written and the average number of times it has been written is greater than a first preset difference (e.g., 20 times). Alternatively, the voiceprint data writing process determines whether to delay writing the file identifier of the first file file_A to the shared queue based on whether the difference between the amount of data already written to the first file file_A and the average amount of data is greater than a second preset difference (e.g., 30MB).
[0125] When the current state of the first file does not meet the second threshold condition, the voiceprint data writing process delays the write operation. For example, after a preset delay, the operation of writing the first file identifier into the shared queue is performed. This preset delay can be positively correlated with the difference.
[0126] In this embodiment, before the voiceprint data writing process rewrites the retrieved file identifier into the shared queue, it first determines whether to delay writing the file identifier based on whether the current state of the voiceprint file meets the second threshold condition. This can limit the timing of the file identifier being rewritten. By adding the file identifier back to the shared queue at the appropriate time, the occurrence of frequent writing of the voiceprint file can be reduced, thereby helping to improve the balance of voiceprint file writing data.
[0127] In one embodiment, in the above steps, the voiceprint data writing process determines whether to delay writing the first file identifier to the shared queue based on whether the current state of the first file meets the second threshold condition, including:
[0128] The voiceprint data writing process determines whether to delay writing the first file identifier to the shared queue based on whether the difference between the number of times the first file has been written and the average number of times it has been written is greater than a first preset difference; the average number of times it has been written is the ratio of the total number of times all voiceprint files have been written to the total number of voiceprint files.
[0129] Alternatively, the voiceprint data writing process determines whether to delay writing the first file identifier to the shared queue based on whether the difference between the amount of data already written to the first file and the average amount of data is greater than a second preset difference; the average amount of data is the ratio of the total amount of data already written to all voiceprint files to the total number of voiceprint files.
[0130] For example, when the difference between the number of times the first file is written and the average number of times it is written is greater than a first preset difference, the first file identifier is delayed in being written to the shared queue.
[0131] When the difference between the number of times the first file is written and the average number of times it is written is less than or equal to a first preset difference, the first file identifier is directly written to the shared queue.
[0132] For example, when the difference between the amount of data already written in the first file and the average amount of data is greater than a second preset difference, the voiceprint data writing process delays writing the first file identifier to the shared queue.
[0133] When the difference between the amount of data already written in the first file and the average amount of data is less than or equal to a second preset difference, the voiceprint data writing process directly writes the first file identifier to the shared queue.
[0134] In this embodiment, when the voiceprint data writing process completes writing the voiceprint data to the first file, it determines whether to delay writing the first file identifier to the shared queue based on the number of times the first file has been written and the average number of times all voiceprint files have been written, or based on the amount of data already written to the file and the average amount of data already written to all voiceprint files. This can effectively alleviate the problem of voiceprint files being written to frequently and the amount of data already written being too large, thereby ensuring the balance of voiceprint file writing data.
[0135] In one embodiment, the method further includes:
[0136] If the voiceprint data writing process determines that it will not write the voiceprint data to the first file, it will not perform the operation of rewriting the first file identifier to the shared queue, and will re-execute steps 102a to 102b above.
[0137] In this embodiment, when the value obtained by adding one to the number of times the first file has been written is greater than the preset maximum number of writes, or when the value obtained by adding the amount of data already written to the first file to the preset amount of data is greater than the preset maximum amount of data to be written, the voiceprint data writing process will no longer perform the operation of rewriting the first file identifier to the shared queue. In this way, by stopping the operation of rewriting the file identifier of the first file to the shared queue after the first file exceeds the limit, it can be ensured that the first file will not cause data overflow due to overwriting. In addition, it can also avoid the situation where other processes cannot write voiceprint data to the first file normally because the first file identifier is rewritten to the shared queue, ensuring that each process can perform voiceprint data writing operations efficiently.
[0138] In one embodiment, the method further includes:
[0139] The queue process transmits the status information of each voiceprint file, which is periodically monitored, to each voiceprint data writing process through an inter-process communication mechanism; the status information of the voiceprint file includes the number of times the voiceprint file has been written and / or the amount of data that has been written.
[0140] In this embodiment, the queue process can monitor the status of each voiceprint file at preset time intervals to obtain the latest status information of each voiceprint file, and transmit the monitored latest status information to each voiceprint data writing process through an inter-process communication mechanism (e.g., shared memory).
[0141] The status information of each voiceprint file includes: the number of times the voiceprint file has been written and / or the amount of data that has been written. The number of times the voiceprint file has been written reflects the frequency of write operations on the voiceprint file, while the amount of data that has been written reflects the current size and remaining capacity of the voiceprint file.
[0142] In this embodiment, after receiving the status information of each voiceprint file, the voiceprint data writing process can determine whether to continue the writing operation based on a preset threshold condition, thereby reducing the occurrence of data overflow or uneven data writing.
[0143] In some embodiments, a queue-based voiceprint data processing method is provided, which may include the following steps:
[0144] A shared queue is provided through a separate process, which stores unused voiceprint file identifiers;
[0145] When the voiceprint extraction process needs to write voiceprint information, it calls the queue through the port to obtain a voiceprint file identifier, writes the voiceprint information into the voiceprint file corresponding to the identifier, and then pushes the identifier back into the shared queue after writing.
[0146] In this embodiment, the voiceprint files are pre-opened in each voiceprint extraction process, and it is necessary to ensure that they are written to disk promptly after being written. This way, although the efficiency of voiceprint extraction in each process may not be the same, the amount of data ultimately written to each voiceprint file is basically balanced.
[0147] In this embodiment, there is only one queue process (i.e., the queue process in the previous embodiment), which is responsible for creating the voiceprint file to be written, creating a shared queue, providing an interface for cross-process calls, and pushing the identifier of the file to be written into the queue. The queue process starts before the voiceprint extraction process, the queue is of the first-in-first-out type, and has a blocking function.
[0148] There are multiple voiceprint extraction processes (i.e., the voiceprint data writing processes in the aforementioned embodiments). During initialization, each process obtains a list of files to be written based on the startup parameters, opens these files in append mode and saves the file handles, while simultaneously connecting to the queue of the queue process. When the voiceprint extraction process needs to write voiceprint data, it first obtains a voiceprint file identifier from the aforementioned voiceprint file queue, finds the corresponding voiceprint file based on the identifier, writes the voiceprint data to that file, and finally pushes the voiceprint file identifier that was just used back into the voiceprint file queue.
[0149] Only one process can open a specific voiceprint file at a time, ensuring no write conflicts occur. The sizes of different voiceprint files remain relatively consistent after multiple writes (e.g., 10,000 times).
[0150] In summary, the technical solution provided in this embodiment can solve the problems of write conflicts, low write efficiency, and unbalanced write data when multiple processes (such as Python processes) write to multiple voiceprint files at the same time, and realizes high-efficiency and balanced sharing of multiple processes writing to multiple voiceprint files.
[0151] In some embodiments, the method further includes:
[0152] Get the write status of the second file pointed to by the second file identifier;
[0153] If the writing status of the second file is abnormal, an error alert will pop up.
[0154] If the writing status of the second file is normal, the second file identifier is pushed back into the shared queue.
[0155] For example, write status can indicate the number of writes to the second file, write frequency, write rate, and write duration. When the write status indicates prolonged write failures, excessively frequent writes, or excessively slow write rates—meaning these write parameters are outside the normal write baseline range—anomaly diagnosis is performed. Anomaly alerts are then issued based on the diagnosed anomalies.
[0156] Possible anomalies include, but are not limited to, at least one of the following:
[0157] Insufficient disk space: If an IOError is caught when attempting to write to a file and the error code corresponds to insufficient disk space, it is considered an exception.
[0158] File permission issues: If you are unable to write to a file due to insufficient permissions, a PermissionError exception will be caught.
[0159] Network interruption: If the request to write data to a server file via an AJAX request fails with a status code of 0, it may be due to a network interruption.
[0160] Server Error: If the server returns a status code of 500 or other error status codes, it indicates that there was a problem when writing files on the server side.
[0161] External storage unavailable: When attempting to write to external storage, check if the external storage status is MEDIA_MOUNTED; if not, it is considered an exception.
[0162] File in use: If a FileNotFoundException is caught when opening a file for writing and it is caused by the file being used by another process, it is considered an exception.
[0163] File system corruption: If an OSError is captured while writing a file and the error message indicates a file system-related problem, it is considered an exception.
[0164] Concurrent write conflict: If a file is written in a multi-threaded or multi-process environment and a ResourceWarning or other concurrency-related exception is caught, it is considered an exception.
[0165] Because this disclosure embodiment performs targeted processing for concurrent writes, especially when a file is detected to be occupied and / or concurrent write conflicts are detected, abnormal events are recorded and the maintenance of the current shared queue is diagnosed as normal. If it is not normal, the shared queue is reset.
[0166] The abnormal notifications in this disclosure include, but are not limited to, immediate notifications. A notification is understood to be one that the user can immediately perceive, such as a pop-up notification and / or a message notification, and may specifically include, but is not limited to, at least one of the following:
[0167] Modal pop-ups: In desktop, web, or mobile applications, modal pop-ups force user interaction, preventing the user from operating other interface elements until the pop-up information is processed. For example, in Windows systems, a "Program Not Responding" pop-up appears when a program error occurs; the user must choose "Close Program" or "Wait" to continue operating the computer. Similarly, when making a payment on a webpage, if the payment fails, a "Payment Failed, Please Check Payment Information" pop-up appears; only after clicking "OK" can the user return to modify payment information or retry the payment.
[0168] Non-modal pop-ups: These do not block user interaction with other interfaces and are typically used to display non-urgent information. Examples include update prompts in some software, allowing users to choose to update later and continue using the software; and lyrics pop-ups in music playback software that appear when switching songs, without affecting user control over playback progress, volume, etc. The choice between modal and non-modal pop-ups depends on the type of exception. For instance, a serious exception might prevent the writing of voiceprint data. Modal pop-ups are suitable for file corruption or insufficient disk space, while non-modal pop-ups are better for network interruptions or server errors. In short, non-modal pop-ups are used for exceptions that can be resolved by retries, while non-modal pop-ups are used for exceptions that cannot be resolved by retries.
[0169] System messages: The operating system's built-in message notification mechanism. For example, on Android phones, when the phone's battery is low, the system will display a low battery reminder in the notification bar; in Windows systems, when the network connection is abnormal, a network connection disconnection prompt icon and message will pop up in the taskbar.
[0170] In-app messages: Messages sent by the application itself. In social media apps, when a new message, friend request, or system notification (such as an account login notification from a different location) is received, it will be displayed in the app's message center or via a pop-up window; e-commerce apps will push notifications to users when product prices drop or order status changes (such as shipped or refunded).
[0171] Figure 2This is a schematic diagram of a voiceprint data writing device according to an embodiment of the present invention. The device is applied to an electronic device having a queue process and multiple voiceprint data writing processes.
[0172] Reference Figure 2 As shown, the voiceprint data writing device 200 includes:
[0173] The queue management module 210 is used to create and maintain a shared queue, wherein the shared queue is a first-in-first-out queue with blocking function, and is used to store the voiceprint file identifier queue;
[0174] The process initialization module 220 is used to open the voiceprint file to be written in append mode at once during the initialization phase of the voiceprint data writing process, cache the handle of the voiceprint file, and connect it to the shared queue.
[0175] The writing module 230 is used to, when there is a writing requirement, the voiceprint data writing process obtains a first file identifier from the shared queue and uses the handle of the first file represented by the first file identifier to write the first voiceprint data corresponding to the writing requirement into the first file.
[0176] The queue management module 210 is also used to put the first file identifier back into the shared queue after completing the writing of the first voiceprint data.
[0177] In one embodiment, the queue management module is used by the operating system's thread scheduler to create a queue process and use the queue process to create and maintain the shared queue; or, the operating system's thread scheduler creates and maintains the shared queue.
[0178] In one embodiment, the device further includes: a setting module, configured to set a data buffer for each voiceprint data writing process, the data buffer being used to temporarily store voiceprint data to be written;
[0179] Writing the first voiceprint data corresponding to the writing requirement to the first file includes: writing the first voiceprint data to the data buffer; when the amount of data in the data buffer reaches a set threshold, writing the first voiceprint data to the first file in batches.
[0180] In one embodiment, the data cache includes a first cache and a second cache; writing the first voiceprint data into the data cache includes: writing the first voiceprint data into the first cache; when the data volume in the first cache reaches a set threshold, writing the first voiceprint data into the second cache; when the data volume in the data cache reaches the set threshold, batch writing the first voiceprint data into the first file includes: when the data volume in the second cache reaches a preset threshold, batch writing the first voiceprint data in the second cache into the first file.
[0181] In one embodiment, the apparatus further includes an adjustment module for dynamically adjusting the size of the data buffer based on the operating rate of each of the voiceprint data writing processes.
[0182] In one embodiment, the setting module is further configured to set a queue buffer; the queue buffer is used to temporarily store voiceprint file identifiers. The voiceprint data writing process obtains the first file identifier from the shared queue, including: the voiceprint data reads the first file identifier from the queue buffer and deletes the first file identifier read from the queue buffer.
[0183] Maintaining the shared queue includes: reading voiceprint file identifiers from the shared queue and writing them sequentially into the queue cache.
[0184] In one embodiment, the queue management module is further configured to manage the queue buffer according to a preset mechanism, such that the order in which the voiceprint data writing process reads the voiceprint file identifier from the queue buffer is the same as the order in which it reads the voiceprint file identifier from the shared queue.
[0185] In one embodiment, the device further includes:
[0186] The acquisition module is used to obtain the write status of the second file pointed to by the second file identifier;
[0187] The pop-up module is used to display an error alert if the writing status of the second file is abnormal.
[0188] The queue management module is also used to push the second file identifier back into the shared queue if the writing status of the second file is normal.
[0189] In one embodiment, the writing module is further configured to determine whether to write voiceprint data to the first file based on whether the current state of the first file meets a first threshold condition. In one embodiment, the writing module is further configured to determine whether to write voiceprint data to the first file based on whether the value obtained by adding one to the number of times the first file has been written is less than or equal to a preset maximum number of writes. In one embodiment, the writing module is further configured to determine whether to write voiceprint data to the first file based on whether the value obtained by adding a preset amount of data to the amount of data already written to the first file is less than or equal to a preset maximum amount of data to be written. In one embodiment, the writing module is further configured to determine whether to delay writing the first file identifier to the shared queue based on whether the current state of the first file meets a second threshold condition. In one embodiment, the writing module is further configured to determine whether to delay writing the first file identifier to the shared queue based on whether the difference between the number of times the first file has been written and the average number of times it has been written is greater than a first preset difference; wherein the average number of times it has been written is the ratio of the total number of times all voiceprint files have been written to the total number of voiceprint files. In one embodiment, the writing module is further configured to determine whether to delay writing the first file identifier to the shared queue based on whether the difference between the amount of data already written to the first file and the average data amount is greater than a second preset difference; wherein the average data amount is the ratio of the total amount of data already written to all voiceprint files to the total number of voiceprint files. In one embodiment, the writing module is further configured to, if the voiceprint data writing process determines that it will not write the voiceprint data to the first file, not to perform the operation of rewriting the first file identifier to the shared queue, and to re-execute the operation steps. In one embodiment, the queue process module is further configured to transmit the status information of each voiceprint file periodically monitored to each voiceprint data writing process through an inter-process communication mechanism; wherein the status information of the voiceprint file includes the number of times the voiceprint file has been written and / or the amount of data already written.
[0190] It should be noted that the description of the above embodiments of the voiceprint data writing device is similar to the description of the above embodiments of the queue-based voiceprint data processing method, and the beneficial effects of the same method will not be repeated. For technical details not disclosed in the embodiments of the voiceprint data writing device of the present invention, please refer to the description of the embodiments of the queue-based voiceprint data processing method of the present invention.
[0191] Combination Figure 3As shown, this application embodiment provides an electronic device including a processor 10 and a memory 11. Optionally, the device may further include a communication interface 12 and a bus 9. The processor 10, communication interface 12, and memory 11 can communicate with each other via the bus 9. The communication interface 12 can be used for information transmission. The processor 10 can call logical instructions in the memory 11 to execute the queue-based voiceprint data processing method of the above embodiment.
[0192] Furthermore, the logical instructions in the aforementioned memory 11 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.
[0193] The memory 11, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this application. The processor 10 executes functional applications and data processing by running the program instructions / modules stored in the memory 11, that is, implementing the queue-based voiceprint data processing method in the above embodiments.
[0194] The memory 11 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 11 may include high-speed random access memory and may also include non-volatile memory.
[0195] This application provides a storage medium storing computer-executable instructions configured to execute the above-described queue-based voiceprint data processing method.
[0196] This application provides a computer program product, which includes a computer program stored on a storage medium. The computer program includes program instructions, which, when executed by a computer, cause the computer to perform the above-described queue-based voiceprint data processing method.
[0197] The aforementioned computer-readable storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.
[0198] The technical solutions of this application embodiment can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this application embodiment. The aforementioned storage medium can be a non-transitory storage medium, including various media capable of storing program code such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks, or it can be a transient storage medium.
[0199] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0200] The embodiments or examples disclosed in this application are not exhaustive, but merely illustrative of some embodiments or examples, and are not intended to limit the scope of protection of this disclosure. Unless contradictory, each step in a particular embodiment or example can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a particular embodiment or example can also be implemented as an independent embodiment, and the order of the steps in a particular embodiment or example can be arbitrarily interchanged. Furthermore, optional methods or examples in a particular embodiment or example can be arbitrarily combined; moreover, embodiments or examples can be arbitrarily combined. For example, some or all steps of different embodiments or examples can be arbitrarily combined, and a particular embodiment or example can be arbitrarily combined with optional methods or examples of other embodiments or examples.
[0201] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.
[0202] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0203] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0204] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.
[0205] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A queue-based voiceprint data processing method, characterized in that, include: A shared queue is created and maintained, wherein the shared queue is a first-in-first-out queue with a blocking function, used to store the voiceprint file identifier queue; the blocking function is as follows: when the shared queue is empty, the operation of the voiceprint data writing process to retrieve the file identifier from the shared queue will be blocked until a file identifier is added to the shared queue, at which point the first file identifier is retrieved from the shared queue; During the initialization phase, the voiceprint data writing process opens the voiceprint file containing the voiceprint data to be written in append mode all at once, caches the handle of the voiceprint file, and connects it to the shared queue. When there is a write requirement, the voiceprint data writing process obtains the first file identifier from the shared queue and uses the handle of the first file represented by the first file identifier to write the first voiceprint data corresponding to the write requirement into the first file. After the first voiceprint data is written, the first file identifier is returned to the shared queue.
2. The method according to claim 1, characterized in that, The creation and maintenance of the shared queue includes: The operating system's thread scheduler creates a queue process, which then creates and maintains the shared queue; or, The operating system's thread scheduler creates and maintains the shared queue.
3. The queue-based voiceprint data processing method according to claim 1 or 2, characterized in that, Also includes: A data buffer is set up for each voiceprint data writing process, and the data buffer is used to temporarily store the voiceprint data to be written; Write the first voiceprint data corresponding to the writing requirement into the first file, including: Write the first voiceprint data into the data cache area; When the amount of data in the data cache reaches a set threshold, the first voiceprint data is written to the first file in batches.
4. The method according to claim 3, characterized in that, The data cache includes a first cache and a second cache; writing the first voiceprint data into the data cache includes: writing the first voiceprint data into the first cache; when the data volume in the first cache reaches a set threshold, writing the first voiceprint data into the second cache; when the data volume in the data cache reaches the set threshold, batch writing the first voiceprint data into the first file includes: when the data volume in the second cache reaches a preset threshold, batch writing the first voiceprint data in the second cache into the first file.
5. The method according to claim 3, characterized in that, The method further includes: The size of the data buffer is dynamically adjusted according to the operation rate of each voiceprint data writing process.
6. The method according to claim 1 or 2, characterized in that, The method further includes: Set up a queue cache area; the queue cache area is used to temporarily store the voiceprint file identifier; The voiceprint data writing process obtains a first file identifier from the shared queue, including: the voiceprint data reads the first file identifier from the queue buffer and deletes the first file identifier read from the queue buffer; Maintaining the shared queue includes: The voiceprint file identifier is read from the shared queue and written sequentially to the queue cache.
7. The method according to claim 6, characterized in that, The method further includes: The queue buffer is managed according to a preset mechanism, such that the order in which the voiceprint data writing process reads the voiceprint file identifier from the queue buffer is the same as the order in which it reads the voiceprint file identifier from the shared queue.
8. The method according to claim 6, characterized in that, The method further includes: Get the write status of the second file pointed to by the second file identifier; If the writing status of the second file is abnormal, an error alert will pop up. If the writing status of the second file is normal, the second file identifier is pushed back into the shared queue.
9. A queue-based voiceprint data processing device, characterized in that, Applications include systems with voiceprint data writing capabilities, including: The queue management module is used to create and maintain a shared queue, wherein the shared queue is a first-in-first-out queue with a blocking function, used to store the voiceprint file identifier queue; the blocking function is as follows: when the shared queue is empty, the operation of the voiceprint data writing process to retrieve the file identifier from the shared queue will be blocked until a file identifier is added to the shared queue, at which point the first file identifier is retrieved from the shared queue; The process initialization module is used to open the voiceprint file to be written in append mode at once during the initialization phase of the voiceprint data writing process, cache the handle of the voiceprint file, and connect it to the shared queue. The writing module is used to, when there is a writing requirement, obtain a first file identifier from the shared queue, use the handle of the first file represented by the first file identifier to write the first voiceprint data corresponding to the writing requirement into the first file; after completing the writing of the first voiceprint data, put the first file identifier back into the shared queue.
10. An electronic device, characterized in that, include: A processor and a memory, the memory storing computer-executable instructions executable by the processor, the processor executing the computer-executable instructions to implement the queue-based voiceprint data processing method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Automatic driving image sending method and device, electronic equipment and computer medium
CN115589527A
Method and system for storing multiple concurrent large files
CN119396789A