Methods and equipment for capturing voice keywords and assisting in evidence retrieval in interrogation scenarios

Abstract

This invention provides a method and device for assisting in voice keyword capture and evidence retrieval in interrogation scenarios. The method includes: S1. System initialization and parameter configuration; S2. Voice signal acquisition and preprocessing; S3. Voice-to-text conversion and hot word enhancement; S4. Keyword recognition and semantic understanding; S5. Evidence database retrieval and multi-algorithm fusion matching; S6. Multi-evidence fuzzy matching processing; S7. Automatic evidence display. This invention improves the accuracy of voice transcription and reduces the keyword omission rate; shortens the multi-evidence screening time, automatically adjusts the evidence similarity after judge confirmation, and improves the accuracy of subsequent similar queries; it also improves the accuracy of voice-controlled case file page turning command recognition and reduces the number of court interruptions.

Description

Technical Field

[0001] This invention relates to the field of speech and text processing technology, and in particular to a method and device for capturing speech keywords and assisting in evidence retrieval in interrogation scenarios. Background Technology

[0002] Currently, the following technical bottlenecks exist in voice keyword capture and evidence retrieval during interrogation:

[0003] Inefficient speech processing: Traditional solutions rely on court clerks to manually record court proceedings, which can lead to problems such as missing keywords (e.g., the legal term "intentional homicide" is mistakenly recorded as "intentional injury") and recording delays (missed when the speech is too fast).

[0004] Semantic ambiguity in evidence retrieval: Existing retrieval systems are mostly based on keyword word form matching (such as matching only the text of "dagger"), lacking semantic understanding capabilities and unable to associate the semantic equivalence between "murder weapon" and "dagger".

[0005] Lack of multi-evidence matching processing: When there are multiple similar pieces of evidence (such as multiple reports related to the "murder weapon"), traditional systems cannot automatically sort and recommend them, requiring judges to manually review each one, which increases the average time spent per case.

[0006] The case file interaction method is outdated: turning pages in the case file relies on manual operation and cannot respond to voice commands. The court proceedings are inefficient due to frequent interruptions, which affects the continuity of the court proceedings.

[0007] Limitations of existing technical solutions:

[0008] Limitations of single retrieval algorithms: Using only bag-of-words models such as TF-IDF cannot capture the semantic association between "tool of the crime" and "murder weapon", resulting in low retrieval accuracy for semantically ambiguous words (such as "dagger" and "fruit knife").

[0009] Insufficient application of clustering technology: Semantic clustering of the evidence database was not performed, and the time required for full database retrieval increased exponentially with the increase in data volume, which could not meet the real-time requirements of court hearings.

[0010] Lack of human-machine collaboration mechanism: The lack of a feedback and optimization mechanism after human confirmation means that historical retrieval errors cannot be accumulated and learned, making it difficult to continuously improve the system's accuracy. Summary of the Invention

[0011] This invention provides a method and device for capturing voice keywords and retrieving evidence in interrogation scenarios to solve one or more of the problems mentioned above.

[0012] To achieve the above objectives, the present invention adopts the following technical solution:

[0013] Methods for capturing key words in interrogation scenarios and assisting in evidence retrieval include:

[0014] S1. System initialization and parameter configuration: Start the hardware device and load the software module, collect and store the court hearing hot words database entered by the court clerk, and at the same time perform clustering preprocessing on the evidence database to generate a hot word database in memory and an evidence cluster index table;

[0015] S2. Speech signal acquisition and preprocessing: Acquire speech signals from interrogation scenarios, perform noise reduction, framing, and windowing processing, and extract Mel frequency cepstral coefficient feature sequences;

[0016] S3. Speech-to-Text and Hot Word Enhancement: The Mel frequency cepstral coefficient feature sequence is transcribed into a text stream using the Transformer model. Combined with a hot word library in memory, hot word enhanced text is generated through prefix tree matching and score recalculation.

[0017] S4. Keyword Recognition and Semantic Understanding: Keyword recognition is obtained from hot word-enhanced text using a Bi-LSTM+CRF model, and semantics are parsed using a BERT model to generate keywords and their type labels.

[0018] S5. Evidence Base Retrieval and Multi-Algorithm Fusion Matching: Based on keywords and their type labels, semantic cluster information is obtained from the evidence cluster index table. Candidate clusters are filtered through DBSCAN clustering. VSM, DSSM and clustering ranking algorithms are fused to generate a weighted fused evidence retrieval score list.

[0019] S6. Fuzzy matching of multiple evidences: When the highest score in the evidence retrieval score list exceeds the first preset threshold, the evidence is automatically selected; when the highest score does not exceed the first preset threshold but the scores of the first 3 pieces of evidence all exceed the second preset threshold and the difference between adjacent scores is less than the third preset threshold, the manual confirmation process is triggered, the confirmation result is recorded and the similarity of the evidence database is updated.

[0020] S7. Automatic Evidence Display: Based on the evidence manually confirmed or automatically matched in S6, generate corresponding display content and visualize it on the display terminal.

[0021] In this specification, the clustering preprocessing of the evidence database in S1 includes: generating semantic vectors from the evidence database text using the DBSCAN algorithm, constructing a KD tree index and dividing it into semantic clusters, and calculating the cluster centroids and storing them in the evidence cluster index table.

[0022] In this specification, the noise reduction process in S2 uses wavelet transform thresholding method. After multi-level decomposition of the speech signal, the high-frequency coefficients are processed using a soft thresholding function.

[0023] In this specification, the hot word enhancement in S3 includes: constructing a prefix tree for hot words according to their character length, matching hot words in a sliding window of the transcribed text, calculating the weighted score of the acoustic model and the language model, and forcibly replacing them with hot words when the score exceeds a threshold.

[0024] In this specification, the keyword recognition in S4 adopts the Bi-LSTM+CRF model, which extracts features through a bidirectional long short-term memory network, combines it with a conditional random field for sequence labeling, and outputs the BIOES tags of the keywords.

[0025] In this specification, the multi-algorithm fusion matching in S5 includes: converting query keywords into DSSM vectors, filtering candidate clusters based on the evidence cluster index table, calculating TF-IDF weights for evidence within clusters and adjusting them in conjunction with cluster relevance, and then performing three-level weighted fusion through DSSM semantic matching and cluster ranking scores.

[0026] In this specification, obtaining semantic cluster information from the evidence cluster index table in step S5 includes: reading the cluster centroid vector and keyword tags from the evidence cluster index table, and calculating the cosine similarity between the query vector and the cluster centroid to filter candidate clusters.

[0027] In this specification, the first preset threshold in S6 is 0.9, the second preset threshold is 0.7, and the third preset threshold is 0.1.

[0028] In this manual, S7 also includes file-based page turning processing: voice commands are converted into text using the S3 transcription model, page numbers or chapter commands are parsed using regular expressions, the file system API is called to achieve automatic page turning, and the accuracy of command recognition is improved by fine-tuning through the calibration library.

[0029] An auxiliary device for capturing voice keywords and retrieving evidence in interrogation scenarios, applying any one of the methods described above, wherein the auxiliary device for capturing voice keywords and retrieving evidence in interrogation scenarios comprises:

[0030] The hardware processing unit is used to perform system initialization and hardware self-test, including a microphone array, a GPU server and a 4K display terminal. The microphone array is used to collect voice signals in the interrogation scene.

[0031] The storage module includes an in-memory database and a relational database. The in-memory database is used to store the court hearing hot words database entered by the court clerk, and the relational database is used to store the evidence database and the evidence cluster index table generated by cluster preprocessing.

[0032] The speech processing engine is used to perform noise reduction, framing, and windowing of speech signals, and to extract the Mel frequency cepstral coefficient feature sequence.

[0033] The natural language processing engine includes a Transformer model and a Bi-LSTM+CRF model. The Transformer model is used to transcribe the Mel frequency cepstral coefficient feature sequence into a text stream and combine it with a hot word library to generate hot word enhanced text. The Bi-LSTM+CRF model is used to identify keywords from the hot word enhanced text and combine it with a BERT model to parse semantics.

[0034] The evidence retrieval engine is used to obtain semantic cluster information from the evidence cluster index table based on keywords and their type labels, filter candidate clusters through DBSCAN clustering, and generate an evidence retrieval score list by integrating VSM, DSSM and clustering ranking algorithms.

[0035] The interactive processing unit is used to trigger the manual confirmation process when the search evidence retrieval score list meets the preset conditions, record the confirmation results and update the evidence database similarity, and display the evidence content on the display terminal according to the confirmation results.

[0036] The case file interface unit is used to interface with the electronic case file system to enable voice command control for page turning.

[0037] In summary, the present invention has at least the following beneficial effects:

[0038] Speech processing accuracy: Improve speech transcription accuracy and reduce keyword miss rate by using wavelet denoising and hot word enhancement.

[0039] Evidence retrieval efficiency: By integrating three algorithms (VSM+DSSM+clustering and sorting), the cross-domain evidence recall rate is improved, the retrieval response time of the evidence database is reduced, and the F1 score of physical evidence queries is increased.

[0040] Fuzzy matching efficiency: The dynamic clustering and manual confirmation mechanism shortens the screening time for multiple evidences. After the judge confirms the evidence, the similarity is automatically adjusted, which improves the accuracy of subsequent similar queries.

[0041] Case file interaction efficiency: Improved the accuracy of voice-controlled case file page turning commands, reducing the number of court interruptions.

[0042] Full-process automation: It shortens the processing time of the entire chain from voice collection to evidence presentation, improves efficiency, and saves trial time for a single case. Attached Figure Description

[0043] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart illustrating the interrogation scenario voice keyword capture and evidence retrieval assistance method and device involved in this invention.

[0045] Figure 2 This is a schematic diagram of the speech processing and transcription process involved in this invention.

[0046] Figure 3 This is a schematic diagram of the evidence retrieval and fusion process involved in this invention.

[0047] Figure 4 This is a schematic diagram of the multi-evidence matching process involved in this invention. Detailed Implementation

[0048] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the embodiments of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0049] The following disclosure provides many different implementations or examples for carrying out different structures of the embodiments of the present invention. To simplify the disclosure of the embodiments of the present invention, specific examples of components and arrangements are described below. Of course, these are merely examples and are not intended to limit the embodiments of the present invention. Furthermore, reference numerals and / or reference letters may be repeated in different examples of the embodiments of the present invention; such repetition is for simplification and clarity and does not in itself indicate a relationship between the various implementations and / or arrangements discussed.

[0050] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0051] like Figure 1 As shown, this embodiment provides a method for capturing voice keywords and assisting in evidence retrieval in interrogation scenarios, including:

[0052] S1. System initialization and parameter configuration: Start the hardware device and load the software module, collect and store the court hearing hot words database entered by the court clerk, and at the same time perform clustering preprocessing on the evidence database to generate a hot word database in memory and an evidence cluster index table;

[0053] S2. Speech signal acquisition and preprocessing: Acquire speech signals from interrogation scenarios, perform noise reduction, framing, and windowing processing, and extract Mel frequency cepstral coefficient feature sequences;

[0054] S3. Speech-to-Text and Hot Word Enhancement: The Mel frequency cepstral coefficient feature sequence is transcribed into a text stream using the Transformer model. Combined with a hot word library in memory, hot word enhanced text is generated through prefix tree matching and score recalculation.

[0055] S4. Keyword Recognition and Semantic Understanding: Keyword recognition is obtained from hot word-enhanced text using a Bi-LSTM+CRF model, and semantics are parsed using a BERT model to generate keywords and their type labels.

[0056] S5. Evidence Base Retrieval and Multi-Algorithm Fusion Matching: Based on keywords and their type labels, semantic cluster information is obtained from the evidence cluster index table. Candidate clusters are filtered through DBSCAN clustering. VSM, DSSM and clustering ranking algorithms are fused to generate a weighted fused evidence retrieval score list.

[0057] S6. Fuzzy matching of multiple evidences: When the highest score in the evidence retrieval score list exceeds the first preset threshold, the evidence is automatically selected; when the highest score does not exceed the first preset threshold but the scores of the first 3 pieces of evidence all exceed the second preset threshold and the difference between adjacent scores is less than the third preset threshold, the manual confirmation process is triggered, the confirmation result is recorded and the similarity of the evidence database is updated.

[0058] S7. Automatic Evidence Display: Based on the evidence manually confirmed or automatically matched in S6, generate corresponding display content and visualize it on the display terminal.

[0059] In some embodiments, the clustering preprocessing of the evidence base in S1 includes: generating semantic vectors from the evidence base text using the DBSCAN algorithm, constructing a KD tree index and dividing it into semantic clusters, and calculating the cluster centroids and storing them in the evidence cluster index table.

[0060] In some embodiments, the noise reduction process in S2 employs wavelet transform thresholding, which decomposes the speech signal into multiple layers and then processes the high-frequency coefficients using a soft thresholding function.

[0061] In some embodiments, the hot word enhancement in S3 includes: constructing a prefix tree for hot words according to their character length, matching hot words in a sliding window of the transcribed text, calculating the weighted score of the acoustic model and the language model, and forcibly replacing them with hot words when the score exceeds a threshold.

[0062] In some embodiments, keyword recognition in S4 uses a Bi-LSTM+CRF model, which extracts features through a bidirectional long short-term memory network, combines a conditional random field for sequence labeling, and outputs the BIOES tags of the keywords.

[0063] In some embodiments, the multi-algorithm fusion matching in S5 includes: converting query keywords into DSSM vectors, filtering candidate clusters based on the evidence cluster index table, calculating TF-IDF weights for evidence within clusters and adjusting them in conjunction with cluster relevance, and then performing three-level weighted fusion through DSSM semantic matching and cluster ranking scores.

[0064] In some embodiments, obtaining semantic cluster information from the evidence cluster index table in step S5 includes: reading the cluster centroid vector and keyword tags in the evidence cluster index table, and calculating the cosine similarity between the query vector and the cluster centroid to filter candidate clusters.

[0065] In some embodiments, the first preset threshold in S6 is 0.9, the second preset threshold is 0.7, and the third preset threshold is 0.1.

[0066] In some embodiments, S7 further includes file-based page turning processing: the voice commands are converted into text using the transcription model of S3, page numbers or chapter commands are parsed using regular expressions, the file system API is called to achieve automatic page turning, and the accuracy of command recognition is improved by fine-tuning through the calibration library.

[0067] The technical concept of this invention is as follows:

[0068] S1. System Initialization and Parameter Configuration:

[0069] Processing steps: Start the hardware cluster (including an 8-channel M-AudioAIR 192|14 microphone array, an NVIDIA A100 GPU server, and a 27-inch 4K LCD display terminal), load the modular software system, including a Python-based speech signal processing engine, a TensorFlow-based natural language understanding module, an Elasticsearch-optimized evidence retrieval engine, and an interactive interface developed using Vue.js. The court clerk initializes the court hearing hot word database through the graphical interface. The database covers core elements of the case, including:

[0070] Parties involved: such as the names of natural persons like "Zhang San", "Li Si", and "Wang Wu", and the names of legal entities like "a certain technology company";

[0071] Physical evidence includes concrete items such as "dagger," "vehicle," and "drugs," as well as abstract categories such as "tools of the crime" and "murder weapon."

[0072] Legal provisions: such as specific citations of "Article 232 of the Criminal Law" and "Article 108 of the Civil Procedure Law";

[0073] Time-related terms: such as "June 1, 2024" or "the day of the incident".

[0074] The system stores the hot word library in JSON format in a Redis distributed in-memory database (with a cluster configuration of 3 masters and 3 slaves), achieving millisecond-level retrieval response. Simultaneously, it triggers a clustering preprocessing flow for the evidence library, employing the DBSCAN algorithm to perform semantic clustering of the evidence library, thus building a semantic cluster index for subsequent retrieval.

[0075] Data flow: Hot word library input (interaction with clerk) → Redis in-memory database storage → DBSCAN evidence library clustering preprocessing → generation of cluster index table (EvidenceCluster).

[0076] Specific processing steps:

[0077] 1. Hardware Self-Test Process: A custom device status inspection protocol (developed based on the TCP / IP protocol stack) is used to perform sampling rate calibration on the microphone array (default setting is 44.1kHz, allowing ±0.5kHz error), server CPU load detection (threshold set to ≤70%), and GPU memory usage detection (threshold set to ≤80%). Upon successful self-test, the device status manager outputs a binary signal "0x01" indicating system readiness; otherwise, a red alarm light flashes, and a detailed error record containing error codes and timestamps is generated in the log system.

[0078] 2. Hot word library construction (supports two input methods):

[0079] Interactive item-by-item addition: Enter keyword text and type label through the form interface, such as entering "Zhang San" and selecting the "Party Affiliation" type, and setting the priority to "High";

[0080] Batch import: Parse an Excel template file. The template contains three columns: "Keywords", "Type", and "Priority". Example data is as follows:

[0081] Keywords type Priority Zhang San party high The dagger used in the crime Physical evidence high Article 232 of the Criminal Law Legal terms middle

[0082] 3. Evidence library clustering preprocessing:

[0083] Upon initialization, full DBSCAN clustering is automatically triggered:

[0084] Extract the text content of all evidence in the evidence library (including PDF document text, image OCR recognition text, and audio / video subtitle text), and generate a 128-dimensional semantic vector using the DSSM model;

[0085] Calculate the Euclidean distance between all vectors, and take the median of the distance distribution as the neighborhood radius ε (e.g., when the median distance is 0.8, ε = 0.8). Set the minimum number of samples to... (N is the total number of pieces of evidence. When N = 10000, MinPts = 100).

[0086] A KD-tree index was constructed using the FLANN library (the vector dimension was reduced to 64 dimensions using PCA while retaining 95% of the variance), and the evidence was divided into several semantic clusters and stored in the EvidenceCluster table.

[0087] S2. Speech signal acquisition and preprocessing:

[0088] Processing Procedure: Real-time acquisition of interrogation scene speech signals (sampling rate 44.1kHz, 16-bit quantization depth) via an 8-channel microphone array. Environmental noise suppression, time-domain framing, frequency-domain transformation, and feature parameter extraction are performed sequentially to generate a Mel-frequency cepstral coefficient (MFCC) sequence suitable for speech transcription. The processing must ensure a signal-to-noise ratio improvement of ≥15dB for the speech signal, providing high-quality input for subsequent transcription.

[0089] Data flow: microphone array raw signal (time domain waveform) → wavelet transform noise reduction → frame segmentation and windowing processing → FFT frequency domain transformation → Mel filter bank processing → DCT transform → MFCC feature vector sequence (dimension is 13×T, T is the total number of frames).

[0090] Specific processing steps:

[0091] 1. Wavelet transform noise reduction:

[0092] The input speech signal x(t) is decomposed into 5-level Daubechies wavelet (db4) decompositions to obtain the high-frequency coefficients c. j,k and low-frequency coefficient d N1,k Where j and N1 represent the wavelet decomposition levels, and k represents the index of the coefficients in that level. A soft thresholding function is used to process high-frequency coefficients to suppress environmental noise (such as air conditioning noise and keyboard typing sounds).

[0093]

[0094] in Here are the high-frequency wavelet coefficients after soft thresholding; λ is the threshold, calculated using the following formula: σ is the noise standard deviation, estimated using the median method (σ = 1.4826 × median(|c j,k |)); L is the signal length; I represents the indicator function. Taking a 10-second voice signal as an example, assuming σ = 0.2 and L = 441000, then

[0095] The signal-to-noise ratio after noise reduction is calculated as follows:

[0096]

[0097] Where x clean (t) represents the clean speech signal after noise reduction, where t is the index of the time sampling point;

[0098] Actual tests show that this processing can improve the SNR of voice signals in an office environment from 10dB to over 25dB.

[0099] 2. Frame division and windowing:

[0100] The noise-reduced signal is divided into frames with a frame length of 25ms (corresponding to 1102 sampling points at a sampling rate of 44.1kHz) and a frame shift of 10ms (441 sampling points), resulting in frame sequence x. n (m)(n is the frame index, m is the sample index). A Hamming window is applied to each frame to eliminate boundary discontinuities:

[0101]

[0102] Where w(m) is the value of the Hamming window function at the m-th sample point, and M is the total number of samples in a single frame of speech signal; for example, for the n-th frame signal, the expression after windowing is x n (m)·w(m), effectively reducing spectrum leakage.

[0103] 3. MFCC Feature Extraction:

[0104] Fast Fourier Transform (FFT): Perform a 512-point FFT on each windowed frame to convert the time-domain signal into a frequency-domain representation X. n (k1):

[0105]

[0106] Where X n (k1) is the frequency domain representation of the nth frame of the speech signal, corresponding to the complex value of the k1th frequency point; e is the natural constant (Euler number), approximately equal to 2.71828; h1 is the imaginary unit.

[0107] Mel filter bank: converts linear frequency f to Mel frequency f mel =2595log 10 (1+f / 700) is used to construct 40 triangular filters covering the 300Hz-8000Hz frequency band. Taking 800Hz as an example, its Mel frequency is 2595log. 10 (1+800 / 700)≈2595×0.168≈436mel;

[0108] Logarithmic and DCT Transform: Taking the natural logarithm of the filter output and performing a 13th-order Discrete Cosine Transform (DCT) yields the MFCC coefficients c. n (l), where l is the dimension index of the MFCC coefficients, l = 1,...,13. Dimensions 1-12 reflect the spectral envelope characteristics, and dimension 13 represents the logarithmic energy characteristics.

[0109] S3. Speech-to-text and hot word enhancement:

[0110] Processing content: Use a pre-trained Transformer speech-to-text model to convert the MFCC feature sequence into a text stream, and perform post-processing in combination with the hot word library preset by the court clerk to correct misidentifications of professional terms (such as misidentifying "murder weapon" as "chest weapon"), and improve the transcription accuracy to over 95%. The processing process includes four links: model inference, hot word matching, score recalculation, and forced replacement.

[0111] Data flow: MFCC feature sequence (13×T) → Transformer encoder-decoder inference → initial text stream → hot word prefix tree matching → score calculation → hot word enhanced text.

[0112] Specific processing process:

[0113] 1. Transformer model construction: Adopt a 6-layer encoder-6-layer decoder architecture, and each layer contains:

[0114] Multi-head self-attention mechanism: 8 attention heads, the key vector dimension is 64, and the calculation method is:

[0115] [[ID=1​​​​​​​​​​​​​​​​​​​​​​​​​​​​​Layer Normalization (LayerNorm): Apply LayerNorm(x+SubLayer(x)) after each sublayer to stabilize the training process. SubLayer(x) is the output of the sublayer function.

[0122] 2. Model training process:

[0123] Pre-training phase: 100,000 hours of general-purpose speech data (including court transcripts, court debates, legal lectures, etc.) are used, with cross-entropy as the loss function.

[0124]

[0125] Where x1 is an MFCC sequence, y t Let p(y) be the actual character at time t, T2 be the sequence length, and p(y) be the actual character at time t. t |y1,...,y t-1 x1) represents the conditional probability. Training uses the Adam optimizer with an initial learning rate of 5e-4 and a cosine decay strategy.

[0126] Fine-tuning phase: For 5000 hours of annotated interrogation voice data (including hot word location annotations), the first 4 layers of the encoder were frozen, and only the last 2 layers and the decoder were fine-tuned, with the learning rate reduced to 1e-4.

[0127] 3. Hot word enhancement processing:

[0128] Trie construction: Hot words are inserted into the tree structure in descending order of character length. For example, if the hot word library contains "intentional homicide", "murder weapon", and "Zhang San", then in the tree structure, "intentional homicide" is the longest path (5 characters), "murder weapon" (2 characters) and "Zhang San" (2 characters) are the shorter paths.

[0129] Text traversal matching: The transcribed text is traversed by sliding a window (window size = maximum length of hot words, e.g., 5) character by character to find matching paths in the Trie tree. For example, the text "He used a chest weapon to injure people" matches "chest weapon" when the window size is 2, but the Trie tree can find the longer hot word "murder weapon".

[0130] Score recalculation: For matching hot words, calculate the acoustic model score S. acoustic (The sum of the output probabilities of MFCC through the acoustic model for the corresponding speech frame) and the language model score S language The weighted sum of the predicted probabilities of hot words by the Transformer language model:

[0131] S = 0.6·S acoustic +0.4·S language ;

[0132] Assume that the acoustic score of the "murder weapon" is 0.7 and the language model score is 0.9, then S = 0.6×0.7 + 0.4×0.9 = 0.42 + 0.36 = 0.78;

[0133] Forced replacement: Replacement is performed when S > 0.8. In the above example where S = 0.78 < 0.8, there is no replacement; if S = 0.85, then "chest weapon" is replaced with "murder weapon".

[0134] S1, S2, S3 are the voice processing and transcription processes, as Figure 2 shown.

[0135] S4. Keyword recognition and semantic understanding:

[0136] Processing content: Identify key information units (such as parties, physical evidence, time, etc.) from the hot word enhanced text, perform sequence labeling through the Bi-LSTM + CRF model, and combine with the BERT semantic model to parse the context semantics of the keywords to determine their logical types and roles in the case. The processing needs to achieve a keyword boundary recognition accuracy of 95% and a type classification accuracy of over 92%.

[0137] Data flow: Hot word enhanced text → Character-level Word2Vec embedding → Bi-LSTM feature extraction → CRF sequence labeling → BERT semantic encoding → Keyword type tags (such as "party", "physical evidence", "time").

[0138] Specific processing process: <00​​​​​​​​​​​​​​​​​​​​​​​​​ (L1 is the number of tags, +2 is the starting tag) <start>and termination label <end>The optimal label sequence is obtained using the Viterbi algorithm.

[0145]

[0146] in Let y1 represent the optimal label sequence, y2 represent any possible label sequence, and arg max represent the maximum possible label sequence. y1 To find the solution that maximizes the objective function for variable y1, where t1 represents the time step index, T3 represents the sequence length, P(y t1 |h t1 Given the hidden state h t1 At that time, label y t1 The probability of emission.

[0147] Tagging system: The BIOES tagging method is used. Specific tags include:

[0148] B - Parties (Party's Substantive Relationship Begins)

[0149] I - Parties (Substantive Continuation of Parties)

[0150] E-Party (Substantive Conclusion of the Party)

[0151] S-Party (Single-character party entity)

[0152] B-evidence, I-evidence, E-evidence, S-evidence

[0153] B-time, I-time, E-time, S-time, etc.

[0154] 2. Model training process:

[0155] Data annotation: 5000 interrogation records, annotated with keyword boundaries and types by legal experts, with a tag accuracy rate of ≥98%. For example, the text "Zhang San committed the crime using a dagger on June 1, 2024" is annotated as:

[0156] "Zhang San" / B - party involved, "on" / O, "June 1, 2024" / B - time, "used" / O, "dagger" / B - physical evidence, "committed the crime" / O;

[0157] Loss function: CRF negative log-likelihood, combined with L2 regularization (λ1 = 0.001) to prevent overfitting.

[0158] 3. Deepening semantic understanding:

[0159] BERT encoding: Using a BERT model specifically designed for the legal domain (fine-tuned based on Chinese-Law-BERT), the input keyword and its context of five words before and after it generate a 768-dimensional semantic vector. For example, the context of the keyword "dagger" is "[using] dagger [committing a crime]", which, when input into BERT, yields a vector.

[0160] Type classification: Semantic vectors are classified using softmax after passing through a fully connected layer (768→128→L3 (L3 is the total number of keyword type classifications)). The loss function is cross-entropy. Taking time type classification as an example, when the input is "June 1, 2024", the fully connected layer outputs [0.02, 0.95, 0.03], corresponding to the probabilities of "parties involved", "time", and "physical evidence". The maximum value of 0.95 is taken as the type.

[0161] Rule engine supplement: Preset regular expression rules match specific types, such as:

[0162] Time expression: / \d{4}year\d{1,2}month\d{1,2}day / ;

[0163] Matching logic:

[0164] \d{4}: Matches a 4-digit year (e.g., 2024);

[0165] \d{1,2}: Matches 1-2 digits of month / date (e.g., June, January 1st);

[0166] The separators "year", "month", and "day" are fixed characters to ensure format accuracy.

[0167] Amount expression: / ￥? \d+(\.\d{1,2})? yuan? / ;

[0168] Matching logic:

[0169] ￥?: You can choose the RMB symbol (￥), such as "￥5000" or "5000 yuan" will both match;

[0170] \d+: Matches one or more digits (the integer part);

[0171] (\.\d{1,2})?: Optional decimal part (1-2 digits), such as ".00" or ".5";

[0172] Yuan?: You can select the unit "Yuan", such as "5000" or "5000 Yuan", both are valid.

[0173] Physical evidence name: / [knife, spear, club, axe, halberd, hook, fork]+\s*murder weapon? / ;

[0174] Matching logic:

[0175] [[Swords, spears, cudgels, staffs, axes, tomahawks, hooks, and tridents]]+: Match at least one weapon character (e.g., "sword", "spear", "axe");

[0176] \s*: Zero or more spaces (compatible with extra spaces in "dagger weapon");

[0177] Weapon??: Optional "weapon" suffix (e.g., both "machete" and "machete weapon" are matched).

[0178] The results of rule matching and deep learning are fused with a weight of 0.3.

[0179] S5. Evidence Base Retrieval and Multi-Algorithm Fusion Matching:

[0180] Processing Content: Based on the keywords and types identified in S4, candidate evidence clusters are screened through DBSCAN semantic clustering, and the vector space model (VSM), deep semantic matching (DSSM), and clustering sorting algorithms are fused to generate a weighted fusion evidence retrieval result. This step needs to solve the problem of cross-domain evidence association (such as associating "weapon photos" in an "intentional injury" case) and improve the accuracy of semantic retrieval.

[0181] Data Flow: Keyword Set → DSSM Query Vector Generation → Cluster Similarity Calculation → Candidate Cluster Screening → VSM Retrieval → DSSM Fine Screening → Clustering Sorting → Three-Level Fusion Score Calculation → Evidence List (sorted in descending order of scores).

[0182] The specific processing process is as Figure 3 shown:

[0183] 1. Evidence Base Clustering Preprocessing and Dynamic Maintenance:

[0184] DBSCAN Clustering Model:

[0185] Model Construction: Using the evidence semantic vectors generated by DSSM as features, the determination method of core parameters:​​​​​​​​​​​​​​​​Use the FLANN library to build a KD-tree index (reduced to 64 dimensions by PCA while retaining 95% of the variance) to accelerate neighborhood search;

[0191] Traverse each vector and calculate the number of points in its ε-neighborhood. If the number of points is greater than or equal to MinPts, mark it as a core point; otherwise, mark it as a boundary point or noise point.

[0192] Core points and their density reachable points form cluster C k2 Calculate the centroid

[0193] Where k2 is the cluster index, μ k2 For cluster C k2 The centroid vector, f D (d) is the output vector of the Deep Semantic Matching (DSSM) model for evidence d.

[0194] Application process:

[0195] Incremental Update: Incremental clustering is triggered when the number of new / deleted evidence entries in the evidence database exceeds 10%.

[0196] Calculate the distance between the new evidence vector and the centroid of the existing cluster. If the distance is less than ε, add the corresponding cluster and update the centroid.

[0197]

[0198] Where n k2 Represents the k2th cluster C k2 The amount of evidence in the Central Plains, f D (d new ) is new evidence d new DSSM semantic vectors;

[0199] For example, cluster C1 originally has 100 pieces of evidence, with centroid μ1 = [0.1, 0.2, ..., 0.6]. A new evidence vector f is added. D =[0.12,0.21,...,0.59], then Otherwise, as a new cluster, MinPts is adjusted to 80% of the original MinPts (i.e., 57);

[0200] Noise handling: For evidence marked as noise (such as irrelevant financial statements), a search weight decay factor of 0.5 is set to reduce its search score.

[0201] 2. Query-oriented cluster filtering and VSM retrieval:

[0202] Cluster selection mechanism:

[0203] Query vectorization: The keyword set output by S4 (e.g., ["Zhang San", "dagger", "damage"]) is used to generate a 128-dimensional query vector f through the DSSM model. Q (q);

[0204] Cluster similarity calculation: Traverse all cluster centroids μ k2 Calculate the cosine similarity:

[0205]

[0206] Where q is the set of keywords;

[0207] For example, query vector f Q The dot product with the centroid μ1 of cluster C1 is 80, f Q If the norm is 10 and the μ1 norm is 8, then Sim = 80 / (10 × 8) = 1.0;

[0208] Candidate cluster generation: Select the top 5 clusters with similarity > 0.4 as C. cand If none of the conditions are met, the search is expanded to the entire database.

[0209] VSM search enhancement:

[0210] TF-IDF weight calculation:

[0211]

[0212] Where t2 is the target keyword, ∑ t2' n t2',d D represents the total number of words in the evidence database;

[0213] For example, if the keyword "dagger" appears 3 times in evidence d, and the total number of words in d is 100, then TF = 3 / 100 = 0.03; if 10 out of 1000 pieces of evidence in the evidence database contain "dagger", then IDF = log(1000 / 11) ≈ log(90.91) ≈ 4.51, and TF - IDF = 0.03 × 4.51 ≈ 0.135;

[0214] Cluster relevance adjustment: Based on the similarity between the query and the cluster, Sim(q,C) k2 To enhance the TF-IDF weights:

[0215] TF-IDF'(t2,d)=TF-IDF(t2,d)×(1+0.5·Sim(q,C k2 ));

[0216] If Sim = 0.8, then the adjusted TF-IDF' = 0.135 × (1 + 0.5 × 0.8) = 0.135 × 1.4 = 0.189;

[0217] Similarity calculation: Constructing query vector Q and evidence vector D d Cosine similarity of ':

[0218]

[0219] Where D d ' is the adjusted TF-IDF vector of evidence d;

[0220] 3. DSSM semantic filtering and three-level fusion:

[0221] DSSM similarity calculation:

[0222] Semantic matching of evidence and query:

[0223]

[0224] Assume f D (d)·f Q (q) = 100, |f D (d)|=10,|f Q (q)|=10, then Sim D =1.0;

[0225] Cluster affiliation correction: The closer the semantic distance between the evidence and its cluster, the higher its credibility.

[0226] Sim D '(d,q)=Sim D (d,q)×(1+0.3·Sim(d,C k2 ));

[0227] Where Sim(d,C) k2 )=1-||f D (d)-μ k2 || / max d istance is the maximum intra-cluster distance. If ||f D (d)-μ k2 ||=0.5, max d if istance = 1.0, then Sim(d,C) k2 ) = 0.5, Sim D =1.0×(1+0.3×0.5)=1.15.

[0228] Clustering-assisted sorting:

[0229] Intra-cluster relative distance: measures the relative position of evidence within the cluster compared to the query.

[0230]

[0231] Adding a minimal constant ∈ = 1e-6 to the denominator avoids division by zero, for example, ||f D -f Q ||=0.8,||f D -μ k2 If || = 0.4, then ClusterDist = 0.8 / (0.4 + 1e-6) = 2.0;

[0232] Score ranking: The closer the distance, the higher the score, exhibiting exponential decay.

[0233] RankScore(d,q)=exp(-ClusterDist(d,q))=exp(-2.0)≈0.135;

[0234] Three-level fusion model:

[0235] Dynamic weight allocation: Automatically adjusted based on the keyword type distribution output by S4.

[0236] Weighting of physical evidence queries ("physical evidence" tag percentage > 40%): 0.4, 0.5, 0.1;

[0237] Text-based query weights (indicating occurrences of "contract," "agreement," etc.): 0.3, 0.6, 0.1;

[0238] General query weights: 0.3, 0.3, 0.4;

[0239] Fusion formula:

[0240] FinalScore(d,q) = w1·Sim V '(d,q)+w2·Sim D '(d,q)+w3·RankScore(d,q);

[0241] Where w1, w2, and w3 are the fusion weight coefficients;

[0242] For example, Sim V =0.8, Sim D =1.15, RankScore =0.135, evidence class weight (0.4,0.5,0.1), then FinalScore =0.4×0.8+0.5×1.15+0.1×0.135=0.32+0.575+0.0135=0.9085.

[0243] S6. Multi-evidence fuzzy matching processing:

[0244] Processing steps: Based on the evidence fusion score list output by S5, automatic matching is first performed. If the score meets the high confidence condition, the evidence is directly selected; otherwise, fuzzy matching detection is performed, triggering manual confirmation or negative feedback processing. The process needs to implement hierarchical decision-making between automatic matching and manual confirmation to ensure the efficiency and accuracy of evidence selection.

[0245] Data flow: Evidence fusion score list (FinalScore sorting) → Automatic matching condition detection → Automatic selection of evidence ID / Manual confirmation of evidence ID → Evidence ID output to S7 → Evidence display rendering.

[0246] Specific processing steps, such as Figure 4 As shown:

[0247] 1. Automatic matching condition detection:

[0248] Automatic selection of high-confidence evidence: When the highest score in the list, FinalScore_max, is greater than 0.9, the system automatically selects that evidence without manual intervention. For example, if a piece of evidence has a fusion score of 0.92, it is directly marked as selected, and the following actions are performed:

[0249] Records automatic selection logs: including evidence ID, selection time, score, and triggering condition (FinalScore). max >0.9);

[0250] The evidence display module (S7) is invoked for rendering, and the similarity of the evidence is increased by 0.1 (formula: new). sim =old sim +0.1), strengthens the matching priority of subsequent similar queries;

[0251] If the score of this evidence is compared with that of similar historical cases, and the difference exceeds 15% (e.g., the current score is 0.92, the historical average is 0.78, and the difference is 17.9% > 15%), it will be marked as "pending review" and will be reviewed by the court clerk.

[0252] Threshold explanation: The 0.9 threshold is based on historical data statistics. When the score is >0.9, the accuracy rate of manual review reaches 98.7%, and it can be safely and automatically selected.

[0253] 2. Fuzzy matching requires manual confirmation:

[0254] Conditional check: Manual confirmation is triggered if the following conditions are met:

[0255] FinalScore(d1,q)>0.7;

[0256] FinalScore(d2,q)>0.7;

[0257] FinalScore(d1,q)-FinalScore(d2,q)<0.1;

[0258] FinalScore(d2,q)-FinalScore(d3,q)<0.1;

[0259] Where d1, d2, and d3 represent the top three pieces of evidence with the highest fusion scores in the search results;

[0260] For example, the scores of the first three pieces of evidence are 0.78, 0.77, and 0.76, all of which are greater than 0.7 and the adjacent score differences are 0.01 and 0.01 < 0.1, thus triggering confirmation.

[0261] Threshold explanation: 0.7 is the medium confidence threshold, and a difference of 0.1 ensures that the relevance of the evidence is close; according to the test, the necessity of manual confirmation reaches 83.5% under this condition.

[0262] Manual confirmation process:

[0263] The interface displays the first 5 pieces of evidence (including score, type, and preview), with keywords highlighted;

[0264] Supports three operations:

[0265] Selecting evidence: Record the person who confirmed it and the time; the score increases by 0.2 (formula: new). sim =old sim +0.2);

[0266] No match: Record negative feedback, add related keyword combinations to the blacklist, and reduce the search weight by 0.3 in subsequent searches;

[0267] Skip: Do not process for now, return to the search results list:

[0268] 3. Evidence ID Output Mechanism: Regardless of whether selection is automatic or manual verification, a unique evidence ID (e.g., "EVID-20250630-001") is generated and synchronized to the S7 module via a message queue (RabbitMQ) to ensure: EvidenceID S6输出 =EvidenceID S7输入 ;

[0269] 4. Exception handling: If no confirmation receipt is received from S7 within 30 seconds, the evidence ID will be automatically resent, with a maximum of 3 retries and a 1-second interval between retries.

[0270] S7. Automatic evidence display and case file browsing:

[0271] Processing content: Based on evidence confirmed by S6 or automatically matched high-scoring evidence, render and display it by type on the display terminal (e.g., highlighting keywords in PDF documents, marking areas in images). Simultaneously, connect to the electronic case file system to achieve automatic page turning and positioning of case files via voice commands. Support operations such as page number jump and chapter query; command recognition accuracy must reach over 90%.

[0272] Data flow: Confirm evidence ID → Read from distributed file system → Process by rendering engine → Visualization;

[0273] Voice command → S3 transcription model → command text → regular expression parsing → dossier system API call → page number control.

[0274] Specific processing steps:

[0275] 1. Evidence Presentation Module:

[0276] File reading: Reads raw files from the Ceph distributed file system based on the evidence ID. Supported formats include:

[0277] Document types: PDF, DOCX, RTF, TXT;

[0278] Image formats: JPG, PNG, TIFF, BMP;

[0279] Audio and video formats: MP4, AVI, WAV, MP3.

[0280] Rendering strategy:

[0281] Document-based: PDF files are rendered using PDF.js, and text highlighting is achieved through keyword position information (character offsets) output by S4. For example, if the keyword "dagger" is located at character 120 on page 3 of the PDF, a yellow background with a red border will be added during rendering.

[0282] Image-based: Use Canvas to draw, and mark areas based on keyword semantic matching results. For example, "dagger" corresponds to a rectangle in the image with coordinates (100, 200) - (300, 400), with a red border and a line width of 2px;

[0283] For audio and video: Analyze keyframes and mark the timestamps of keyword appearances on the timeline (error ≤ 500ms). For example, if the keyword "dagger" appears at 00:01:23 in the video, a red marker will appear at that position on the timeline, which can be clicked to jump to the desired location.

[0284] 2. File browsing function:

[0285] Command Recognition: Speech-to-text conversion uses the S3 Transformer model, with a newly added case file-specific hot word library, including: page number related: "Page 1", "Page 10", "Page 123"; chapter related: "Chapter 1", "Appendix A", "Witness Testimony Section"; operation related: "Turn to", "View", "Back". Text Standardization: "Turn to page 3" is converted to "Turn to page 3", unifying the format for easier parsing.

[0286] Instruction parsing:

[0287] Page number command: Use the regular expression / turn to page (\dY+) / capture the target page number p. For example: if you enter "please turn to page 5", the regular expression will match p = 5; dY is the page number.

[0288] Chapter command: Regular expression matching / view(.*?)chapter?section? / , such as "view Chapter 3" captures the chapter name s="Chapter 3", queries the dossier index table (structure: {chapter:"Chapter 3",page:25}) to get the corresponding page number p. s =25.

[0289] Command execution: Calls the dossier system API: scrollToPage(p), supporting smooth scrolling animation (lasts 500ms);

[0290] Accuracy calibration: Record the text i1 of each instruction and the actual page number j1. If |p-j1|>1, store it in the calibration library (format: {instruction:"turn to page 5",actual_page:7,timestamp:"2024-06-0110:30:00"}).

[0291] The language model portion of the Transformer model is automatically fine-tuned daily at midnight using calibration library data. Loss function:

[0292]

[0293] Where C2 is the calibration library sample, p i1 Let j1 be the page number prediction distribution for instruction i1, and j1 be the correct page number.

[0294] An auxiliary device for capturing voice keywords and retrieving evidence in interrogation scenarios, applying any one of the methods described above, wherein the auxiliary device for capturing voice keywords and retrieving evidence in interrogation scenarios comprises:

[0295] The hardware processing unit is used to perform system initialization and hardware self-test, including a microphone array, a GPU server and a 4K display terminal. The microphone array is used to collect voice signals in the interrogation scene.

[0296] The storage module includes an in-memory database and a relational database. The in-memory database is used to store the court hearing hot words database entered by the court clerk, and the relational database is used to store the evidence database and the evidence cluster index table generated by cluster preprocessing.

[0297] The speech processing engine is used to perform noise reduction, framing, and windowing of speech signals, and to extract the Mel frequency cepstral coefficient feature sequence.

[0298] The natural language processing engine includes a Transformer model and a Bi-LSTM+CRF model. The Transformer model is used to transcribe the Mel frequency cepstral coefficient feature sequence into a text stream and combine it with a hot word library to generate hot word enhanced text. The Bi-LSTM+CRF model is used to identify keywords from the hot word enhanced text and combine it with a BERT model to parse semantics.

[0299] The evidence retrieval engine is used to obtain semantic cluster information from the evidence cluster index table based on keywords and their type labels, filter candidate clusters through DBSCAN clustering, and generate an evidence retrieval score list by integrating VSM, DSSM and clustering ranking algorithms.

[0300] The interactive processing unit is used to trigger the manual confirmation process when the search evidence retrieval score list meets the preset conditions, record the confirmation results and update the evidence database similarity, and display the evidence content on the display terminal according to the confirmation results.

[0301] The case file interface unit is used to interface with the electronic case file system to enable voice command control for page turning.

[0302] The embodiments described above are for illustrative purposes only and are not intended to limit the invention. Therefore, any changes in numerical values ​​or substitutions of equivalent elements should still fall within the scope of this invention.

[0303] The above detailed description will enable those skilled in the art to understand that the present invention can indeed achieve the aforementioned objectives and has complied with the provisions of the Patent Law.

[0304] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention. The above descriptions are merely preferred embodiments of the invention and are not intended to limit the invention. It should be noted that any modifications, equivalent substitutions, and improvements made within the spirit and principles of the invention should be included within the scope of protection of the invention.

[0305] It should be noted that the above description of the process is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art can make various modifications and changes to the process under the guidance of this specification. However, these modifications and changes remain within the scope of this specification.

[0306] The basic concepts have been described above. Obviously, for those skilled in the art who have read this application, the above disclosure is merely illustrative and does not constitute a limitation of this application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this application. Such modifications, improvements, and corrections are suggested in this application, and therefore, such modifications, improvements, and corrections still fall within the spirit and scope of the exemplary embodiments of this application.

[0307] Furthermore, this application uses specific terms to describe its embodiments. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic related to at least one embodiment of this application. Therefore, it should be emphasized and noted that "an embodiment," "one embodiment," or "an alternative embodiment" mentioned twice or more in different positions in this specification do not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this application can be appropriately combined.

[0308] Furthermore, those skilled in the art will understand that aspects of this application can be described and illustrated through several patentable types or situations, including any new and useful combination of processes, machines, products, or substances, or any new and useful improvements thereof. Therefore, aspects of this application can be implemented entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or a combination of hardware and software. All of the above hardware or software can be referred to as a "unit," "module," or "system." Furthermore, aspects of this application can take the form of a computer program product embodied in one or more computer-readable media, wherein computer-readable program code is contained therein.

[0309] The computer program code required for the operation of each part of this application can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, and Python; general programming languages ​​such as C; Visual Basic, Fortran2103, Perl, COBOL2102, PHP, and ABAP; dynamic programming languages ​​such as Python, Ruby, and Groovy; or other programming languages. This program code can run entirely on the user's computer, or as a standalone software package on the user's computer, or partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via any network, such as a local area network (LAN) or wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service such as Software as a Service (SaaS).

[0310] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this application are not intended to limit the order of the processes and methods of this application. Although some currently considered useful embodiments of the invention have been discussed in the foregoing disclosure by way of various examples, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the substance and scope of the embodiments of this application. For example, although the implementation of the various components described above can be embodied in a hardware device, it can also be implemented as a purely software solution, such as an installation on an existing server or mobile device.

[0311] Similarly, it should be noted that, in order to simplify the description of the present application and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of the embodiments of the present application sometimes combines multiple features into a single embodiment, drawing, or description thereof. However, this approach of the present application should not be construed as reflecting an intention that the claimed subject matter requires more features than expressly recited in each claim. Rather, the subject of the invention should possess fewer features than in any single embodiment described above.< / end> < / start>

Claims

1. A method for capturing voice keywords and assisting in evidence retrieval in interrogation scenarios, characterized in that, include: S1. System initialization and parameter configuration: Start the hardware device and load the software module, collect and store the court hearing hot words database entered by the court clerk, and at the same time perform clustering preprocessing on the evidence database to generate a hot word database in memory and an evidence cluster index table; S2. Speech signal acquisition and preprocessing: Acquire speech signals from interrogation scenarios, perform noise reduction, framing, and windowing processing, and extract Mel frequency cepstral coefficient feature sequences; S3. Speech-to-Text and Hot Word Enhancement: The Mel frequency cepstral coefficient feature sequence is transcribed into a text stream using the Transformer model. Combined with a hot word library in memory, hot word enhanced text is generated through prefix tree matching and score recalculation. S4. Keyword Recognition and Semantic Understanding: Keyword recognition is obtained from hot word-enhanced text using a Bi-LSTM+CRF model, and semantics are parsed using a BERT model to generate keywords and their type labels. S5. Evidence Base Retrieval and Multi-Algorithm Fusion Matching: Based on keywords and their type labels, semantic cluster information is obtained from the evidence cluster index table. Candidate clusters are filtered through DBSCAN clustering. VSM, DSSM and clustering ranking algorithms are fused to generate a weighted fused evidence retrieval score list. S6. Fuzzy matching of multiple evidences: When the highest score in the evidence retrieval score list exceeds the first preset threshold, the evidence is automatically selected; when the highest score does not exceed the first preset threshold but the scores of the first 3 pieces of evidence all exceed the second preset threshold and the difference between adjacent scores is less than the third preset threshold, the manual confirmation process is triggered, the confirmation result is recorded and the similarity of the evidence database is updated. S7. Automatic Evidence Display: Based on the evidence manually confirmed or automatically matched in S6, generate corresponding display content and visualize it on the display terminal.

2. The method for capturing voice keywords and assisting in evidence retrieval in interrogation scenarios according to claim 1, characterized in that, The clustering preprocessing of the evidence database in S1 includes: using the DBSCAN algorithm to generate semantic vectors for the evidence database text, constructing a KD tree index and dividing it into semantic clusters, and calculating the cluster centroids and storing them in the evidence cluster index table.

3. The method for capturing voice keywords and assisting in evidence retrieval in interrogation scenarios according to claim 1, characterized in that, The noise reduction process in S2 uses wavelet transform thresholding. After multi-level decomposition of the speech signal, a soft thresholding function is used to process the high-frequency coefficients.

4. The method for capturing voice keywords and assisting in evidence retrieval in interrogation scenarios according to claim 1, characterized in that, The hot word enhancement in S3 includes: constructing a prefix tree for hot words according to their character length, matching hot words in a sliding window of the transcribed text, calculating the weighted score of the acoustic model and the language model, and forcibly replacing them with hot words when the score exceeds a threshold.

5. The method for capturing voice keywords and assisting in evidence retrieval in interrogation scenarios according to claim 1, characterized in that, In S4, keyword recognition uses a Bi-LSTM+CRF model, which extracts features through a bidirectional long short-term memory network, combines a conditional random field for sequence labeling, and outputs the BIOES tags of the keywords.

6. The method for capturing voice keywords and assisting in evidence retrieval in interrogation scenarios according to claim 1, characterized in that, The multi-algorithm fusion matching in S5 includes: converting query keywords into DSSM vectors, filtering candidate clusters based on the evidence cluster index table, calculating TF-IDF weights for evidence within clusters and adjusting them in conjunction with cluster relevance, and then performing three-level weighted fusion through DSSM semantic matching and cluster ranking scores.

7. The method for capturing voice keywords and assisting in evidence retrieval in interrogation scenarios according to claim 1, characterized in that, The step S5 involves obtaining semantic cluster information from the evidence cluster index table by: reading the cluster centroid vector and keyword tags from the evidence cluster index table, and calculating the cosine similarity between the query vector and the cluster centroid to filter candidate clusters.

8. The method for capturing voice keywords and assisting in evidence retrieval in interrogation scenarios according to claim 1, characterized in that, In S6, the first preset threshold is 0.9, the second preset threshold is 0.7, and the third preset threshold is 0.

1.

9. The method for capturing voice keywords and assisting in evidence retrieval in interrogation scenarios according to claim 1, characterized in that, S7 also includes file-based page turning: it generates text from voice commands using the S3 transcription model, parses page numbers or chapter commands using regular expressions, calls the file system API to achieve automatic page turning, and fine-tunes the command recognition accuracy through a calibration library.

10. An auxiliary device for capturing voice keywords and retrieving evidence in interrogation scenarios, characterized in that it applies the auxiliary method for capturing voice keywords and retrieving evidence in interrogation scenarios according to any one of claims 1 to 9, wherein the auxiliary device for capturing voice keywords and retrieving evidence in interrogation scenarios comprises: The hardware processing unit is used to perform system initialization and hardware self-test, including a microphone array, a GPU server and a 4K display terminal. The microphone array is used to collect voice signals in the interrogation scene. The storage module includes an in-memory database and a relational database. The in-memory database is used to store the court hearing hot words database entered by the court clerk, and the relational database is used to store the evidence database and the evidence cluster index table generated by cluster preprocessing. The speech processing engine is used to perform noise reduction, framing, and windowing of speech signals, and to extract the Mel frequency cepstral coefficient feature sequence. The natural language processing engine includes a Transformer model and a Bi-LSTM+CRF model. The Transformer model is used to transcribe the Mel frequency cepstral coefficient feature sequence into a text stream and combine it with a hot word library to generate hot word enhanced text. The Bi-LSTM+CRF model is used to identify keywords from the hot word enhanced text and combine it with a BERT model to parse semantics. The evidence retrieval engine is used to obtain semantic cluster information from the evidence cluster index table based on keywords and their type labels, filter candidate clusters through DBSCAN clustering, and generate an evidence retrieval score list by integrating VSM, DSSM and clustering ranking algorithms. The interactive processing unit is used to trigger the manual confirmation process when the search evidence retrieval score list meets the preset conditions, record the confirmation results and update the evidence database similarity, and display the evidence content on the display terminal according to the confirmation results. The case file interface unit is used to interface with the electronic case file system to enable voice command control for page turning.

Images