Method and equipment for assisting in capturing voice keywords and calling evidence in interrogation scene

Abstract

The invention provides an interrogation scene voice keyword capturing and evidence calling auxiliary method and device. The method comprises the steps that S1, system initialization and parameter configuration are carried out; s2, voice signal acquisition and preprocessing; s3, converting voice into text and enhancing hot words; s4, keyword recognition and semantic understanding; s5, performing evidence library retrieval and multi-algorithm fusion matching; s6, performing multi-evidence fuzzy matching processing; and S7, automatically displaying the evidence. According to the invention, the voice transfer accuracy is improved, and the keyword omission ratio is reduced; the multi-evidence screening time is shortened, the evidence similarity is automatically adjusted after confirmation by a judge, and the accuracy of subsequent similar query is improved; the instruction recognition accuracy of voice control file page turning is improved, and court trial interruption times are reduced.

Description

Technical Field

[0001] The present invention relates to the field of speech and text processing technology, and in particular to an auxiliary method and device for capturing speech keywords and retrieving evidence in an interrogation scenario. Background Art

[0002] During the current interrogation process, voice keyword capture and evidence retrieval face the following technical bottlenecks:

[0003] Inefficient voice processing: Traditional solutions rely on clerks to manually record court audio, which can lead to problems such as missing keywords (such as the legal term "intentional homicide" being mistakenly recorded as "intentional injury") and recording delays (missing information when speaking too quickly).

[0004] Semantic ambiguity in evidence retrieval: Existing retrieval systems are mostly based on keyword word form matching (such as only matching the text of "dagger"), lacking semantic understanding capabilities and unable to associate the semantic equivalence between "murder weapon" and "dagger".

[0005] Lack of multi-evidence matching processing: When there are multiple similar pieces of evidence (such as multiple reports related to "murder weapons"), the traditional system cannot automatically sort and recommend them. Judges need to manually review them one by one, which increases the average time required for a single case.

[0006] The interactive mode of the case file is lagging behind: turning the pages of the case file relies on manual operation and cannot respond to voice commands. The trial process is reduced in efficiency due to frequent interruptions, affecting the continuity of the trial.

[0007] Limitations of existing technical solutions:

[0008] Defects of a single retrieval algorithm: Using only bag-of-words models such as TF-IDF cannot capture the semantic association between "crime tool" and "murder weapon", resulting in low retrieval accuracy for semantically ambiguous words (such as "dagger" and "fruit knife").

[0009] Insufficient application of clustering technology: semantic clustering has not been performed on the evidence database, and the time required to search the entire database increases exponentially with the growth of data volume, which cannot meet the real-time requirements of the trial.

[0010] Lack of human-computer collaboration mechanism: There is a lack of feedback optimization mechanism after manual confirmation, and historical search errors cannot be accumulated and learned, resulting in difficulty in continuously improving the system accuracy. Summary of the Invention

[0011] The present invention provides an auxiliary method and device for capturing voice keywords and retrieving evidence in an interrogation scenario to solve one or more of the above problems.

[0012] To achieve the above object, the present invention adopts the following technical solutions:

[0013] Auxiliary methods for capturing voice keywords and retrieving evidence in interrogation scenarios include:

[0014] S1. System initialization and parameter configuration: Start the hardware and load the software module. Collect and store the hot word database entered by the court clerk. Perform clustering preprocessing on the evidence database to generate a hot word in-memory database and evidence cluster index table.

[0015] S2. Speech Signal Acquisition and Preprocessing: Collect speech signals from interrogation scenarios, perform noise reduction, frame segmentation, and windowing, and extract Mel-frequency cepstral coefficient feature sequences.

[0016] S3. Speech-to-text and hotword enhancement: The Transformer model is used to transcribe the Mel-frequency cepstral coefficient feature sequence into a text stream. This is then combined with the hotword in-memory database through prefix tree matching and score recalculation to generate hotword-enhanced text.

[0017] S4. Keyword Identification and Semantic Understanding: We use the Bi-LSTM+CRF model to identify keywords from hot-word augmented text, and then combine it with the BERT model to analyze semantics and generate keywords and their type labels.

[0018] S5. Evidence database retrieval and multi-algorithm fusion matching: Based on keywords and their type labels, semantic cluster information is obtained from the evidence cluster index table. Candidate clusters are screened through DBSCAN clustering. The VSM, DSSM, and cluster ranking algorithms are integrated to generate a weighted fusion evidence retrieval score list.

[0019] S6. Multi-evidence fuzzy matching processing: When the highest score in the evidence retrieval score list exceeds the first preset threshold, the evidence is automatically selected. When the highest score does not exceed the first preset threshold, but the scores of the first three pieces of evidence all exceed the second preset threshold and the difference between adjacent scores is less than the third preset threshold, the manual confirmation process is triggered, the confirmation results are recorded, and the similarity of the evidence library is updated;

[0020] S7. Automatic display of evidence: Generate corresponding display content based on the evidence manually confirmed or automatically matched in S6, and display it visually on the display terminal.

[0021] In this specification, the clustering preprocessing of the evidence library in S1 includes: using the DBSCAN algorithm to generate semantic vectors for the evidence library text, constructing a KD tree index and dividing it into semantic clusters, and calculating the cluster centroids and storing them in the evidence cluster index table.

[0022] In this specification, the noise reduction process in S2 adopts the wavelet transform threshold method, and after performing multi-layer decomposition on the speech signal, the high-frequency coefficients are processed using a soft threshold function.

[0023] In this specification, the hot word enhancement in S3 includes: building a prefix tree for the hot words according to the character length, matching the hot words with the sliding window of the transcribed text, calculating the weighted scores of the acoustic model and the language model, and forcibly replacing them with hot words when the threshold is exceeded.

[0024] In this specification, the keyword recognition in S4 adopts the Bi-LSTM+CRF model, extracts features through a bidirectional long short-term memory network, combines sequence labeling with conditional random fields, and outputs the BIOES tag of the keyword.

[0025] In this specification, the multi-algorithm fusion matching in S5 includes: converting query keywords into DSSM vectors, screening candidate clusters based on the evidence cluster index table, calculating TF-IDF weights for evidence within the cluster and adjusting them in combination with cluster relevance, and then performing three-level weighted fusion through DSSM semantic matching and cluster ranking scores.

[0026] In this specification, obtaining semantic cluster information from the evidence cluster index table in S5 includes: reading cluster centroid vectors and keyword labels in the evidence cluster index table, and calculating the cosine similarity between the query vector and the cluster centroid to screen candidate clusters.

[0027] In this specification, in S6, the first preset threshold is 0.9, the second preset threshold is 0.7, and the third preset threshold is 0.1.

[0028] In this manual, S7 also includes file turning processing: voice commands are converted into text through S3's transcription model, page numbers or chapter instructions are parsed through regular expressions, the file system API is called to achieve automatic page turning, and the accuracy of instruction recognition is improved through fine-tuning of the calibration library.

[0029] An interrogation scene voice keyword capture and evidence retrieval auxiliary device, applying any of the above-mentioned interrogation scene voice keyword capture and evidence retrieval auxiliary methods, the interrogation scene voice keyword capture and evidence retrieval auxiliary device comprising:

[0030] A hardware processing unit, used to perform system initialization and hardware self-tests, includes a microphone array, a GPU server, and a 4K display terminal. The microphone array is used to collect voice signals in interrogation scenarios.

[0031] A storage module, including an in-memory database and a relational database. The in-memory database is used to store the hot word library of the trial input by the clerk, and the relational database is used to store the evidence library and the evidence cluster index table generated by clustering preprocessing;

[0032] Speech processing engine, used to implement noise reduction, framing, and windowing of speech signals, and extract Mel-frequency cepstral coefficient feature sequences;

[0033] The natural language processing engine includes a Transformer model and a Bi-LSTM+CRF model. The Transformer model is used to transcribe the Mel-frequency cepstral coefficient feature sequence into a text stream and generate hot word enhanced text in combination with the hot word library. The Bi-LSTM+CRF model is used to identify keywords from the hot word enhanced text and parse the semantics in combination with the BERT model.

[0034] Evidence retrieval engine, which is used to obtain semantic cluster information from the evidence cluster index table based on keywords and their type labels, screen candidate clusters through DBSCAN clustering, and integrate VSM, DSSM and cluster ranking algorithms to generate a list of evidence retrieval scores;

[0035] An interactive processing unit, configured to trigger a manual confirmation process when the list of retrieved evidence retrieval scores meets preset conditions, record the confirmation results and update the similarity of the evidence library, and display the evidence content on a display terminal according to the confirmation results;

[0036] The file interface unit is used to connect to the electronic file system and realize voice command control of file page turning.

[0037] In summary, the present invention has at least the following beneficial effects:

[0038] Speech processing accuracy: Wavelet noise reduction and hot word enhancement are used to improve speech transcription accuracy and reduce keyword missed detection rates.

[0039] Evidence retrieval efficiency: Through the fusion of three algorithms (VSM+DSSM+clustering sorting), the recall rate of cross-domain evidence is improved, the retrieval response time of the evidence library is reduced, and the F1 value of physical evidence queries is improved.

[0040] Fuzzy matching efficiency: Dynamic clustering and manual confirmation mechanisms shorten the time for screening multiple pieces of evidence. After the judge confirms, the similarity of evidence is automatically adjusted, which improves the accuracy of subsequent similar queries.

[0041] File interaction efficiency: The accuracy of voice-controlled file page turning command recognition has been improved, reducing the number of courtroom interruptions.

[0042] Full-process automation: shortens the entire chain of processing time from voice collection to evidence presentation, improves efficiency, and saves trial time for a single case. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 The figure is a flow chart of the auxiliary method and device for voice keyword capture and evidence retrieval in interrogation scenarios involved in the present invention.

[0045] Figure 2 Schematic diagram of the flow of speech processing and transcription involved in the present invention.

[0046] Figure 3 The figure is a flowchart of the evidence retrieval and fusion involved in the present invention.

[0047] Figure 4 Schematic diagram of the process of multi-evidence matching involved in the present invention. DETAILED DESCRIPTION

[0048] Hereinafter, only certain exemplary embodiments are briefly described. As will be appreciated by those skilled in the art, the described embodiments may be modified in various ways without departing from the spirit or scope of the embodiments of the present invention. Therefore, the drawings and description are to be regarded as illustrative in nature and not restrictive.

[0049] The disclosure below provides many different embodiments or examples for implementing different structures of the embodiments of the present invention. In order to simplify the disclosure of the embodiments of the present invention, the components and configurations of specific examples are described below. Of course, these are merely examples and are not intended to limit the embodiments of the present invention. In addition, the embodiments of the present invention may repeat reference numerals and / or reference letters in different examples. Such repetition is for the purpose of simplicity and clarity and does not in itself indicate the relationship between the various embodiments and / or configurations discussed.

[0050] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0051] like Figure 1 As shown, this embodiment provides an auxiliary method for capturing voice keywords and retrieving evidence in an interrogation scenario, including:

[0052] S1. System initialization and parameter configuration: Start the hardware and load the software module. Collect and store the hot word database entered by the court clerk. Perform clustering preprocessing on the evidence database to generate a hot word in-memory database and evidence cluster index table.

[0053] S2. Speech Signal Acquisition and Preprocessing: Collect speech signals from interrogation scenarios, perform noise reduction, frame segmentation, and windowing, and extract Mel-frequency cepstral coefficient feature sequences.

[0054] S3. Speech-to-text and hotword enhancement: The Transformer model is used to transcribe the Mel-frequency cepstral coefficient feature sequence into a text stream. This is then combined with the hotword in-memory database through prefix tree matching and score recalculation to generate hotword-enhanced text.

[0055] S4. Keyword Identification and Semantic Understanding: We use the Bi-LSTM+CRF model to identify keywords from hot-word augmented text, and then combine it with the BERT model to analyze semantics and generate keywords and their type labels.

[0056] S5. Evidence database retrieval and multi-algorithm fusion matching: Based on keywords and their type labels, semantic cluster information is obtained from the evidence cluster index table. Candidate clusters are screened through DBSCAN clustering. The VSM, DSSM, and cluster ranking algorithms are integrated to generate a weighted fusion evidence retrieval score list.

[0057] S6. Multi-evidence fuzzy matching processing: When the highest score in the evidence retrieval score list exceeds the first preset threshold, the evidence is automatically selected. When the highest score does not exceed the first preset threshold, but the scores of the first three pieces of evidence all exceed the second preset threshold and the difference between adjacent scores is less than the third preset threshold, the manual confirmation process is triggered, the confirmation results are recorded, and the similarity of the evidence library is updated;

[0058] S7. Automatic display of evidence: Generate corresponding display content based on the evidence manually confirmed or automatically matched in S6, and display it visually on the display terminal.

[0059] In some embodiments, the clustering preprocessing of the evidence library in S1 includes: using the DBSCAN algorithm to generate semantic vectors for the evidence library text, constructing a KD tree index and dividing it into semantic clusters, and calculating the cluster centroids and storing them in the evidence cluster index table.

[0060] In some embodiments, the noise reduction process in S2 adopts a wavelet transform threshold method, and after performing multi-layer decomposition on the speech signal, a soft threshold function is used to process the high frequency coefficients.

[0061] In some embodiments, the hot word enhancement in S3 includes: constructing a prefix tree for the hot words according to the character length, matching the hot words with a sliding window of the transcribed text, calculating the weighted scores of the acoustic model and the language model, and forcibly replacing them with hot words when the threshold is exceeded.

[0062] In some embodiments, the keyword recognition in S4 adopts the Bi-LSTM+CRF model, extracts features through a bidirectional long short-term memory network, combines conditional random fields for sequence labeling, and outputs the BIOES tag of the keyword.

[0063] In some embodiments, the multi-algorithm fusion matching in S5 includes: converting query keywords into DSSM vectors, screening candidate clusters based on the evidence cluster index table, calculating TF-IDF weights for evidence within the cluster and adjusting them in combination with cluster relevance, and then performing three-level weighted fusion through DSSM semantic matching and cluster ranking scores.

[0064] In some embodiments, obtaining semantic cluster information from the evidence cluster index table in S5 includes: reading cluster centroid vectors and keyword labels in the evidence cluster index table, and calculating the cosine similarity between the query vector and the cluster centroid to screen candidate clusters.

[0065] In some embodiments, in S6, the first preset threshold is 0.9, the second preset threshold is 0.7, and the third preset threshold is 0.1.

[0066] In some embodiments, S7 also includes file turning processing: generating text from voice instructions through S3's transcription model, parsing page numbers or chapter instructions through regular expressions, calling the file system API to achieve automatic page turning, and fine-tuning the calibration library to improve the accuracy of instruction recognition.

[0067] The technical concept of the present invention is as follows:

[0068] S1. System initialization and parameter configuration:

[0069] Processing: A hardware cluster (including an 8-channel M-AudioAIR 192|14 microphone array, an NVIDIA A100 GPU server, and a 27-inch 4K LCD display) was launched, and a modular software system was loaded, including a Python-based voice signal processing engine, a TensorFlow-based natural language understanding module, an Elasticsearch-optimized evidence retrieval engine, and an interactive user interface developed with Vue.js. The courtroom clerk used the graphical interactive interface to initialize and configure the trial's hot word library, which covers the core elements of the case, including:

[0070] Party type: such as the names of natural persons such as "Zhang San", "Li Si", "Wang Wu", and legal person names such as "Xuan Technology Co., Ltd."

[0071] Physical evidence: such as specific items like "dagger", "vehicle", "drugs", and abstract categories like "crime tools" and "murder weapon";

[0072] Legal clauses: such as "Article 232 of the Criminal Law" and "Article 108 of the Civil Procedure Law";

[0073] Time type: such as "June 1, 2024", "the day of the incident", and other time expressions.

[0074] The system stores the hot word library in JSON format in a Redis distributed in-memory database (configured as a 3-master 3-slave cluster), achieving millisecond-level retrieval responses. Simultaneously, it triggers the evidence library's clustering preprocessing process, employing the DBSCAN algorithm to perform semantic clustering on the evidence library and establish a semantic cluster index for subsequent retrieval.

[0075] Data flow: hot word library input (clerk interaction) → Redis memory database storage → evidence library DBSCAN clustering preprocessing → cluster index table (EvidenceCluster) generation.

[0076] Specific processing process:

[0077] 1. Hardware Self-Test Process: Utilizing a custom device status inspection protocol (developed based on the TCP / IP protocol stack), the system calibrates the microphone array's sampling rate (default setting: 44.1kHz, with a tolerance of ±0.5kHz), checks server CPU load (threshold set to ≤70%), and GPU memory usage (threshold set to ≤80%). If the self-test passes, the device status manager outputs a binary signal "0x01" indicating system readiness. Otherwise, a red alarm light flashes and a detailed error record containing an error code and timestamp is generated in the log system.

[0078] 2. Hot word library construction (supports two input methods):

[0079] Interactively add one by one: Enter the keyword text and type label through the form interface. For example, enter "Zhang San" and select the "Party" type, and set the priority to "High";

[0080] Batch import: Parse the Excel template file. The template contains three columns: "Keywords", "Type", and "Priority". The sample data is as follows:

[0081] Keywords type Priority Zhang San party high Dagger used in the crime physical evidence high Article 232 of the Criminal Law Legal Terms middle

[0082] 3. Evidence database clustering preprocessing:

[0083] When initialized for the first time, full DBSCAN clustering is automatically triggered:

[0084] Extract the text content of all evidence in the evidence library (including PDF document text, image OCR recognized text, audio and video subtitle text), and generate a 128-dimensional semantic vector through the DSSM model;

[0085] Calculate the Euclidean distance between all vectors, take the median of the distance distribution as the neighborhood radius ε (for example, when the median distance is 0.8, ε = 0.8), and set the minimum number of samples to (N is the total number of evidences, when N = 10000, MinPts = 100);

[0086] The FLANN library is used to construct a KD tree index (the vector dimension is reduced to 64 dimensions through PCA, retaining 95% variance), and the evidence is divided into several semantic clusters and stored in the EvidenceCluster table.

[0087] S2. Voice signal acquisition and preprocessing:

[0088] Processing: An 8-channel microphone array collects interrogation scene speech signals in real time (sampling rate 44.1kHz, 16-bit quantization depth). Ambient noise suppression, time-domain framing, frequency-domain transformation, and feature parameter extraction are performed sequentially to generate a Mel-Frequency Cepstral Coefficient (MFCC) sequence suitable for speech transcription. This processing ensures that the signal-to-noise ratio of the speech signal is improved by ≥15dB to provide high-quality input for subsequent transcription.

[0089] Data flow: microphone array original signal (time domain waveform) → wavelet transform denoising → frame segmentation and windowing processing → FFT frequency domain transform → Mel filter bank processing → DCT transform → MFCC feature vector sequence (dimension is 13×T, T is the total number of frames).

[0090] Specific processing process:

[0091] 1. Wavelet transform noise reduction:

[0092] Perform 5-layer Daubechies wavelet (db4) decomposition on the input speech signal x(t) to obtain the high-frequency coefficient c j,k and the low-frequency coefficient d N1,k , where j and N1 represent the number of layers of wavelet decomposition, and k represents the index of the coefficient of the layer. A soft threshold function is used to process the high-frequency coefficients to suppress environmental noise (such as air conditioning noise and keyboard tapping):

[0093]

[0094] in is the high-frequency wavelet coefficient after soft threshold processing; λ is the threshold, and the calculation formula is σ is the noise standard deviation, estimated by the median method (σ=1.4826×median(|c j,k |)); L is the signal length; I represents the indicator function. Taking a 10-second speech signal as an example, assuming σ = 0.2, L = 441000, then

[0095] The signal-to-noise ratio after noise reduction is calculated as follows:

[0096]

[0097] where x clean (t) is the clean speech signal after noise reduction processing, and t is the time sampling point index;

[0098] Actual measurements show that this processing can increase the SNR of voice signals in office environments from 10dB to over 25dB.

[0099] 2. Framing and windowing:

[0100] The noise reduction signal is divided into frames with a frame length of 25ms (corresponding to 1102 sampling points at a sampling rate of 44.1kHz) and a frame shift of 10ms (441 sampling points), and the frame sequence x is obtained. n (m) (n is the frame index, m is the sample index). A Hamming window is applied to each frame to eliminate boundary discontinuities:

[0101]

[0102] Where w(m) is the value of the Hamming window function at the mth sample point, and M is the total number of samples of a single frame of speech signal; for example, for the nth frame signal, the expression after windowing is x n (m)·w(m), effectively reducing spectrum leakage.

[0103] 3. MFCC feature extraction:

[0104] Fast Fourier Transform (FFT): Perform a 512-point FFT on each windowed frame to convert the time domain signal into a frequency domain representation X n (k1):

[0105]

[0106] where X n (k1) is the frequency domain representation of the n-th frame speech signal, corresponding to the complex value of the k1-th frequency point; e is a natural constant (Euler number), approximately equal to 2.71828; h1 is an imaginary unit.

[0107] Mel filter bank: Converts linear frequency f to Mel frequency f mel =2595log 10 (1+f / 700), construct 40 triangular filters covering the frequency band of 300Hz-8000Hz. Taking 800Hz as an example, its Mel frequency is 2595log 10 (1+800 / 700)≈2595×0.168≈436 mel;

[0108] Logarithm and DCT transform: Take the natural logarithm of the filter output and perform a 13th-order discrete cosine transform (DCT) to obtain the MFCC coefficient c n (l), l is the dimension index of the MFCC coefficient, l = 1,..., 13. The 1st to 12th dimensions reflect the spectral envelope characteristics, and the 13th dimension is the logarithmic energy feature.

[0109] S3. Speech-to-text and hotword enhancement:

[0110] Processing content: Use a pre-trained Transformer speech-to-text model to convert the MFCC feature sequence into a text stream, and perform post-processing in combination with the hot word library preset by the court clerk to correct recognition errors of professional terms (such as misrecognizing "murder weapon" as "chest weapon"), and improve the transcription accuracy to over 95%. The processing process includes four links: model inference, hot word matching, score recalculation, and forced replacement.

[0111] Data flow: MFCC feature sequence (13×T) → Transformer encoder-decoder inference → initial text stream → hot word prefix tree matching → score calculation → hot word enhanced text.

[0112] Specific processing process:

[0113] 1. Transformer model construction: Adopt a 6-layer encoder-6-layer decoder architecture, and each layer contains:

[0114] Multi-head self-attention mechanism: 8 attention heads, the key vector dimension is 64, and the calculation method is:

[0115] MultiHead(Q,K,V)=Concat(head1,...,head8)W O ;

[0116] Among them, Q is the query vector, K is the key vector, V is the value vector, and W O Output projection matrix;

[0117]

[0118] where head i is the output of the i-th attention head, T1 represents the transpose of the matrix, and the correlation degree between Q and K is calculated through the scaled dot-product attention mechanism. Taking the query vector key vector as an example, the scaling factor avoids the disappearance of the Softmax gradient caused by the excessive dot-product result;

[0119] Feed-forward neural network: Two fully connected layers (dimension 64→256→64), the activation function is ReLU, and the calculation expression is FFN(x)=max(0,xW1+b1)W2+b2;

[0120] where x is the input vector, W1 is the weight matrix of the first layer, b1 is the bias vector of the first layer, W2 is the weight matrix of the second layer, and b2 is the bias vector of the second layer.

[0121] Layer Normalization (LayerNorm): Apply LayerNorm(x+SubLayer(x)) after each sublayer to stabilize the training process. SubLayer(x) is the output of the sublayer function.

[0122] 2. Model training process:

[0123] Pre-training phase: 100,000 hours of general speech data (including trial records, court debates, legal lectures, etc.) is used, and the loss function is cross entropy:

[0124]

[0125] Where x1 is the MFCC sequence, y t is the real character at time t, T2 is the sequence length, p(y t |y1,...,y t-1 ,x1) is the conditional probability. The training uses the Adam optimizer with an initial learning rate of 5e-4 and a cosine decay strategy;

[0126] Fine-tuning phase: For 5000 hours of annotated interrogation speech data (including hot word position annotations), freeze the first 4 layers of the encoder, and only fine-tune the last 2 layers and the decoder, with the learning rate reduced to 1e-4.

[0127] 3. Hot word enhancement processing:

[0128] Prefix tree (Trie) construction: Hot words are inserted into the tree structure in descending order of character length. For example, if the hot word library contains "intentional homicide", "murder weapon", and "Zhang San", then "intentional homicide" will have the longest path (5 characters), while "murder weapon" (2 characters) and "Zhang San" (2 characters) will have shorter paths.

[0129] Text traversal matching: A sliding window (window size = maximum length of hot words, e.g., 5) is applied to the transcribed text, searching for matching paths in the Trie tree. For example, the text "He used chest weapons to injure people" matches "chest weapons" when the window size is 2, but the longer hot word "murder weapon" can be found through the Trie tree.

[0130] Score recalculation: Calculate the acoustic model score S for matching hot words acoustic (The output probability of the MFCC of the corresponding speech frame through the acoustic model and the language model score S language The weighted sum of the Transformer language model's predicted probability of hot words:

[0131] S=0.6·S acoustic +0.4·S language ;

[0132] Assuming the acoustic score of "murder weapon" is 0.7 and the language model score is 0.9, then S = 0.6 × 0.7 + 0.4 × 0.9 = 0.42 + 0.36 = 0.78;

[0133] Forced Replacement: When S > 0.8, the replacement is performed. For example, if S = 0.78 < 0.8 in the above example, no replacement is performed. However, if S = 0.85, "Breast Weapon" is replaced with "Weapon".

[0134] S1, S2, and S3 are the speech processing and transcription processes, such as Figure 2 shown.

[0135] S4. Keyword recognition and semantic understanding:

[0136] Processing: Identify key information units (such as parties, physical evidence, and time) from hot-word-enhanced text, perform sequence labeling using a Bi-LSTM+CRF model, and combine the BERT semantic model to analyze the keyword context and determine its logical type and role in the case. This process must achieve an accuracy rate of 95% or higher in keyword boundary identification and 92% or higher in type classification.

[0137] Data flow: hot word enhanced text → character-level Word2Vec embedding → Bi-LSTM feature extraction → CRF sequence labeling → BERT semantic encoding → keyword type label (such as "party", "physical evidence", "time").

[0138] Specific processing process:

[0139] 1. Bi-LSTM+CRF model construction:

[0140] Word embedding layer: Use 300-dimensional Word2Vec vectors (pre-trained on a 1 billion-word interrogation text corpus). OOV words are initialized with random Gaussian vectors with a mean of 0 and a variance of 0.1. For example, the embedding vector for the uncommon word "cyanide" is randomly generated.

[0141] Bi-LSTM layer: Bidirectional long short-term memory network, hidden layer dimension 256, forward and reverse output concatenation:

[0142]

[0143] in represents the hidden state of the forward LSTM at time t1, represents the hidden state of the reverse LSTM at time t1, x t1 is the embedding vector of the character at time t1, h t1 is the concatenated feature vector;

[0144] CRF layer: defines the label transfer matrix (L1 is the number of labels, +2 is the starting label <start>and the termination tag <end>), solve the optimal label sequence through the Viterbi algorithm:

[0145]

[0146] in represents the optimal label sequence, y1 represents any possible label sequence, arg max y1 To find the solution that maximizes the objective function for variable y1, t1 represents the time step index, T3 represents the sequence length, P(y t1 ∣h t1 Given a hidden state h t1 When the label y t1 The probability of emission.

[0147] Labeling system: Using BIOES labeling method, specific labels include:

[0148] B-Party (Party Entity Start)

[0149] I-Party (Party Entity Continuation)

[0150] E-Party (Party entity end)

[0151] S-Party (single-character party entity)

[0152] B-Physical evidence, I-Physical evidence, E-Physical evidence, S-Physical evidence

[0153] B-time, I-time, E-time, S-time, etc.

[0154] 2. Model training process:

[0155] Annotated data: 5,000 interrogation records, annotated by legal experts with keyword boundaries and types, with a labeling accuracy of ≥98%. For example, the text "Zhang San used a dagger to commit a crime on June 1, 2024" is annotated as:

[0156] "Zhang San" / B-party, "on" / O, "June 1, 2024" / B-time, "used" / O, "dagger" / B-physical evidence, "committed the crime" / O;

[0157] Loss function: CRF negative log-likelihood, combined with L2 regularization (λ1=0.001) to prevent overfitting.

[0158] 3. Deepening semantic understanding:

[0159] BERT encoding: Using the legal field-specific BERT model (based on Chinese-Law-BERT fine-tuning), input the keyword and its context of 5 words before and after it to generate a 768-dimensional semantic vector. For example, the context of the keyword "dagger" is "[use] dagger [commit a crime]", and after inputting it into BERT, the vector is obtained.

[0160] Type classification: The semantic vector is passed through a fully connected layer (768→128→L3 (L3 is the total number of keyword type categories)) and then softmax classification is performed. The loss function is cross entropy. Taking the time type classification as an example, when the input is "June 1, 2024", the fully connected layer outputs [0.02, 0.95, 0.03], corresponding to the probabilities of "party", "time", and "physical evidence". The maximum value of 0.95 is taken as the type.

[0161] Rule engine supplement: preset regular rules to match specific types, such as:

[0162] Time expression: / \d{4}year\d{1,2}month\d{1,2}day / ;

[0163] Matching logic:

[0164] \d{4}: matches a 4-digit year (such as 2024);

[0165] \d{1,2}: matches 1-2 digits of month / day (such as June, 01);

[0166] The separators "year", "month" and "day" are fixed characters to ensure format accuracy.

[0167] Amount expression: / ￥? \d+(\.\d{1,2})? yuan? / ;

[0168] Matching logic:

[0169] ￥?: The RMB symbol (￥) can be selected, such as "￥5000" or "5000 yuan";

[0170] \d+: matches more than one digit (integer part);

[0171] (\.\d{1,2})?: Optional decimal part (1-2 digits), such as ".00" or ".5";

[0172] Yuan? : You can choose the unit "Yuan", such as "5000" or "5000 Yuan", both are valid.

[0173] Name of physical evidence: / [knife, gun, stick, axe, hook, fork]+\s*murder weapon? / ;

[0174] Matching logic:

[0175] [Knives, guns, clubs, axes, tomahawks, hooks, and tridents]+: Match at least one weapon character (e.g., "knife", "gun", "axe");

[0176] \s*: Zero or more spaces (compatible with extra spaces in "dagger weapon");

[0177] Weapon??: Optional "weapon" suffix (e.g., both "machete" and "machete weapon" are matched).

[0178] The results of rule matching and deep learning are fused with a weight of 0.3.

[0179] S5. Evidence base retrieval and multi - algorithm fusion matching:

[0180] Processing content: Based on the keywords and types identified in S4, candidate evidence clusters are screened through DBSCAN semantic clustering, and the vector space model (VSM), deep semantic matching (DSSM), and clustering sorting algorithms are fused to generate a weighted - fusion evidence retrieval result. This step needs to solve the problem of cross - domain evidence association (such as associating "weapon photos" in an "intentional injury" case) and improve the semantic retrieval accuracy.

[0181] Data flow: Keyword set → DSSM query vector generation → Cluster similarity calculation → Candidate cluster screening → VSM retrieval → DSSM fine - screening → Clustering sorting → Three - level fusion score calculation → Evidence list (sorted in descending order of scores).

[0182] The specific processing process is as Figure 3 shown:

[0183] 1. Evidence base clustering pre - processing and dynamic maintenance:

[0184] DBSCAN clustering model:

[0185] Model construction: Using the evidence semantic vectors generated by DSSM as features, the determination method of core parameters:

[0186] Neighborhood radius ε: Determined by the inflection point of the K - distance graph (K1 = 5), taking the upper quartile of the Euclidean distances of all vectors. For example, when the upper quartile of the distance distribution is 1.2, ε = 1.2;

[0187] Minimum number of samples MinPts: (N is the total number of evidences, when N = 5000, MinPts = 71).

[0188] Training process:

[0189] ]>For all evidences in the evidence base, generate 128 - dimensional semantic vectors through the DSSM model (with the same architecture as the query vector);

[0190] Build a KD - tree index using the FLANN library (reduce the dimension to 64 through PCA, retaining 95% variance) to accelerate neighborhood search;

[0191] Traverse each vector, calculate the number of points within its ε - neighborhood. If it is ≥ MinPts, mark it as a core point; otherwise, it is a boundary point or a noise point;

[0192] The core points and their density - reachable points form a cluster C k2 , calculate the centroid

[0193] where k2 is the cluster index, μ k2 is the centroid vector of cluster C k2 , and f D (d) is the output vector of the deep semantic matching (DSSM) model for evidence d.

[0194] Application process:

[0195] Incremental update: When the amount of new / removed evidence in the evidence base > 10%, trigger incremental clustering:

[0196] Calculate the distance between the new evidence vector and the centroids of existing clusters. If < ε, add it to the corresponding cluster and update the centroid:

[0197]

[0198] where n k2 represents the number of original evidence in the k2 - th cluster C k2 , and f D (d new ) is the DSSM semantic vector of the new evidence d new ;

[0199] For example, if the k2 - th cluster C1 originally has 100 pieces of evidence, the centroid μ1 = [0.1, 0.​​​​​​​​​​​​​​​​ Q (q);

[0204] Cluster similarity calculation: traverse all cluster centroids μ k2 , calculate the cosine similarity:

[0205]

[0206] Where q is the keyword set;

[0207] For example, the query vector f Q The dot product with the centroid μ1 of cluster C1 is 80, f Q The norm is 10, the μ1 norm is 8, then Sim = 80 / (10×8) = 1.0;

[0208] Candidate cluster generation: Select the top 5 clusters with similarity > 0.4 as C cand If none of the conditions are met, the search will be expanded to the entire database.

[0209] VSM search enhancements:

[0210] TF-IDF weight calculation:

[0211]

[0212] Where t2 is the target keyword, ∑ t2' n t2',d is the total number of evidence words, and D is the total number of evidence in the evidence database;

[0213] For example, the keyword "dagger" appears 3 times in evidence d, and the total number of words in d is 100, then TF = 3 / 100 = 0.03; if 10 of the 1000 pieces of evidence in the evidence database contain "dagger", then IDF = log(1000 / 11) ≈ log(90.91) ≈ 4.51, and TF-IDF = 0.03×4.51 ≈ 0.135;

[0214] Cluster relevance adjustment: According to the similarity Sim(q,C k2 ), enhance the TF-IDF weight:

[0215] TF-IDF'(t2,d)=TF-IDF(t2,d)×(1+0.5·Sim(q,C k2 ));

[0216] If Sim = 0.8, then adjusted TF-IDF' = 0.135 × (1 + 0.5 × 0.8) = 0.135 × 1.4 = 0.189;

[0217] Similarity calculation: construct query vector Q and evidence vector D d 's cosine similarity:

[0218]

[0219] Among them D d ' is the TF-IDF vector of the adjusted evidence d;

[0220] 3.DSSM semantic screening and three-level fusion:

[0221] DSSM similarity calculation:

[0222] Semantic matching of evidence to query:

[0223]

[0224] Assume f D (d)·f Q (q) = 100, |f D (d)|=10,|f Q (q)|=10, then Sim D =1.0;

[0225] Cluster belonging degree correction: The closer the semantic distance between the evidence and its cluster, the higher the credibility:

[0226] Sim D '(d,q)=Sim D (d,q)×(1+0.3·Sim(d,C k2 ));

[0227] Among them, Sim(d,C k2 )=1-||f D (d)-μ k2 || / max d istance is the maximum distance within the cluster. If ||f D (d)-μ k2 ||=0.5,max d If istance=1.0, then Sim(d,C k2 )=0.5,Sim D '=1.0×(1+0.3×0.5)=1.15.

[0228] Cluster-assisted sorting:

[0229] Intra-cluster relative distance: measures the relative position of the evidence to the query within the cluster:

[0230]

[0231] Add a very small constant ∈=1e-6 to the denominator to avoid division by zero, for example, ||f D -f Q ||=0.8,||f D -μ k2 || = 0.4, then ClusterDist = 0.8 / (0.4 + 1e-6) = 2.0;

[0232] Sorting score: the closer the distance, the higher the score, which decays exponentially:

[0233] RankScore(d,q)=exp(-ClusterDist(d,q))=exp(-2.0)≈0.135;

[0234] Three-level fusion model:

[0235] Dynamic weight allocation: Automatically adjust according to the keyword type distribution output by S4:

[0236] Physical evidence query weights ("physical evidence" tag > 40%): 0.4, 0.5, 0.1;

[0237] Text query weights ("contract", "agreement", etc.): 0.3, 0.6, 0.1;

[0238] General query weight: 0.3, 0.3, 0.4;

[0239] Fusion formula:

[0240] FinalScore(d,q)=w1·Sim V '(d,q)+w2·Sim D '(d,q)+w3·RankScore(d,q);

[0241] Among them, w1, w2, and w3 are fusion weight coefficients;

[0242] For example, Sim V '=0.8,Sim D '=1.15, RankScore=0.135, weights of physical evidence (0.4, 0.5, 0.1), then FinalScore=0.4×0.8+0.5×1.15+0.1×0.135=0.32+0.575+0.0135=0.9085.

[0243] S6. Multi-evidence fuzzy matching processing:

[0244] Processing: Based on the evidence fusion score list output by S5, automatic matching is first performed. If the score meets the high-confidence criteria, evidence is directly selected. If not, fuzzy matching is performed, triggering manual confirmation or negative feedback. This process requires a hierarchical decision-making process of automatic matching and manual confirmation to ensure the efficiency and accuracy of evidence selection.

[0245] Data flow: Evidence fusion score list (FinalScore sorting) → Automatic matching condition detection → Automatic selection of evidence ID / manual confirmation of evidence ID → Output of evidence ID to S7 → Evidence display and rendering.

[0246] Specific processing procedures, such as Figure 4 As shown:

[0247] 1. Automatic matching condition detection:

[0248] High confidence automatic selection: When the highest score in the list, FinalScore_max, is greater than 0.9, the system automatically selects the evidence without manual intervention. For example, if a piece of evidence has a fusion score of 0.92, it is directly marked as selected and the following operations are performed:

[0249] Record automatic selection log: including evidence ID, selection time, score value, trigger condition (FinalScore max >0.9);

[0250] Call the evidence display module (S7) to render and increase the similarity of the evidence by 0.1 (formula: new sim =old sim +0.1), strengthening the matching priority of subsequent similar queries;

[0251] Compare the scores of the evidence with those of similar historical cases. If the difference exceeds 15% (e.g., the current score is 0.92, the historical average is 0.78, and the difference is 17.9% > 15%), the evidence will be marked as "pending review" and reviewed by the clerk.

[0252] Threshold Description: The 0.9 threshold is based on historical data statistics. When the score is > 0.9, the accuracy of manual review reaches 98.7%, which can be safely selected automatically.

[0253] 2. Manual confirmation of fuzzy matching:

[0254] Condition detection: Manual confirmation is triggered if the following conditions are met:

[0255] FinalScore(d1,q)>0.7;

[0256] FinalScore(d2,q)>0.7;

[0257] FinalScore(d1,q)-FinalScore(d2,q)<0.1;

[0258] FinalScore(d2,q)-FinalScore(d3,q)<0.1;

[0259] Among them, d1, d2, and d3 represent the top three pieces of evidence with the highest fusion scores in the retrieval results;

[0260] For example, the scores of the first three pieces of evidence are 0.78, 0.77, and 0.76, all of which are greater than 0.7 and the adjacent score differences are 0.01 and 0.01<0.1, triggering confirmation.

[0261] Threshold Description: 0.7 is the medium confidence threshold, and a difference of 0.1 ensures close evidence relevance. After testing, the necessity of manual confirmation under this condition reached 83.5%.

[0262] Manual confirmation process:

[0263] The interface displays the first five pieces of evidence (including score, type, and preview), with keywords highlighted;

[0264] Three operations are supported:

[0265] Select evidence: record the confirmer and time, score increases by 0.2 (formula: new sim =old sim +0.2);

[0266] No match: negative feedback is recorded, the relevant keyword combination is added to the blacklist, and the subsequent search weight is reduced by 0.3;

[0267] Skip: Do not process for now and return to the search result list:

[0268] 3. Evidence ID output mechanism: Whether automatically selected or manually confirmed, a unique evidence ID (such as "EVID-20250630-001") is generated and synchronized to the S7 module through the message queue (RabbitMQ) to ensure: Evidence ID S6输出 =EvidenceID S7输入 ;

[0269] 4. Exception handling: If no confirmation receipt is received from S7 within 30 seconds, the evidence ID will be automatically resent, with a maximum of 3 retries and a retry interval of 1 second.

[0270] S7. Automatic display of evidence and automatic file translation:

[0271] Processing: Based on evidence confirmed by S6 or automatically matched with high-scoring evidence, the system displays the evidence by type on the display terminal (e.g., highlighting keywords in PDF documents or marking areas in images). The system also connects to the electronic dossier system to automatically turn pages and locate files via voice commands. Operations such as page jumps and chapter searches are supported, with command recognition accuracy exceeding 90%.

[0272] Data flow: confirm evidence ID → read from distributed file system → process in rendering engine → visualize;

[0273] Voice command → S3 transcription model → Instruction text → Regular parsing → File system API call → Page number control.

[0274] Specific processing process:

[0275] 1. Evidence display module:

[0276] File reading: Read the original file from the Ceph distributed file system according to the evidence ID. Supported formats include:

[0277] Document types: PDF, DOCX, RTF, TXT;

[0278] Image types: JPG, PNG, TIFF, BMP;

[0279] Audio / video types: MP4, AVI, WAV, MP3.

[0280] Rendering strategy:

[0281] Document types: Render PDF files using PDF.js, and highlight text through the keyword position information (character offset) output by S4. For example, if the keyword "dagger" is at the 120th character position on page 3 of the PDF, a yellow background + red border mark is added during rendering;

[0282] Image types: Draw using Canvas and mark the area according to the keyword semantic matching result. For example, "dagger" corresponds to a rectangular box with coordinates (100, 200)-(300, 400) in the image, and the border color is red with a line width of 2px;

[0283] Audio / video types: Analyze key frames and mark the time points where keywords appear on the timeline (error ≤ 500ms). For example, if the keyword "dagger" appears at 00:01:23 in the video, a red mark point is displayed at that position on the timeline, and clicking it can jump.

[0284] 2. File system follow - up turning function: <于

[0285] Instruction recognition: For voice - to - text conversion, use the Transformer model of S3, and add a special hot - word library for the file system, including: Page - related: "Page 1", "Page 10", "Page 123"; Chapter - related: "Chapter 1", "Appendix A", "Witness testimony section"; Operation - related: "Turn to", "View", "Return". Text normalization: Convert "Turn to the third page" to "Turn to Page 3" to unify the format for parsing.

[0286] Instruction parsing:

[0287] Page number instruction: Use regular expression / turn to page (\dY+) / capture target page number p. For example, if you input "Please turn to page 5", p = 5 after regular expression matching; dY is the page number.

[0288] Chapter command: Regular match / view (.*?) chapter? section? / , such as "view chapter 3" captures the chapter name s = "Chapter 3", query the file directory index table (structure: {chapter: "Chapter 3", page: 25}) to get the corresponding page number p s =25.

[0289] Instruction execution: Call the file system API: scrollToPage(p), supporting smooth scrolling animation (lasting 500ms);

[0290] Accuracy calibration: Record the text i1 of each instruction and the actual jump page j1. If |p-j1|>1, save it in the calibration library (format: {instruction:"Turn to page 5", actual_page:7, timestamp:"2024-06-01 10:30:00"});

[0291] The language model part of the Transformer model is automatically fine-tuned using the calibration library data every morning. The loss function is:

[0292]

[0293] Where C2 is the calibration library sample, p i1 is the model’s predicted distribution of page numbers for instruction i1, and j1 is the correct page number.

[0294] An interrogation scene voice keyword capture and evidence retrieval auxiliary device, applying any of the above-mentioned interrogation scene voice keyword capture and evidence retrieval auxiliary methods, the interrogation scene voice keyword capture and evidence retrieval auxiliary device comprising:

[0295] A hardware processing unit, used to perform system initialization and hardware self-tests, includes a microphone array, a GPU server, and a 4K display terminal. The microphone array is used to collect voice signals in interrogation scenarios.

[0296] A storage module, including an in-memory database and a relational database. The in-memory database is used to store the hot word library of the trial input by the clerk, and the relational database is used to store the evidence library and the evidence cluster index table generated by clustering preprocessing;

[0297] Speech processing engine, used to implement noise reduction, framing, and windowing of speech signals, and extract Mel-frequency cepstral coefficient feature sequences;

[0298] The natural language processing engine includes a Transformer model and a Bi-LSTM+CRF model. The Transformer model is used to transcribe the Mel-frequency cepstral coefficient feature sequence into a text stream and generate hot word enhanced text in combination with the hot word library. The Bi-LSTM+CRF model is used to identify keywords from the hot word enhanced text and parse the semantics in combination with the BERT model.

[0299] Evidence retrieval engine, which is used to obtain semantic cluster information from the evidence cluster index table based on keywords and their type labels, screen candidate clusters through DBSCAN clustering, and integrate VSM, DSSM and cluster ranking algorithms to generate a list of evidence retrieval scores;

[0300] An interactive processing unit, configured to trigger a manual confirmation process when the list of retrieved evidence retrieval scores meets preset conditions, record the confirmation results and update the similarity of the evidence library, and display the evidence content on a display terminal according to the confirmation results;

[0301] The file interface unit is used to connect to the electronic file system and realize voice command control of file page turning.

[0302] The above embodiments are intended to illustrate the present invention, not to limit the present invention. Therefore, changes in illustrative values ​​or substitutions of equivalent components should still fall within the scope of the present invention.

[0303] From the above detailed description, it will be clear to those skilled in the art that the present invention can indeed achieve the aforementioned objectives and is in compliance with the provisions of the Patent Law.

[0304] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic inventive concepts. Therefore, the appended claims are intended to be interpreted as covering the preferred embodiments and all changes and modifications that fall within the scope of the invention. The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

[0305] It should be noted that the above description of the relevant processes is for illustration and explanation only and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to the processes under the guidance of this specification. However, such modifications and changes are still within the scope of this specification.

[0306] The basic concepts have been described above. It will be apparent to those skilled in the art after reading this application that the above disclosures are merely illustrative and do not constitute limitations on this application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and amendments to this application. Such modifications, improvements, and amendments are suggested in this application and remain within the spirit and scope of the exemplary embodiments of this application.

[0307] At the same time, this application uses specific terms to describe the embodiments of this application. For example, "one embodiment," "an embodiment," and / or "some embodiments" refer to a feature, structure, or characteristic related to at least one embodiment of this application. Therefore, it should be emphasized and noted that "one embodiment," "an embodiment," or "an alternative embodiment" mentioned twice or more in different places in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics of one or more embodiments of this application may be appropriately combined.

[0308] In addition, it will be understood by those skilled in the art that various aspects of the present application can be illustrated and described by a number of patentable categories or situations, including any new and useful process, machine, product or combination of substances, or any new and useful improvement thereof. Therefore, various aspects of the present application can be implemented entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software can all be referred to as "units", "modules" or "systems". In addition, various aspects of the present application can take the form of a computer program product embodied in one or more computer-readable media, wherein computer-readable program code is contained therein.

[0309] The computer program code required for the operation of each part of the application can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python, conventional procedural programming languages ​​such as C programming language, VisualBasic, Fortran2103, Perl, COBOL2102, PHP, ABAP, dynamic programming languages ​​such as Python, Ruby and Groovy or other programming languages. The program code can be run completely on the user's computer, or run on the user's computer as an independent software package, or run partly on the user's computer and partly on a remote computer, or run completely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any network form, such as a local area network (LAN) or a wide area network (WAN), or be connected to an external computer (such as by the Internet), or in a cloud computing environment, or used as a service such as software as a service (SaaS).

[0310] In addition, unless expressly stated in the claims, the order of the processing elements and sequences described in this application, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this application. Although the above disclosure discusses some embodiments of the invention that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this application. For example, although the implementation of the various components described above can be embodied in a hardware device, it can also be implemented as a pure software solution, for example, installation on an existing server or mobile device.

[0311] Similarly, it should be noted that in order to simplify the presentation of this disclosure and thereby facilitate understanding of one or more of the invention's embodiments, the foregoing descriptions of the embodiments of this disclosure sometimes combine multiple features into a single embodiment, figure, or description thereof. However, this approach should not be interpreted as reflecting an intention that the claimed subject matter requires more features than expressly recited in each claim. Rather, the subject matter of the invention may possess fewer features than the single embodiment described above.< / end> < / start>

Claims

1. An auxiliary method for capturing speech keywords and retrieving evidence in interrogation scenarios, characterized by: include: S1. System initialization and parameter configuration: Start the hardware and load the software module. Collect and store the hot word database entered by the court clerk. Perform clustering preprocessing on the evidence database to generate a hot word in-memory database and evidence cluster index table. S2. Speech Signal Acquisition and Preprocessing: Collect speech signals from interrogation scenarios, perform noise reduction, frame segmentation, and windowing, and extract Mel-frequency cepstral coefficient feature sequences. S3. Speech-to-text and hotword enhancement: The Transformer model is used to transcribe the Mel-frequency cepstral coefficient feature sequence into a text stream. This is then combined with the hotword in-memory database through prefix tree matching and score recalculation to generate hotword-enhanced text. S4. Keyword Identification and Semantic Understanding: We use the Bi-LSTM+CRF model to identify keywords from hot-word augmented text, and then combine it with the BERT model to analyze semantics and generate keywords and their type labels. S5. Evidence database retrieval and multi-algorithm fusion matching: Based on keywords and their type labels, semantic cluster information is obtained from the evidence cluster index table. Candidate clusters are screened through DBSCAN clustering. The VSM, DSSM, and cluster ranking algorithms are integrated to generate a weighted fusion evidence retrieval score list. S6. Multi-evidence fuzzy matching processing: When the highest score in the evidence retrieval score list exceeds the first preset threshold, the evidence is automatically selected. When the highest score does not exceed the first preset threshold, but the scores of the first three pieces of evidence all exceed the second preset threshold and the difference between adjacent scores is less than the third preset threshold, the manual confirmation process is triggered, the confirmation results are recorded, and the similarity of the evidence library is updated; S7. Automatic display of evidence: Generate corresponding display content based on the evidence manually confirmed or automatically matched in S6, and display it visually on the display terminal.

2. The method for assisting in capturing voice keywords and retrieving evidence in interrogation scenarios according to claim 1, characterized in that: The clustering preprocessing of the evidence library in S1 includes: using the DBSCAN algorithm to generate semantic vectors for the evidence library text, constructing a KD tree index and dividing it into semantic clusters, and calculating the cluster centroids and storing them in the evidence cluster index table.

3. The method for assisting in capturing speech keywords and retrieving evidence in interrogation scenarios according to claim 1, characterized in that: The noise reduction process in S2 adopts the wavelet transform threshold method, and after performing multi-layer decomposition on the speech signal, the high-frequency coefficients are processed using a soft threshold function.

4. The method for assisting voice keyword capture and evidence retrieval in interrogation scenarios according to claim 1 is characterized in that: The hot word enhancement in S3 includes: constructing a prefix tree for the hot words according to the character length, matching the hot words with the sliding window of the transcribed text, calculating the weighted scores of the acoustic model and the language model, and forcibly replacing them with hot words when the threshold is exceeded.

5. The method for assisting in capturing voice keywords and retrieving evidence in interrogation scenarios according to claim 1, characterized in that: The keyword recognition in S4 adopts the Bi-LSTM+CRF model, extracts features through a bidirectional long short-term memory network, combines conditional random fields for sequence labeling, and outputs the BIOES tag of the keyword.

6. The method for assisting voice keyword capture and evidence retrieval in interrogation scenarios according to claim 1 is characterized in that: The multi-algorithm fusion matching in S5 includes: converting query keywords into DSSM vectors, screening candidate clusters based on the evidence cluster index table, calculating TF-IDF weights for evidence within the cluster and adjusting them in combination with cluster relevance, and then performing three-level weighted fusion through DSSM semantic matching and cluster ranking scores.

7. The method for assisting voice keyword capture and evidence retrieval in interrogation scenarios according to claim 1 is characterized in that: The step of obtaining semantic cluster information from the evidence cluster index table in S5 includes: reading cluster centroid vectors and keyword labels in the evidence cluster index table, and calculating the cosine similarity between the query vector and the cluster centroid to screen candidate clusters.

8. The method for assisting voice keyword capture and evidence retrieval in interrogation scenarios according to claim 1 is characterized in that: In S6, the first preset threshold is 0.9, the second preset threshold is 0.7, and the third preset threshold is 0.

1.

9. The method for assisting voice keyword capture and evidence retrieval in interrogation scenarios according to claim 1 is characterized in that: S7 also includes on-the-fly file turning processing: voice commands are converted into text through S3's transcription model, page numbers or chapter instructions are parsed through regular expressions, the file system API is called to achieve automatic page turning, and the accuracy of instruction recognition is improved through fine-tuning of the calibration library.

10. An interrogation scene voice keyword capture and evidence retrieval auxiliary device, characterized in that the interrogation scene voice keyword capture and evidence retrieval auxiliary method according to any one of claims 1 to 9 is applied, and the interrogation scene voice keyword capture and evidence retrieval auxiliary device comprises: A hardware processing unit, used to perform system initialization and hardware self-tests, includes a microphone array, a GPU server, and a 4K display terminal. The microphone array is used to collect voice signals in interrogation scenarios. A storage module, including an in-memory database and a relational database. The in-memory database is used to store the hot word library of the trial input by the clerk, and the relational database is used to store the evidence library and the evidence cluster index table generated by clustering preprocessing; Speech processing engine, used to implement noise reduction, framing, and windowing of speech signals, and extract Mel-frequency cepstral coefficient feature sequences; The natural language processing engine includes a Transformer model and a Bi-LSTM+CRF model. The Transformer model is used to transcribe the Mel-frequency cepstral coefficient feature sequence into a text stream and generate hot word enhanced text in combination with the hot word library. The Bi-LSTM+CRF model is used to identify keywords from the hot word enhanced text and parse the semantics in combination with the BERT model. Evidence retrieval engine, which is used to obtain semantic cluster information from the evidence cluster index table based on keywords and their type labels, screen candidate clusters through DBSCAN clustering, and integrate VSM, DSSM and cluster ranking algorithms to generate a list of evidence retrieval scores; An interactive processing unit, configured to trigger a manual confirmation process when the list of retrieved evidence retrieval scores meets preset conditions, record the confirmation results and update the similarity of the evidence library, and display the evidence content on a display terminal according to the confirmation results; The file interface unit is used to connect to the electronic file system and realize voice command control of file page turning.

Images