A power dispatch recording key information automatic extraction and retrieval method, system, device and medium

Abstract

The application discloses a power dispatch recording key information automatic extraction and retrieval method, system, device and medium, and the method comprises the steps of obtaining a power dispatch recording file, preprocessing the power dispatch recording file to obtain a standardized audio file, using a domain adaptive speech recognition engine to perform speech recognition on the standardized audio file to obtain recognized text, performing information extraction on the recognized text to obtain a structured information record, converting the structured information record into a multi-modal index storage structure, obtaining a structured query condition, parallelly retrieving the multi-modal index storage structure based on the structured query condition to obtain a candidate record set, calculating the candidate record set to obtain a target record, intercepting a corresponding recording segment according to audio positioning information of the target record and returning a retrieval result, realizing automatic extraction and retrieval of power dispatch recording key information, and effectively supporting dispatch decision and accident tracing.

Description

Technical Field

[0001] This invention relates to the intersection of power system information technology and artificial intelligence technology, and in particular to a method, system, device and medium for automatic extraction and retrieval of key information from power dispatch recordings. Background Technology

[0002] Power dispatch centers generate a large number of dispatch call recordings daily, containing crucial information such as fault reports, operational instructions, and status confirmations. Currently, the utilization of these recordings mainly relies on manual listening and transcription, which is inefficient, time-consuming, and labor-intensive, and cannot achieve rapid information location and retrieval in emergency situations. While general speech-to-text services exist, they have significant shortcomings in the professional scenario of power dispatching. These include low accuracy in recognizing power-related technical terms, equipment numbers, and location names; the inability to automatically extract key information items from long, transcribed texts; and a lack of ability to efficiently and multi-dimensionally retrieve the extracted key information. This results in massive amounts of dispatch recording data becoming "data silos," with their core value not being effectively tapped.

[0003] Therefore, there is an urgent need for a method that can automatically, accurately, and efficiently extract structured key information from power dispatch recordings and provide intelligent retrieval, in order to improve the informatization level of dispatch operations and emergency response capabilities. This invention provides a method for automatically extracting and retrieving key information from power dispatch recordings. Its purpose is to achieve automatic transcription of dispatch recordings, accurate structured extraction of key information, and rapid intelligent retrieval of the extracted information, thereby significantly improving the automation level and utilization efficiency of dispatch information processing. Summary of the Invention

[0004] In view of the above-mentioned existing problems, the present invention provides a method, system, device and medium for automatic extraction and retrieval of key information from power dispatch recordings, which solves the problems of low recognition accuracy of professional terms, equipment numbers and place names in the power field, inability to automatically extract key information items from long transcribed texts in a structured manner, and lack of ability to perform efficient and multi-dimensional retrieval of the extracted key information.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for automatically extracting and retrieving key information from power dispatch recordings, including: Obtain power dispatch recording files, preprocess the power dispatch recording files to obtain standardized audio files; The standardized audio file is used to perform speech recognition using a domain-adaptive speech recognition engine to obtain the recognized text. Information is extracted from the identified text to obtain structured information records, and the structured information records are converted into a multimodal index storage structure; Obtain structured query conditions, and retrieve the multimodal index storage structure in parallel based on the structured query conditions to obtain a candidate record set; The target record is obtained by calculating the candidate record set. The corresponding recording segment is extracted according to the audio positioning information of the target record and the retrieval result is returned, so as to realize the automatic extraction and retrieval of key information of power dispatch recording.

[0006] As a preferred embodiment of the method for automatically extracting and retrieving key information from power dispatch recordings according to the present invention, the power dispatch recording file is preprocessed to obtain a standardized audio file, including: An adaptive filter is used to perform adaptive noise reduction on the power dispatch recording files to obtain the noise-reduced frequency signal; Automatic gain control is applied to the noise-reduced audio signal to obtain a normalized audio signal; The normalized audio signal is classified using a speech activity detection model to obtain valid speech segments; The effective speech segments are divided into call sessions based on the length of the silence interval to obtain standardized audio files.

[0007] As a preferred embodiment of the method for automatically extracting and retrieving key information from power dispatch recordings according to the present invention, the standardized audio file is subjected to speech recognition to obtain the recognized text, which includes: Extract acoustic feature sequences from standardized audio files; The acoustic feature sequence is input into the acoustic model for decoding to obtain the posterior probability of the phoneme state; A list of hot words for power dispatch is constructed and compiled into a bias graph. A language model in the power dispatch domain is integrated, and the posterior probability of phoneme state, the bias score of the bias graph and the language probability of the language model are jointly decoded to output the optimal text sequence. The optimal text sequence is timestamped and labeled with confidence to obtain the recognized text with timestamps.

[0008] The advantages of this preferred technical solution are that it can adapt to electrical engineering terminology, improve transcription accuracy, and accurately annotate audio timestamps.

[0009] As a preferred embodiment of the method for automatically extracting and retrieving key information from power dispatch recordings according to the present invention, the method involves extracting information from the identified text to obtain structured information records, including: The identified text is preprocessed to obtain normalized text; The normalized text is encoded using a pre-trained language model to obtain a sequence of context semantic vectors; Based on the context semantic vector sequence, named entities in the normalized text are identified using sequence labeling to obtain an entity set; Classify the entity set by relation to obtain the entity relation triplet set; Based on the predefined key information template, the entity set and entity relationship triple set are filled into the corresponding fields to generate structured information records.

[0010] The beneficial effect of this preferred technical solution is that it can automatically extract electrical entities and relationships to generate standard structured records.

[0011] As a preferred embodiment of the automatic extraction and retrieval method for key information from power dispatch recordings described in this invention, the structured information record is transformed into a multimodal index storage structure, comprising: Establish a mapping triplet between structured information records and original audio files; Differentiated word segmentation is performed on the content of each field in the structured information record, and the field source is marked to construct an inverted index table; Based on the concatenated text generated by splicing the fields in the structured information record, a semantic vector is obtained by encoding using a pre-trained language model, and a semantic vector index is constructed. Construct a time-dimensional index based on the fault time field in the structured information records; The mapping triples, inverted index table, semantic vector index, and time dimension index are permanently stored to form a multimodal index storage structure.

[0012] The beneficial effects of this preferred technical solution are that it supports multi-dimensional retrieval through multi-index collaboration, taking into account both accuracy and semantic query, and quickly locating audio segments.

[0013] As a preferred embodiment of the automatic extraction and retrieval method for key information in power dispatch recordings described in this invention, the method involves parallel retrieval of the multimodal index storage structure to obtain a candidate record set, including: In response to user input of natural language queries, the natural language queries are parsed to obtain structured query conditions; Based on the entity constraint field, query the inverted index table using the composite index key to obtain the exact matching record set; The inverted index table is queried based on the word segmentation results of the natural language query to obtain the full-text keyword recall record set; The semantic vector index is retrieved based on the query vector obtained from the natural language query encoding to obtain the semantic recall record set; Query the time dimension index based on the time constraint field to obtain the time-matched record set; The exact match record set, the full-text keyword recall record set, the semantic recall record set, and the time-match record set are merged according to the record identifier to obtain the candidate record set.

[0014] As a preferred embodiment of the automatic extraction and retrieval method for key information from power dispatch recordings described in this invention, the method involves: calculating the target record from the candidate record set, extracting the corresponding recording segment based on the audio location information of the target record, and returning the retrieval results, including: Calculate the exact match score, keyword hit score, semantic similarity score, and time match score for each candidate record in the candidate record set. Based on the query characteristics of natural language queries, determine keyword weight, semantic weight, time weight, and precision weight; The weighted sum of each candidate record is calculated to obtain the overall score for each candidate record; The candidate records are sorted by their overall scores, and the record with the highest overall score is selected as the target record. Read the mapping triplet corresponding to the target record, extract the corresponding audio segment from the original recording file based on the recording file path and start and end time offset in the mapping triplet, associate the structured information of the target record with the audio segment, and return the search results.

[0015] Secondly, the present invention provides a system for automatically extracting and retrieving key information from power dispatch recordings, comprising: The preprocessing module is used to acquire power dispatch recording files, preprocess the power dispatch recording files, and obtain standardized audio files. The speech recognition module is used to perform speech recognition on the standardized audio file using a domain-adaptive speech recognition engine to obtain the recognized text; The information extraction and structure transformation module is used to extract information from the identified text to obtain structured information records, and to transform the structured information records into a multimodal index storage structure. The parallel retrieval module is used to obtain structured query conditions and, based on the structured query conditions, perform parallel retrieval of the multimodal index storage structure to obtain a candidate record set. The retrieval result return module is used to calculate the target record from the candidate record set, extract the corresponding recording segment based on the audio location information of the target record, and return the retrieval result, thereby realizing the automatic extraction and retrieval of key information from power dispatch recordings.

[0016] Thirdly, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the steps of the method for automatically extracting and retrieving key information from power dispatch recordings.

[0017] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the method for automatically extracting and retrieving key information from power dispatch recordings.

[0018] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention achieves new functions such as fully automated processing of dispatch recordings, natural language retrieval, segment-level localization, and cross-recording association; this invention adopts a domain-adaptive ASR model, improving the recognition rate of power industry terminology and accents, laying a high-quality text foundation for subsequent information extraction. This invention achieves fully automated processing from raw recordings to structured key information, requiring no manual intervention and greatly freeing up manpower. This invention uses a deep learning NLP model to accurately convert unstructured dialogue text into predefined formatted structured data, facilitating computer processing and analysis; this invention combines keyword inverted indexes and semantic vector indexes, supporting fast and accurate retrieval and flexible natural language semantic retrieval, enabling rapid location of key information in massive recordings. This invention's system closely integrates with the needs of power dispatching operations, with extracted information items directly corresponding to business forms such as dispatching logs and fault reports, effectively supporting dispatching decisions and accident tracing. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the overall process logic of a method for automatically extracting and retrieving key information from power dispatch recordings, provided in one embodiment of the present invention. Figure 2 A flowchart of voice activity detection and call session segmentation for an automatic extraction and retrieval method of key information from power dispatch recordings, provided in an embodiment of the present invention; Figure 3 The flowchart below illustrates a method for automatically extracting and retrieving key information from power dispatch recordings, based on an embodiment of the present invention, which includes multi-level matching of power equipment entities and ledger association discrimination. Detailed Implementation

[0021] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0022] Example 1, refer to Tables 1-2, Figures 1-3 As an embodiment of the present invention, a method for automatically extracting and retrieving key information from power dispatch recordings is provided, comprising: S100: Obtain power dispatch recording files, preprocess the power dispatch recording files to obtain standardized audio files; S200: Uses a domain-adaptive speech recognition engine to perform speech recognition on standardized audio files and obtain the recognized text; In one optional embodiment, the domain-adaptive speech recognition engine can be a recurrent neural network transcription engine, which extracts standardized audio Mel filter bank features and obtains an acoustic feature sequence through linear dimensionality reduction; the acoustic features are encoded through a multi-layer recurrent neural network, and a single-layer recurrent prediction network is used to output historical text representations; the bias map of power dispatch-specific hot words and the domain language model are jointly decoded, and the scheduling text with millisecond timestamps is output using beam search; the confidence of single-word recognition is calculated, and verification marks are added to words below the threshold of 0.7 to complete the recognition output; In another optional embodiment, the domain-adaptive speech recognition engine includes a convolutional transform hybrid engine that extracts Mel-spectral features from standardized audio frames, and convolutional layers capture local short-term acoustic features of the audio; a multi-head self-attention network models long-distance speech context associations and outputs high-level acoustic representations; a power scheduling vocabulary weight map is loaded to correct the decoding cost, and a pre-trained language model of the scheduling text is combined to collaboratively calculate the text probability; forced alignment of phonemes and characters is performed, the start and end offsets of each word's audio are marked, and transcribed text with confidence is generated; In this embodiment of the invention, the domain-adaptive speech recognition engine includes an ASR engine; Specifically, the process involves acquiring standardized audio, extracting and reducing the dimensionality of Mel filter bank (FBank) features to generate an acoustic feature sequence; feeding this sequence into an acoustic model fine-tuned by power dispatch data to output posterior phoneme probabilities; loading a weighted finite state converter (WFST) bias map of power hot words and combining it with a dispatch domain language model for joint logarithmic weighted decoding; outputting the optimal text through Viterbi beam search to achieve forced alignment between text and audio, and annotating with millisecond-level timestamps; calculating word confidence based on acoustic and linguistic probabilities, marking words below 0.7 for verification, and outputting transcribed text with confidence scores.

[0023] S300: Extract information from the recognized text to obtain structured information records, and transform the structured information records into a multimodal index storage structure; S400: Obtain structured query conditions, and perform parallel retrieval of the multimodal index storage structure based on the structured query conditions to obtain a candidate record set; S500: Calculates the target record from the candidate record set, extracts the corresponding recording segment based on the audio location information of the target record, and returns the retrieval results, realizing the automatic extraction and retrieval of key information from power dispatch recordings.

[0024] This invention optimizes audio quality through preprocessing and significantly improves the recognition accuracy of power industry professional terms by combining domain-adaptive ASR; it automatically extracts structured information and builds a multimodal index, supports multi-dimensional parallel retrieval, eliminates the need for manual listening to recordings, can quickly locate corresponding audio segments, has a high degree of automation, greatly reduces scheduling manpower costs, improves fault tracing and operation verification efficiency, and solves the pain point of inefficient use of recording data.

[0025] In this embodiment of the invention, step S100 includes the following sub-steps A1-A4; In A1: Adaptive filtering is used to perform adaptive noise reduction on the power dispatch recording file to obtain a noise-reduced frequency signal; In A2: Automatic gain control is applied to the noise-reduced audio signal to obtain a normalized audio signal; In A3: The normalized audio signal is classified using a speech activity detection model to obtain valid speech segments; In A4: The effective speech segments are divided into call sessions based on the silence interval length to obtain standardized audio files.

[0026] In embodiments of the present invention, such as Figure 2 As shown, the WAV format audio file output by the dispatch recording telephone system is acquired in near real-time via the JMS message bus. After each dispatch call ends, the dispatch recording telephone system automatically encapsulates the call recording into a WAV format file, including a 16kHz sampling rate, 16-bit quantization depth, and mono recording. This, along with call metadata such as the caller's number, called number, call start time, call end time, channel number, and seat number, is then pushed to the target topic on the JMS message bus. As a JMS consumer, subscribing to this topic and listening for message arrival events, the system acquires the file and its associated metadata in near real-time (with a delay of no more than 3 seconds) after the recording file is generated. The original WAV file is temporarily stored in a local circular buffer for subsequent preprocessing steps. The JMS message bus uses a publish-subscribe model, supporting multiple systems consuming the same recording file in parallel. Each channel is distinguished by its dispatch console seat number.

[0027] The first 200ms of silent segment, i.e. the line silence segment before ringing or connection, is extracted from the acquired WAV file as the background noise baseline estimate. The reference noise and mixed audio signal are converted to the frequency domain by Fast Fourier Transform, and the average power spectrum of environmental noise at each frequency point is calculated as the basis for noise estimation in subsequent adaptive filtering.

[0028] In one optional embodiment, the adaptive filter can be a minimum mean square adaptive filter, which extracts the 200ms silent segment before recording as a noise reference signal; sets a fixed iteration step size, uses the noise reference as the desired input; iterates and updates the filter weights frame by frame to cancel out the noise components in the mixed audio; and outputs clean audio after filtering out line noise and call noise.

[0029] In another alternative embodiment, the adaptive filter can be a recursive least squares adaptive filter, which extracts the silent noise segment to construct the initial correlation matrix; introduces a forgetting factor to adapt to the real-time small changes in noise; recursively solves for the optimal filter weights, quickly converges to suppress power frequency noise; completes frequency domain noise cancellation, and outputs the noise-reduced scheduled audio.

[0030] In this embodiment of the invention, the adaptive filter includes a normalized minimum mean square error adaptive filter. Specifically, a normalized minimum mean square error adaptive filter is used to denoise the mixed audio signal. The noise spectrum of the extracted first 200ms silent segment is used as the reference noise spectrum. Spectral subtraction or adaptive filtering is then applied to the entire audio segment. The filter order is 512, and the step size factor is... After filtering, the noise-reduced audio signal is output.

[0031] Energy detection is performed on the denoised audio signal to calculate the root mean square value of the current frame signal. , and the target value (default Compare and calculate the gain compensation amount. It applies gain compensation to the current frame and limits the rate of gain change to within 0.5dB / ms to avoid auditory abruptness, outputting a volume-normalized audio signal.

[0032] In one optional embodiment, the speech activity detection model can be a convolutional neural network for speech activity detection, which extracts Mel-frequency cepstral (MFCC) features frame by frame from normalized audio; multiple convolutional layers extract short-term audio noise or local speech features, and pooling layers compress the feature dimensions; fully connected layers output speech probabilities, and a threshold of 0.6 distinguishes between valid speech frames and silent frames; continuous speech frames are spliced ​​to form speech segments, and noise and silent segments before and after are removed. In another optional embodiment, the speech activity detection model can also be used for self-attention transform network speech activity detection, extracting normalized audio Mel filter bank (FBank) features and encoding position information; multi-head self-attention modeling audio temporal global noise and speech difference features; classification head outputting the speech confidence of each frame, dividing speech / silence with a threshold of 0.6; removing isolated short noise frames, and outputting complete and valid speech intervals; In this embodiment of the invention, the speech activity detection model includes a speech activity detection model based on a gated recurrent unit (GRU). Specifically, a speech activity detection model based on gated recurrent units (GRU) is used to classify the gain-normalized audio signal frame by frame. The model input is the Mel-frequency cepstral coefficient feature of each audio frame, and the output is the probability value that the frame belongs to valid speech. If the probability value is greater than a threshold, the output is considered valid. ,default The frame is marked as a valid speech frame. A series of valid speech frames constitute a speech segment. The silent segments and noise segments before and after the segment are removed.

[0033] The length of the silence interval between consecutive speech segments is detected, and when the silence interval exceeds a threshold... ,default At each second, the point is determined as the dividing point between two independent calls. The long audio file is then segmented into independent call session recording files based on this dividing point. Each segmented recording file is appended with a metadata header, including: session number, start timestamp (accurate to milliseconds), end timestamp, channel number, sampling rate, and quantization bit depth, and output as a standardized audio file in standard WAV format.

[0034] It should be noted that this invention uses the JMS message bus to obtain pre-packaged WAV files from the scheduling recording telephone system. This method avoids the engineering complexity of independently deploying acquisition hardware for each channel, and achieves near real-time reliable transmission of recording files through enterprise-level message middleware. Furthermore, the recording files themselves contain business metadata such as call start timestamp, calling and called numbers, and seat numbers, providing a data foundation for subsequent time-dimensional indexing and call correlation analysis.

[0035] In this embodiment of the invention, step S200 includes the following sub-steps B1-B4; In B1: Extract acoustic feature sequences from standardized audio files; In B2: The acoustic feature sequence is input into the acoustic model for decoding to obtain the posterior probability of the phoneme state; In B3: Construct a list of hot words for power dispatch and compile it into a bias graph. Integrate the language model in the power dispatch domain, and jointly decode the posterior probability of phoneme state, the bias score of the bias graph and the language probability of the language model to output the optimal text sequence. In B4: The optimal text sequence is timestamped and labeled with confidence to obtain the recognized text with timestamps.

[0036] In this embodiment of the invention, the WAV file obtained from the JMS message bus is preprocessed and then FBank features are extracted with a frame length of 25ms and a frame shift of 10ms.

[0037] Specifically, each frame sequentially performs pre-emphasis, Hamming windowing, fast Fourier transform, 80-channel Mel filter bank filtering, and logarithmic energy calculation, outputting an 80-dimensional FBank vector; the FBank vectors of five adjacent frames are concatenated to form a 400-dimensional vector, which is then reduced to 200 dimensions by linear discriminant analysis, serving as the acoustic feature sequence.

[0038] In one optional embodiment, the acoustic model can be a Conformer acoustic model, which inputs the acoustic feature sequence into a convolutional module to extract short-term local acoustic features of the audio; a multi-head self-attention layer models long-term speech dependencies and fuses global acoustic information; a feedforward network completes feature transformation and outputs optimized acoustic representations; and a decoding network outputs the posterior probabilities of each phoneme for subsequent joint decoding. In another alternative embodiment, the acoustic model can also be a deep speech model. The acoustic feature sequence is fed into multiple one-dimensional convolutional layers to compress redundant features; a multi-layer bidirectional recurrent neural network (BiRNN) extracts contextual speech features; a fully connected classification layer outputs phoneme classification probabilities; and the output phoneme posterior probabilities are supplied to the language model for collaborative decoding. In this embodiment of the invention, the acoustic model includes a Transformer-Transducer architecture; Specifically, the acoustic model adopts a Transformer-Transducer architecture, comprising an encoder, a prediction network, and a joint network. The encoder consists of 12 stacked Transformer layers, each containing an 8-head self-attention mechanism and a feedforward network with 2048 hidden layers, used to encode the input feature sequence into a high-dimensional acoustic representation. The prediction network consists of two LSTM layers, each with 1024 hidden units, used for iterative prediction of previously output non-blank symbols. The joint network concatenates the acoustic and prediction representations, then passes them through a 1024-unit fully connected layer and a Softmax classification layer, outputting the posterior probability distribution of each phoneme state at the current moment. This acoustic model was fine-tuned and trained using at least 200 hours of labeled power dispatch call data.

[0039] The acoustic model employs a Transformer-Transducer architecture, comprising an encoder, a prediction network, and a joint network. The encoder consists of 12 stacked Transformer layers, each containing an 8-head self-attention mechanism and a feedforward network with 2048 hidden layers, used to encode the input feature sequence into a high-dimensional acoustic representation. The prediction network consists of two LSTM layers (1024 hidden units per layer), used for iterative prediction of previously output historical non-blank symbols. The joint network concatenates the acoustic and prediction representations, then passes them through a 1024-unit fully connected layer and a Softmax classification layer, outputting the posterior probability distribution of each phoneme state at the current time. This acoustic model was fine-tuned and trained using at least 200 hours of labeled power dispatch call data.

[0040] A language model from the power dispatching domain is integrated for joint decoding. This language model employs a Transformer architecture with 12 layers, 12 attention heads, and 768 hidden layers. It is trained using at least 500 hours of transcribed power dispatching calls to calculate the language model probability score for recognizing text sequences. During joint decoding, the acoustic model posterior probability, hot word bias score, and language model probability are logarithmically and linearly weighted, with acoustic weights of 0.6, language weights of 0.3, and hot word bias weights of 0.1. A Viterbi beam search with a beam width of 256 is then used to search for the optimal text sequence in the decoding space.

[0041] Each word in the decoded output corresponds to a consecutive frame interval in the original audio. Forced alignment maps the word boundaries to the audio timeline, obtaining the start and end time offsets for each word. The confidence score of each word is jointly determined based on the mean acoustic posterior probability of its corresponding audio frame interval and the language model probability. A confidence score below a threshold of 0.7 is marked as "low confidence verification." The final output includes the recognized text with millisecond-level timestamps and confidence score annotations.

[0042] In this embodiment of the invention, step S300 includes the following sub-steps C1-C5; In C1: The recognized text is preprocessed to obtain normalized text; In C2: The normalized text is encoded using a pre-trained language model to obtain a sequence of context semantic vectors; In C3: Based on the context semantic vector sequence, the named entities in the normalized text are identified using sequence labeling to obtain the entity set; In C4: The entity set is classified into relational categories to obtain a set of entity relation triples; In C5: Based on the predefined key information template, the entity set and entity relationship triple set are filled into the corresponding fields to generate structured information records.

[0043] In the embodiment of the present invention, as Figure 3 shown, the ASR recognized text is cleaned based on the power dispatching spoken stop word lexicon. This lexicon contains three types of words: modal filler words, such as "um", "ah", "that", "that is to say", repeated redundant segments, such as consecutively repeated "okay okay", "received received", single-occurrence reserved segments and slip-of-the-tongue correction segments, such as "put into... uh... exit", where the content after correction is taken as the standard and the content before correction is deleted. During cleaning, the text is traversed sequentially, entries in the lexicon are matched, and deletion or replacement operations are executed.

[0044] Regular replacement is performed on the cleaned text, converting Chinese numerals such as "zero, one, two, three, four, five, six, seven, eight, nine" into Arabic numerals "0-9"; mapping colloquial unit expressions to standard unit expressions, such as "kilovolt" → "kV", "kilovar" → "kvar", "megawatt" → "MW"; converting unstructured time expressions into ISO format time strings, such as "ten twenty-five today" → current date T10:25:00.

[0045] Colloquial device descriptions are mapped to the standard device naming format of dispatching regulations, and the mapping rules are based on the power dispatching terminology dictionary, which stores the correspondence between spoken expressions and standard terms in the form of key-value pairs, such as "main transformer" → "main power transformer", "switch" → "circuit breaker", "blade switch" → "disconnector".

[0046] In an optional embodiment, the pre-trained language model may be a RoBERTa model, which adds segment markers to the cleaned dispatching text, removes the next sentence prediction task, and retains only the masked pre-training task; a multi-layer multi-head self-attention network is input to extract the contextual semantics of power entities and relationships; the top linear classification layer outputs entity label probabilities to complete entity recognition; entities are paired two by two and sent to the classification network to output the semantic relationships between entities; In another optional embodiment, the pre-trained language model may also be a knowledge-enhanced pre-training model, which inputs standardized dispatching text and fuses the knowledge embeddings of power entities such as substations and lines; superimposes multi-layer Transformer encoders to learn the contextual features of professional terms; performs BIO sequence labeling based on the output semantic vectors to extract various power entities; uses the knowledge-enhanced features to perform entity relationship classification to generate entity triples; In the embodiment of the present invention, the pre-trained language model includes a BERT model fine-tuned with power dispatching corpus as the encoder; Specifically, this model is a BERT-base-Chinese version, containing 12 stacked Transformer encoder layers, each layer with 12 multi-head self-attention mechanisms. The hidden layer dimension is 768, the fully connected feedforward network dimension is 3072, and the total number of parameters is approximately 110 million. The responsibilities of each attention head are allocated as follows: The bottom layer (layers 1-4) focuses on capturing local word order and phrase structure, achieved through the distribution of attention weights between adjacent words, with weights concentrated within a range of 2-3 adjacent words; the middle layer (layers 5-8) focuses on capturing long-distance dependencies, achieved through the distribution of attention weights across sentence segments, with weights distributed throughout the entire sentence; the top layer (layers 9-12) focuses on capturing semantic features strongly related to entity recognition, and after fine-tuning on labeled data, the attention weight distribution is biased towards entity boundary words and relation trigger words. The specific principle of the multi-head self-attention mechanism for capturing contextual semantics is as follows: For each word in the input text, its word embedding and position encoding are added to generate a query vector, a key vector, and a value vector, all of which are 64-dimensional. Each attention head calculates the attention weight distribution in parallel. The weight distribution is obtained by normalizing the dot product of the query vector and the key vector using Softmax. The results calculated by each head are concatenated and output after a linear transformation, so that the final representation of each word incorporates the contextual information related to the current word in the entire text.

[0047] The preprocessed text sequence is input into the BERT model described above. A [CLS] tag is appended to the beginning of the text sequence to aggregate global information, and sentences are separated by a [SEP] tag. After 12 layers of Transformer encoding, a 768-dimensional context semantic vector sequence corresponding to each word is output.

[0048] A linear classification layer is overlaid on top of the BERT model, with a 768-dimensional input and the number of label categories as the output. The fine-tuning training utilizes an annotated corpus from the power dispatching domain, consisting of no fewer than 10,000 transcribed dispatch calls. Each text has been annotated with entity boundaries and type labels by power industry professionals. The cross-entropy loss function is used as the optimization objective, and the loss function is calculated as follows: in, For sequence length, For the number of tag categories, One-hot encoding of the real label. The learning rate is set to 2 × 10 to predict probabilities for the model. -5 The batch size is 32, and the number of training rounds is 5.

[0049] A 768-dimensional semantic vector sequence is input into the top-level linear classification layer. The classification layer outputs the probability distribution of each word on each category label, and the label corresponding to the highest probability is taken as the entity type annotation result for that word. The annotation adopts the BIO format, and the label set includes: substation name start (B-SUB), substation name internal (I-SUB), voltage level start (B-VOL), voltage level internal (I-VOL), equipment number start (B-EQP), equipment number internal (I-EQP), protection type start (B-PRO), protection type internal (I-PRO), operation action start (B-ACT), operation action internal (I-ACT), time expression start (B-TIM), time expression internal (I-TIM), personnel name start (B-PER), personnel name internal (I-PER), fault phenomenon start (B-FLT), fault phenomenon internal (I-FLT), and non-entity (O). Finally, the output consists of consecutive word fragments annotated with the same entity type, forming a named entity set. Each entity contains entity text, entity type, start word index, and end word index.

[0050] Each identified entity is paired, and the results are input into a relation classifier to determine the semantic relationship between them. The relation classifier structure consists of two fully connected network layers superimposed on the top layer of the BERT model. The first layer is 768 to 384 dimensions with the ReLU activation function; the second layer is 384 to R dimensions, where R is the number of relation types, and outputs the probability distribution of each relation type. The relation type label set includes: equipment / substation location attribution (located_at), protection / equipment ownership relationship (belongs_to), action application object (act_on), fault occurrence location (occur_at), time sequence (before / after), and cause trigger relationship (trigger).

[0051] Output the set of entity-relation-entity triples ,in .

[0052] Based on a predefined key information template, entity sets and entity relation triples are populated into corresponding fields. The key information template includes fields such as: fault time, substation name, equipment number, protection type, operation action, operator, and fault description. The population rules are as follows: iterate through entity set E, filling the entity text into the corresponding fields of the template according to entity type; iterate through triple set T, supplementing the relationship information between fields according to relation type. For unidentified fields, fill with empty values ​​and mark them as "to be completed". Fields corresponding to terms marked "low confidence review" are marked with a "review" tag in the structured record.

[0053] The identified equipment entities are matched against the standard equipment ledger for power dispatching, mapping colloquial or abbreviated descriptions to the full path names of the standard equipment. The standard equipment ledger is organized in a hierarchical tree structure, with each equipment node recording the following fields: full path name of the standard equipment, equipment ID, voltage level, substation, parent equipment, and equipment type. The specific implementation process for entity link matching is as follows: precise matching involves a complete string comparison between the text of the equipment entity and the equipment name field in the standard equipment ledger. If they match completely, a direct association is established, and the standard equipment ID is entered as a foreign key into the structured record.

[0054] For abbreviated alias fuzzy matching, if exact matching fails, the following fuzzy matching process is performed on the equipment entity text: First, the entity text is normalized, removing equipment type suffixes such as "switch", "knife switch", "protection" and voltage level prefixes such as "220kV" and "110kV", and extracting core equipment identification numbers such as "1011" and "Liantang Line"; the normalized entity text is then compared with the equipment abbreviation field and equipment alias field of each record in the standard equipment ledger by calculating the string edit distance. When the edit distance is ≤2, it is considered a successful match; when there are multiple candidate matches, the voltage level information extracted from the entity text is used for filtering, retaining only candidates with the same voltage level. If there are still multiple candidates, the record with the smallest edit distance is selected.

[0055] Context-assisted disambiguation: If fuzzy matching still fails or multiple candidates cannot be distinguished, the extracted entity relationship information is used to assist disambiguation: when the device to be matched has an located_at relationship with a substation that has been successfully matched, only the list of devices under that substation is retrieved in the ledger to narrow the matching range before a second matching is performed.

[0056] If no match is found, and all matching processes fail, the entity is marked as "Not Matched in Ledger". The corresponding device field in the structured record is filled with the original recognized text and marked "Pending Manual Verification". The reason for the matching failure is also recorded, such as failure of exact match / no candidate for fuzzy match / failure of disambiguation. After all matching processes are completed, the output device entities are all associated with structured records of standard device IDs.

[0057] In this embodiment of the invention, after completing steps C1-C5, step S300 further includes steps C6-C10; In C6: Establish a mapping triplet between structured information records and the original audio files; In C7: Differentiated word segmentation is performed on the content of each field in the structured information record, and the field source is marked to construct an inverted index table; In C8: according to the spliced text generated by splicing each field in the structured information record, a pre-trained language model is used for encoding to obtain semantic vectors, and a semantic vector index is constructed; In C9: a time dimension index is constructed according to the fault time field in the structured information record; In C10: the mapping triples, inverted index table, semantic vector index and time dimension index are solidified and stored to form a multimodal index storage structure.

[0058] In the embodiment of the present invention, according to the structured record containing fields such as fault time, substation name, equipment number, protection type, operation action, operator, fault description and the corresponding standard equipment ID, the storage association between the record and the original audio file is established. Each record corresponds to a recording file, which is named with a unique session number in the format of "YYYYMMDD_HHMMSS_channel number.wav", and the complete storage path of the recording in the file system is recorded at the same time. From the timestamped recognized text, extract the start time offset and end time offset of the call session corresponding to the record, and store them in the audio positioning field of the structured record in the form of an interval of the start offset and end offset, so as to form an accurate mapping triple of "record ID→(recording file path, start offset, end offset)". This mapping relationship is stored in the mapping table of the relational database, and the fields include record ID, file path, start offset, end offset, and record generation timestamp.

[0059] Perform differentiated word segmentation on the content of each field in the structured record according to the field type. The substation name field and the equipment number field are accurately segmented using the power dispatching dictionary, for example, "110kV Fenghua Substation" is segmented into "110kV / Fenghua Substation", and the complete field content is retained as an integral term at the same time; the fault description field and the operation action field are segmented by a general word segmenter combined with a power dictionary, and stop words such as "de", "le", "shi" are removed.

[0060] Each term is attached with its source field name tag when being stored in the database, forming a composite index key of "field name: term", such as "substation name: Fenghua Substation", "equipment number: Zunhuaping Line", which ensures that the retrieval scope can be limited to a specific field during retrieval.

[0061] Using "field name: term" as the key and the record ID list as the value, write into the inverted index table, which is stored in the relational database, and the table structure includes term key, record ID list, and term frequency. When a new record is added to the database, the new record ID is appended to the record ID list corresponding to the existing term, and a new record ID list is created if the term does not exist.

[0062] The content of each field in the structured record is concatenated into a text sequence in a preset order. The concatenation order is "fault time + substation name + equipment number + protection type + operation action + fault description". The fields are separated by the [SEP] mark. The length of the concatenated text sequence shall not exceed 512 characters.

[0063] The concatenated text is input into the BERT model, and the 768-dimensional output vector corresponding to the [CLS] marker position is taken as the global semantic vector representation of that record. No gradient updates are performed during the encoding process; only forward inference is executed.

[0064] Record IDs are used as keys, and 768-dimensional semantic vectors are used as values ​​to store data in a vector database. The vector database uses the FAISS library to build its index structure, with an index type of IVF-PQ (Inverted File and Product Quantization), nlist=1024, m=8, supporting approximate nearest neighbor retrieval for large-scale vectors.

[0065] After the vectors are added to the database, the HNSW graph structure is constructed. The number of layers is L=log2(N) (N is the total number of records), the maximum number of connections per layer is M_max=16, the dynamic list size is ef_construction=200, the distance metric is cosine similarity, and the constructed structure supports vector retrieval with O(log2(N)) complexity.

[0066] The fault time field is extracted from the structured records, converted to Unix timestamp format, and a B+ tree index is built using the timestamp as the key and the record ID as the value, and stored in the relational database. The B+ tree has an order m=128. Non-leaf nodes store timestamps and pointers to child nodes, while leaf nodes store a list of record IDs in ascending order of timestamps, supporting single-point queries and range queries.

[0067] The relationships between the three indexes are embedded in the storage layer. The inverted index table is stored in a relational database (table name: inv_idx), the semantic vector index is stored in a vector database (set name: vec_idx), and the time index shares the same relational database with the inverted index table in B+ tree form. The three are cross-linked through record IDs. After retrieving a record ID from any search path, complete information can be quickly retrieved by querying other indexes using that record ID. The record ID is a 32-bit hexadecimal string in UUID format, generated by concatenating the first 16 digits of the record's timestamp with the last 16 digits of a random number, ensuring global uniqueness and including time sorting information.

[0068] In this embodiment of the invention, step S400 includes the following sub-steps D1-D6; In D1: In response to the user's input of a natural language query, the natural language query is parsed to obtain structured query conditions; In D2: Based on the entity constraint field, query the inverted index table using the composite index key to obtain the exact matching record set; In D3: Based on the word segmentation results of the natural language query, the inverted index table is queried to obtain the full-text keyword recall record set; In D4: The semantic vector index is retrieved based on the query vector obtained from the natural language query encoding to obtain the semantic recall record set; In D5: Query the time dimension index based on the time constraint field to obtain the time-matched record set; In D6: The exact match record set, the full-text keyword recall record set, the semantic recall record set, and the time-match record set are merged according to the record identifier to obtain the candidate record set.

[0069] In this embodiment of the invention, a natural language query statement input by a user is received, a preprocessing process is performed, including the removal of colloquial interjections, standardization of numbers and units, and standardization of power terminology, and a normalized query text is output.

[0070] The normalized query text is input into the BERT model, which extracts a 768-dimensional contextual semantic vector for each word. This vector, after being processed by a trained linear classification layer, outputs BIO tags for each word to identify the entity type in the query. Simultaneously, time constraints are extracted through rule matching; the time keyword mapping table is shown in Table 1. Table 1 Time Keyword Mapping Table

[0071] The identified entity types and extracted time constraints are combined into a structured query condition set. When any field is empty, it indicates that no constraints are set for that dimension. The complete content of the original query text is also preserved for subsequent semantic vector encoding.

[0072] The identified non-empty constraint fields, including substation name, equipment number, protection type, operation action, and personnel name, are used to construct query keys in the format of "field name:term". The inverted index table is then queried key by key. The intersection of the record ID sets of the search results for each field yields the exact matching record set. If all constraint fields are empty, skip this search path.

[0073] The normalized query text is fully segmented into words, and stop words are removed to obtain a keyword list. The inverted index table is queried one by one (without limiting the field range), and the recalled records are sorted in descending order by the number of keywords hit, taking the top few records. Default =200, thus obtaining the full-text keyword recall record set. The keyword hit score for each record is calculated as: number of hit words / total number of queries.

[0074] The normalized query text is concatenated and encoded. The 768-dimensional vector corresponding to the [CLS] tag is taken as the query vector. Approximate nearest neighbor retrieval is performed on the constructed vector index to recall the vector with the highest cosine similarity to the query vector. Default record =200, thus obtaining the semantic recall record set. Each record is accompanied by a semantic similarity score. , cosine similarity value, with a range of [-1, 1].

[0075] When the extracted time_range is not empty, a range query is performed in the B+ tree index built with the upper and lower limits of the time range to obtain the time-matched record set. Each record includes a time matching score. The hit time precision is 1 point for a day, 2 points for an hour, and 3 points for a minute. After scoring at the finest granularity, the scores are normalized to the [0,1] interval.

[0076] In this embodiment of the invention, step S500 includes the following sub-steps E1-E5; In E1: Calculate the exact match score, keyword hit score, semantic similarity score, and temporal match score for each candidate record in the candidate record set. In E2: Based on the query characteristics of natural language queries, determine keyword weight, semantic weight, time weight, and precision weight; In E3: The weighted sum of each candidate record is calculated to obtain the overall score of each candidate record; In E4: Sort the candidate records according to their overall score, and select the record with the highest overall score as the target record; In E5: Read the mapping triplet corresponding to the target record, extract the corresponding audio segment from the original recording file based on the recording file path and start and end time offset in the mapping triplet, associate the structured information of the target record with the audio segment, and return the search results.

[0077] In this embodiment of the invention, The four recall records are merged according to their record IDs to form a candidate record set. Only one duplicate record will be retained.

[0078] For each record in the candidate record set Calculate the following scores: exact match bonus :like ;otherwise .

[0079] Keyword Hit Score ,like quilt If recalled, the corresponding value is taken; otherwise... .

[0080] Semantic similarity score If r is If recalled, the corresponding value is taken (after ReLU activation to ensure non-negativity); otherwise... =0.

[0081] Time Matching Score If r is If recalled, the corresponding value is taken; otherwise... =0.

[0082] The weighting coefficients are dynamically adjusted based on the explicitness of various constraints in the query statement, as shown in Table 2. Table 2. Multi-dimensional Search Dynamic Weight Allocation Comparison Table

[0083] The device serial number is identified by a regular expression matching "^[A-Za-z0-9]+ "And the length is ≥4, the substation name is identified based on the entity type label being SUB. Query feature judgment priority: contains equipment number / substation name > contains explicit time range > contains only vague descriptive words > default."

[0084] The comprehensive score for candidate record r is calculated as follows: in, The overall score is calculated based on the total score.

[0085] If r is not recalled by any search, it exists only as a deduplication result. Set the value to 0 and remove it from the candidate set. After all records have been calculated, press [the appropriate button]. Sort in descending order, and extract the first N results. The default value is N=20, which will be used as the final search results list.

[0086] Based on the final search result list, for each record, the stored mapping triplet includes the recording file path, start offset, and end offset. The file storage system is accessed based on the recording file path. The corresponding audio segment is extracted from the original WAV file according to the start and end offsets. The extraction method is to read the PCM audio data within the interval from the sampling point corresponding to the start offset to the sampling point corresponding to the end offset, and then regenerate an independent WAV file. A return data packet is generated for each search result, containing a structured record summary, a list of field name-value pairs, the extracted audio segment file (temporarily stored in a cache directory), a download link, the recording file name, and start and end time offset information. All return data packets are encapsulated in JSON format and output to the user interface, allowing the scheduler to view the text summary one by one and click to play the corresponding audio segment.

[0087] Example 2, referring to Table 3, based on the above examples, provides an application simulation test of a method for automatically extracting and retrieving key information from power dispatch recordings, to verify the feasibility and advantages of the present invention.

[0088] Take a real fault report recording as an example; At 2:23 PM on a certain day, the dispatch console received a telephone report from maintenance personnel at the 110kV Suiyang Substation. The system, through the JMS message bus, acquired the WAV format audio file output by the dispatch recording telephone system in real time, continuously monitored the dispatch console's audio stream, and captured the call signal at a 16kHz sampling rate and 16-bit quantization depth. After adaptive noise reduction and automatic gain control, the system segmented the effective speech segments using voice activity detection. A silence interval exceeding 2 seconds was considered the end of the call, and the entire call was saved as a standardized audio file named "20251025_1423_Suiyang Substation Fault Report.wav". The file header included metadata such as the session number, start timestamp 2025-10-25 14:23:06.328, end timestamp 2025-10-25 14:24:12.547, and channel number (CH03).

[0089] The standardized audio files were fed into the ASR engine. The engine first extracted 80-dimensional FBank features with a 25ms frame length and a 10ms frame shift. Five adjacent frames were concatenated and then reduced to 200 dimensions using LDA to form an acoustic feature sequence. This acoustic feature sequence was input into a Transformer-Transducer acoustic model fine-tuned with 200 hours of power dispatch call data, outputting the posterior probability of phoneme states. During decoding, a dynamic hotword loader compiled the power dispatch hotword list into a WFST bias graph with a bias weight of 1.5. This graph was then jointly decoded with a power dispatch domain language model trained on 500 hours of dispatch transcribed text, with an acoustic weight of 0.6, a language weight of 0.3, and a hotword bias weight of 0.1. The optimal text sequence was output through a Viterbi beam search with a beam width of 256.

[0090] The original text output after ASR transcription of the recording is: "Hello, this is Zhang San from 110kV Suiyang Substation. The work on modifying the main protection of the 35kV Fengsui Mao Line has been completed. Now, the main protections of the 35kV Fengsui Mao Line need to be activated. The line is ready for power supply. I will report the specific load situation: the current load is 8.5 MW. Please confirm." After forced alignment, each word is accompanied by a millisecond-level start and end time offset. "110kV Suiyang Substation" corresponds to the audio from 2.3 seconds to 4.1 seconds, "35kV Fengsui Mao Line" corresponds to 4.5 seconds to 6.8 seconds, and "Main Protection" corresponds to 7.0 seconds to 8.2 seconds. The confidence of each word is higher than 0.7, and no low-confidence review flag is triggered.

[0091] The ASR output text first enters preprocessing. Preprocessing performs three cleaning operations: first, removal of colloquial content, removing filler words like "hello," the address "Zhang San," and redundant suffixes like "I'll report," resulting in a normalized text; second, standardization of numbers and units, converting "eight point five megawatts" to "8.5MW"; and third, standardization of electrical terminology, confirming "main protection 1" and "main protection 2" as standard terms. The cleaned text is: "The modification work of the main protection 1 of the 110kV Suiyang substation and the 35kV Fengsui Mao line has been completed. The main protection 1 and 2 of the 35kV Fengsui Mao line need to be activated. Power supply is ready. The current load is 8.5MW. Please confirm." The cleaned text input was fine-tuned using a BERT-base-Chinese encoder based on a power dispatch corpus. This encoder consisted of 12 layers, including a 768-dimensional hidden layer, and was encoded using a 12-head self-attention mechanism. The output was a 768-dimensional context semantic vector corresponding to each word. The top-level linear classification layer, taking a 768-dimensional input, output a 17-dimensional label probability, annotating entity types word-by-word in BIO format. The recognition results are shown in Table 3. Table 3 Recognition Results

[0092] The identified entities are paired and input into the relationship classifier, which outputs the following triples: (35kV Fengsui Mao Line, belongs_to, 110kV Suiyang Substation), (Main Protection 1, belongs_to, 35kV Fengsui Mao Line), (Main Protection 2, belongs_to, 35kV Fengsui Mao Line), (Activated, act_on, 35kV Fengsui Mao Line). The entity linking submodule matches "110kV Suiyang Substation" with the standard substation record in the ledger and returns the standard equipment ID "SUB-110-SY-001"; it matches "35kV Fengsui Mao Line" to the 35kV outgoing line ledger record under this substation and returns the standard equipment ID "EQ-35-FSM-015".

[0093] The system generates structured records based on predefined key information templates, which include the following fields: Record a unique identifier: Automatically generate a 32-bit UUID, with the first 16 bits encoded by the generation timestamp and the last 16 bits being a random number; Original recording file name: 20251025_1423_Suiyang Substation Fault Report.wav; The original ASR recognized text and the cleaned and normalized text; Structured extraction results: The substation name is uniformly "110kV Suiyang Substation", and the standard equipment ID is SUB-110-SY-001; the equipment number is uniformly "35kV Fengsui-Mao Line", and the standard equipment ID is EQ-35-FSM-015; the protection type is "Main Protection 1" and "Main Protection 2"; the operation action is "Activated"; the status description is "Energy supply conditions met"; the load information is "8.5MW"; Audio location information: storage path of the recording file and start and end time offset of the entire conversation; The start and end character positions of each entity in the original text and their confidence scores; Standard device ID association results during entity linking; The record generates a timestamp and verification markers for each field. In this embodiment, the confidence level of all fields is higher than the threshold, so no verification markers are needed. Details of error correction operations performed during text cleaning, including the conversion record of "eight point five megawatts" being standardized to "8.5MW".

[0094] Index building: After the structured records are generated, the system synchronously performs three types of index building: For the inverted index, "110kV Suiyang Substation" is indexed by the substation name field, "35kV Fengsui Mao Line" is indexed by the equipment number field, "Main Protection 1" and "Main Protection 2" are indexed by the protection type field, "Activated" is indexed by the operation action field, and "8.5MW" is indexed by the load information field. Each term is stored in the form of a "field name:term" composite key, and the record identifier is recorded and written into the inverted index table of the relational database.

[0095] For semantic vector indexing, the content of each field is concatenated in the order of "fault time + substation name + equipment number + protection type + operation action + fault description", with each field separated by a delimiter to generate concatenated text. The concatenated text is input into a vector encoding model, and the encoder takes the 768-dimensional output vector corresponding to the first and second markers as the global semantic vector representation of the record. It is stored in the vector database with the record identifier as the key, and an approximate nearest neighbor index structure is constructed based on the HNSW algorithm.

[0096] For time indexing, the fault time field (2025-10-25 14:23:00) is extracted from the record, converted into a Unix timestamp, and written into a B+ tree index structure with the timestamp as the key and the record identifier as the value.

[0097] Intelligent Search and Result Return: The next day, the dispatcher enters a natural language query into the search interface: "What was the status of the main protection of the Fengsui Mao Line of Suiyang Substation yesterday?" After receiving the query, the system performs parsing: preprocessing removes interjections and identifies "Suiyang Substation" as a substation name constraint, "Fengsui Mao Line" as an equipment number constraint, "Main Protection" as a protection type constraint, and "Yesterday" as a time constraint (mapped to 00:00:00 to 23:59:59 of the previous day). The system then performs three parallel searches: an inverted index for precise retrieval of the intersection records of "Substation Name: Suiyang Substation", "Equipment Number: Fengsui Mao Line", and "Protection Type: Main Protection"; a semantic vector search to recall the top 200 records with the highest cosine similarity in the vector database using the query vector; and a B+ tree time index to lock records within the entire time range of the previous day. After taking the union of the three results, the exact match bonus is 1.0, the keyword hit score is 0.83, the semantic similarity score is 0.91, the time match score is 1.0, and the highest comprehensive score is 0.4×0.83+0.35×0.91+0.15×1.0+0.1×1.0=0.901, which is the highest score in this candidate record set.

[0098] The system extracts an audio segment from seconds 2.3 to 8.2 of the original recording file based on the path of the highest-scoring record and the audio location triplet. This segment is then repackaged into an independent WAV file and pushed to the search interface along with a complete structured record summary. The dispatcher can click the play button to replay the original audio clip containing the line name and protection type reported by maintenance personnel, confirming the specific content that "the main protection change work has been completed." The entire search and playback process is completed within one second.

[0099] Example 3 illustrates a schematic scheme for an automatic extraction and retrieval method for key information from power dispatch recordings according to this embodiment. It should be noted that the technical solution of this automatic extraction and retrieval method for key information from power dispatch recordings is based on the same concept as the aforementioned method. Details not described in detail in the technical solution of this automatic extraction and retrieval method for key information from power dispatch recordings can be found in the description of the aforementioned method.

[0100] This embodiment of a system for automatically extracting and retrieving key information from power dispatch recordings includes: The preprocessing module is used to acquire power dispatch recording files, preprocess the power dispatch recording files, and obtain standardized audio files. The speech recognition module is used to perform speech recognition on the standardized audio file using a domain-adaptive speech recognition engine to obtain the recognized text; The information extraction and structure transformation module is used to extract information from the identified text to obtain structured information records, and to transform the structured information records into a multimodal index storage structure. The parallel retrieval module is used to obtain structured query conditions and, based on the structured query conditions, perform parallel retrieval of the multimodal index storage structure to obtain a candidate record set. The retrieval result return module is used to calculate the target record from the candidate record set, extract the corresponding recording segment based on the audio location information of the target record, and return the retrieval result, thereby realizing the automatic extraction and retrieval of key information from power dispatch recordings.

[0101] This embodiment also provides a computer device applicable to a method for automatically extracting and retrieving key information from power dispatch recordings, including: The system includes a memory and a processor. The memory stores computer-executable instructions, and the processor executes these instructions to implement an automatic extraction and retrieval method for key information from power dispatch recordings, as proposed in the above embodiments.

[0102] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, it implements a method for automatically extracting and retrieving key information from power dispatch recordings as proposed in the above embodiment.

[0103] The storage medium proposed in this embodiment and the method for automatically extracting and retrieving key information from power dispatch recordings proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0104] Based on the above description of the implementation methods, those skilled in the art will clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computing device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0105] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for automatically extracting and retrieving key information from power dispatch recordings, characterized in that, include: Obtain power dispatch recording files, preprocess the power dispatch recording files to obtain standardized audio files; The standardized audio file is used to perform speech recognition using a domain-adaptive speech recognition engine to obtain the recognized text. Information is extracted from the identified text to obtain structured information records, and the structured information records are converted into a multimodal index storage structure; Obtain structured query conditions, and retrieve the multimodal index storage structure in parallel based on the structured query conditions to obtain a candidate record set; The target record is obtained by calculating the candidate record set. The corresponding recording segment is extracted according to the audio positioning information of the target record and the retrieval result is returned, so as to realize the automatic extraction and retrieval of key information of power dispatch recording.

2. The method for automatic extraction and retrieval of key information from power dispatch recordings as described in claim 1, characterized in that, The power dispatch recording files are preprocessed to obtain standardized audio files, including: An adaptive filter is used to perform adaptive noise reduction on the power dispatch recording files to obtain the noise-reduced frequency signal; Automatic gain control is applied to the noise-reduced audio signal to obtain a normalized audio signal; The normalized audio signal is classified using a speech activity detection model to obtain valid speech segments; The effective speech segments are divided into call sessions based on the length of the silence interval to obtain standardized audio files.

3. The method for automatic extraction and retrieval of key information from power dispatch recordings as described in claim 2, characterized in that, Speech recognition is performed on the standardized audio file to obtain the recognized text, which includes: Extract acoustic feature sequences from standardized audio files; The acoustic feature sequence is input into the acoustic model for decoding to obtain the posterior probability of the phoneme state; A list of hot words for power dispatch is constructed and compiled into a bias graph. A language model in the power dispatch domain is integrated, and the posterior probability of phoneme state, the bias score of the bias graph and the language probability of the language model are jointly decoded to output the optimal text sequence. The optimal text sequence is timestamped and labeled with confidence to obtain the recognized text with timestamps.

4. The method for automatically extracting and retrieving key information from power dispatch recordings as described in claim 3, characterized in that, Information extraction is performed on the identified text to obtain structured information records, including: The identified text is preprocessed to obtain normalized text; The normalized text is encoded using a pre-trained language model to obtain a sequence of context semantic vectors; Based on the context semantic vector sequence, named entities in the normalized text are identified using sequence labeling to obtain an entity set; Classify the entity set by relation to obtain the entity relation triplet set; Based on the predefined key information template, the entity set and entity relationship triple set are filled into the corresponding fields to generate structured information records.

5. The method for automatically extracting and retrieving key information from power dispatch recordings as described in claim 4, characterized in that, The transformation of the structured information records into a multimodal index storage structure includes: Establish a mapping triplet between structured information records and original audio files; Differentiated word segmentation is performed on the content of each field in the structured information record, and the field source is marked to construct an inverted index table; Based on the concatenated text generated by splicing the fields in the structured information record, a semantic vector is obtained by encoding using a pre-trained language model, and a semantic vector index is constructed. Construct a time-dimensional index based on the fault time field in the structured information records; The mapping triples, inverted index table, semantic vector index, and time dimension index are permanently stored to form a multimodal index storage structure.

6. The method for automatically extracting and retrieving key information from power dispatch recordings as described in claim 5, characterized in that, Parallel retrieval of the multimodal index storage structure yields a candidate record set including: In response to user input of natural language queries, the natural language queries are parsed to obtain structured query conditions; Based on the entity constraint field, query the inverted index table using the composite index key to obtain the exact matching record set; The inverted index table is queried based on the word segmentation results of the natural language query to obtain the full-text keyword recall record set; The semantic vector index is retrieved based on the query vector obtained from the natural language query encoding to obtain the semantic recall record set; Query the time dimension index based on the time constraint field to obtain the time-matched record set; The exact match record set, the full-text keyword recall record set, the semantic recall record set, and the time-match record set are merged according to the record identifier to obtain the candidate record set.

7. A method for automatically extracting and retrieving key information from power dispatch recordings as described in claim 1 or 6, characterized in that, The candidate record set is calculated to obtain the target record. Based on the audio location information of the target record, the corresponding audio segment is extracted and the retrieval results are returned, including: Calculate the exact match score, keyword hit score, semantic similarity score, and time match score for each candidate record in the candidate record set. Based on the query characteristics of natural language queries, determine keyword weight, semantic weight, time weight, and precision weight; The weighted sum of each candidate record is calculated to obtain the overall score for each candidate record; The candidate records are sorted by their overall scores, and the record with the highest overall score is selected as the target record. Read the mapping triplet corresponding to the target record, extract the corresponding audio segment from the original recording file based on the recording file path and start and end time offset in the mapping triplet, associate the structured information of the target record with the audio segment, and return the search results.

8. A system for automatically extracting and retrieving key information from power dispatch recordings, comprising the method for automatically extracting and retrieving key information from power dispatch recordings as described in any one of claims 1-7, characterized in that, include: The preprocessing module is used to acquire power dispatch recording files, preprocess the power dispatch recording files, and obtain standardized audio files. The speech recognition module is used to perform speech recognition on the standardized audio file using a domain-adaptive speech recognition engine to obtain the recognized text; The information extraction and structure transformation module is used to extract information from the identified text to obtain structured information records, and to transform the structured information records into a multimodal index storage structure. The parallel retrieval module is used to obtain structured query conditions and, based on the structured query conditions, perform parallel retrieval of the multimodal index storage structure to obtain a candidate record set. The retrieval result return module is used to calculate the target record from the candidate record set, extract the corresponding recording segment based on the audio location information of the target record, and return the retrieval result, thereby realizing the automatic extraction and retrieval of key information from power dispatch recordings.

9. A computer device, characterized in that, include: The system includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method for automatically extracting and retrieving key information from power dispatch recordings as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a processor, implements the steps of the method for automatically extracting and retrieving key information from power dispatch recordings according to any one of claims 1 to 7.

Images