A speech recognition text editing processing method

Abstract

The present invention provides a speech recognition text editing processing method, which innovatively designs a user interface to clearly separate the manual editing area and the automatic text generation area. Users can freely modify the text in the manual editing area without affecting the cursor position in the automatic text generation area, ensuring that the two areas operate independently and improving work efficiency and accuracy. In order to solve the problem of cursor position confusion caused by automatic text generation during manual modification, the present invention introduces a cursor locking mechanism. When the user operates in the manual editing area, the lock is enabled by triggering a button or shortcut key to pause the automatic text generation in the editing area. After the modification is completed and confirmed, the lock is released and the automatic text generation in the editing area is resumed. This mechanism ensures that the cursor position is stable during manual modification and avoids operational errors caused by interference from automatic generation.

Description

Technical Field

[0001] The present invention relates to the field of speech-to-text editing, and more particularly to a speech recognition-based text editing processing method. Background Art

[0002] In today's digital age, the combined application of speech recognition and text editing is becoming increasingly popular. However, in practical application, many problems still exist. Due to technical limitations, the automatically generated text often contains errors when speech recognition is used to generate text. When users manually correct erroneous information, the cursor position often changes. This phenomenon seriously affects the continuity and accuracy of speech recognition. In traditional text processing workflows, speech recognition is typically a separate and prior step. That is, the speech recognition system first completes the conversion of all speech before the user can begin to review and correct the converted text. In this process, users need to wait for speech recognition to complete, which can result in a waste of time, especially when the speech recognition results require extensive correction.

[0003] Existing technologies have shown obvious deficiencies when dealing with such problems. On the one hand, there is a lack of effective division between the editing area and the generation area, which leads to mutual interference between manual editing and automatic generation. On the other hand, there is no dedicated cursor locking mechanism, so that automatic text generation is still in progress during the manual modification process, causing confusion in the cursor position. In addition, the processing of text merging and conflict detection is not perfect, and problems such as duplication and inconsistency are prone to occur, which affects the quality of the final text, increases the user's workload and work difficulty, leads to poor user experience, and limits the application and promotion of this technology in a wider range of fields. Summary of the Invention

[0004] The present invention aims to overcome the many defects in the existing application of combining speech recognition and text editing. In order to solve the mutual interference problem caused by the ineffective division of editing area and generation area in the existing technology, a speech recognition text editing processing method is proposed, which includes:

[0005] Step 1: construct a user interface on the speech-to-text generation interface, wherein the user interface includes a manual editing area and an automatic text generation area;

[0006] Step 2: Recognize the input voice signal, display the first text information generated by the recognition in the automatic text generation area, and synchronously copy the first text information to the manual editing area to generate second text information;

[0007] Step 3: When the second text information is edited in the manual editing area, triggering a cursor lock operation to stop copying the first text information in the automatic text generation area to the manual editing area;

[0008] Step 4: After the editing of the second text information in the manually edited area is completed, unlocking the cursor, copying the first text information in the automatic text generation area after the editing completion time to the manually edited area, and returning to Step 3 until the voice signal recognition is completed and the editing of the second text information in the manually edited area is completed, and then executing Step 5;

[0009] Step 5: Merge the first text information in the automatic text generation area and the second text information in the manual editing area, and finally output the second text information in the manual editing area.

[0010] The present invention provides a method for editing text using speech recognition, which has the following beneficial effects:

[0011] (1) An innovative user interface is designed to clearly separate the manual editing area and the automatic text generation area. Users can freely modify text in the manual editing area without affecting the cursor position in the automatic text generation area, ensuring that the two areas operate independently and improving work efficiency and accuracy.

[0012] (2) To address the issue of cursor position confusion caused by automatic text generation during manual editing, the present invention introduces a cursor lock mechanism. When the user operates in the manual editing area, they can activate the lock by triggering a button or shortcut key, pausing the automatic text generation in the editing area. After the modification is confirmed, the lock is released and automatic text generation in the editing area is resumed. This mechanism ensures that the cursor position is stable during manual editing and avoids operational errors caused by interference from automatic text generation.

[0013] (3) After both manual modification and automatic text generation are completed, the texts generated by both can be intelligently merged into a coherent document to ensure the integrity and coherence of the output document. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a flow chart of a speech recognition text editing processing method provided by the present invention. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. In addition, the technical features in the various embodiments or single embodiments provided by the present invention can be arbitrarily combined with each other to form a feasible technical solution. This combination is not restricted by the sequence of steps and / or structural composition mode, but must be based on the ability of ordinary technicians in this field to implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0016] In existing speech recognition applications, manual corrections must wait until speech recognition is complete, preventing simultaneous operations. This wastes time. The inability to perform manual intervention during speech recognition makes the entire process disjointed and inefficient.

[0017] Based on this, the present invention proposes a method of separating the text editing area from the text generation area, fundamentally eliminating the mutual interference between the two. In practical applications, this separation allows users to fully focus on the modification work when performing manual editing without having to worry about the automatically generated text interfering with it. At the same time, the automatic text generation area can also be unaffected by manual editing and continuously and stably generate content. This not only greatly improves work efficiency, but also significantly improves the accuracy of editing and generation, reducing errors caused by mutual interference.

[0018] Figure 1 This is a flow chart of an intelligent speech recognition text editing processing method provided by the present invention, such as Figure 1 As shown, the method includes:

[0019] Step 1: construct a user interface on the voice-to-text generation interface, wherein the user interface includes a manual editing area and an automatic text generation area.

[0020] It is understandable that on the voice-to-text generation interface, a left-right structured user interface is constructed, which displays the manual editing area and the automatic text generation area respectively. The left side displays the automatic text generation area, and the right side displays the manual editing area.

[0021] The automatic text generation area is specifically designed to receive and display text generated by the speech recognition algorithm. The manual editing area not only receives and displays text generated by the speech recognition algorithm, but also provides an operation toolbar in the upper right corner of the interface for operations such as locking, unlocking, inserting, deleting, copying, pasting, and annotating. Its layout and operation methods have been carefully optimized to meet user operating habits and needs.

[0022] These two areas are independent in layout and clearly distinguished in the display. The manual editing area uses a brighter background color or border to highlight its importance and independence. The automatic text generation area is displayed in a softer color scheme to minimize distraction. Any user operation in the editing area will not affect the cursor position in the generation area, and vice versa. This independent operation mechanism greatly reduces interference and significantly improves work efficiency.

[0023] Step 2: Recognize the input voice signal, display the first text information generated by the recognition in the automatic text generation area, and synchronously copy the first text information to the manual editing area to generate second text information.

[0024] It is understandable that after constructing independent automatic text generation areas and manual editing areas, it is necessary to detect the automatic text generation areas and manual editing areas on the user interface so that the first text information generated by voice recognition can be accurately displayed in the automatic text generation area, and the copied first text information can be accurately displayed in the manual editing area.

[0025] In a possible implementation of the present invention, step 2 of recognizing the input voice signal, displaying the first text information generated by the recognition in the automatic text generation area, and synchronously copying the first text information to the manual editing area to generate the second text information includes:

[0026] Based on the edge detection algorithm, the constructed manual editing area and the automatic text generation area are detected respectively, the first text information generated by speech recognition is displayed in the automatic text generation area, and the first text information is synchronously copied to the manual editing area.

[0027] Specifically, the user interface is separated from the automatically generated text area and the manually edited area based on an edge detection algorithm. The edge detection algorithm mainly includes the following steps: Gaussian smoothing, calculating gradient magnitude and direction, non-maximum suppression, double threshold processing, and edge connection.

[0028] In one embodiment of the present invention, the detecting of the constructed manual editing area and the automatically generated text area based on an edge detection algorithm includes:

[0029] Step 21: Perform Gaussian smoothing on the user interface image.

[0030] The Gaussian smoothing process specifically includes:

[0031] During image acquisition and transmission, images may be affected by noise, which can lead to erroneous edge detection results. Gaussian smoothing reduces the effects of noise by convolving the image with a Gaussian kernel. The Gaussian kernel is a two-dimensional normal distribution function that takes a weighted average of each pixel in the image and its neighborhood, with pixels closer to the center pixel receiving a greater weight.

[0032] Therefore, calculate the Gaussian value of each pixel of the user interface image:

[0033] ;

[0034] Among them, (x, y) is the pixel coordinate, G (x, y) is the Gaussian value of the pixel, is the standard deviation of the Gaussian function, which controls the width of the Gaussian kernel. Pixels with Gaussian values ​​less than the preset threshold are removed, while pixels with Gaussian values ​​greater than or equal to the preset threshold are retained to achieve the purpose of denoising.

[0035] Step 22: Calculate the gradient magnitude and gradient direction of each pixel in the user interface image after Gaussian smoothing.

[0036] Specifically, for each pixel after Gaussian processing, the gradient magnitude and gradient direction are calculated. Edges typically correspond to regions of the image with large changes in grayscale values, so these regions can be found by calculating the image's gradient. The gradient represents the rate of change of the image in each direction, with the gradient magnitude indicating the magnitude of the change, and the gradient direction indicating the direction of the change. A common method is to use the Sobel operator to calculate the gradient of the image in the x and y directions. The Sobel operator is a 3x3 convolution kernel that calculates the horizontal and vertical gradients, respectively.

[0037] Based on the Sobel operator, the gradient of each pixel in the x and y directions is calculated according to the horizontal operator and the vertical operator, where the horizontal gradient of the pixel point G x for:

[0038] ;

[0039] The vertical gradient G of the pixel y for:

[0040] ;

[0041] in, I is the pixel coordinate (x, y);

[0042] Calculate the gradient magnitude G of each pixel:

[0043] ;

[0044] Calculate the gradient direction of each pixel:

[0045] .

[0046] In step 23, preliminary edge pixels are detected using non-maximum suppression according to the gradient magnitude and gradient direction of each pixel.

[0047] In a possible implementation of the present invention, detecting preliminary edge pixels using non-maximum suppression based on the gradient magnitude and gradient direction of each pixel includes:

[0048] For any pixel point, multiple pixels in the local neighborhood are found in the gradient direction to form local area pixels. The pixel with the largest gradient amplitude among all pixels in the local area is found as the preliminary edge pixel point.

[0049] Specifically, after calculating the gradient magnitude and direction, each pixel in the image has a gradient value. However, pixels with large gradient magnitudes do not necessarily correspond to edges, because there may be multiple pixels with large gradient magnitudes near the edge. The purpose of non-maximum suppression is to find the local maximum in the gradient direction and set the pixel values ​​of non-local maxima to 0, thereby refining the edge. For each pixel, the gradient magnitudes of its adjacent pixels in the gradient direction are compared according to its gradient direction. If the gradient magnitude of the pixel is not a local maximum, it is set to 0.

[0050] Step 24: edge connections are obtained based on double threshold processing, and final edge pixels are selected from the preliminary edge pixels.

[0051] In a possible implementation manner of the present invention, the step of extracting edge connections based on dual threshold processing and selecting final edge pixels from the preliminary edge pixels includes:

[0052] Setting a high threshold and a low threshold, comparing the gradient amplitude of each preliminary edge pixel with the high threshold and the low threshold, respectively, marking pixels with a gradient amplitude greater than the high threshold as strong edge pixels, marking pixels with a gradient amplitude less than the low threshold as non-edge pixels, and marking pixels with a gradient amplitude between the high threshold and the low threshold as weak edge pixels;

[0053] For the weak edge pixel points, the weak edge pixel points connected to the strong edge pixel points are retained, and the weak edge pixel points not connected to the strong edge pixel points are removed. The strong edge pixel points and the retained weak edge pixel points constitute the final edge pixel points.

[0054] Specifically, after non-maximum suppression, the edges in the image have been refined. However, there may still be some false edges caused by noise or other factors. Double threshold processing further filters the edges by setting two gradient amplitude thresholds (high threshold and low threshold). Pixels with gradient amplitudes higher than the high threshold are considered strong edge pixels, pixels with gradient amplitudes lower than the low threshold are considered non-edge pixels, and pixels between the two thresholds are considered weak edge pixels. Strong edge pixels are definitely true edge pixels, while weak edge pixels may be true edge pixels or noise. Through edge connection, weak edge pixels connected to strong edge pixels are retained, and weak edge pixels not connected to strong edge pixels are removed.

[0055] The above method is used to detect the automatic text generation area and the manual editing area respectively. Using a speech recognition algorithm and a deep learning-based speech recognition model, the input speech signal is converted into a first text message. The automatically generated first text message is placed in the automatic text generation area. The first text message in the automatic text generation area is then synchronously copied to the manual editing area. The text information in the manual editing area is called the second text message and is available for the user to view and edit.

[0056] Speech recognition specifically involves using the Mel-frequency cepstral coefficient algorithm to convert speech signals into feature vectors. First, the speech signal is pre-emphasized to enhance the high-frequency component. Next, frame-by-frame windowing is performed to segment the speech signal into short frames and window them to reduce spectral leakage. A fast Fourier transform (FFT) is then performed to obtain the spectrum. The linear spectrum is then converted to a Mel-frequency scale using a Mel filter bank. Finally, a logarithmic operation and a discrete cosine transform (DCT) are performed to obtain MFCC features. The deep neural network speech recognition model, based on a multi-layer perceptron (MLP), is trained using a large amount of speech data. The input layer receives the speech feature vector, and the intermediate hidden layer transforms and learns the features using nonlinear activation functions (such as ReLU). The output layer outputs the probability distribution of each phoneme or word.

[0057] In a possible implementation of the present invention, in step 2, recognizing the input voice signal includes:

[0058] Step 21 ′: pre-emphasize the input speech signal.

[0059] Specifically, the spectrum of a speech signal usually has the characteristic of high-frequency energy attenuation. The purpose of pre-emphasis is to enhance the high-frequency part of the speech signal, making the spectrum of the signal flatter and easier to process. In the time domain, the calculation formula for pre-emphasis is ,in is the original speech signal, is the pre-emphasized signal, and i represents the frame index.

[0060] Step 22': performing frame division and windowing processing on the pre-emphasized speech signal.

[0061] Specifically, the speech signal is divided into multiple short frames so that the spectrum analysis can be performed on each frame. The speech signal is segmented according to a certain frame length (such as 25 milliseconds) and frame shift (such as 10 milliseconds). Assume that the sampling frequency of the speech signal is , the frame length is N sampling points, the frame shift is M sampling points, then the starting position of the i-th frame is i*M, and the signal in the frame is .

[0062] The speech signal after framing will be discontinuous at the frame boundary. The purpose of windowing is to reduce the impact of this discontinuity on spectrum analysis and make the signal in the frame gradually transition to zero at the boundary. Use Hamming window for windowing operation: ,in , N is the length of the window function. The windowing operation is completed by multiplying each frame of signal by the window function.

[0063] Step 23 ′: Perform Fourier transform on the windowed speech signal to obtain a linear spectrum.

[0064] Specifically, the windowed time-domain signal is converted to the frequency domain, yielding the spectrum of each frame. An N-point FFT transform is performed on each frame of the windowed signal, where N is typically a power of 2 to improve computational efficiency. The FFT transform converts the time-domain signal to the frequency domain, yielding the signal's amplitude and phase spectra.

[0065] Step 24 ′: convert the linear spectrum into a Mel frequency spectrum based on multiple Mel filters to obtain the output energy of each Mel filter.

[0066] Specifically, the linear spectrum is converted into a Mel-frequency spectrum to simulate the human ear's perception of sounds of different frequencies. A Mel filter bank is designed, typically consisting of 20-40 triangular filters. The linear spectrum of each frame is filtered through the Mel filter bank to obtain the output energy of each filter.

[0067] Step 25 ′: take the logarithm of the output energy of each Mel filter and perform discrete cosine transform to obtain the MFCC feature vector.

[0068] Specifically, the output energy of each Mel filter is logarithmized to convert the multiplicative relationship of energy into an additive relationship, while simulating the logarithmic perception characteristics of the human ear to the sound intensity. Taking the logarithm, we get , j is the index of the Mel filter.

[0069] The logarithmic energy spectrum is subjected to discrete cosine transform to obtain MFCC parameters. DCT can convert the highly correlated logarithmic energy spectrum into less correlated cepstral coefficients while preserving the main characteristics of the speech signal.

[0070] Step 26 ′: input the MFCC feature vector into a deep neural network to obtain the first text information generated by speech recognition.

[0071] Specifically, text recognition is performed on the MFCC feature vector based on a deep neural network to obtain the first text information generated by speech recognition, and the recognized first text information is displayed in the automatic text generation area.

[0072] It's important to note that during the training of the deep neural network (speech recognition model), user-corrected text is fed into the model as real-time context to adjust the decoding strategy. For example, our prosody prediction technology, by integrating text feature corrections, can be expanded to weight user corrections during the decoding phase to suppress similar errors.

[0073] The error correction model architecture adopted uses short sentences corrected by users as training samples, updates the language feature encoder parameters through online learning, and realizes dynamic optimization of the speech recognition model.

[0074] Error pattern analysis and incremental training: We build an error log library, collect statistics on frequently misrecognized words (e.g., a keyword list), and regularly trigger incremental training. For example, we target common noise interference words in enclosed spaces (e.g., mistaking "valve" for "boring") to expand training data and fine-tune the speech recognition model.

[0075] The combined cascade model design adds a feedback loop after the speech recognition model, aligning user corrections with the original audio before feeding it back to the speech recognition module, creating a closed-loop optimization. Manually marked typos serve as context to continuously improve recognition quality.

[0076] Step 3: When the second text information is edited in the manual editing area, triggering a cursor lock operation to stop copying the first text information in the automatic text generation area to the manual editing area.

[0077] Specifically, the first text information generated by speech recognition may contain errors, so the speech-recognized text information needs to be manually edited and corrected. In an embodiment of the present invention, the first text information generated by speech recognition in the automatic text generation area is synchronously copied to the manual editing area, and the second text information is manually edited and corrected in the manual editing area. However, during the manual editing process, the continuous synchronous copying of the text information in the automatic text generation area to the manual editing area will interfere with the editing process and cause confusion.

[0078] Based on this, in an embodiment of the present invention, when editing the second text information in the manual editing area, a cursor lock operation is triggered. The cursor lock mechanism is optimized so that it is only effective in the editing area. This allows speech recognition to continue during the lock period without affecting the performance of speech recognition in the automatic text generation area. When the user begins to modify the manual editing area, the lock is triggered, restricting cursor movement only in the editing area.

[0079] It should be noted that, during the process of editing the second text information in the manual editing area, if the second text information is modified, the position coordinates of each word in the second text information in the manual editing area need to be updated so that the speech recognition text in the subsequent text automatic generation area and the second text information in the manual editing area can be copied and aligned.

[0080] In a possible real-time mode of the present invention, when the second text information is edited in the manual editing area, step 3 further includes updating the position coordinates of each word in the edited second text information:

[0081] The upper left corner of the manually edited area is taken as the origin of the coordinate system, the rightward direction is the positive x-axis direction, and the downward direction is the positive y-axis direction;

[0082] When the manual editing area When a new value is inserted at a position coordinate, the original position coordinates of each subsequent word Update, the updated position coordinates are ;

[0083] The coordinate system of the manually edited area is expanded into a one-dimensional sequence by row, and the position number of the character is calculated starting from the upper left corner in the original coordinate system. The original point position of each character is expressed as: , the point position of the inserted word is expressed as: ,in, The maximum x-axis coordinate value of the manually edited area;

[0084] if , the point position of each subsequent word needs to be shifted right one position, ;

[0085] Convert the point position of each character back to two-dimensional coordinates: , ;

[0086] When the manually edited area When the word at the position coordinate is deleted, the original position coordinates of each subsequent word Update, the updated position coordinates are ;

[0087] The coordinate system of the manually edited area is expanded into a one-dimensional sequence by row, and the position number of the word is calculated starting from the upper left corner in the original coordinate system. The original point position of each word is expressed as: , the point position of the deleted word is expressed as: ;

[0088] if , then the point position of each subsequent word needs to be shifted left by one position, ;

[0089] Convert the point position of each character back to two-dimensional coordinates: , .

[0090] Each time you insert text during locked editing, the following text will execute the above logic, change its coordinate information, and record the total number of changed points.

[0091] In a possible implementation of the present invention, in step 3, when the second text information is edited in the manual editing area, a cursor lock operation is triggered to stop copying the first text information in the automatic text generation area to the manual editing area:

[0092] Step 31: Setting a cursor operation button on the user interface, and triggering a cursor lock operation by clicking the cursor operation button when editing the second text information in the manual editing area; or

[0093] Step 32, respectively obtain the cursor position (x0, y0) at the previous moment and the cursor position (x1, y1) at the current moment, wherein the upper left corner of the automatic text generation area is the origin of the coordinate system, the right is the positive direction of the x-axis, and the downward is the positive direction of the y-axis; when y1 < y0, or y1 = y0 and x1 > x0, it indicates that the second text information in the manual editing area at that moment is being edited, triggering the cursor lock operation.

[0094] Specifically, when triggering the cursor lock operation, a cursor operation button can be provided on the user interface. When the second text information in the manual editing area needs to be edited, the cursor lock operation can be triggered by clicking the cursor operation button. Alternatively, it can be detected whether the second text information in the manual editing area is being manually edited by detecting the change in the cursor position. When it is detected that the second text information in the manual editing area is being edited, the cursor lock mechanism is automatically triggered.

[0095] Specifically, with the upper left corner of the automatic text generation area as the origin of the coordinate system, the rightward direction is the positive x-axis direction, and the downward direction is the positive y-axis direction. The cursor position at the previous moment (x0, y0) and the current moment (x1, y1) are obtained respectively. When y1 < y0, or y1 = y0 and x1 > x0, it indicates that the second text information in the manual editing area is being edited at that moment, triggering the cursor lock operation, stopping the copying of the first text information in the automatic text generation area to the corresponding position in the manual editing area.

[0096] Step 4: After the editing of the second text information in the manually edited area is completed, the cursor is unlocked, and the first text information after the editing completion moment in the automatic text generation area is copied to the manually edited area. Return to step 3 until the voice signal recognition is completed and the editing of the second text information in the manually edited area is completed, and then execute step 5.

[0097] Specifically, after the editing of the second text information in the manually edited area is completed, the cursor is unlocked and can be unlocked by clicking the cursor operation button. In terms of unlocking design, when the user completes editing and confirms unlocking, the voice recognition results after the unlocking moment are automatically seamlessly connected with the content of the editing area. By real-time monitoring and analysis of the rhythm and semantics of voice input, it is ensured that subsequent voice recognition can be continuously effective in the editing area. For example, using timestamp and semantic matching technology, the unlocked voice content can be accurately inserted into the appropriate position in the editing area while maintaining the coherence and logic of the overall text.

[0098] It should be noted that when the first text information in the automatic text generation area after the editing is completed is copied to the manual editing area, the similarity between the second text information copied to the manual editing area and the original first text information in the automatic text generation area is calculated. If the similarity is less than the preset similarity threshold, the second text information in the manual editing area is copied to the manual editing area again.

[0099] When calculating the similarity between two text messages, cosine similarity is used to measure the semantic similarity between the edit area text and the speech recognition text. Assume that the word vector of the edit area text is represented as , the word vector of speech recognition text is represented as , then the cosine similarity The calculation formula is: ;

[0100] Among them, n is the dimension of word vector, and They are and The i-th component of .

[0101] Step 5: Merge the first text information in the automatic text generation area and the second text information in the manual editing area, and finally output the second text information in the manual editing area.

[0102] It is understandable that after voice recognition is completed and the second text information in the manual editing area is edited, since the first text information after the cursor is released is copied to the manual editing area when the cursor is released, the second text information in the manual editing area lacks the text information from the time when the cursor is locked to the time when the cursor is released, compared to the first text information in the automatic text generation area, the space for the text information in this time period is left blank in the manual editing area. In order to ensure the integrity of the output text information, the first text information in the automatic text generation area and the second text information in the manual editing area are merged, and the merged second text information is output.

[0103] This embodiment of the present invention develops an intelligent text merging algorithm that efficiently merges text from manually edited and automatically generated areas into a coherent document. During the merging process, the manually edited and automatically generated texts are first subjected to grammatical and semantic analysis to identify duplicate or conflicting sections. Duplicate content is filtered and retained based on context and importance. Inconsistent wording is unified and optimized using a vocabulary library and semantic understanding to ensure the quality of the merged document.

[0104] Specifically, in an embodiment of the present invention, step 5 of merging the first text information of the automatically generated text area and the second text information of the manually edited area, and finally outputting the second text information of the manually edited area, includes:

[0105] Comparing the first text information of the automatically generated text area with the second text information of the manually edited area word by word in sequence based on a recurrent neural network, and finding the first text information that is missing from the manually edited area compared to the automatically generated text area;

[0106] The missing first text information is copied to the corresponding position of the manual editing area, and finally the second text information of the manual editing area is output.

[0107] It should be noted that during the process of copying the missing first text information to the corresponding position in the manually edited area, the copied text information is tested. Specifically, the similarity between the second text information copied to the manually edited area and the original first text information in the automatically generated text area is calculated. If the similarity is less than a preset similarity threshold, the second text information in the manually edited area is copied back to the manually edited area, and finally a complete and correct second text information is output.

[0108] The speech recognition text editing method provided by the embodiment of the present invention has the following advantages:

[0109] (1) Improve user experience:

[0110] This invention reduces interference between the manual editing area and the automatic text generation area by clearly demarcating them. Users can freely modify text in the editing area without affecting the cursor position in the generation area. This design allows users to focus more on editing, reducing the distractions caused by cursor position changes, thereby improving the user experience.

[0111] (2) Improve text processing efficiency:

[0112] The simultaneous execution of speech recognition and manual repair significantly shortens the entire text processing time. Users no longer need to wait for speech recognition to complete before repairing, greatly improving text processing efficiency.

[0113] The introduction of a cursor lock mechanism separates the user editing process from the automatic text generation process, preventing cursor position confusion caused by interference from automatic text generation. This mechanism ensures a stable cursor position during editing, improving editing accuracy and efficiency. Furthermore, the text merging algorithm efficiently processes both manually edited and automatically generated text, reducing conflicts and errors during the merging process and further improving text processing efficiency.

[0114] It should be noted that, in the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0115] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0117] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0118] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0119] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0120] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A speech recognition text editing processing method, characterized in that: include: Step 1: construct a user interface on the speech-to-text generation interface, wherein the user interface includes a manual editing area and an automatic text generation area; Step 2: Recognize the input voice signal, display the first text information generated by the recognition in the automatic text generation area, and synchronously copy the first text information to the manual editing area to generate second text information; Step 3: When the second text information is edited in the manual editing area, triggering a cursor lock operation to stop copying the first text information in the automatic text generation area to the manual editing area; Step 4: After the editing of the second text information in the manually edited area is completed, unlocking the cursor, copying the first text information in the automatic text generation area after the editing completion time to the manually edited area, and returning to Step 3 until the voice signal recognition is completed and the editing of the second text information in the manually edited area is completed, and then executing Step 5; Step 5: Merge the first text information in the automatically generated text area and the second text information in the manually edited area, and finally output the second text information in the manually edited area; When the second text information is edited in the manual editing area, step 3 further includes updating the position coordinates of each word in the edited second text information: The upper left corner of the manually edited area is taken as the origin of the coordinate system, the rightward direction is the positive x-axis direction, and the downward direction is the positive y-axis direction; When the manual editing area When a new value is inserted at a position coordinate, the original position coordinates of each subsequent word Update, the updated position coordinates are ; The coordinate system of the manually edited area is expanded into a one-dimensional sequence by row, and the position number of the character is calculated starting from the upper left corner in the original coordinate system. The original point position of each character is expressed as: , the point position of the inserted word is expressed as: ,in, The maximum x-axis coordinate value of the manually edited area; if , the point position of each subsequent word needs to be shifted right one position, ; Convert the point position of each character back to two-dimensional coordinates: , ; When the manually edited area When the word at the position coordinate is deleted, the original position coordinates of each subsequent word Update, the updated position coordinates are ; The coordinate system of the manually edited area is expanded into a one-dimensional sequence by row, and the position number of the word is calculated starting from the upper left corner in the original coordinate system. The original point position of each word is expressed as: , the point position of the deleted word is expressed as: ; if , then the point position of each subsequent word needs to be shifted left by one position, ; Convert the point position of each character back to two-dimensional coordinates: , .

2. The speech recognition text editing processing method according to claim 1, characterized in that: The step 2 of recognizing the input voice signal, displaying the first text information generated by the recognition in the automatic text generation area, and synchronously copying the first text information to the manual editing area to generate the second text information includes: Based on an edge detection algorithm, the constructed manual editing area and the automatic text generation area are detected respectively, the first text information generated by speech recognition is displayed in the automatic text generation area, and the first text information is synchronously copied to the manual editing area; The detecting of the constructed manual editing area and the automatic text generation area based on the edge detection algorithm includes: Perform Gaussian smoothing on the user interface image; Calculate the gradient magnitude and gradient direction of each pixel in the user interface image after Gaussian smoothing; According to the gradient amplitude and gradient direction of each pixel, the preliminary edge pixels are detected using non-maximum suppression. Edge connections are obtained based on double threshold processing, and final edge pixels are screened out from the preliminary edge pixels.

3. The speech recognition text editing processing method according to claim 2, characterized in that: The performing Gaussian smoothing on the user interface image includes: Calculate the Gaussian value of each pixel of the user interface image ; Among them, (x, y) is the pixel coordinate, G (x, y) is the Gaussian value of the pixel, is the standard deviation of the Gaussian function; Pixels with Gaussian values ​​less than a preset threshold are removed, and pixels with Gaussian values ​​greater than or equal to the preset threshold are retained.

4. The speech recognition text editing processing method according to claim 2, characterized in that: The calculating of the gradient magnitude and gradient direction of each pixel in the user interface image after Gaussian smoothing includes: Based on the Sobel operator, the gradient of each pixel in the x and y directions is calculated according to the horizontal operator and the vertical operator, where the horizontal gradient of the pixel point G x for: ； The vertical gradient G of the pixel y for: ； Where I is the pixel coordinate (x, y); Calculate the gradient magnitude G of each pixel: ； Calculate the gradient direction of each pixel: 。 5. The speech recognition text editing processing method according to claim 2 or 4, characterized in that: The method of detecting preliminary edge pixels using non-maximum suppression based on the gradient magnitude and gradient direction of each pixel includes: For any pixel point, find multiple pixel points in the local neighborhood along the gradient direction to form the local area pixel point, and find the pixel point with the largest gradient amplitude among all the pixels in the local area as the preliminary edge pixel point; The step of processing edge connections based on double thresholds and selecting final edge pixels from the preliminary edge pixels includes: Setting a high threshold and a low threshold, comparing the gradient amplitude of each preliminary edge pixel with the high threshold and the low threshold, respectively, marking pixels with a gradient amplitude greater than the high threshold as strong edge pixels, marking pixels with a gradient amplitude less than the low threshold as non-edge pixels, and marking pixels with a gradient amplitude between the high threshold and the low threshold as weak edge pixels; For the weak edge pixel points, the weak edge pixel points connected to the strong edge pixel points are retained, and the weak edge pixel points not connected to the strong edge pixel points are removed. The strong edge pixel points and the retained weak edge pixel points constitute the final edge pixel points.

6. The speech recognition text editing processing method according to claim 1, characterized in that: In step 2, the input voice signal is recognized, including: Pre-emphasize the input voice signal; Performing frame division and windowing processing on the pre-emphasized speech signal; Perform Fourier transform on the windowed speech signal to obtain a linear spectrum; Converting the linear spectrum into a Mel frequency spectrum based on multiple Mel filters to obtain output energy of each Mel filter; Take the logarithm of the output energy of each Mel filter and perform discrete cosine transform to obtain the MFCC feature vector; The MFCC feature vector is input into a deep neural network to obtain the first text information generated by speech recognition.

7. The speech recognition text editing processing method according to claim 1, characterized in that: In step 3, when the second text information is edited in the manual editing area, a cursor lock operation is triggered to stop copying the first text information in the automatic text generation area to the manual editing area. A cursor operation button is provided on the user interface, and when the second text information is edited in the manual editing area, the cursor lock operation is triggered by clicking the cursor operation button; or Get the cursor position of the previous moment (x0, y0) and the cursor position of the current moment (x1, y1) respectively; The upper left corner of the automatic text generation area is taken as the origin of the coordinate system, the right side is the positive x-axis direction, and the downward side is the positive y-axis direction; When y1<y0, or y1=y0 and x1>x0, it indicates that the second text information in the manual editing area is being edited at that moment, triggering a cursor locking operation.

8. The speech recognition text editing processing method according to claim 1, characterized in that: The step 5 of merging the first text information of the automatically generated text area and the second text information of the manually edited area, and finally outputting the second text information of the manually edited area, includes: Comparing the first text information of the automatically generated text area with the second text information of the manually edited area word by word in sequence based on a recurrent neural network, and finding the first text information that is missing from the manually edited area compared to the automatically generated text area; The missing first text information is copied to the corresponding position of the manual editing area, and finally the second text information of the manual editing area is output.

Images