Speech recognition text editing processing method

Abstract

According to the speech recognition text editing processing method provided by the invention, a user interface is innovatively designed, and a manual editing region and an automatic text generation region are clearly separated. The user can freely modify the text in the manual editing area, the cursor position of the automatic text generation area is not affected, independent operation of the two areas is guaranteed, and the working efficiency and accuracy are improved. In order to solve the problem that the cursor position is disordered due to automatic text generation in the manual modification process, a cursor locking mechanism is introduced. When the user operates in the manual editing area, locking is started through a trigger button or a shortcut key, and automatic text generation in the editing area is paused. Unlocking after modification confirmation is completed, and recovering automatic text generation in the editing area. The mechanism ensures that the cursor position is stable during manual modification, and operation errors caused by interference generated automatically are avoided.

Description

Technical Field

[0001] The present invention relates to the field of speech text editing, and more specifically, to a method for editing and processing speech recognition text. Background Art

[0002] In today's digital age, the combined application of speech recognition and text editing has been continuously popularized. However, in actual use, there are still many problems. When speech recognition generates text, due to technical limitations, the automatically generated text often contains errors. When users manually correct the error information, the cursor position often changes. This phenomenon seriously affects the continuity and accuracy of speech recognition. In the traditional text processing process, speech recognition is usually an independent and prior step. That is, the speech recognition system first completes the conversion of all speech, and then users can start to view and repair the converted text. In this process, users need to wait for the speech recognition to complete, which may lead to a waste of time, especially when the speech recognition results require more repairs.

[0003] The existing technologies show obvious deficiencies in dealing with such problems. On the one hand, there is a lack of effective division between the editing area and the generated area, resulting in interference between manual editing and automatic generation. On the other hand, there is no dedicated cursor locking mechanism, so that during the manual modification process, automatic text generation is still in progress, resulting in chaos in the cursor position. In addition, the processing of text merging and conflict detection is not perfect enough, and problems such as duplication and inconsistency are likely to occur, affecting the quality of the final text, increasing the workload and difficulty of users, resulting in poor user experience, and restricting the application and promotion of this technology in a wider range of fields. Summary of the Invention

[0004] The present invention aims to overcome many defects in the combined application of existing speech recognition and text editing. Aiming at the interference problem caused by the ineffective division between the editing area and the generated area in the prior art, a method for editing and processing speech recognition text is proposed, including: Step 1, on the speech text generation interface, construct a user interface, the user interface includes a manual editing area and an automatic text generation area; Step 2, recognize the input speech signal, display the first text information generated by the recognition in the automatic text generation area, and synchronously copy the first text information to the manual editing area to generate the second text information; Step 3, when editing the second text information in the manual editing area, trigger a cursor locking operation to stop copying the first text information in the automatic text generation area to the manual editing area; Step 4, after the editing of the second text information in the manual editing area is completed, unlock the cursor, copy the first text information after the editing completion time in the automatic text generation area to the manual editing area, and return to Step 3 until the speech signal recognition is completed and the editing of the second text information in the manual editing area is completed, then execute Step 5; Step 5, merge the first text information in the automatic text generation area and the second text information in the manual editing area, and finally output the second text information in the manual editing area.

[0005] A speech recognition text editing and processing method provided by the present invention has the following beneficial effects: (1) Innovatively design a user interface, clearly separating the manual editing area and the automatic text generation area. Users can freely modify the text in the manual editing area without affecting the cursor position in the automatic text generation area, ensuring the independent operation of the two areas and improving work efficiency and accuracy.

[0006] (2) To solve the problem of cursor position chaos caused by automatic text generation during the manual modification process, the present invention introduces a cursor locking mechanism. When the user operates in the manual editing area, the locking is enabled by triggering a button or a shortcut key, and the automatic text generation in the editing area is paused. After the modification is confirmed, the locking is released and the automatic text generation in the editing area is restored. This mechanism ensures the stability of the cursor position during the manual modification and avoids operation errors caused by the interference of automatic generation.

[0007] (3) After both the manual modification and the automatic text generation are completed, the texts generated by the two can be intelligently merged into a coherent document, ensuring the integrity and coherence of the output document. Brief Description of the Drawings

[0008] Figure 1 It is a flowchart of a speech recognition text editing and processing method provided by the present invention. Detailed Embodiments

[0009] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention. In addition, the technical features in each embodiment or a single embodiment provided by the present invention can be combined with each other arbitrarily to form a feasible technical solution. Such combination is not restricted by the order of steps and / or the pattern of structural composition, but must be based on the fact that those of ordinary skill in the art can implement it. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0010] In the application of existing speech recognition technology, since it is necessary to wait until the speech recognition is converted into text and ends before manual repair can be carried out, synchronous operation cannot be achieved, which results in a waste of time. Because manual intervention cannot be carried out while the speech recognition is in progress, the entire process becomes discontinuous and inefficient.

[0011] Based on this, the present invention proposes a method of separating the text editing area and the text generation area, fundamentally eliminating the mutual interference between the two. In practical applications, this separation enables users to fully focus on the modification work during manual editing without worrying about the interference caused by the automatically generated text. At the same time, the automatic text generation area can also continuously and stably generate content without being affected by manual editing. This not only greatly improves work efficiency but also significantly enhances the accuracy of editing and generation, reducing errors caused by mutual interference.

[0012] Figure 1 The following is a flowchart of an intelligent speech recognition text editing and processing method provided by the present invention. As Figure 1 shown, the method includes: Step 1, on the speech text generation interface, construct a user interface, where the user interface includes a manual editing area and an automatic text generation area.

[0013] It can be understood that on the speech text generation interface, a user interface with a left-right structure is constructed, which respectively displays the manual editing area and the automatic text generation area. The automatic text generation area is displayed on the left, and the manual editing area is displayed on the right.

[0014] The automatic text generation area is specifically used to receive and display the text results converted by the speech recognition algorithm. The manual editing area can not only receive and display the text results generated by the speech recognition algorithm, but also provides an operation toolbar in the upper right corner of the interface for operations such as locking, unlocking, inserting, deleting, copying, pasting, and annotating. Its layout and operation methods have been carefully optimized to meet the operation habits and needs of users.

[0015] These two areas are independent of each other in layout and have obvious distinguishing marks in display. The manual editing area uses a more distinct background color or border to highlight its importance and independence. The automatic text generation area is displayed in a softer color tone to reduce the distraction of the user's attention. Any operation performed by the user in the editing area will not interfere with the cursor position in the generation area, and vice versa. This independent operation mechanism greatly reduces the interference between them and significantly improves work efficiency.

[0016] Step 2: Recognize the input speech signal, display the first text information generated by the recognition in the automatic text generation area, and synchronously copy the first text information to the manual editing area to generate the second text information.

[0017] It can be understood that after constructing the independent automatic text generation area and manual editing area, it is necessary to detect the automatic text generation area and manual editing area on the user interface so that the first text information generated by speech recognition can be accurately displayed in the automatic text generation area and the copied first text information can be accurately displayed in the manual editing area.

[0018] In a possible implementation manner of the present invention, in the step 2, recognizing the input speech signal, displaying the first text information generated by the recognition in the automatic text generation area, and synchronously copying the first text information to the manual editing area to generate the second text information includes: Based on the edge detection algorithm, respectively detect the constructed manual editing area and automatic text generation area, display the first text information generated by speech recognition in the automatic text generation area, and synchronously copy the first text information to the manual editing area.

[0019] Specifically, the automatic text generation area and the manual editing area are distinguished from the user interface based on the edge detection algorithm. Among them, the main steps of the edge detection algorithm include: Gaussian smoothing, calculating the gradient magnitude and direction, non-maximum suppression, and double-threshold processing and edge connection.

[0020] In an embodiment of the present invention, the detecting, based on the edge detection algorithm, of the constructed manual editing area and automatic text generation area respectively includes: Step 21, perform Gaussian smoothing on the user interface image.

[0021] Among them, the Gaussian smoothing process specifically includes: During the acquisition and transmission of the image, it may be disturbed by noise, and the noise will cause incorrect results in edge detection. Gaussian smoothing reduces the influence of noise by performing a convolution operation on the image and using a Gaussian kernel. The Gaussian kernel is a two-dimensional normal distribution function, which performs weighted averaging on each pixel in the image and its neighborhood, and the pixels closer to the central pixel have larger weights.

[0022] Therefore, calculate the Gaussian value of each pixel point in the user interface image: ; Among them, (x, y) is the pixel point coordinate, and G(x, y) is the Gaussian value of the pixel point. is the standard deviation of the Gaussian function, which controls the width of the Gaussian kernel. Remove the pixel points with Gaussian values less than the preset threshold and retain the pixel points with Gaussian values greater than or equal to the preset threshold to achieve the purpose of denoising.

[0023] Step 22, calculate the gradient magnitude and gradient direction of each pixel point in the Gaussian-smoothed user interface image.

[0024] Specifically, for the pixel points after Gaussian processing, calculate the gradient magnitude and gradient direction of each pixel point. Specifically, edges usually correspond to regions with large changes in gray values in the image, so these regions can be found by calculating the gradient of the image. The gradient represents the rate of change of the image in each direction, the gradient magnitude represents the magnitude of the change, and the gradient direction represents the direction of the change. A commonly used method is to use the Sobel operator to calculate the gradients of the image in the x and y directions. The Sobel operator is a 3*3 convolution kernel, which is used to calculate the gradients in the horizontal and vertical directions respectively.

[0025] Based on the Sobel operator, calculate the gradients of each pixel point in the x and y directions according to the horizontal operator and the vertical operator respectively. Among them, the horizontal direction gradient G x of the pixel point is: ; The vertical direction gradient G y of the pixel point is: ; Among them, I is the pixel point coordinate (x, y); Calculate the gradient magnitude G of each pixel point: ; Calculate the gradient direction of each pixel point: 。

[0026] Step 23: According to the gradient magnitude and gradient direction of each pixel, non-maximum suppression is used to detect preliminary edge pixels.

[0027] In a possible implementation manner of the present invention, the step of using non-maximum suppression to detect preliminary edge pixels according to the gradient magnitude and gradient direction of each pixel includes: For any pixel, find multiple pixels in the local neighborhood in the gradient direction to form local region pixels, and find the pixel with the largest gradient magnitude among all the pixels in the local region as the preliminary edge pixel.

[0028] Specifically, after calculating the gradient magnitude and direction, each pixel in the image has a gradient value. However, pixels with larger gradient magnitudes do not necessarily correspond to edges, because there may be multiple pixels with relatively large gradient magnitudes near the edge. The purpose of non-maximum suppression is to find the local maximum in the gradient direction and set the pixel values of non-local maxima to 0, thereby refining the edge. For each pixel, according to its gradient direction, compare the gradient magnitudes of its adjacent pixels in the gradient direction. If the gradient magnitude of this pixel is not the local maximum, set it to 0.

[0029] Step 24: Based on double-threshold processing for edge connection, filter out the final edge pixels from the preliminary edge pixels.

[0030] In a possible implementation manner of the present invention, the step of filtering out the final edge pixels from the preliminary edge pixels based on double-threshold processing for edge connection includes: Set a high threshold and a low threshold, compare the gradient magnitude of each preliminary edge pixel with the high threshold and the low threshold respectively. Mark the pixels with gradient magnitudes greater than the high threshold as strong edge pixels, mark the pixels with gradient magnitudes less than the low threshold as non-edge pixels, and mark the pixels with gradient magnitudes between the high threshold and the low threshold as weak edge pixels; For the weak edge pixels, retain the weak edge pixels connected to the strong edge pixels and remove the weak edge pixels not connected to the strong edge pixels. The strong edge pixels and the retained weak edge pixels constitute the final edge pixels.

[0031] Specifically, after non-maximum suppression, the edges in the image have been refined. However, there may still be some spurious edges caused by noise or other factors. Double-threshold processing further filters the edges by setting two gradient magnitude thresholds (a high threshold and a low threshold). Pixels with a gradient magnitude higher than the high threshold are considered strong edge pixel points, pixels with a gradient magnitude lower than the low threshold are considered non-edge pixel points, and pixels between the two thresholds are considered weak edge pixel points. Strong edge pixel points are definitely real edge pixel points, while weak edge pixel points may be real edge pixel points or noise. Through edge connection, the weak edge pixel points connected to the strong edge pixel points are retained, and the weak edge pixel points not connected to the strong edge pixel points are removed.

[0032] The automatic text generation area and the manual editing area are detected respectively by the above method. Using the speech recognition algorithm and the speech recognition model based on deep learning, the input speech signal is converted into the first text information. The automatically generated first text information is placed in the automatic text generation area, and then the first text information in the automatic text generation area is synchronously copied to the manual editing area. The text information in the manual editing area is called the second text information for the user to view and edit.

[0033] Speech recognition specifically includes: adopting the Mel Frequency Cepstral Coefficient (MFCC) algorithm to convert the speech signal into a feature vector. First, the speech signal is pre-emphasized to enhance the high-frequency part; then it is framed and windowed to segment the speech signal into short frames and windowed to reduce spectral leakage. Next, the Fast Fourier Transform (FFT) is performed to obtain the spectrum, and then the linear spectrum is converted to the Mel frequency scale through the Mel filter bank. Finally, logarithmic operation and Discrete Cosine Transform (DCT) are performed to obtain the MFCC features. The deep neural network speech recognition model is based on the Multi-Layer Perceptron (MLP) and is trained with a large amount of speech data. The input layer receives the speech feature vector, the intermediate hidden layer performs feature transformation and learning through a non-linear activation function (such as ReLU), and the output layer outputs the probability distribution of each phoneme or vocabulary.

[0034] In a possible implementation manner of the present invention, in step 2, recognizing the input speech signal includes: Step 21', pre-emphasize the input speech signal.

[0035] Specifically, the spectrum of the speech signal usually has the characteristic of high-frequency energy attenuation. The purpose of pre-emphasis is to enhance the high-frequency part of the speech signal, make the spectrum of the signal flatter, and facilitate subsequent processing. In the time domain, the calculation formula for pre-emphasis is , where is the original speech signal, is the pre-emphasized signal, and i represents the frame index.

[0036] Step 22’, frame and window the pre-emphasized speech signal.

[0037] Specifically, divide the speech signal into multiple short frames for spectral analysis of each frame. Divide the speech signal according to a certain frame length (such as 25 milliseconds) and frame shift (such as 10 milliseconds). Assume the sampling frequency of the speech signal is , the frame length is N sampling points, and the frame shift is M sampling points. Then the starting position of the i-th frame is i*M, and the signal within the frame is .

[0038] At the frame boundaries of the framed speech signal, discontinuities will occur. The purpose of windowing is to reduce the impact of such discontinuities on spectral analysis and make the signal within the frame gradually transition to zero at the boundaries. Use a Hamming window for windowing operation: , where , and N is the length of the window function. Multiply each frame of the signal by the window function to complete the windowing operation.

[0039] Step 23’, perform Fourier transform on the windowed speech signal to obtain the linear spectrum.

[0040] Specifically, convert the windowed time-domain signal to the frequency-domain signal to obtain the spectrum of each frame. Perform an N-point FFT transform on the windowed signal of each frame. Usually, N is taken as a power of 2 to improve the calculation efficiency. The FFT transform converts the time-domain signal to the frequency-domain signal to obtain the amplitude spectrum and phase spectrum of the signal.

[0041] Step 24’, convert the linear spectrum to the Mel frequency spectrum based on multiple Mel filters to obtain the output energy of each Mel filter.

[0042] Specifically, convert the linear spectrum to the Mel frequency spectrum to simulate the perception characteristics of the human ear for sounds of different frequencies. Design a set of Mel filter banks, usually containing 20 - 40 triangular filters. Filter the linear spectrum of each frame through the Mel filter bank to obtain the output energy of each filter.

[0043] Step 25’, take the logarithm of the output energy of each Mel filter and perform discrete cosine transform to obtain the MFCC feature vector.

[0044] Specifically, take the logarithm of the output energy of each Mel filter to convert the multiplicative relationship of the energy into an additive relationship, and at the same time simulate the logarithmic perception characteristics of the human ear for sound intensity. Take the logarithm of the output energy of each Mel filter to obtain , where j is the index of the Mel filter.

[0045] The logarithmic energy spectrum is subjected to discrete cosine transform to obtain MFCC parameters. DCT can transform the logarithmic energy spectrum with strong correlation into cepstrum coefficients with weak correlation while retaining the main features of the speech signal.

[0046] Step 26': input the MFCC feature vector into a deep neural network to obtain the first text information generated by speech recognition.

[0047] Specifically, text recognition is performed on the MFCC feature vector based on a deep neural network to obtain the first text information generated by speech recognition, and the recognized first text information is displayed in the automatic text generation area.

[0048] It should be noted that during the training of the deep neural network (speech recognition model), the user-corrected text is input into the speech recognition model as context in real time to adjust the decoding strategy. For example, the prosody prediction technology can be extended to weight the user-corrected content in the decoding stage to suppress similar errors by fusing the text feature correction results.

[0049] The error correction model architecture adopted takes the short sentences corrected by users as training samples, updates the language feature encoder parameters through online learning, and realizes dynamic optimization of the speech recognition model.

[0050] Error pattern analysis and incremental training: Establish an error log library, count high-frequency misrecognition words (such as a keyword list), and regularly trigger incremental training. For example, for common noise interference words in closed places (such as "valve" mistakenly written as "boring"), expand training data in a targeted manner and fine-tune the speech recognition model.

[0051] The combined cascade model design adds a feedback loop after the speech recognition model, aligns the user's correction content with the original audio and then feeds it back to the speech recognition module, forming a closed-loop optimization. Manually marked typos are used as context to continuously improve recognition quality.

[0052] Step 3: When the second text information is edited in the manual editing area, a cursor locking operation is triggered to stop copying the first text information in the automatic text generation area to the manual editing area.

[0053] Specifically, the first text information of speech recognition may contain errors, so it is necessary to manually edit and correct the text information of speech recognition. In an embodiment of the present invention, the first text information generated by speech recognition in the automatic text generation area is synchronously copied to the manual editing area, and the second text information is manually edited and corrected in the manual editing area. However, in the process of manual editing, since the text information in the automatic text generation area is constantly synchronously copied to the manual editing area, it will interfere with the editing process and cause confusion.

[0054] Based on this, in the embodiments of the present invention, when editing the second text information in the manual editing area, a cursor locking operation is triggered. For the cursor locking mechanism, an optimized design is carried out to make it only effective for the editing area. In this way, during the locking period, the voice can continue without affecting the effect of speech recognition in the automatic text generation area. When the user starts to modify in the manual editing area, locking is triggered, and only the cursor movement in the editing area is restricted.

[0055] It should be noted that during the process of editing the second text information in the manual editing area, if the second text information is modified, the position coordinates of each character in the second text information in the manual editing area need to be updated to facilitate the copying and alignment of the speech recognition text in the subsequent automatic text generation area and the second text information in the manual editing area.

[0056] In a possible real-time manner of the present invention, when editing the second text information in the manual editing area in step 3, it further includes updating the position coordinates of each character in the second text information after the editing modification: Taking the upper left corner of the manual editing area as the origin of the coordinate system, the right direction is the positive x-axis direction, and the downward direction is the positive y-axis direction; When inserting a new value at the position coordinate in the manual editing area, for the original position coordinates of each subsequent character are updated, and the updated position coordinates are ; Among them, the coordinate system of the manual editing area is expanded into a one-dimensional sequence by rows, and the position serial number of the character is calculated starting from the upper left corner in the original coordinate system. The original point position of each character is expressed as: , and the point position of the inserted character is expressed as: , where is the maximum coordinate value of the x-axis of the manual editing area; If , the point position of each subsequent character needs to be shifted one place to the right, ; Converting the point position of each character back to two-dimensional coordinates is: , ; When deleting the character at the position coordinate in the manual editing area, for the original position coordinates of each subsequent character are updated, and the updated position coordinates are ; Unfold the coordinate system of the manually edited area row by row into a one-dimensional sequence, and calculate the position serial number of the character starting from the upper left corner in the original coordinate system. The original point position of each character is expressed as: , and the point position of the deleted character is expressed as: ; If , then the point position of each subsequent character needs to be shifted one position to the left. ; Convert the point position of each character back to two-dimensional coordinates as: , .

[0057] During each lock edit, when performing an insert text operation, the subsequent text will execute the above logic, change its coordinate information, and record the total number of changed point positions.

[0058] In a possible implementation manner of the present invention, in step 3, when editing the second text information in the manually edited area, trigger a cursor lock operation to stop copying the first text information in the automatic text generation area to the manually edited area: Step 31, set a cursor operation button on the user interface. When editing the second text information in the manually edited area, trigger the cursor lock operation by clicking the cursor operation button; or, Step 32, respectively obtain the cursor position (x0, y0) at the previous moment and the cursor position (x1, y1) at the current moment. Among them, taking the upper left corner of the automatic text generation area as the origin of the coordinate system, the right direction is the positive x-axis direction, and the downward direction is the positive y-axis direction; when y1 < y0, or y1 = y0 and x1 > x0, it indicates that the second text information in the manually edited area is being edited at that moment, and trigger the cursor lock operation.

[0059] Specifically, when triggering the cursor lock operation, a cursor operation button can be set on the user interface. When it is necessary to edit the second text information in the manually edited area, the cursor lock operation can be triggered by clicking the cursor operation button. It is also possible to detect whether the second text information in the manually edited area is being edited manually by detecting the change in the cursor position. When it is detected that the second text information in the manually edited area is being edited, the cursor lock mechanism is automatically triggered.

[0060] Specifically, taking the upper left corner of the automatic text generation area as the origin of the coordinate system, with the right direction as the positive x-axis direction and the downward direction as the positive y-axis direction. The cursor positions at the previous moment (x0, y0) and the current moment (x1, y1) are obtained respectively. When y1 < y0, or when y1 = y0 and x1 > x0, it indicates that the second text information in the manual editing area is being edited at that moment, triggering the cursor locking operation and stopping copying the first text information in the automatic text generation area to the corresponding position in the manual editing area.

[0061] Step 4, after the editing of the second text information in the manual editing area is completed, unlock the cursor, copy the first text information after the editing completion moment in the automatic text generation area to the manual editing area, and return to Step 3 until the voice signal recognition is completed and the editing of the second text information in the manual editing area is completed, then execute Step 5.

[0062] Specifically, when the editing of the second text information in the manual editing area is completed this time, unlock the cursor, which can be achieved by clicking the cursor operation button. In terms of the unlocking design, when the user completes the editing and confirms the unlocking, the voice recognition result after the unlocking moment is automatically seamlessly connected with the content in the editing area. By real-time monitoring and analyzing the rhythm and semantics of the voice input, it is ensured that the subsequent voice recognition can be continuous and effective in the editing area. For example, using timestamp and semantic matching technology, accurately insert the unlocked voice content into the appropriate position in the editing area while maintaining the coherence and logic of the overall text.

[0063] It should be noted that when copying the first text information after the editing completion moment in the automatic text generation area to the manual editing area, calculate the similarity between the second text information copied to the manual editing area and the original first text information in the automatic text generation area. If the similarity is less than the preset similarity threshold, copy the second text information in the manual editing area to the manual editing area again.

[0064] When calculating the similarity between two text information, use cosine similarity to measure the semantic similarity between the text in the editing area and the text of the voice recognition. Assume that the word vector representation of the text in the editing area is , and the word vector representation of the text of the voice recognition is , then the cosine similarity The calculation formula is: ; where n is the dimension of the word vector, and are respectively and the i-th component of.

[0065] Step 5: Merge the first text information in the automatic text generation area and the second text information in the manual editing area, and finally output the second text information in the manual editing area.

[0066] It can be understood that when the speech recognition is completed and the second text information in the manual editing area is also edited, since after the cursor is released, the first text information after the cursor release moment is copied to the manual editing area, the second text information in the manual editing area is missing the text information in the period from the cursor locking moment to the cursor release moment compared with the first text information in the automatic text generation area. There is a blank space for the text information in this period in the manual editing area. To ensure the integrity of the output text information, the first text information in the automatic text generation area and the second text information in the manual editing area are merged, and the merged second text information is output.

[0067] The embodiment of the present invention develops an intelligent text merging algorithm, which can efficiently merge the texts in the manual editing area and the automatic generation area into a coherent document. When merging, first perform syntax and semantic analysis on the manually modified and automatically generated texts to find the repeated or conflicting parts. For the repeated content, screen and retain it according to the context and importance; for the inconsistent words, unify and optimize them through the vocabulary library and semantic understanding to ensure the quality of the merged document.

[0068] Specifically, in the embodiment of the present invention, in Step 5, merging the first text information in the automatic text generation area and the second text information in the manual editing area, and finally outputting the second text information in the manual editing area includes: Based on the recurrent neural network, compare each word in the first text information in the automatic text generation area and the second text information in the manual editing area in sequence to find the first text information missing in the manual editing area compared with the automatic text generation area; Copy the missing first text information to the corresponding position in the manual editing area, and finally output the second text information in the manual editing area.

[0069] It should be noted that during the process of copying the missing first text information to the corresponding position in the manual editing area, the copied text information is detected. Specifically, calculate whether the similarity between the second text information copied to the manual editing area and the original first text information in the automatic text generation area is less than a preset similarity threshold. If the similarity is less than the preset similarity threshold, copy the second text information in the manual editing area back to the manual editing area again, and finally output the complete and correct second text information.

[0070] The speech recognition text editing and processing method provided by the embodiments of the present invention has the following advantages: (1) Improve user experience: By clearly dividing the manual editing area and the automatic text generation area, the present invention reduces the interference between the two. Users can freely modify the text in the editing area without affecting the cursor position in the generation area. This design enables users to focus more on the editing work, reduces the trouble caused by the change of the cursor position, and thus improves the user experience.

[0071] (2) Improve text processing efficiency: The synchronization of speech recognition and manual repair is realized, which can significantly shorten the time of the entire text processing. Users do not need to wait for the completion of all speech recognition before performing repair, greatly improving the efficiency of text processing.

[0072] The introduction of the cursor locking mechanism separates the user's editing process from the automatic text generation process, avoiding the chaos of the cursor position caused by the interference of automatic generation. This mechanism ensures a stable cursor position for the user during editing, improving the accuracy and efficiency of editing. At the same time, the text merging algorithm can efficiently process the manually edited and automatically generated text, reducing conflicts and errors during the merging process, and further improving the efficiency of text processing.

[0073] It should be noted that in the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0074] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0075] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices produce means for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0076] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including instruction means that implement the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0077] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0078] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0079] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these changes and modifications.

Claims

1. A method for editing and processing speech recognition text, characterized in that, include: Step 1, constructing a user interface on the speech text generation interface, wherein the user interface includes a manual editing area and an automatic text generation area; Step 2, recognizing the input voice signal, displaying the first text information generated by the recognition in the automatic text generation area, and synchronously copying the first text information to the manual editing area to generate second text information; Step 3, when the second text information is edited in the manual editing area, triggering a cursor lock operation to stop copying the first text information in the automatic text generation area to the manual editing area; Step 4, when the editing of the second text information in the manual editing area is completed, unlock the cursor, copy the first text information after the editing completion time in the automatic text generation area to the manual editing area, return to step 3, until the voice signal recognition is completed and the editing of the second text information in the manual editing area is completed, and execute step 5; Step 5: merge the first text information of the automatic text generation area and the second text information of the manual editing area, and finally output the second text information of the manual editing area.

2. The voice recognition text editing and processing method according to claim 1, wherein The step 2, identifying the input voice signal, displaying the first text information generated by the identification in the automatic text generation area, and synchronously copying the first text information to the manual editing area to generate the second text information, includes: Based on an edge detection algorithm, the constructed manual editing area and the automatic text generation area are detected respectively, the first text information generated by speech recognition is displayed in the automatic text generation area, and the first text information is synchronously copied to the manual editing area; The step of detecting the constructed manual editing area and the automatic text generation area based on the edge detection algorithm comprises: Gaussian smoothing of the user interface image; Calculate the gradient magnitude and gradient direction of each pixel in the user interface image after Gaussian smoothing; According to the gradient amplitude and gradient direction of each pixel, the preliminary edge pixels are detected by non-maximum suppression. The edge connection is obtained based on the double threshold processing, and the final edge pixel points are screened out from the preliminary edge pixel points.

3. The speech recognition text editing and processing method according to claim 2, characterized in that, The Gaussian smoothing of the user interface image comprises: Calculate the Gaussian value of each pixel of the user interface image ; where (x, y) are the coordinates of the pixel point, and G(x, y) is the Gaussian value of the pixel point, is the standard deviation of the Gaussian function; Pixels with Gaussian values less than the preset threshold are removed, and pixels with Gaussian values greater than or equal to the preset threshold are retained.

4. The voice recognition text editing and processing method according to claim 2, characterized in that The step of calculating the gradient magnitude and gradient direction of each pixel in the user interface image after Gaussian smoothing includes: Based on the Sobel operator, calculate the gradients of each pixel point in the x and y directions according to the horizontal operator and the vertical operator respectively. Among them, the horizontal direction gradient G x is as follows: ； The vertical gradient G of the pixel point y is as follows: ； Among them, I is the pixel point coordinate (x, y); Calculate the gradient magnitude G of each pixel: ； Calculate the gradient direction of each pixel: 。 5. The speech recognition text editing and processing method according to claim 2 or 4, characterized in that, The method of detecting preliminary edge pixels using non-maximum suppression according to the gradient amplitude and gradient direction of each pixel point includes: For any pixel point, find multiple pixel points in the local neighborhood in the gradient direction to form local area pixels, and find the pixel point with the largest gradient amplitude among all the pixels in the local area as the preliminary edge pixel point; The step of processing edge connections based on double thresholds and selecting final edge pixel points from the preliminary edge pixel points includes: Set a high threshold and a low threshold, and compare the gradient magnitude of each of the preliminary edge pixel points with the high threshold and the low threshold respectively. Mark the pixel points with a gradient magnitude greater than the high threshold as strong edge pixel points, mark the pixel points with a gradient magnitude less than the low threshold as non-edge pixel points, and mark the pixel points with a gradient magnitude between the high threshold and the low threshold as weak edge pixel points; For the weak edge pixel points, retain the weak edge pixel points connected to the strong edge pixel points, and remove the weak edge pixel points not connected to the strong edge pixel points. The strong edge pixel points and the retained weak edge pixel points constitute the final edge pixel points.

6. The voice recognition text editing and processing method according to claim 1, wherein In step 2, the recognition of the input speech signal includes: Perform pre-emphasis on the input speech signal; Perform frame segmentation and windowing on the pre-emphasized speech signal; Perform Fourier transform on the windowed speech signal to obtain a linear spectrum; Based on multiple Mel filters, convert the linear spectrum into a Mel frequency spectrum to obtain the output energy of each Mel filter; Take the logarithm of the output energy of each Mel filter and perform discrete cosine transform to obtain MFCC feature vectors; Input the MFCC feature vectors into a deep neural network to obtain the first text information generated by speech recognition.

7. The voice recognition text editing and processing method according to claim 1, characterized in that When editing the second text information in the manual editing area in step 3, it also includes updating the position coordinates of each character in the second text information after editing and modification: Take the upper left corner of the manual editing area as the origin of the coordinate system, with the right direction as the positive x-axis direction and the downward direction as the positive y-axis direction; When inserting a new value at the position coordinates of the manually edited area, for the original position coordinates of each subsequent character update them, and the updated position coordinates are ; Among them, the coordinate system of the manually edited area is expanded into a one-dimensional sequence row by row, and the position serial number of the character is calculated starting from the upper left corner in the original coordinate system. The original point position of each character is expressed as: , and the point position of the inserted character is expressed as: , where is the maximum coordinate value of the x-axis of the manually edited area; If , the dot position of each subsequent character needs to be shifted one position to the right, ; Converting the dot positions of each character back to two-dimensional coordinates gives: , ; When deleting the character at the position coordinates of the manually edited area, for the original position coordinates of each subsequent character are updated, and the updated position coordinates are ; Expand the coordinate system of the manually edited area into a one-dimensional sequence row by row, and calculate the position serial number of the character starting from the upper left corner in the original coordinate system. The original point position of each character is expressed as: , and the point position of the deleted character is expressed as: ; If , the dot position of each subsequent character needs to be shifted one position to the left, ; Converting the dot positions of each character back to two-dimensional coordinates gives: , .

8. The voice recognition text editing and processing method according to claim 1, characterized in that, In step 3, when editing the second text information in the manual editing area, trigger the cursor locking operation to stop copying the first text information in the automatic text generation area to the manual editing area: Set a cursor operation button on the user interface. When editing the second text information in the manual editing area, trigger the cursor locking operation by clicking the cursor operation button; or, Respectively obtain the cursor position (x0, y0) at the previous moment and the cursor position (x1, y1) at the current moment; Take the upper left corner of the automatic text generation area as the origin of the coordinate system, with the right direction as the positive x-axis direction and the downward direction as the positive y-axis direction; When y1 < y0, or y1 = y0 and x1 > x0, it indicates that the second text information in the manual editing area is being edited at that moment, and trigger the cursor locking operation.

9. The voice recognition text editing and processing method according to claim 1, characterized in that In step 5, merge the first text information in the automatic text generation area and the second text information in the manual editing area, and finally output the second text information in the manual editing area, including: Based on a recurrent neural network, compare the first text information in the automatic text generation area and the second text information in the manual editing area word by word in sequence to find the first text information missing in the manual editing area compared to the automatic text generation area; Copy the missing first text information to the corresponding position in the manual editing area, and finally output the second text information in the manual editing area.

10. The method for editing and processing speech recognition text according to claim 9, wherein In step 4, copying the first text information in the automatic text generation area after the editing completion time to the manual editing area, and copying the missing first text information to the corresponding position in the manual editing area further includes: Calculate the similarity between the second text information copied to the manual editing area and the original first text information in the automatic text generation area. If the similarity is less than the preset similarity threshold, copy the second text information in the manual editing area to the manual editing area again.

Images