Intelligent text error correction method based on mouse interaction

Abstract

The invention relates to the technical field of intelligent text error correction, in particular to an intelligent text error correction method based on mouse interaction, and the method comprises the following steps: S1, displaying a character recognition result in an APP; s2, judging an error existing in the identification result in the APP input box; s3, starting a text error correction program through a key on the mouse; s4, popping up an error correction pop-up window moving along with the cursor; s5, a cursor is moved to the position near one wrong character through a mouse; s6, acquiring information such as a handle, input box content, cursor context content and a cursor position of the APP; s7, judging a current wrong entry and generating confused candidate entries; s8, screening out N optimal confused candidate entries from the confused candidate entries, and displaying the N optimal confused candidate entries in an error correction popup window; s9, clicking from the error correction popup window through a mouse to select a correct entry; s10, replacing the wrong characters in the APP with the selected correct entries, and exiting the text error correction mode through a key on the mouse; and efficient and accurate error correction of character recognition is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent text error correction, and in particular to an intelligent text error correction method based on mouse interaction. Background Art

[0002] With the development of artificial intelligence, artificial intelligence tools such as speech recognition and image recognition have made great progress and gradually become popular.

[0003] In practical applications, speech recognition (ASR) models and image recognition (OCR) models often exhibit certain text recognition errors. For example, when using an intelligent voice mouse for voice input, incorrect text is often recognized. To correct these misrecognized text, users must perform numerous mouse and keyboard operations, frequently alternating between these operations. This significantly reduces text input efficiency.

[0004] Therefore, there is an urgent need for an intelligent text error correction method based on mouse interaction, which can achieve efficient correction of erroneous text by using mouse interaction and intelligent models. Summary of the Invention

[0005] The purpose of the present invention is to provide an intelligent text correction method based on mouse interaction, which is used to solve the problem of low efficiency in correcting text with intelligent recognition errors in the prior art.

[0006] To achieve the above object, the present invention provides the following technical solution: an intelligent text error correction method based on mouse interaction, comprising the following steps:

[0007] S1. The AI ​​recognition model generates the text recognition result and displays it in the input box of the APP.

[0008] S2. Determine if there are any errors in the recognition results in the APP input box;

[0009] When there are no errors in the text of the recognition result, the error correction process is terminated directly;

[0010] S3. Use the mouse button to wake up the text correction mode to start the text correction program;

[0011] S4. A correction pop-up window will pop up as the cursor moves.

[0012] S5. Move the cursor to the vicinity of an error text using the mouse;

[0013] S6. Get the APP handle, input box content, cursor context content, cursor position and other information;

[0014] S7, judging the current erroneous term and generating confused candidate terms based on the acquired cursor context content and cursor position;

[0015] S8. Filtering N optimal obfuscation candidate entries from the obfuscation candidate entries and displaying them in an error correction pop-up window;

[0016] S9. Use the mouse to click on the correct entry in the correction pop-up window;

[0017] S10. Replace the incorrect text in the APP with the selected correct entry, and exit the text correction mode by pressing the button on the mouse.

[0018] Preferably, the APP in step S1 includes but is not limited to Word, WPS, Notepad, WeChat, browser and other APPs in Windows and MacOS systems.

[0019] Preferably, the AI ​​recognition model in step S1 is one of a speech recognition ASR model and an image recognition OCR model.

[0020] Preferably, when the cursor is moved in step S5, the cursor is moved to a position in front of or behind the erroneous text.

[0021] Preferably, the method for determining the current erroneous entry based on the cursor position and context in step S7 includes the following steps:

[0022] S7.1. The cursor position and context are provided to the AI ​​big model simultaneously. The AI ​​big model uses this information to calculate the offset of the erroneous entry relative to the current cursor position.

[0023] S7.2. Give all possible confusion candidate terms;

[0024] S7.3 merges and removes duplicates from the given confusion candidate terms to obtain a confusion candidate list of incorrect terms.

[0025] Preferably, the AI ​​big model includes but is not limited to one of the Qwen big model and the DeepSeek big model.

[0026] Preferably, the confusion candidate terms given in step S7.2 include but are not limited to speech recognition results after historical speech positioning extracts the wrong terms, a confusion dictionary of the wrong text, a dictionary of homophone terms of the wrong text, and confusion terms given by the AI ​​large model based on the wrong terms and all the contents in the entire input box.

[0027] Preferably, in step S8, all confused term candidates, erroneous terms, and contextual content are simultaneously input into the AI ​​big model, which then sorts the confused term candidate list from high to low according to the possibility of term replacement, and displays the top N terms after sorting in the error correction window.

[0028] Preferably, when the correct entry is not displayed in the error correction pop-up window of step S8, all confused candidate entries are classified according to the set filtering mechanism and displayed in the error correction pop-up window, and then the filtering conditions are operated in the y direction of the error correction pop-up window by the mouse, and the confused candidate entries under the filtering conditions are operated in the x direction of the error correction pop-up window to select the correct entry.

[0029] Preferably, the screening mechanism is one of pinyin and radical.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] 1. The present invention relates to an intelligent text error correction method based on mouse interaction, which can complete the entire error correction process only by mouse operation, avoiding keyboard switching, and greatly improving the efficiency of text recognition and error correction.

[0032] 2. The present invention relates to an intelligent text error correction method based on mouse interaction, which adopts the method of integrating intelligent models with multi-source data to ensure the accuracy of error recognition and candidate word recommendation.

[0033] 3. The present invention involves an intelligent text correction method based on mouse interaction that supports cross-platform (Windows, MacOS) and multiple application scenarios (Word, WPS, WeChat, browsers, etc.); it adopts commonly used pinyin index and radical index to reduce user selection costs and can adapt to different correction scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A flowchart of an intelligent text error correction method based on mouse interaction according to the present invention;

[0035] Figure 2 This is a schematic diagram of an error correction pop-up window in an embodiment of an intelligent text error correction method based on mouse interaction of the present invention. DETAILED DESCRIPTION

[0036] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0037] Example, see Figure 1-2 , an intelligent text error correction method based on mouse interaction, comprising the following steps:

[0038] S1. The results recognized by the ASR model are displayed in the APP. The APP includes, but is not limited to, APPs in Windows and MacOS systems such as Word, WPS, Notepad, WeChat, browsers, etc.

[0039] S2. The user determines whether there are errors in the current recognition results. If there are no errors, then directly end the current error correction process.

[0040] S3. The user presses the error correction mode button on the mouse to wake up the text error correction mode, and the program opens an error correction pop-up window.

[0041] S4. Pop up an error correction pop-up window that follows the cursor movement;

[0042] S5. The user moves the cursor to near a misspelled word by the mouse. In this embodiment, the user moves the cursor to near the two characters "妄想" in the text "合肥妄想城的人气很汪". The specific position can be determined according to the user's preference, that is, the cursor is moved to one of the front or rear of the misspelled word.

[0043] S6. Obtain information such as the handle of the APP, the content of the input box, the context content of the cursor, and the cursor position. The information returned in this embodiment is {handle: 12345, input box content: 合肥妄想城的人气很汪, context content: 合肥妄想城的人气很汪, cursor position: 0, 3}.

[0044] S7. According to the obtained context content of the cursor and the cursor position, judge the current error entry and generate confused candidate entries;

[0045] Here, the cursor position and the context content are simultaneously provided to the Qwen or DeepSeek large model, and the large model will give the offset of the misspelled entry relative to the current cursor position according to this information. For example, in this embodiment, it is (-1, +1, +2).

[0046] Then, this program will give all possible confused candidate entries. Here, several methods are used simultaneously to ensure the comprehensiveness of the confused candidates, including but not limited to: a) the speech recognition results after the misspelled entry is extracted by historical voice positioning; b) the confusion dictionary of the misspelled text; c) the homophone entry dictionary of the misspelled text; d) the confused entries given by the Qwen or DeepSeek large model according to the misspelled entry and all the content in the entire input box.

[0047] By combining these confusion candidate entries and removing duplicates, a comprehensive confusion entry candidate list of the current erroneous entry is obtained. For example, in this embodiment, it is "Vientiane City, Wanxiang City, Wanxiang City, delusion, play thread, play formation..."

[0048] S8. Filter out N optimal confusion candidate terms from the confusion candidate terms and display them in an error correction pop-up window.

[0049] Here, all obfuscated term candidates, incorrect terms, and context are simultaneously fed into a Qwen or DeepSeek large model. The large model then sorts the list of obfuscated term candidates from highest to lowest likelihood of replacement. The top N ranked terms are then displayed in the error correction window. For example, in this example, "Vanxiang City, Wanxiang City, Delusion" is displayed.

[0050] S9. Use the mouse to click on the correct entry in the error correction pop-up window.

[0051] This program classifies all confusion candidate entries according to the set screening mechanism (pinyin or radical) and displays them in the correction pop-up window, such as Figure 2 shown.

[0052] The user uses the mouse to operate the error correction pop-up window, operates the filtering conditions in the y direction, operates the confused candidate terms under the filtering conditions in the x direction, and selects the correct term.

[0053] The user clicks on the correct candidate entry, which is achieved by calling the system-level text message interface and keyboard key events. In this implementation, the user clicks on the entry "Vientiane City".

[0054] S10. The incorrect text in the APP is replaced with the selected correct entry.

[0055] S11. The user operates the mouse to move the cursor to the location of the next erroneous text, and the process is executed from S4 to S10 until all errors have been corrected.

[0056] S12. Press the error correction mode button on the mouse to exit the error correction mode, and the error correction process ends.

[0057] The system of the error correction method comprises: an interactive module for receiving a mouse trigger signal, controlling an error correction mode switch and tracking a cursor position;

[0058] The information acquisition module obtains the handle, text content, and cursor position through the operating system API (such as WM_GETTEXT and EM_GETSEL in Windows and AXUIElement in MacOS);

[0059] Intelligent analysis module, which generates error entries and offsets based on cursor context and intelligent models, and integrates candidate words from multiple sources;

[0060] The sorting and display module sorts the candidate words and displays them in a pop-up window, supporting pinyin indexing and filtering;

[0061] Replacement module, which implements text replacement through system-level interfaces such as EM_SETSEL, EM_REPLACESEL, or AXUIElementSetAttributeValue.

[0062] When the user presses the error correction mode button, the mouse sends a key data packet to this program via the 2.4G or Bluetooth protocol. Then this program obtains the currently active window handle through the system interface.

[0063] Use the GetForegroundWindow interface in Win and apply the following code:

[0064] HWND hwnd=GetForegroundWindow();

[0065] Use the following code in MacOS:

[0066]

[0067]

[0068] Get all the text in the input box of the APP through the system-level API.

[0069] In Win, use the WM_GETTEXT message and the SendMessage interface to get all the text using the following code.

[0070]

[0071]

[0072] Get the input cursor position in the APP through the system-level API

[0073] In the Win system, use the EM_GETSEL message and the SendMessage interface to obtain the information. The application is as follows:

[0074] HWND hwndEdit = / *target window handle* / ;

[0075] DWORD start,end;

[0076] SendMessage(hwndEdit,EM_GETSEL,(WPARAM)&start,(LPARAM)&end);

[0077] / / start == end indicates the insertion point; otherwise it is the selection area

[0078] In MacOS, use Apple's accessibility framework AXUIElement to obtain the application as follows

[0079] AXUIElementRef systemWide=AXUIElementCreateSystemWide();

[0080] AXUIElementRef focusedApp=NULL;

[0081] AXUIElementRef focusedUIElement=NULL;

[0082] AXUIElementCopyAttributeValue(systemWide,

[0083] kAXFocusedApplicationAttribute,(CFTypeRef*)&focusedApp);

[0084] AXUIElementCopyAttributeValue(systemWide,

[0085] kAXFocusedUIElementAttribute,(CFTypeRef*)&focusedUIElement);

[0086] / / Get the row number or range where the insertion point is located

[0087] CFTypeRef value = NULL;

[0088] AXUIElementCopyAttributeValue(focusedUIElement,

[0089] CFSTR("AXSelectedTextRange"),&value);

[0090] Calculate error message for input cursor position

[0091] Organize the text content in 2 and the input cursor position in 3 into json format, as follows

[0092] {

[0093] "content":"Hefei Fantasy City is very popular",

[0094] “caretpos”:”(3,3,0)”

[0095] }

[0096] Add the following prompt words

[0097] You will receive a text content and a tuple `(start, end, y)` indicating the position of the input cursor. The text may come from speech recognition and may contain recognition errors.

[0098] Please complete the following tasks:

[0099] 1. Based on the cursor position, analyze the characters before and after it to identify possible speech recognition errors (such as semantic incomprehension, common sense errors, typos, etc.).

[0100] 2. Find the wrong character (maybe one or more).

[0101] 3. Output the offset of these erroneous characters relative to the cursor starting position (i.e.: character subscript - cursor starting subscript).

[0102] Please return in the following JSON format:

[0103]

[0104] Input the json content and prompt words into the qwen / deepseek model at the same time, and the qwen / deepseek model will give the answer, such as:

[0105] {

[0106] "wrong":"Dream City",

[0107] "offset":[-1,0,1]

[0108] }

[0109] Get all confusion candidates for the wrong term

[0110] In order to cover all obfuscation candidates as much as possible, this program uses the following obfuscation candidate sources.

[0111] Speech recognition results after historical speech localization extracts incorrect terms

[0112] Obfuscation dictionary for incorrect text

[0113] Dictionary of phonetic entries for incorrect text

[0114] The Qwen / DeepSeek large model generates confusing entries based on the incorrect entry and all the content in the entire input box.

[0115] The prompt words for the Qwen / DeepSeek large model are as follows:

[0116] You are a speech recognition error analysis assistant. You receive a speech recognition text and an incorrectly recognized term. Based on the context, phonetic similarity, and common confusion phenomena, please provide possible correct candidates (confused terms) for the incorrect term.

[0117] Here is an example:

[0118]

[0119] The results are as follows

[0120] {

[0121] "candidates":["Vientiane City","Wangxiang City","Wangxiang City"]

[0122] }

[0123] Calculate the ranking of all obfuscation candidates

[0124] All obfuscation candidates from the five sources are put into the same list and deduplicated. The resulting obfuscation candidate list is then sorted using the Qwen / DeepSeek large model.

[0125] The prompt words are as follows

[0126] You are a speech recognition post-processing assistant.

[0127] Now the input is:

[0128] content: Complete speech recognition text content

[0129] wrong_word: Identify incorrect words

[0130] Candidates: Multiple candidate terms provided by the user or the system, which may be the original words of the wrong terms

[0131] Please sort the terms in candidates by the probability of being the most likely correct term, taking into account factors such as contextual semantics, pronunciation similarity, language fluency, and common collocations, and return the sorted list.

[0132] Here is an example:

[0133]

[0134] The output example is as follows

[0135] {

[0136] "sorted_candidates":["Vientiane City","Dream City","Wangxiang City","Wangxiang City","Wangxiang City"]

[0137] }

[0138] A pop-up window will appear near the cursor position, showing confusion candidates and indexing based on pinyin.

[0139] Use the open-source tool pypinyin to annotate all six confusion candidates, and use the annotated results to index the confusion candidates. Based on the cursor position, a pop-up window displays the candidates and their pinyin index near the cursor. Furthermore, the corresponding confusion candidates are displayed based on the pinyin.

[0140] After the user selects the correct entry, the system interface is called to replace the incorrect entry with the correct entry.

[0141] In win, use the following code

[0142] / / Set up a new selection

[0143] SendMessage(hwndEdit,EM_SETSEL,start,end);

[0144] / / Replace with new string

[0145] SendMessage(hwndEdit,EM_REPLACESEL,TRUE,(LPARAM)L"new content");

[0146] In macOS, use code like the following

[0147] AXUIElementRefsystemWide=AXUIElementCreateSystemWide();

[0148] AXUIElementReffocusedElement=NULL;

[0149] / / Get the currently focused control (such as an input box)

[0150] AXUIElementCopyAttributeValue(systemWide,

[0151] kAXFocusedUIElementAttribute,(CFTypeRef*)&focusedElement);

[0152] / / Get the current cursor position (e.g. location=5, length=0)

[0153] CFTypeRefselectedRange=NULL;

[0154] AXUIElementCopyAttributeValue(focusedElement,

[0155] CFSTR("AXSelectedTextRange"),&selectedRange);

[0156] / / Assuming it is a CFRange or similar structure wrapped by NSValue

[0157] CFRange range = ...; / / parse the value of selectedRange

[0158] / / Extend forward 1, expand backward 1

[0159] range.location=MAX(0,range.location-1);

[0160] range.length=range.length+2;

[0161] / / Set up a new selection

[0162] AXUIElementSetAttributeValue(focusedElement,CFSTR("AXSelectedTextRange"),(__bridge CFTypeRef)([NSValue valueWithRange:range]));

[0163] / / Replace content (note that AXValue corresponds to the entire input box text, but some controls support replacing the current selection)

[0164] AXUIElementSetAttributeValue(focusedElement,CFSTR("AXValue"),CFSTR("Replacement content"));

[0165] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0166] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent text error correction method based on mouse interaction, characterized in that: The following steps are involved: S1. The AI ​​recognition model generates the text recognition result and displays it in the input box of the APP. S2. Determine if there are any errors in the recognition results in the APP input box; When there are no errors in the text of the recognition result, the error correction process is terminated directly; S3. Use the mouse button to wake up the text correction mode to start the text correction program; S4. A correction pop-up window will pop up as the cursor moves. S5. Move the cursor to the vicinity of an error text using the mouse; S6. Get the APP handle, input box content, cursor context content, cursor position and other information; S7, judging the current erroneous term and generating confused candidate terms based on the acquired cursor context content and cursor position; S8. Filtering N optimal obfuscation candidate entries from the obfuscation candidate entries and displaying them in an error correction pop-up window; S9. Use the mouse to click on the correct entry from the error correction pop-up window; S10. Replace the incorrect text in the APP with the selected correct entry, and exit the text correction mode by pressing the button on the mouse.

2. The intelligent text error correction method based on mouse interaction according to claim 1, characterized in that: The APPs in step S1 include but are not limited to Word, WPS, Notepad, WeChat, browsers and other APPs in Windows and MacOS systems.

3. The intelligent text error correction method based on mouse interaction according to claim 1, characterized in that: The AI ​​recognition model in step S1 is one of a speech recognition ASR model and an image recognition OCR model.

4. The intelligent text error correction method based on mouse interaction according to claim 1, characterized in that: When the cursor is moved in step S5, the cursor moves to a position before or after the erroneous text.

5. The intelligent text error correction method based on mouse interaction according to claim 1, characterized in that: The method for determining the current erroneous entry based on the cursor position and context in step S7 includes the following steps: S7.

1. The cursor position and context are provided to the AI ​​big model simultaneously. The AI ​​big model uses this information to calculate the offset of the erroneous entry relative to the current cursor position. S7.

2. Give all possible confusion candidate terms; S7.3 merges and removes duplicates from the given confusion candidate terms to obtain a confusion candidate list of incorrect terms.

6. The intelligent text error correction method based on mouse interaction according to claim 5, characterized in that: AI big models include but are not limited to one of the Qwen big models and the DeepSeek big model.

7. The intelligent text error correction method based on mouse interaction according to claim 5, characterized in that: The confusion candidate entries given in step S7.2 include but are not limited to the speech recognition results after the historical speech positioning extracts the wrong entry, the confusion dictionary of the wrong text, the dictionary of homophone entries of the wrong text, and the confusion entry given by the AI ​​large model based on the wrong entry and all the contents in the entire input box.

8. The intelligent text error correction method based on mouse interaction according to claim 1, characterized in that: In step S8, all the confused term candidates, erroneous terms, and contextual content are simultaneously input into the AI ​​big model, which then sorts them from high to low based on the likelihood of term replacement in the confused candidate term list, and displays the top N terms after sorting in the error correction window.

9. The intelligent text error correction method based on mouse interaction according to claim 1, characterized in that: When the correct entry is not displayed in the error correction pop-up window of step S8, all confused candidate entries are classified according to the set filtering mechanism and displayed in the error correction pop-up window. Then, the filtering conditions are operated in the y direction of the error correction pop-up window by the mouse, and the confused candidate entries under the filtering conditions are operated in the x direction of the error correction pop-up window to select the correct entry.

10. The intelligent text error correction method based on mouse interaction according to claim 9, characterized in that: The filtering mechanism is one of pinyin and radical.

Images