A semantic recognition system and method for image text

Abstract

The application discloses a kind of for image character semantic recognition system and method, it is related to intelligent identification technical field, including control module, the control module is connected with database, data acquisition module, direction correction module, semantic recognition module and early warning module;The database is used to store the characteristic value of word, the semantics of word, the identification error record of user feedback and the record of unrecognizable;The data acquisition module is used to obtain the word to be identified, and generates corresponding early warning signal according to the word to be identified obtained;The direction correction module is used for direction correction to the word to be identified;The semantic recognition module is used for semantic recognition to the word to be identified, and generates corresponding early warning signal in the identification process;The early warning module is used for processing early warning signal;The system is corrected to the word to be identified by direction correction module, so that subsequent semantic recognition is more accurate.

Description

Technical Field

[0001] This invention relates to the field of intelligent recognition technology, specifically a semantic recognition system and method for image text. Background Technology

[0002] Artificial intelligence recognition technology refers to the technical means of automatically acquiring and recognizing target instructions, data, and other information through devices such as computers, cameras, and scanners. Image recognition technology is an important field of artificial intelligence recognition. It refers to the technology of object recognition in images to identify targets and objects of various patterns. Image recognition technology may be based on the main features of an image. Every image has its features, such as the letter A having a point, P having a circle, and Y having an acute angle at its center. Studies on eye movements during image recognition show that the gaze is always focused on the main features of the image, that is, on the places where the curvature of the image contour is greatest or where the contour direction changes abruptly; these places contain the most information. Moreover, the scanning path of the eye always moves sequentially from one feature to another. Therefore, in the process of image recognition, the perceptual mechanism must filter out redundant input information and extract key information. At the same time, there must be a mechanism in the brain responsible for integrating information, which can organize the information acquired in stages into a complete perceptual image.

[0003] In the existing technology, recognizing text in images has become a relatively mature technology, but there are still some obstacles in recognizing text from different angles. Therefore, a semantic recognition system and method for image text is provided. Summary of the Invention

[0004] The purpose of this invention is to provide a semantic recognition system and method for image text.

[0005] The objective of this invention can be achieved through the following technical solution: a semantic recognition system for image text, comprising a control module, wherein the control module is connected to a database, a data acquisition module, a direction correction module, a semantic recognition module, and an early warning module;

[0006] The database is used to store the feature values ​​of words, the semantics of words, user feedback records of recognition errors and unrecognizable records;

[0007] The data acquisition module is used to acquire the words to be identified and generate corresponding warning signals based on the acquired words to be identified.

[0008] The orientation correction module is used to correct the orientation of the words to be recognized.

[0009] The semantic recognition module is used to perform semantic recognition on the words to be recognized and generate corresponding warning signals during the recognition process;

[0010] The early warning module is used to process early warning signals.

[0011] Furthermore, the database includes a character library, semantic units, and recognition record units;

[0012] The character library is used to store the feature values ​​of words, and the feature values ​​are composed of the coordinates of multiple feature points;

[0013] The semantic unit is used to store the semantics of words; the words and semantics are not limited to expressing the meaning of other languages ​​in one language, that is, equivalent words are semantically related to each other;

[0014] The identification and recording unit includes an error identification area and an unidentifiable area;

[0015] The error recognition area is used to store records of recognition errors reported by users; the unrecognizable area is used to store records of unrecognizable information from the semantic recognition module.

[0016] Furthermore, the data acquisition module is equipped with both click-based and drag-and-drop acquisition methods.

[0017] The click-based acquisition means that when a user clicks on the screen, the words within a certain range of the user's click coordinates are automatically acquired.

[0018] The drag-and-drop acquisition means that when a user drags on the screen, the words within the drag range are automatically acquired.

[0019] Furthermore, the specific process of obtaining words through click-based methods is as follows:

[0020] Get the user's click coordinates on the screen;

[0021] Set the extension values ​​x and y;

[0022] The selected area is generated by extending horizontally by x units and vertically by y units from the center of the click coordinates.

[0023] Set the threshold z;

[0024] Get the pixel values ​​in the selected area. When the percentage of pixel values ​​exceeds the threshold z, mark the pixel value as to be valued.

[0025] The pixel value proportion is the ratio of the number of pixels within a certain pixel value range to the total number of pixels;

[0026] For example, if there are N pixels with pixel values ​​between [135, 140] and the total number of pixels is M, then the percentage of pixels with pixel values ​​between [135, 140] is... ,like If the value exceeds the threshold z, then the pixel value [135, 140] is to be estimated;

[0027] The image to be evaluated is binarized sequentially, and then the feature values ​​are extracted from the image and matched with the feature values ​​in the character library.

[0028] A threshold T is set. When the degree of matching between the feature values ​​in the image and the feature values ​​in the character library is greater than or equal to the threshold T, the image to be valued is marked as a word to be recognized. When the degree of matching between the feature values ​​in the image and the feature values ​​in the character library is less than the threshold T, the image is submitted to the unrecognizable area in the database for storage, and a first-level warning signal is generated.

[0029] At the same time, set the supplementary value u;

[0030] Based on the selected area, expand outward by u units to generate a supplementary area. Check if there are any pixels with values ​​to be estimated in the supplementary area. If not, no operation is required. If they are, add the pixels with values ​​to be estimated from the supplementary area to the word to be recognized. After the supplementation is completed, expand outward by u units again to generate a supplementary area. Repeat the above supplementation operation until there are no pixels with values ​​to be estimated in the supplementary area.

[0031] Furthermore, the implementation process of drag-and-drop word retrieval is as follows:

[0032] Obtain the area selected by the user on the screen, obtain the value to be valued, and perform binarization on the value to be valued in turn.

[0033] The original pixel values ​​are extracted to form the feature values ​​of the image composed of the pixels to be valued. These feature values ​​are then matched with the feature values ​​in the character library. If the matching degree between the feature values ​​in the image and the feature values ​​in the character library is greater than or equal to a threshold T, the image composed of the pixels to be valued is marked as a word to be recognized. If the matching degree is less than the threshold T, the image is submitted to the unrecognizable area in the database for storage, and a first-level warning signal is generated.

[0034] Furthermore, the process by which the direction correction module corrects the direction of the word to be recognized is as follows:

[0035] Obtain the feature values ​​of the words to be identified;

[0036] Set the difference value based on the feature value; submit a query request to the character library in the database to obtain the character feature value within the difference value range;

[0037] The feature point coordinates of two words to be identified are randomly selected, and the deviation value between the feature point coordinates and the corresponding feature point coordinates in the character is obtained. The deviation angle is generated based on the deviation value. After obtaining the deviation angle, the word to be identified will be adjusted according to the deviation angle.

[0038] Obtain the coordinates of the feature points of the word to be identified. Establish a two-dimensional coordinate system with the coordinates of a certain feature point as the origin. Based on the tangent value, adjust the angles of all feature point coordinates except the origin relative to the origin.

[0039] Furthermore, the semantic recognition module performs semantic recognition on the words to be recognized, including:

[0040] After receiving the word to be identified, a matching request is submitted to the semantic unit in the database, that is, the word to be identified is matched with the words in the semantic unit. If the match is successful, a language query is sent to the user to ask the user for the target language.

[0041] After the user submits the target language, a query request is submitted to the semantic unit to query the semantics of the word to be identified in the target language environment, and the semantics are fed back to the user.

[0042] If the match fails, a level 2 warning signal will be generated.

[0043] Furthermore, a semantic intelligent recognition method for smart devices includes the following steps:

[0044] Step S1: Obtain the area selected by the user by clicking or dragging on the screen, obtain the value to be valued within the area, and perform binarization processing on the value to be valued in sequence. Extract the feature value of the image after binarization processing, and match it with the feature value of the character in the character library. If the matching degree is greater than or equal to the threshold, the matching is considered successful, and the image is marked as the word to be recognized.

[0045] Step S2: Set the phase difference value, obtain the character feature values ​​within the phase difference value range in the character library, randomly select the feature point coordinates of two words to be recognized, obtain the deviation value between the feature point coordinates and the corresponding feature point coordinates in the characters, generate the deviation angle based on the deviation value, and adjust the angle of the words to be recognized according to the deviation value angle.

[0046] Step S3: Match the word to be identified after angle adjustment with the words in the semantic unit. If the match is successful, obtain the semantics of the word to be identified and send the semantics to the user. If the match fails, report the failure to obtain the semantics to the user.

[0047] Compared with the prior art, the beneficial effects of the present invention are:

[0048] This system obtains the click coordinates of the user on the screen, sets an extension value, extends the center according to the extension value with the click coordinates as the center, marks it as the selected area, and filters out the words to be recognized in the selected area through binarization processing and eigenvalue matching. At the same time, a supplementary value is set to generate a supplementary area to supplement the words to be recognized, greatly improving the accuracy of the words to be recognized.

[0049] This system randomly extracts the characteristic point coordinates of two words to be recognized, obtains the deviation value between the characteristic point coordinates and the corresponding characteristic point coordinates in the character, and then generates a deviation angle according to the deviation value; adjusts the angle of the words to be recognized through the deviation angle, improving the discrimination of some words with angular distortion. Brief Description of the Drawings

[0050] Figure 1 It is the schematic diagram of the present invention. Detailed Embodiment

[0051] As Figure 1 shown, a semantic recognition system for image text includes a control module, and the control module is connected to a database, a data acquisition module, a direction correction module, a semantic recognition module, and an early warning module;

[0052] The database is used to store the characteristic values of words, the semantics of words, the recognition error records and unrecognizable records feedback by users. The specific process is as follows:

[0053] The database includes a character library, a semantic unit, and a recognition record unit;

[0054] The character library is used to store the characteristic values of words, and the characteristic values are composed of the coordinates of multiple characteristic points;

[0055] The semantic unit is used to store the semantics of words; the words and semantics are not limited to expressing the meanings of other languages in a certain language; that is, synonymous words are semantic to each other; for example, the semantics of "严重" in English is "serious", and the semantics of "serious" in Chinese is "严重";

[0056] The recognition record unit includes an error recognition area and an unrecognizable area;

[0057] The error recognition area is used to store the recognition error records feedback by users; the unrecognizable area is used to store the unrecognizable records of the semantic recognition module;

[0058] Subsequently, the word recognition and semantic recognition can be further optimized according to the information recorded in the error recognition area and the unrecognizable area;

[0059] The data acquisition module is used to acquire words that need to be semantically recognized. The specific process is as follows:

[0060] The data acquisition module is equipped with both click-based and drag-and-drop acquisition methods.

[0061] The click-based acquisition means that when a user clicks on the screen, the words within a certain range of the user's click coordinates are automatically acquired.

[0062] The drag-and-drop acquisition means that when a user drags on the screen, the words within the drag range are automatically acquired.

[0063] The specific implementation process of the click-based word acquisition is as follows:

[0064] Get the user's click coordinates on the screen;

[0065] Set the extension values ​​x and y;

[0066] The selected area is generated by extending horizontally by x units and vertically by y units from the center of the click coordinates.

[0067] Set the threshold z;

[0068] Get the pixel values ​​in the selected area. When the percentage of pixel values ​​exceeds the threshold z, mark the pixel value as to be valued.

[0069] The pixel value proportion is the ratio of the number of pixels within a certain pixel value range to the total number of pixels;

[0070] For example, if there are N pixels with pixel values ​​between [135, 140] and the total number of pixels is M, then the percentage of pixels with pixel values ​​between [135, 140] is... ,like If the value exceeds the threshold z, then the pixel value [135, 140] is to be estimated;

[0071] The pixels to be valued are binarized sequentially, that is, the pixels with the value to be valued are changed to pixels with the value of 0, and the pixels not to be valued are changed to pixels with the value of 255.

[0072] Then, the feature values ​​of the image are formed by extracting the pixels with a pixel value of 0, and matched with the feature values ​​in the character library;

[0073] A threshold T is set. When the degree of matching between the feature values ​​in the image and the feature values ​​in the character library is greater than or equal to the threshold T, that is, when the number of coordinates of feature points in the image that match the coordinates of characters in the character library is greater than or equal to the total number of character coordinates, the image to be valued is marked as a word to be recognized. When the degree of matching between the feature values ​​in the image and the feature values ​​in the character library is less than the threshold T, the image is submitted to the unrecognizable area in the database for storage. At the same time, a first-level warning signal is generated and submitted to the warning module for corresponding processing.

[0074] At the same time, set the supplementary value u;

[0075] Based on the selected area, expand outward by u units to generate a supplementary area. Perform a supplementary operation on the word to be recognized, that is, check whether there are pixels with values ​​to be estimated in the supplementary area. If not, no operation is required. If they exist, add the pixels with values ​​to be estimated from the supplementary area to the word to be recognized. After the supplement is completed, expand outward by u units again to generate a supplementary area. Repeat the above supplementary operation until there are no pixels with values ​​to be estimated in the supplementary area.

[0076] The specific implementation process of the drag-and-drop word retrieval method is as follows:

[0077] Obtain the area selected by the user on the screen, obtain the value to be valued, and perform binarization on the value to be valued in turn.

[0078] The original pixel values ​​are extracted to form the feature values ​​of the image composed of the pixels to be valued. These feature values ​​are then matched with the feature values ​​in the character library. If the matching degree between the feature values ​​in the image and the feature values ​​in the character library is greater than or equal to a threshold T, the image composed of the image to be valued is marked as a word to be recognized. If the matching degree is less than the threshold T, the image is submitted to the unrecognizable area in the database for storage. At the same time, a first-level warning signal is generated and submitted to the warning module for appropriate processing.

[0079] The word to be recognized is submitted to the orientation correction module, which then performs the next step of processing.

[0080] This module differentiates between different ways users touch the screen and uses different methods to acquire words for different touch methods, thus improving the flexibility of word acquisition. By adding extended and supplementary areas, it improves the accuracy of click-based word acquisition and avoids missing words due to the extension area being too small.

[0081] The orientation correction module is used to correct the orientation of the words to be recognized, making subsequent semantic recognition more accurate. The specific process is as follows:

[0082] Obtain the feature values ​​of the words to be identified;

[0083] Set the phase difference value P according to the aforementioned feature value;

[0084] Submit a query request to the character library in the database to obtain character feature values ​​within the range of difference value P;

[0085] The feature point coordinates of two words to be identified are randomly selected, the deviation value between the feature point coordinates and the corresponding feature point coordinates in the character is obtained, and the deviation angle is generated based on the deviation value.

[0086] It should be further explained that the meaning of "corresponding" does not refer to a complete correspondence, that is, a correspondence within the range of the difference value P is sufficient; for example, if the feature point coordinates of the word to be identified are (a, b), then the feature point coordinates corresponding to these feature point coordinates in the character are the feature point coordinates in the range of (a+P, b+P) to (aP, bP).

[0087] Let the coordinates of the feature point of the word to be recognized be (q, w), and the coordinates of the corresponding feature point of the character be (e, r). Based on the tangent value, the deviation angle is obtained;

[0088] After obtaining the deviation angle, the word to be recognized will be angled according to the deviation angle.

[0089] Obtain the feature point coordinates of the word to be identified, establish a two-dimensional coordinate system with a certain feature point coordinate as the origin, and adjust the angle of all feature point coordinates except the origin relative to the origin based on the tangent value.

[0090] If the vocabulary to be identified contains feature point coordinates (f, g) and (h, j), and if the feature point coordinates (f, g) are taken as the origin, then the feature point coordinates (h, j) will become (hf, jg). Based on the tangent value, the feature point coordinates (hf, jg) are adjusted to (b, n), making... ;

[0091] The adjusted words to be recognized are sent to the semantic recognition module for further processing.

[0092] This module increases the tolerance for errors in the words to be identified by setting the phase difference value, making the subsequent angle adjustment of the words to be identified more representative, so as to facilitate word matching of the words to be identified in the future; at the same time, by obtaining the deviation angle, the angle of the words to be identified is adjusted, which improves the flexibility of direction correction and avoids the high workload of training a large number of distorted characters.

[0093] The semantic recognition module is used to perform semantic recognition on the words to be recognized. The specific process is as follows:

[0094] After receiving the word to be identified, a matching request is submitted to the semantic unit in the database, that is, the word to be identified is matched with the words in the semantic unit. If the match is successful, a language query is sent to the user to ask the user for the target language. If the language of the word to be identified is A, and the word to be identified needs to be translated into a synonym of language B, then language B is the target language.

[0095] After the user submits the target language, a query request is submitted to the semantic unit to query the semantics of the word to be identified in the target language environment, and the semantics are fed back to the user.

[0096] At the same time, a result query request is submitted to the user to ask whether the semantics of the feedback to the user are correct. If the user selects the correct answer, no operation is performed. If the user selects the wrong answer, the word to be recognized is sent to the error recognition area in the recognition record unit for storage.

[0097] If the match fails, the word to be identified is sent to the unrecognizable area in the recognition record unit, and a secondary warning signal is generated and sent to the warning module for appropriate processing.

[0098] The early warning module is used to process early warning signals, and the specific process is as follows:

[0099] When a Level 1 warning signal is received, a warning message is sent to the user, informing the user that the character cannot be recognized and to request that the character be selected again.

[0100] When a Level 2 warning signal is received, a warning message is sent to the user, informing the user that the word cannot be recognized and semantic acquisition has failed. The user is asked to check whether the selected area is complete and correct.

[0101] A semantic recognition system for image text includes the following steps:

[0102] Step S1: Obtain the area selected by the user by clicking or dragging on the screen, obtain the value to be valued within the area, and perform binarization processing on the value to be valued in sequence. Extract the feature value of the image after binarization processing, and match it with the feature value of the character in the character library. If the matching degree is greater than or equal to the threshold, the matching is considered successful, and the image is marked as the word to be recognized.

[0103] Step S2: Set the phase difference value, obtain the character feature values ​​within the phase difference value range in the character library, randomly select the feature point coordinates of two words to be recognized, obtain the deviation value between the feature point coordinates and the corresponding feature point coordinates in the characters, generate the deviation angle based on the deviation value, and adjust the angle of the words to be recognized according to the deviation value angle.

[0104] Step S3: Match the word to be identified after angle adjustment with the words in the semantic unit. If the match is successful, obtain the semantics of the word to be identified and send the semantics to the user. If the match fails, report the failure to obtain the semantics to the user.

[0105] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A semantic recognition system for image text, comprising a control module, characterized in that, The control module is connected to a database, a data acquisition module, a direction correction module, a semantic recognition module, and an early warning module; The database is used to store the feature values ​​of words, the semantics of words, user feedback records of recognition errors and unrecognizable records; The data acquisition module is used to acquire the words to be identified and generate corresponding warning signals based on the acquired words to be identified. The orientation correction module is used to correct the orientation of the words to be recognized. The semantic recognition module is used to perform semantic recognition on the words to be recognized and generate corresponding warning signals during the recognition process; The early warning module is used to process early warning signals; The database includes a character library, semantic units, and recognition record units; The character library is used to store the feature values ​​of words, and the feature values ​​are composed of the coordinates of multiple feature points; The semantic unit is used to store the semantics of words; The identification and recording unit includes an error identification area and an unidentifiable area; The error recognition area is used to store records of recognition errors reported by users; the unrecognizable area is used to store records of unrecognizable information from the semantic recognition module. The data acquisition module is equipped with both click-based and drag-and-drop acquisition methods. The click-based acquisition means that when a user clicks on the screen, the words within the coordinate range of the user's click are automatically acquired. The drag-and-drop acquisition means that when a user drags on the screen, the words within the drag range are automatically acquired. The process of obtaining words by clicking is as follows: Get the user's click coordinates on the screen; Set the extension values ​​x and y; The selected area is generated by extending horizontally by x units and vertically by y units from the center of the click coordinates. Set the threshold z; Get the pixel values ​​in the selected area. When the percentage of pixel values ​​exceeds the threshold z, mark the pixel value as to be valued. The pixel value proportion is the ratio of the number of pixels in the pixel value range to the total number of pixels; The image to be evaluated is binarized sequentially, and then the feature values ​​are extracted from the image and matched with the feature values ​​in the character library. A threshold T is set. When the degree of matching between the feature values ​​in the image and the feature values ​​in the character library is greater than or equal to the threshold T, the image to be valued is marked as a word to be recognized. When the degree of matching between the feature values ​​in the image and the feature values ​​in the character library is less than the threshold T, the image is submitted to the unrecognizable area in the database for storage, and a first-level warning signal is generated. At the same time, set the supplementary value u; Based on the selected area, expand outward by u units to generate a supplementary area. Check if there are any pixels with values ​​to be estimated in the supplementary area. If not, no operation is required. If they are, add the pixels with values ​​to be estimated from the supplementary area to the word to be recognized. After the supplementation is completed, expand outward by u units again to generate a supplementary area. Repeat the above supplementation operation until there are no pixels with values ​​to be estimated in the supplementary area. The process of dragging and dropping to retrieve words is as follows: Obtain the area selected by the user on the screen, obtain the value to be valued, and perform binarization on the value to be valued in turn. The original pixel values ​​are extracted to form the feature values ​​of the image composed of the pixels to be valued. These feature values ​​are then matched with the feature values ​​in the character library. If the matching degree between the feature values ​​in the image and the feature values ​​in the character library is greater than or equal to a threshold T, the image composed of the image to be valued is marked as a word to be recognized. If the matching degree is less than the threshold T, the image is submitted to the unrecognizable area in the database for storage, and a first-level warning signal is generated. The process by which the orientation correction module corrects the orientation of the word to be recognized is as follows: Obtain the feature values ​​of the words to be identified; Set the difference value based on the feature value; submit a query request to the character library in the database to obtain the character feature value within the difference value range; The feature point coordinates of two words to be identified are randomly selected, and the deviation value between the feature point coordinates and the corresponding feature point coordinates in the character is obtained. The deviation angle is generated based on the deviation value. After obtaining the deviation angle, the word to be identified will be adjusted according to the deviation angle. If there are N pixels with values ​​between [135, 140] and M pixels in total, then the percentage of pixels with values ​​between [135, 140] is [percentage missing]. ,like If the value exceeds the threshold z, then the pixel value [135, 140] is to be estimated; The warning signal is submitted to the warning module, which then takes appropriate action.

2. The semantic recognition system for image text according to claim 1, characterized in that, The semantic recognition module performs semantic recognition on the words to be recognized, including the following process: After receiving the word to be identified, a matching request is submitted to the semantic unit in the database, that is, the word to be identified is matched with the words in the semantic unit. If the match is successful, a language query is sent to the user to ask the user for the target language. After the user submits the target language, a query request is submitted to the semantic unit to query the semantics of the word to be identified in the target language environment, and the semantics are fed back to the user. If the match fails, a level 2 warning signal will be generated.

3. A recognition method for a semantic recognition system for image text according to any one of claims 1 to 2, characterized in that, Includes the following steps: Step S1: Obtain the area selected by the user by clicking or dragging on the screen, obtain the value to be valued within the area, and perform binarization processing on the value to be valued in sequence. Extract the feature value of the image after binarization processing, and match it with the feature value of the character in the character library. If the matching degree is greater than or equal to the threshold, the matching is considered successful, and the image is marked as the word to be recognized. Step S2: Set the phase difference value, obtain the character feature values ​​within the phase difference value range in the character library, randomly select the feature point coordinates of two words to be recognized, obtain the deviation value between the feature point coordinates and the corresponding feature point coordinates in the characters, generate the deviation angle based on the deviation value, and adjust the angle of the words to be recognized according to the deviation value angle. Step S3: Match the word to be identified after angle adjustment with the words in the semantic unit. If the match is successful, obtain the semantics of the word to be identified and send the semantics to the user. If the match fails, report the failure to obtain the semantics to the user.

Images