Image processing method, electronic circuit, visual impairment assistance device and medium

Abstract

Provided is an image processing method, comprising: performing text line detection on an input image to obtain a text line image comprising a curved text line; determining a plurality of reference points for the curved text line in the text line image; determining a text line curve for the curved text line based on the plurality of reference points; and adjusting the curved text line using an adjustment parameter determined based on the text line curve to obtain a recognized text line corresponding to the curved text line, wherein the recognized text line comprises a plurality of characters displayed horizontally. The method provided by the embodiments of the present disclosure can facilitate and accurately perform curve fitting on a curved text line. By performing segmentation processing on the curved text line, the curved text line comprising a plurality of characters can be adjusted into a text line displayed horizontally, which is easier for character recognition.

Description

Technical Field

[0001] This disclosure relates to the field of image processing, and in particular to an image processing method, electronic circuit, assistive device for the visually impaired, electronic device, storage medium, and program product. Background Technology

[0002] Text recognition can be achieved by processing images, and this text recognition function has wide applications in various fields.

[0003] The methods described in this section are not necessarily methods that had been previously conceived or adopted. Unless otherwise specified, no method described in this section should be assumed to be prior art simply because it is included in this section. Similarly, unless otherwise specified, the issues mentioned in this section should not be considered to be accepted in any prior art. Summary of the Invention

[0004] According to one aspect of this disclosure, an image processing method is provided, comprising: performing text line detection on an input image to obtain a text line image including the curved text lines; determining a plurality of reference points in the text line image for the curved text lines; determining a text line curve for the curved text lines based on the plurality of reference points; and adjusting the curved text lines using adjustment parameters determined based on the text line curves to obtain a recognized text line corresponding to the curved text lines, wherein the recognized text line includes a plurality of horizontally displayed characters.

[0005] According to another aspect of this disclosure, an electronic circuit is provided, comprising: a circuit configured to perform the steps of the above-described method.

[0006] According to another aspect of this disclosure, a visually impaired assistive device is also provided, comprising: a camera configured to acquire an image, wherein the image includes a curved text line; a curved text line correction circuit implemented by the electronic circuitry described above; a circuit configured to perform text detection and / or recognition on the recognized text line obtained by the curved text line correction circuit to obtain text data; a circuit configured to convert the text data into sound data; and a circuit configured to output the sound data.

[0007] According to another aspect of this disclosure, an electronic device is also provided, comprising: a processor; and a memory storing a program, the program including instructions that, when executed by the processor, cause the processor to perform the methods described above.

[0008] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is also provided, the program comprising instructions that, when executed by a processor of an electronic device, cause the electronic device to perform the method described above.

[0009] According to another aspect of this disclosure, a computer program product is also provided, comprising a computer program, wherein the computer program implements the above-described method when executed by a processor. Attached Figure Description

[0010] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0011] Figure 1 A schematic diagram of an exemplary system in which the various methods and apparatus described herein may be implemented according to embodiments of the present disclosure is shown;

[0012] Figure 2 An exemplary flowchart of an image processing method according to an embodiment of the present disclosure is shown;

[0013] Figure 3A An example of a text line image including curved text lines is shown;

[0014] Figure 3B An example of multiple character detection boxes in a text line image obtained through character target detection is shown;

[0015] Figure 3C The image shows an example of a text line region in a text line image obtained through image segmentation;

[0016] Figure 3D An example is shown where a reference point is determined based on the height of the text line region and a predetermined step size;

[0017] Figure 3E An example of a text line curve obtained using B-spline interpolation is shown;

[0018] Figure 4 An exemplary flowchart of a method for adjusting curved lines of text according to embodiments of the present disclosure is shown;

[0019] Figure 5 An exemplary flowchart is shown for a method of determining multiple text sub-regions in a curved text line according to embodiments of the present disclosure;

[0020] Figure 6A An example is shown of the slope at a location corresponding to at least one point on the curve of a text line;

[0021] Figure 6BThe image shows an example of multiple text sub-regions obtained by dividing the text line curve based on the slope at the position corresponding to each reference point;

[0022] Figure 7 An exemplary flowchart of a method for adjusting curved lines of text according to an embodiment of the present disclosure is shown;

[0023] Figure 8 It shows that according to Figure 7 An example of a recognized text line obtained by concatenating multiple adjusted text sub-regions obtained by the method described in the text;

[0024] Figure 9 Another exemplary flowchart is shown for a method of adjusting curved lines of text according to embodiments of the present disclosure;

[0025] Figures 10A-10C It shows that according to Figure 9 The method described in the document identifies examples of text lines;

[0026] Figure 11 An exemplary flowchart of a text recognition process according to an embodiment of the present disclosure is shown;

[0027] Figure 12 An exemplary block diagram of an image processing apparatus according to embodiments of the present disclosure is shown; and

[0028] Figure 13 This is a block diagram illustrating an example of an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0029] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0030] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. Furthermore, the term "and / or" as used in this disclosure covers any one of the listed items and all possible combinations thereof.

[0031] In the process of recognizing text information contained in an image, if the surface containing the text information is curved / deformed, then the text information to be recognized in the image will have a curved shape. Recognizing curved text information is a challenge. Since the text display in curved text lines is irregular, directly training a text recognition model to recognize text data in curved text lines will make the text recognition model quite complex, and the accuracy of text recognition will be lower than that for horizontally displayed text lines.

[0032] The principles of this disclosure are described below using an example of characters in a text line arranged horizontally. "Horizontal direction" refers to the direction consistent with the character arrangement direction of the text line. "Vertical direction" refers to the direction perpendicular to the character arrangement direction of the text line. "Curved text line" refers to a text line where the lines connecting the characters are not on a single horizontal line, for example, the distance between each character and the same horizontal line exceeds a predetermined distance threshold. Using the method provided by this disclosure, a curved text line can be straightened to obtain multiple characters displayed horizontally.

[0033] When it is understood, without departing from the principles of this disclosure, the "horizontal direction" and "vertical direction" may be interchanged to correct the curvature of vertically arranged text columns.

[0034] In horizontally arranged text, "horizontal display" means that all characters are basically on the same horizontal line, that is, the distance between each horizontally displayed character and the same horizontal line does not exceed a predetermined distance threshold. In vertically arranged text, "horizontal display" means that all characters are basically on the same vertical line, that is, the distance between each horizontally displayed character and the same vertical line does not exceed a predetermined distance threshold.

[0035] To accurately and efficiently process curved text in text lines of images, this disclosure provides a novel image processing method. The principles of this disclosure will be described below with reference to the accompanying drawings.

[0036] Figure 1 A schematic diagram of an exemplary system 100 in which the various methods and apparatus described herein can be implemented according to embodiments of this disclosure is shown. Reference Figure 1 The system 100 includes one or more terminal devices 101, a server 120, and one or more communication networks 110 coupling the one or more terminal devices 101 to the server 120. The terminal devices 101 can be configured to execute one or more applications.

[0037] In embodiments of this disclosure, server 120 may run one or more services or software applications that enable the execution of the image processing methods according to this disclosure. In some embodiments, terminal device 101 may also be used to run one or more services or software applications that enable the image processing methods according to this disclosure. In some implementations, terminal device 101 may be implemented as a visually impaired assistive device.

[0038] In some embodiments, server 120 may also provide other services or software applications that may include both non-virtual and virtual environments. In some embodiments, these services may be provided as web-based services or cloud services, such as to users of terminal device 101 under a Software as a Service (SaaS) model.

[0039] exist Figure 1 In the configuration shown, server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or combinations thereof that can be executed by one or more processors. A user operating terminal device 101 can sequentially interact with server 120 using one or more terminal applications to utilize the services provided by these components. It should be understood that various different system configurations are possible and may differ from system 100. Therefore, Figure 1 This is an example of a system used to implement the various methods described herein, and is not intended to be limiting.

[0040] Terminal device 101 can provide an interface that allows users of the terminal device to interact with it. The terminal device can also output information to the user through this interface. Although Figure 1 Only one terminal device is depicted, but those skilled in the art will understand that this disclosure can support any number of terminal devices.

[0041] Terminal device 101 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptops), workstation computers, wearable devices, gaming systems, thin clients, various messaging devices, sensors, or other sensing devices. These computer devices can run various types and versions of software applications and operating systems, such as Microsoft Windows, Apple iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as Google Chrome OS); or include various mobile operating systems, such as Microsoft Windows Mobile OS, iOS, Windows Phone, and Android. Portable handheld devices may include cellular phones, smartphones, tablets, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays and other devices. Gaming systems may include various handheld gaming devices, internet-enabled gaming devices, etc. Terminal devices are capable of executing various applications, such as various internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and can use various communication protocols.

[0042] Network 110 can be any type of network well known to those skilled in the art, and can use any of a variety of available protocols (including but not limited to TCP / IP, SNA, IPX, etc.) to support data communication. By way of example only, one or more networks 110 can be a local area network (LAN), an Ethernet-based network, a token ring network, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0043] Server 120 may include one or more general-purpose computers, special-purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running a virtual operating system, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for servers). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0044] The computing unit in server 120 can run one or more operating systems, including any of the aforementioned operating systems and any commercially available server operating system. Server 120 can also run any of a variety of additional server applications and / or middleware applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0045] In some implementations, server 120 may include one or more applications to analyze and merge data feeds and / or event updates received from a user of terminal device 101. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of terminal device 101.

[0046] In some implementations, server 120 can be a server for a distributed system or a server integrated with blockchain. Server 120 can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0047] System 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. Databases 130 may reside in various locations. For example, a data repository used by server 120 may be local to server 120, or it may be located away from server 120 and may communicate with server 120 via a network-based or dedicated connection. Databases 130 may be of different types. In some embodiments, databases 130 used by server 120 may be relational databases. One or more of these databases may store, update, and retrieve data from and from the databases in response to commands.

[0048] In some embodiments, one or more of the databases 130 may also be used by an application to store application data. The databases used by the application may be of different types, such as key-value stores, object stores, or regular stores supported by a file system.

[0049] Figure 1 The system 100 can be configured and operated in various ways to enable the application of the various methods and apparatus described in this disclosure.

[0050] Figure 2An exemplary flowchart of an image processing method 200 according to an embodiment of the present disclosure is shown. Figure 2 The method shown can be derived from Figure 1 This can be executed using the terminal device 101 or server 120 shown. It can be utilized... Figure 2 The method 200 shown in the figure processes an image containing curved text lines to correct the characters in the curved text lines in the image to be displayed horizontally for use in a further character recognition process.

[0051] In step 202, text line detection can be performed on the input image to obtain a text line image that includes curved text lines.

[0052] In some embodiments, the input image may be acquired by an image acquisition unit (such as a camera) installed on the terminal device. In other embodiments, a pre-acquired image may be read from memory as the input image. The input image may include one or more lines of text, and these lines may be curved.

[0053] The input image can be processed using a pre-existing image processing model for detecting text lines in an image to obtain a text line image that includes a single curved text line. In some embodiments, the input image can be processed using a pre-trained neural network-based text line detection model to obtain a sub-image of the input image that includes a text line as the text line image.

[0054] Figure 3A An example of a text line image including curved text lines is shown. Figure 3A As shown, by performing text line detection on the input image, a text line image containing only one text line can be detected from the input image. Figure 3A The example shown includes a curved text line, meaning that multiple characters in the text line are not displayed horizontally.

[0055] In step S204, multiple reference points in the text line image can be determined for bending the text line.

[0056] The positions of the aforementioned reference points can be the same as the position of at least one character included in the curved text line, or they can be simulated positions of characters included in the curved text line, and do not necessarily correspond to the actual positions of the characters. The characters in the text line can be any form of unit within the text line, such as English words, English letters, Chinese characters, punctuation marks, etc. No restrictions are placed on the text format within the text line.

[0057] In some embodiments, character target detection can be performed on the text line image to obtain multiple reference points for the curved text line. Each reference point indicates the position of a character included in the curved text line.

[0058] In some implementations, a pre-trained neural network-based target detection model for recognizing characters can be used to process the text line image to obtain character detection boxes for each character included in the curved text line.

[0059] Figure 3B An example of multiple character detection boxes in a text line image obtained through character target detection is shown. Each character detection box 301 may include at least one character. Figure 3B As shown, most character detection boxes contain only one character. However, due to the inherent errors in object detection models, some character detection boxes may also contain multiple characters. According to the principles of this disclosure, the number of characters contained in the character detection boxes is not required; it is sufficient that the character detection results can substantially reflect the trend of characters within the curved text line.

[0060] It can be based on, for example Figure 3B The positions of the multiple character detection boxes shown determine the positions of multiple reference points used to bend the text line. For example, the center point of at least one of the detected character detection boxes can be determined as a reference point; that is, the position of the center point of at least one character detection box can be determined as the position of the corresponding reference point.

[0061] In other embodiments, the text line image can be segmented to obtain the text line regions corresponding to the curved text lines. For example, a pre-trained neural network-based image segmentation model for text line segmentation can be used to segment the text line image pixel by pixel to obtain the segmentation results of whether each pixel in the text line image belongs to the text line or not, thereby determining the text line regions corresponding to the curved text lines.

[0062] Figure 3C The image shows an example of text line regions in a text line image obtained through image segmentation. Black areas represent image regions in the text line image that are determined not to contain characters, while white areas represent text line regions in the text line image that are determined to contain characters.

[0063] Utilize Figure 3C The height of the text line region and the predetermined step size shown in the figure can be used to determine multiple reference points in the text line region.

[0064] Figure 3DAn example is shown where a reference point is determined based on the height of a text line region and a predetermined step size. The predetermined step size can indicate a predetermined character width. It is understood that those skilled in the art can arbitrarily set the value of the predetermined step size according to the actual situation, and this predetermined step size may differ from the actual width of the characters in the curved text line. Figure 3D The predetermined step size shown is smaller than the actual width of the characters in the curved text line. In other embodiments, the predetermined step size may also be larger than the actual width of the characters in the curved text line.

[0065] like Figure 3D As shown, the text line region can be segmented based on a predetermined step size to obtain multiple simulated character frames 302 representing simulated character positions. The position of the center point of each simulated character frame can be determined as the position of a reference point. In some embodiments, the x-coordinate of the reference point can be the average of the x-coordinates of the left and right boundaries of the corresponding simulated character frame, and the y-coordinate of the reference point can be the average of the y-coordinates of the points within the simulated character frame.

[0066] In step S206, a text line curve for bending the text line can be determined based on multiple reference points identified in step S304. The text line curve can indicate the specific shape of the character arrangement in the bent text line. By performing mathematical analysis on the text line curve representing the shape of the bent text line, adjustment parameters for straightening the bent text line can be obtained.

[0067] As mentioned earlier, step S204 can be used to obtain the positions of multiple reference points indicating the location of the curved text line. By performing curve fitting on the above multiple reference points, a text line curve can be obtained to simulate the curve where the characters in the curved text line are located.

[0068] In some embodiments, B-spline interpolation can be used to fit curves to the positions of multiple reference points to obtain a mathematical expression for the text line curve. In other embodiments, arbitrary curve fitting methods, such as polynomial fitting, can also be used to fit curves to the positions of multiple reference points.

[0069] Figure 3E An example of a text line curve obtained using B-spline interpolation is shown. From Figure 3E As can be seen in the example shown, the text line curve 303 obtained using the B-spline interpolation method can accurately fit the curve containing the characters in the curved text line. It is understood that, without departing from the principles of this disclosure, those skilled in the art can employ any mathematical method capable of fitting the curved text line well.

[0070] In step S208, the curved text line can be adjusted using adjustment parameters determined based on the text line curve to obtain a corresponding recognition text line. The recognition text line includes multiple characters displayed horizontally.

[0071] Since the characters in the recognized text line are mostly displayed on the same horizontal line, a trained character recognition model can be used to process the recognized text line to obtain the text data within it. It's understandable that, because the recognized text line is obtained by straightening a curved text line, the text data in the recognized text line is the same as the text data in the curved text line.

[0072] The image processing method provided by the embodiments of this disclosure can obtain a text line curve that accurately represents the curved text line based on the position of a reference point used for the curved text line. The curved text line can be straightened by adjusting parameters obtained using the text line curve. Since the text line curve obtained based on the reference point can accurately represent the position of the curved text line, the method provided by this disclosure can achieve better straightening results. In the subsequent text recognition process, the text recognition algorithm can directly perform text recognition on text lines where characters are displayed on a roughly horizontal line. For example, an end-to-end seq2seq deep learning model can be used to recognize text sequences.

[0073] Figure 4 An exemplary flowchart of a method 400 for adjusting curved text lines according to an embodiment of the present disclosure is shown.

[0074] In step S402, multiple text sub-regions for bending the text line can be determined.

[0075] To straighten curved text lines, they can be divided into multiple sub-regions, thus segmenting the curved text lines. For example, the display of each sub-region can be adjusted so that the characters in each sub-region are displayed on roughly the same horizontal line.

[0076] In some embodiments, each of the plurality of text sub-regions may include a single character. In other embodiments, each of the plurality of text sub-regions may include at least two characters. The number of characters included in each text sub-region may be the same or different. In still other embodiments, each of the plurality of text sub-regions may include a width of a single column of pixels. It is understood that the above description is merely an exemplary illustration of segmenting curved text lines and is not intended to limit the scope of this disclosure.

[0077] In step S404, for each of the multiple text sub-regions, the text sub-region can be adjusted based on the adjustment parameters for that text sub-region determined using the text line curve.

[0078] Understandably, in curved text lines, characters in different regions are displayed in different positions on the image, and the characters are not displayed on the same horizontal line. Based on the combination Figure 2 The method described obtains a text line curve, which can determine corresponding adjustment parameters for each text sub-region. These parameters are used to adjust at least one of the display direction and position of the characters in the text sub-region, so that the characters in each text sub-region are displayed horizontally, thereby achieving the effect of straightening the curved text line.

[0079] In some embodiments, the adjustment parameters for each text sub-region may include the arrangement direction of the characters in the text sub-region and the angle in the horizontal direction. This will be discussed in conjunction with... Figure 7 The specific methods and processes for adjusting text sub-regions are described, and will not be repeated here.

[0080] In step S406, the identified text line corresponding to the curved text line can be determined based on the adjusted text sub-region.

[0081] In some embodiments, the adjusted multiple text sub-regions can be scaled so that the adjusted multiple text sub-regions have the same height.

[0082] Because the height of the adjusted text sub-regions depends on the rotation angle during the adjustment process, the heights of multiple adjusted text sub-regions may be different.

[0083] To ensure that the adjusted text sub-regions can be joined into a single text line, the size of the adjusted text sub-regions can be scaled to ensure that they have the same height.

[0084] In some implementations, the dimensions of the adjusted text sub-regions can be scaled only in the height direction. In other implementations, the dimensions of the adjusted text sub-regions can be scaled proportionally in both the height and length directions, so that the adjusted text sub-regions have the same height. For example, the adjusted text sub-regions can be scaled based on a predetermined reference height, so that the scaled text sub-regions all have the reference height.

[0085] Multiple scaled text sub-regions can be concatenated horizontally to obtain a recognized text line, wherein the characters in the recognized text line are displayed horizontally.

[0086] Using the method provided in this disclosure, multiple characters in a curved text line can be segmented, and the curved text line can be straightened based on adjustment parameters determined by the text line curve. The image processing method provided in this disclosure can straighten curved text lines of arbitrary length and obtain recognized text lines with multiple characters displayed horizontally.

[0087] Figure 5 An exemplary flowchart is shown of a method 500 for determining multiple text sub-regions in a curved text line according to an embodiment of the present disclosure.

[0088] In step S502, the slope at the position corresponding to at least one point on the text line curve can be determined.

[0089] In step S504, the curved text line can be divided based on the slope at the position corresponding to at least one of the above points to obtain multiple text sub-regions, wherein adjacent text sub-regions correspond to different slopes.

[0090] As mentioned earlier, the text line curve obtained by fitting multiple reference points can simulate the movement and position of characters in a curved text line. By determining the slope at the position of at least one point in the text line curve, the movement of the character at that point can be obtained.

[0091] If the slopes of the text line curve at two positions corresponding to two adjacent reference points are similar, it indicates that the characters between the two adjacent reference points have similar trends. Characters with similar trends can be grouped into the same text sub-region based on their corresponding slopes.

[0092] Figure 6A An example of the slope at a position on a text line curve corresponding to at least one reference point is shown. Wherein, given the mathematical expression of the text line curve obtained based on the aforementioned method, a point on the text line curve with the same x-coordinate as each reference point can be determined as the position corresponding to that reference point. Figure 6A The arrows shown indicate the different slopes of the text line curve at different locations.

[0093] In some embodiments, the rate of change of the slope between adjacent reference points can be determined based on the slope at each point on the text line curve, and the regions between adjacent points whose rate of change of slope is less than a change threshold can be divided into the same text sub-region. In this case, it can be assumed that the characters in each text sub-region correspond to the same slope, that is, the trend of the characters in each text sub-region is basically the same.

[0094] Figure 6B The image shows an example of multiple text sub-regions divided based on the slope at at least one point on the text line curve. For example... Figure 6BAs shown, the arrangement direction of the characters in each text sub-region is at approximately the same angle as the horizontal direction. For example... Figure 6B As shown, the boundaries of each text sub-region are represented by angled quadrilateral regions. The left and right boundaries of each sub-region are perpendicular to the horizontal direction, and the angles between the top and bottom boundaries and the horizontal direction are the same as the angles between the character trend indicated by the slope of the corresponding text sub-region and the horizontal direction. Furthermore, the height of each text sub-region can be obtained based on the character heights within the curved text lines. For example, the height of the text sub-region can be determined based on the height of the character detection boxes obtained from object detection. Alternatively, the height of the text sub-region can be determined based on the height of the text lines in the text line segmentation result.

[0095] Figure 7 An exemplary flowchart is shown, illustrating a method for adjusting curved lines of text according to embodiments of the present disclosure. This can be utilized... Figure 7 Method 700 pairs shown Figure 6B Adjust the text sub-region shown in the image.

[0096] In step S702, adjustment parameters for each of the multiple text sub-regions of the curved text line can be determined. These adjustment parameters may include the angle between the arrangement direction of the characters within the text sub-region and the horizontal direction, determined based on the slope of the text line curve corresponding to that sub-region.

[0097] In step S704, the text sub-region can be adjusted based on the angle between the arrangement direction of the characters in the text sub-region determined in step S702 and the horizontal direction, so that the characters in the text sub-region are displayed horizontally.

[0098] The entire text line image can be rotated inversely based on the angle between the arrangement direction of the characters in the text sub-region determined in step S702 and the horizontal direction, so that the characters in the text sub-region are displayed horizontally, and the positions of the four vertices of the corresponding text sub-region in the rotated text line image are obtained based on the rotation angle. The minimum bounding rectangle of the four vertices of the rotated text sub-region can be obtained, and the minimum bounding rectangle can be cropped from the rotated text line image to obtain the adjusted text sub-region. The upper and lower boundaries of the minimum bounding rectangle are parallel to the horizontal direction, and the left and right boundaries are parallel to the vertical direction.

[0099] Figure 8 It shows that according to Figure 7 The example shown is a line of recognized text obtained by concatenating multiple adjusted text sub-regions obtained by the method described. The adjusted text sub-regions are concatenated horizontally to obtain the line of recognized text used for text recognition.

[0100] Figure 9 Another exemplary flowchart is shown for a method of adjusting curved lines of text according to an embodiment of the present disclosure.

[0101] In step S902, for each column of pixels in the text line image, adjustment parameters for that column of pixels are determined. These adjustment parameters include the offset between the ordinate of a point on the text line curve within that column of pixels and a reference position. In some embodiments, the reference position in the text line image can be predetermined. For example, the position of the horizontal center line of the text line image can be determined as the reference position. Another example is the position of the horizontal line containing any character in the text line image. Yet another example is the average ordinate of the character detection boxes obtained through character detection in the text line image, which can be determined as the reference position.

[0102] In step S904, the display of each column of pixels in the text line image can be adjusted using adjustment parameters. For example, the position of the point on the text line curve in the column of pixels can be adjusted vertically based on the offset between the ordinate of the point on the text line curve in that column of pixels and the reference position, so that the adjusted vertical position of the point on the text line curve in that column of pixels is consistent with the reference position.

[0103] In step S906, the identified text line can be determined based on the adjusted text line image. For example, the background of the adjusted text line image can be cropped based on the character height to obtain the identified text line.

[0104] Figures 10A-10C It shows that according to Figure 9 The method described in the document identifies examples of text lines.

[0105] like Figure 10A As shown, for column pixel 1001, it can be determined that the point on the text line curve within this text sub-region is located at a distance d below the baseline 1002. In this case, the pixel sequence at height d on the opposite side of the point on the text line curve (i.e., above the baseline 1002 for column pixel 1001) can be cropped. The remaining portion of the cropped text sub-region 1001 is moved upward by a distance d, and the cropped pixel sequence at height d is then filled in reverse below the point on the text line curve, thereby adjusting the position of the point on the text line curve within column pixel 1001 to be consistent with the baseline position.

[0106] Figure 10B It shows the use of combination Figure 10A The method described is the result obtained by adjusting the pixels in each column of the text line image separately. For example... Figure 10BAs shown, all characters in the curved text line are adjusted to be displayed horizontally. Figure 10C It shows the Figure 10B The results shown are the recognized text lines after being cropped according to character height.

[0107] Using the method for adjusting text line images provided in this disclosure, pixels in the text line image can be backfilled column by column based on the parameters of the text line curve and the reference position, so that the pixels corresponding to the characters in each column are basically displayed at the reference position. This method can conveniently straighten curved text lines of arbitrary length.

[0108] Figure 11 An exemplary flowchart of a text recognition process 1100 according to an embodiment of the present disclosure is shown.

[0109] In step 1102, text line detection can be performed on the acquired input image to obtain a text line image that includes a single text line.

[0110] In step S1104, the text lines in the text line image can be corrected for curvature to obtain recognized text lines, wherein the recognized text lines include multiple characters displayed horizontally. The aforementioned combination can be utilized. Figure 2-Figure 10C The process described here involves correcting the curvature of curved text lines in a text line image, which will not be elaborated upon further.

[0111] In step S1106, text recognition can be performed on the identified text line to obtain the text data included in the text line.

[0112] Text lines can be processed using a pre-trained neural network-based character recognition model. Since the characters in the text lines are displayed horizontally, the character recognition model does not need to directly recognize the content within the curved text lines, thus reducing the complexity of the model and improving the accuracy of character recognition.

[0113] By using the text recognition method provided in this disclosure, the text lines that are basically displayed on the same horizontal line can be obtained by first correcting the curved text lines. This can alleviate the recognition pressure of the text recognition model on texts with large curvature / long texts and improve the text recognition performance.

[0114] Figure 12 An exemplary block diagram of an image processing apparatus according to an embodiment of the present disclosure is shown.

[0115] like Figure 12As shown, the image processing apparatus 1200 may include a text line detection unit 1210, a reference point determination unit 1220, a curve determination unit 1230, and a recognized text determination unit 1240. The text line detection unit 1210 may be configured to perform text line detection on an input image to obtain a text line image including curved text lines. The reference point determination unit 1220 may be configured to determine multiple reference points in the text line image for the curved text lines. The curve determination unit 1230 may be configured to determine a text line curve for the curved text lines based on the multiple reference points. The recognized text determination unit 1240 may be configured to adjust the curved text lines using adjustment parameters determined based on the text line curve to obtain a recognized text line corresponding to the curved text lines, wherein the recognized text line includes multiple horizontally displayed characters.

[0116] The text line detection unit 1210, the reference point determination unit 1220, the curve determination unit 1230, and the text recognition determination unit 1240 can be used to achieve the aforementioned combination. Figure 2-Figure 10C The steps of the image processing method described herein will not be repeated here.

[0117] Using the image processing apparatus provided in the embodiments of this disclosure, a text line curve that accurately represents the curved text line can be obtained based on the position of a reference point used for the curved text line. The curved text line can be straightened by adjusting parameters obtained using the text line curve. Since the text line curve obtained based on the reference point accurately represents the position of the curved text line, a better straightening effect can be obtained using the method provided in this disclosure. In the subsequent text recognition process, the text recognition algorithm can directly perform text recognition on text lines where characters are displayed on a substantially horizontal line. For example, an end-to-end seq2seq deep learning model can be used to recognize text sequences.

[0118] Exemplary methods according to this disclosure have been described above with reference to the accompanying drawings. Exemplary embodiments utilizing electronic circuits and electronic devices of this disclosure will now be further described with reference to the accompanying drawings.

[0119] According to another aspect of this disclosure, an electronic circuit is provided, comprising: a circuit configured to perform the steps of the methods described in this disclosure.

[0120] According to another aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory storing a program, the program including instructions that, when executed by the processor, cause the processor to perform the methods described in this disclosure.

[0121] According to another aspect of this disclosure, a computer-readable storage medium is provided that stores a program, the program including instructions that, when executed by a processor of an electronic device, cause the electronic device to perform the methods described in this disclosure.

[0122] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program, wherein the computer program, when executed by a processor, implements the methods described in this disclosure.

[0123] Figure 13 This is a block diagram illustrating an example of an electronic device according to exemplary embodiments of the present disclosure. It should be noted that... Figure 13 The structure shown is merely an example; depending on the specific implementation, the electronic device disclosed herein may include only [specific components]. Figure 13 One or more of the components shown.

[0124] Electronic device 1300 may be, for example, a general-purpose computer (e.g., a laptop computer, tablet computer, etc.), a mobile phone, or a personal digital assistant. According to some embodiments, electronic device 1300 may be an assistive device for the visually impaired. Electronic device 1300 may include a camera and electronic circuitry for correcting curved text lines. The camera may be configured to acquire images including curved text lines, and the electronic circuitry may be configured to perform a combination... Figure 2-Figure 10C The image processing method described is used for text line correction.

[0125] According to some embodiments, the electronic device 1300 may be configured to include an eyeglass frame or be configured to be detachably mounted to an eyeglass frame (e.g., the frame of the eyeglass frame, a connector connecting two frames, temples, or any other part) so as to capture an image that approximately includes the user's field of vision.

[0126] According to some embodiments, the electronic device 1300 can also be mounted on or integrated with other wearable devices. The wearable device may be, for example, a head-mounted device (e.g., a helmet or hat), or a device that can be worn on the ears. According to some embodiments, the electronic device may be implemented as an accessory that can be attached to a wearable device, for example, as an accessory that can be attached to a helmet or hat.

[0127] According to some embodiments, the electronic device 1300 may also have other forms. For example, the electronic device 1300 may be a mobile phone, a general-purpose computing device (e.g., a laptop computer, a tablet computer, etc.), a personal digital assistant, etc. The electronic device 1300 may also have a stand, so that it can be placed on a desktop.

[0128] Electronic device 1300 may include camera 1304 for acquiring images. Camera 1304 may include, but is not limited to, a webcam or camera. Electronic device 1300 may also include curved text line correction circuit (electronic circuit) 1400, which includes steps configured to perform the image method for text line correction as described above (e.g., in conjunction with...). Figure 2-Figure 10C The circuit describes the method steps.

[0129] The electronic device 1300 may further include a text recognition circuit 1305, which is configured to perform text detection and / or recognition (e.g., OCR processing) on ​​corrected text lines contained in an image output by a curved text line correction circuit, thereby obtaining text data. The text recognition circuit 1305 may be implemented, for example, using a dedicated chip. The electronic device 1300 may further include a sound conversion circuit 1306, which is configured to convert the text data into sound data. The sound conversion circuit 1306 may be implemented, for example, using a dedicated chip. The electronic device 1300 may further include a sound output circuit 1307, which is configured to output the sound data. The sound output circuit 1307 may include, but is not limited to, headphones, speakers, or vibrators, and their corresponding driving circuits.

[0130] According to some embodiments, the electronic device 1300 may further include an image processing circuit 1308, which may include circuitry configured to perform various image processing operations on the image. The image processing circuit 1308 may include, for example, one or more of the following: circuitry configured to denoise the image; circuitry configured to deblur the image; circuitry configured to perform geometric correction on the image; circuitry configured to extract features from the image; circuitry configured to perform object detection and / or recognition on objects in the image; circuitry configured to perform text detection on text contained in the image; circuitry configured to extract text lines from the image; circuitry configured to extract text coordinates from the image; circuitry configured to extract object boxes from the image; circuitry configured to extract text boxes from the image; circuitry configured to perform layout analysis (e.g., paragraph segmentation) based on the image; and so on.

[0131] According to some embodiments, the electronic device 1300 may also include a text processing circuit 1309, which may be configured to perform various processing based on extracted text-related information (such as text data, text boxes, paragraph coordinates, text line coordinates, text coordinates, etc.) to obtain processing results such as paragraph sorting, text semantic analysis, and layout analysis results.

[0132] One or more of the various circuits described above (e.g., character recognition circuit 1305, voice conversion circuit 1306, voice output circuit 1307, image processing circuit 1308, character processing circuit 1309, curved text line correction circuit (electronic circuit) 1400) may be implemented using custom hardware, and / or using hardware, software, firmware, middleware, microcode, hardware description language, or any combination thereof. For example, one or more of the various circuits described above may be implemented by programming hardware (e.g., programmable logic circuits including field-programmable gate arrays (FPGAs) and / or programmable logic arrays (PLAs)) using assembly language or hardware programming languages ​​(such as Verilog, VHDL, C++) according to the logic and algorithms of this disclosure.

[0133] According to some embodiments, the electronic device 1300 may also include a communication circuit 1310, which may be any type of device or system that enables communication with external devices and / or with a network, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication devices and / or chipsets, such as Bluetooth devices, 1302.11 devices, WiFi devices, WiMax devices, cellular communication devices and / or the like.

[0134] According to some embodiments, the electronic device 1300 may also include an input device 1311, which may be any type of device capable of inputting information to the electronic device 1300, and may include, but is not limited to, various sensors, mice, keyboards, touch screens, buttons, joysticks, microphones and / or remote controls, etc.

[0135] According to some embodiments, the electronic device 1300 may also include an output device 1312, which may be any type of device capable of presenting information, and may include, but is not limited to, a display, a visual output terminal, a vibrator, and / or a printer. Although the electronic device 1300 is used in some embodiments for assistive devices for the visually impaired, the vision-based output device can facilitate the user's family members or maintenance personnel to obtain output information from the electronic device 1300.

[0136] According to some embodiments, the electronic device 1300 may further include a processor 1301. The processor 1301 may be any type of processor and may include, but is not limited to, one or more general-purpose processors and / or one or more dedicated processors (e.g., special processing chips). The processor 1301 may be, for example, but not limited to, a central processing unit (CPU) or a microprocessor (MPU), etc. The electronic device 1300 may also include a working memory 1302, which may store programs (including instructions) and / or data (e.g., images, text, sound, and other intermediate data) useful to the operation of the processor 1301, and may include, but is not limited to, random access memory and / or read-only memory devices. Electronic device 1300 may also include storage device 1303, which may include any non-transitory storage device. A non-transitory storage device can be any storage device that is non-transitory and capable of storing data, and may include, but is not limited to, disk drives, optical storage devices, solid-state storage, floppy disks, flexible disks, hard disks, magnetic tapes or any other magnetic media, optical discs or any other optical media, ROM (read-only memory), RAM (random access memory), cache memory and / or any other memory chip or cartridge, and / or any other medium from which a computer can read data, instructions, and / or code. Working memory 1302 and storage device 1303 may be collectively referred to as "memory" and in some cases may be used interchangeably.

[0137] According to some embodiments, the processor 1301 can control and schedule at least one of the following: camera 1304, text recognition circuit 1305, audio conversion circuit 1306, audio output circuit 1307, image processing circuit 1308, text processing circuit 1309, communication circuit 1310, curved text line correction circuit (electronic circuit) 1400, and other various devices and circuits included in the electronic device 1300. According to some embodiments, Figure 13 At least some of the components described herein may be interconnected and / or communicate with each other via bus 1313.

[0138] Software elements (programs) may be located in the working memory 1302, including but not limited to operating system 1302a, one or more application programs 1302b, drivers and / or other data and code.

[0139] According to some implementations, the instructions for performing the aforementioned control and scheduling may be included in the operating system 1302a or one or more application programs 1302b.

[0140] According to some implementations, the steps of the method described in this disclosure are performed (e.g., in combination with...). Figure 2-Figure 10CThe instructions for the described method steps may be included in one or more application programs 1302b, and the various modules of the described electronic device 1300 may be implemented by the processor 1301 reading and executing the instructions of one or more application programs 1302b. In other words, the electronic device 1300 may include a processor 1301 and a memory storing a program (e.g., working memory 1302 and / or storage device 1303), the program including instructions that, when executed by the processor 1301, cause the processor 1301 to perform the methods described in the various embodiments of this disclosure.

[0141] According to some implementations, some or all of the operations performed by at least one of the character recognition circuit 1305, voice conversion circuit 1306, image processing circuit 1308, character processing circuit 1309, and curved text line correction circuit (electronic circuit) 1400 can be implemented by the processor 1301 reading and executing instructions from one or more application programs 1302.

[0142] Executable code or source code of the instructions of a software element (program) may be stored in a non-transitory computer-readable storage medium (e.g., the storage device 1303) and may be stored in working memory 1302 during execution (possibly for compilation and / or installation). Therefore, this disclosure provides a computer-readable storage medium storing a program comprising instructions that, when executed by a processor of an electronic device (e.g., a visually impaired assistive device), cause the electronic device to perform the methods described in various embodiments of this disclosure. According to another embodiment, the executable code or source code of the instructions of a software element (program) may also be downloaded from a remote location.

[0143] It should also be understood that various modifications can be made depending on specific requirements. For example, custom hardware can be used, and / or the individual circuits, units, modules, or elements can be implemented using hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. For example, some or all of the circuits, units, modules, or elements included in the disclosed methods and apparatus can be implemented by programming the hardware (e.g., programmable logic circuits including field-programmable gate arrays (FPGAs) and / or programmable logic arrays (PLAs)) using the logic and algorithms according to this disclosure in assembly language or hardware programming languages ​​(such as Verilog, VHDL, C++).

[0144] According to some implementations, the processor 1301 in the electronic device 1300 can be distributed across a network. For example, one processor can perform some processing, while another processor located far away can perform other processing simultaneously. Other modules of the electronic device 1300 can also be distributed similarly. In this way, the electronic device 1300 can be interpreted as a distributed computing system that performs processing in multiple locations.

[0145] The following describes some exemplary aspects of this disclosure.

[0146] Aspect 1. An image processing method, comprising:

[0147] Perform text line detection on the input image to obtain a text line image including the curved text lines;

[0148] Determine multiple reference points in the text line image for the curved text line;

[0149] The text line curve for the curved text line is determined based on the plurality of reference points;

[0150] The curved text line is adjusted using adjustment parameters determined based on the text line curve to obtain a recognition text line corresponding to the curved text line, wherein the recognition text line includes multiple characters displayed horizontally.

[0151] Aspect 2. The image processing method as described in Aspect 1, wherein determining a plurality of reference points in the text line image for the curved text line includes:

[0152] Character target detection is performed on the text line image to obtain multiple reference points for the curved text line, wherein each reference point indicates the position of each character in the curved text line.

[0153] Aspect 3. The image processing method as described in Aspect 1, wherein determining a plurality of reference points in the text line image for the curved text line includes:

[0154] The text line image is segmented to obtain the text line region in the text line image corresponding to the curved text line;

[0155] Based on the height of the text line region and a predetermined step size, multiple reference points in the text line region are determined for the curved text line.

[0156] Aspect 4. The image processing method according to any one of Aspects 1-3, wherein determining the text line curve for the curved text line based on the plurality of reference points includes:

[0157] The positions of the multiple reference points are fitted using a B-spline interpolation method to obtain the text line curve.

[0158] Aspect 5. The image processing method as described in Aspect 1, wherein adjusting the curved text line using adjustment parameters determined based on the text line curve to obtain a recognized text line corresponding to the curved text line includes:

[0159] Identify multiple text sub-regions for the curved text line;

[0160] For each of the plurality of text sub-regions, the text sub-region is adjusted based on the adjustment parameters for that text sub-region determined using the text line curve;

[0161] The identified text line corresponding to the curved text line is determined based on the adjusted text sub-region.

[0162] Aspect 6. The image processing method as described in Aspect 5, wherein determining the plurality of text sub-regions in the curved text line comprises:

[0163] Determine the slope at the location of at least one point on the text line curve;

[0164] The curved text line is divided based on the slope at the location corresponding to the at least one point to obtain multiple text sub-regions, wherein adjacent text sub-regions correspond to different slopes.

[0165] Aspect 7. The image processing method as described in Aspect 6, wherein the adjustment parameters of each of the plurality of text sub-regions include the angle between the arrangement direction of the characters included in the text sub-region and the horizontal direction, determined based on the slope corresponding to the text sub-region.

[0166] Aspect 8. The image processing method as described in Aspect 7, wherein adjusting the text sub-region based on adjustment parameters for the text sub-region determined using the text line curve includes:

[0167] The text sub-region is adjusted based on the aforementioned angle so that the characters in the adjusted text sub-region are displayed horizontally.

[0168] Aspect 9. The image processing method according to any one of Aspects 5-8, wherein determining the recognized text line corresponding to the curved text line based on the adjusted text sub-region comprises:

[0169] The adjusted text sub-regions are scaled so that they have the same height;

[0170] Multiple scaled text sub-regions are concatenated horizontally to obtain the identified text line, wherein the characters in the identified text line are displayed horizontally.

[0171] Aspect 10. The image processing method of aspect 1, wherein adjusting the curved text line using adjustment parameters determined based on the text line curve to obtain a recognized text line corresponding to the curved text line includes:

[0172] For each column of pixels in the text line image, determine the adjustment parameters for that column of pixels;

[0173] The display of each column of pixels in the text line image is adjusted using the adjustment parameters;

[0174] The identified text line is determined based on the adjusted text line image.

[0175] Aspect 11. The image processing method of aspect 10, wherein, for each column of pixels in the text line image, the adjustment parameters for that column of pixels include the offset between the ordinate of a point on the text line curve in that column of pixels and a reference position.

[0176] Aspect 12. The image processing method as described in aspect 11, wherein adjusting the display of the column of pixels using the adjustment parameters includes:

[0177] Based on the offset, the position of the point on the text line curve of the column of pixels is adjusted in the vertical direction so that the adjusted vertical position of the point on the text line curve of the column of pixels is consistent with the reference position.

[0178] Aspect 13. An electronic circuit, comprising:

[0179] A circuit configured to perform the steps of the method according to any one of aspects 1-12.

[0180] Aspect 14. An assistive device for the visually impaired, comprising:

[0181] A camera is configured to acquire images, wherein the images include curved lines of text;

[0182] A curved text line correction circuit implemented by the electronic circuit described in aspect 13;

[0183] A circuit configured to perform text detection and / or recognition on the recognized text lines obtained by the curved text line correction circuit to obtain text data;

[0184] Circuitry configured to convert the text data into sound data; and

[0185] The circuit is configured to output the sound data.

[0186] Aspect 15. An electronic device comprising:

[0187] Processor; and

[0188] A memory that stores a program, the program comprising instructions that, when executed by the processor, cause the processor to perform the method according to any one of aspects 1-12.

[0189] Aspect 16. A non-transitory computer-readable storage medium storing a program, the program comprising instructions that, when executed by a processor of an electronic device, cause the electronic device to perform the method according to any one of Aspects 1-12.

[0190] Aspect 17. A computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the method described in any one of Aspects 1-12.

[0191] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the methods, systems, and devices described above are merely exemplary embodiments or examples, and the scope of the invention is not limited by these embodiments or examples, but only by the granted claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as the technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure.

Claims

1. An image processing method, comprising: Perform text line detection on the input image to obtain a text line image that includes curved text lines; Determine multiple reference points in the text line image for the curved text line. The text line curve for the curved text line is determined based on the plurality of reference points; The curved text line is adjusted using adjustment parameters determined based on the text line curve to obtain a corresponding recognized text line, wherein the recognized text line includes multiple characters displayed horizontally. The determination of multiple reference points in the text line image for the curved text line includes: Character target detection is performed on the text line image to obtain multiple reference points for the curved text line, wherein each reference point indicates the position of each character in the curved text line; or The text image is segmented pixel-by-pixel to obtain text line regions in the text line image corresponding to the curved text lines. Based on the height of the text line regions and a predetermined step size, multiple reference points in the text line regions for the curved text lines are determined. The process of adjusting the curved text line using adjustment parameters determined based on the text line curve to obtain the corresponding recognized text line includes: For each column of pixels in the text line image, an adjustment parameter is determined for that column of pixels, the adjustment parameter including the offset between the ordinate of the point on the text line curve in that column of pixels and the reference position; Based on the offset, the pixels in this column are cropped, and the cropped pixel sequence with a height equal to the offset is filled in reverse below the points on the text line curve, so that the adjusted vertical position of the points on the text line curve in this column of pixels is consistent with the reference position. The identified text line is determined based on the adjusted text line image.

2. The image processing method as claimed in claim 1, wherein, Determining the text line curve for the curved text line based on the multiple reference points includes: The positions of the multiple reference points are fitted using a B-spline interpolation method to obtain the text line curve.

3. The image processing method of claim 1, wherein, Adjusting the curved text line using adjustment parameters determined based on the text line curve to obtain the corresponding recognized text line includes: Identify multiple text sub-regions for the curved text line; For each of the plurality of text sub-regions, the text sub-region is adjusted based on the adjustment parameters for that text sub-region determined using the text line curve; The identified text line corresponding to the curved text line is determined based on the adjusted text sub-region.

4. The image processing method of claim 3, wherein, Determining multiple text sub-regions within the curved text line includes: Determine the slope at the location of at least one point on the text line curve; The curved text line is divided based on the slope at the location corresponding to the at least one point to obtain multiple text sub-regions, wherein adjacent text sub-regions correspond to different slopes.

5. The image processing method of claim 4, wherein, The adjustment parameters for each of the multiple text sub-regions include the angle between the arrangement direction of the characters included in the text sub-region and the horizontal direction, determined based on the slope corresponding to the text sub-region.

6. The image processing method of claim 5, wherein, The adjustment of the text sub-region based on the adjustment parameters determined using the text line curve includes: The text sub-region is adjusted based on the aforementioned angle so that the characters in the adjusted text sub-region are displayed horizontally.

7. The image processing method according to any one of claims 3-6, wherein, The identified text lines corresponding to the curved text lines are determined based on the adjusted text sub-regions, including: The adjusted text sub-regions are scaled so that they have the same height; Multiple scaled text sub-regions are concatenated horizontally to obtain the identified text line, wherein the characters in the identified text line are displayed horizontally.

8. An electronic circuit, comprising: A circuit configured to perform the steps of the method according to any one of claims 1-7.

9. An assistive device for the visually impaired, comprising: A camera is configured to acquire images, wherein the images include curved lines of text; A curved text line correction circuit implemented by the electronic circuit as described in claim 8; A circuit configured to perform text detection and / or recognition on the recognized text lines obtained by the curved text line correction circuit to obtain text data; A circuit configured to convert the text data into sound data; as well as The circuit is configured to output the sound data.

10. An electronic device, comprising: processor; as well as A memory storing a program, the program comprising instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1-7.

11. A non-transitory computer-readable storage medium storing a program, the program comprising instructions that, when executed by a processor of an electronic device, cause the electronic device to perform the method according to any one of claims 1-7.

12. A computer program product comprising a computer program, wherein, The computer program, when executed by a processor, implements the method of any one of claims 1-7.

Images