Image processing method, circuit, visual impairment assistance device, electronic device, and medium

Abstract

An image processing method, circuit, visual impairment auxiliary device, electronic device, and medium are provided. The method includes: obtaining a to-be-processed image and a reference image of the to-be-processed image, the size of the reference image being smaller than the size of the to-be-processed image; performing compression processing on the to-be-processed image according to the size of the reference image; and determining the image quality of the to-be-processed image according to the to-be-processed image after the compression processing.

Description

[0001] This application is a divisional application of the invention patent application filed on December 25, 2019, with application number 201911353598.1 and invention title "Image Processing Method, Circuit, Assistive Device for the Visually Impaired, Electronic Device and Medium". Technical Field

[0002] This disclosure relates to the field of image processing technology, and in particular to an image processing method, electronic circuit, visually impaired assistive device, electronic device, and computer-readable storage medium. Background Technology

[0003] In related technologies, before performing text recognition operations such as OCR (Optical Character Recognition) on an image to be processed, it is often necessary to perform image evaluation preprocessing operations on the image to be processed to assess its image quality. If it is determined that the image quality of the image to be processed is poor, subsequent text recognition operations such as OCR can be omitted, thereby reducing unnecessary computational consumption and saving time and power consumption.

[0004] The methods described in this section are not necessarily methods that had been previously conceived or adopted. Unless otherwise specified, no method described in this section should be assumed to be prior art simply because it is included in this section. Similarly, unless otherwise specified, the issues mentioned in this section should not be considered to be accepted in any prior art. Summary of the Invention

[0005] According to one aspect of this disclosure, an image processing method is provided, comprising: acquiring an image to be processed and a reference image of the image to be processed, the size of the reference image being smaller than the size of the image to be processed; compressing the image to be processed according to the size of the reference image; and determining the image quality of the image to be processed based on the compressed image to be processed.

[0006] According to one aspect of this disclosure, an image processing method is provided, comprising: acquiring an image to be processed and segmenting the image to be processed into multiple image blocks; calculating the Laplacian standard deviation of each image block based on the pixel values ​​of pixels in each image block; comparing the calculated Laplacian standard deviation of each image block with a standard deviation threshold, selecting a first image block whose corresponding Laplacian standard deviation is greater than the standard deviation threshold; and determining the image blur of the image to be processed based on the Laplacian standard deviation of the selected first image block.

[0007] According to another aspect of this disclosure, an electronic circuit is provided, comprising: a circuit configured to perform the steps of the methods described in this disclosure.

[0008] According to another aspect of this disclosure, a visually impaired assistive device is provided, comprising: a camera configured to acquire images; and the electronic circuitry described in this disclosure.

[0009] According to another aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory storing a program, the program including instructions that, when executed by the processor, cause the processor to perform the methods described in this disclosure.

[0010] According to another aspect of this disclosure, a computer-readable storage medium is provided that stores a program, the program including instructions that, when executed by a processor of an electronic device, cause the electronic device to perform the methods described in this disclosure.

[0011] Further features and advantages of this disclosure will become clear from the exemplary embodiments described below in conjunction with the accompanying drawings. Attached Figure Description

[0012] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0013] Figure 1 This is a flowchart illustrating an image processing method according to an exemplary embodiment of the present disclosure;

[0014] Figure 2A This is a schematic diagram showing a preview image according to an exemplary embodiment of the present disclosure;

[0015] Figure 2B This is a schematic diagram illustrating a compressed photographed image according to an exemplary embodiment of the present disclosure;

[0016] Figure 3A This is a schematic diagram illustrating another preview image according to an exemplary embodiment of the present disclosure;

[0017] Figure 3B This is a schematic diagram illustrating another compressed photographic image according to an exemplary embodiment of the present disclosure;

[0018] Figure 4 This is a schematic flowchart illustrating the process of determining the text offset between the compressed image to be processed and the reference image using a grayscale projection method according to an exemplary embodiment of the present disclosure.

[0019] Figure 5 This is a schematic flowchart illustrating the process of determining the image blur of a compressed image using the Laplacian variance algorithm according to an exemplary embodiment of the present disclosure.

[0020] Figure 6 This is a schematic diagram illustrating the overall process of adjusting the standard deviation threshold according to an exemplary embodiment of the present disclosure;

[0021] Figure 7 This is a schematic diagram illustrating the overall flow of an image processing method according to an exemplary embodiment of the present disclosure;

[0022] Figure 8A A schematic diagram illustrating a preview image of a text box that can be detected according to an exemplary embodiment of the present disclosure;

[0023] Figure 8B This diagram illustrates an image to be processed obtained by performing an image capture operation according to an exemplary embodiment of the present disclosure.

[0024] Figure 8C A schematic diagram showing a compressed image to be processed according to an exemplary embodiment of the present disclosure;

[0025] Figure 8D A schematic diagram showing a cropped preview image according to an exemplary embodiment of the present disclosure;

[0026] Figure 8E A schematic diagram showing a cropped image to be processed according to an exemplary embodiment of the present disclosure;

[0027] Figure 9 This is a block diagram illustrating an example of an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0028] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0029] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. Furthermore, the term "and / or" as used in this disclosure covers any one of the listed items and all possible combinations thereof.

[0030] In related technologies, before performing text recognition operations such as OCR (Optical Character Recognition) on an image to be processed, it is often necessary to perform image evaluation preprocessing to assess its image quality. If the image quality of the image to be processed is determined to be poor, subsequent text recognition operations such as OCR can be skipped to reduce unnecessary computational consumption and save time and power. The image evaluation preprocessing operation may include image blur assessment of the image to be processed. However, in order to accommodate the high recognition rate required by subsequent text recognition operations such as OCR, the number of pixels, data volume, storage volume, or sharpness of the image to be processed is usually large. Therefore, the computational load required for performing preprocessing operations such as blur assessment of the image to be processed is usually large, resulting in low image processing efficiency.

[0031] This disclosure provides an image processing method that avoids directly performing image evaluation preprocessing on a relatively large image to be processed. Instead, it compresses the image to be processed based on a relatively small reference image, and then determines the image quality of the image to be processed based on the compressed image. Exemplary embodiments of the image processing method of this disclosure will be further described below with reference to the accompanying drawings.

[0032] Figure 1 This is a flowchart illustrating an image processing method according to an exemplary embodiment of the present disclosure. Figure 1 As shown, the image processing method may include the following steps: acquiring an image to be processed and a reference image of the image to be processed, wherein the size of the reference image is smaller than the size of the image to be processed (step S101); compressing the image to be processed according to the size of the reference image (step S102); and determining the image quality of the image to be processed according to the compressed image to be processed (step S103).

[0033] according to Figure 1 The image processing method shown in the flowchart compresses the image to be processed based on the size of a reference image, and determines the image quality of the image to be processed based on the compressed image. The size of the reference image is smaller than the size of the image to be processed. This avoids the computational burden of directly performing image evaluation preprocessing on a relatively large image to be processed, resulting in a shorter image quality evaluation process, improved image processing efficiency, and reduced processing time.

[0034] According to some embodiments, the image to be processed and the reference image can be electronic images acquired by an image sensor (e.g., a camera, video camera, etc.). The electronic image can be a color image or a grayscale image. The electronic image can also be a still image or a video image. According to some embodiments, the image sensor can be included in a user-held electronic device (e.g., a reading aid, mobile phone, tablet computer, personal digital assistant, wearable electronic device, etc.). The image sensor or the electronic device including the image sensor can be mounted on a user's wearable device or glasses, etc., so that the image to be processed and / or the reference image can be an image acquired by the image sensor containing a scene of a reading material (e.g., a book or magazine) held by the user. According to some implementations, the image to be processed and the reference image can be stored (e.g., cached) in a storage device or storage medium after being acquired by an image sensor, and can be read out for subsequent processing. The image to be processed stored in the storage device or storage medium may include the original image of the image to be processed or include the original image of the image to be processed and at least one copy (the size of the copy is the same as the size of the original image). The stored reference image may include the original image of the reference image or include the original image of the reference image and at least one copy.

[0035] According to some embodiments, the size of each image can be represented by one or more of the following: pixel count, data volume, storage volume, or resolution. For example, taking the size of the image to be processed and the reference image as represented by the number of pixels, the size of the image to be processed can be represented as M1 x N1, and the size of the reference image can be represented as M2 x N2, where the units of M1, N1, M2, and N2 can be pixels. Furthermore, the specific values ​​of M1, N1, M2, and N2 can be determined according to actual conditions, such as the size of the image sensor and the number of pixels, as long as the size of the reference image is smaller than the size of the image to be processed. For example, the size of the reference image can be represented as 320 x 240 (in pixels), and the size of the image to be processed can be represented as 3264 x 2448 (in pixels), and so on.

[0036] According to some embodiments, the compressed image to be processed is obtained by compressing a copy of the image to be processed, rather than the original image, according to the size of the reference image. In this way, the original image to be processed can be preserved as is to avoid affecting subsequent operations that the user may perform on the original image to be processed. According to some embodiments, one copy of the image to be processed stored in a storage device or storage medium can be selected for compression to obtain the compressed image to be processed. Alternatively, the original image to be processed stored in a storage device or storage medium can be copied to obtain a copy of the image to be processed, and then the copy of the image to be processed can be compressed to obtain the compressed image to be processed.

[0037] According to some embodiments, the size of the compressed image to be processed can be the same as the size of the reference image, so as to facilitate more efficient execution of subsequent image quality determination operations. Of course, the size of the compressed image to be processed can also have a certain permissible error compared with the size of the reference image in the length and / or width directions, as long as it does not affect the execution of subsequent image quality determination operations.

[0038] According to some embodiments, the reference image and the image to be processed can be images of the same scene (e.g., the same target or the same area) acquired at different times. The reference image can detect text regions, meaning that this disclosure is applicable to processing scene images containing text content. According to some embodiments, the reference image and the image to be processed can be a preview image and a captured image of the same scene, acquired during the image preview stage and the image capture stage, respectively. In other words, the reference image can be a preview image that detects text regions, acquired through an image preview operation before acquiring the image to be processed through the image capture operation. Since preview images are typically small, the time required for image quality assessment of the image to be processed can be further shortened; for example, the image quality assessment process can be reduced to the millisecond level. Furthermore, by using the preview image acquired through the image preview operation as the reference image for the image to be processed acquired through the subsequent image capture operation, the correlation between images and the convenience of image acquisition can be improved, thereby increasing the accuracy and efficiency of subsequent image quality assessment.

[0039] According to some implementations, the image preview stage is a stage that can be used to perform text box detection. Text box detection can be performed on an image (such as a preview image acquired in the image preview stage) based on a text box detection algorithm, and in response to the detection of text boxes in the image, the image is determined to be an image in which text regions can be detected. The text regions can, for example, be regions containing Chinese, English, Japanese, and / or other language characters.

[0040] For example, in this embodiment of the disclosure, the image to be processed and a reference image of the image to be processed can be obtained by: having an image sensor (e.g., a camera, a video camera, etc.) perform an image preview operation on a set scene (e.g., a scene containing a book held by a user, or other scenes with text recognition requirements, etc.) to obtain a preview image; performing text box detection on the preview image; and, in response to detecting a text box in the preview image, using the preview image as a reference image and having the image sensor perform an image capture operation on the set scene to obtain the image to be processed (i.e., the captured image). According to some embodiments, the text box detection algorithm may include, for example, non-deep learning methods based on extreme regions such as MSER (Maximally Stable Extremal Regions) or connected regions, or deep learning methods such as bounding box regression or segmentation methods. According to some implementations, the text box detection algorithm may employ non-deep learning methods such as methods based on extreme regions (e.g., MSER) ​​or connected regions to reduce the complexity of text box detection and improve the efficiency of image processing.

[0041] According to some embodiments, determining the image quality of the compressed image to be processed, based on the compressed image to be processed, includes any one or a combination of the following steps: determining a text offset between the compressed image to be processed and the reference image, and determining the image quality of the image to be processed based on the text offset; determining the image blur of the compressed image to be processed, and determining the image quality of the image to be processed based on the image blur.

[0042] According to some embodiments, determining the image quality of the image to be processed based on the text offset may include: determining that the image quality of the image to be processed meets a set quality condition in response to the text offset not being greater than an offset threshold. Correspondingly, in response to the text offset being greater than the offset threshold, it may be determined that the image quality of the image to be processed does not meet the set quality condition. According to some embodiments, determining the image quality of the image to be processed based on the image blurriness may include: determining that the image quality of the image to be processed meets a set quality condition in response to the image blurriness not being greater than a blurriness threshold. Correspondingly, in response to the image blurriness being greater than a blurriness threshold, it may be determined that the image quality of the image to be processed does not meet the set quality condition. The offset threshold and the blurriness threshold may be preset, and may also be dynamically adjusted according to actual conditions and / or requirements.

[0043] In other words, in this embodiment of the disclosure, the image quality of the image to be processed can be evaluated based on one or both of the indicators of text offset and image blur, thereby improving the flexibility of image quality evaluation. When the image quality of the image to be processed is evaluated based on both text offset and image blur, in response to the text offset not being greater than an offset threshold and the image blur not being greater than a blur threshold, it can be determined that the image quality of the image to be processed meets the set quality conditions; otherwise, it can be determined that the image quality of the image to be processed does not meet the set quality conditions. Therefore, the accuracy of the image quality evaluation result will be higher. Tests have shown that in this case, the accuracy of image quality evaluation can reach over 90%.

[0044] Furthermore, it should be noted that the metrics used for image quality assessment can be flexibly determined based on the actual situation and / or requirements. For example, for situations requiring high image quality assessment results, the image quality of the image to be processed can be evaluated based on both text offset and image blur. For situations requiring high efficiency in image quality assessment, the image quality of the image to be processed can be evaluated based on either text offset or image blur. As another example, one or both of the metrics, text offset and image blur, can be selected to evaluate the image quality of the image to be processed, either through pre-setting or manual settings, and so on.

[0045] For example, using the reference image as an example Figure 2A The preview image shown is the compressed image to be processed. Figure 2B The image shown is an example of a compressed photograph. Figure 2A and 2BAs shown, the compressed image is relatively clear, but the text offset (or displacement) between it and the preview image is large; for example, the text in the preview image has completely shifted out of the compressed image. In this case, image quality can be evaluated based on the degree of text offset to improve the efficiency of image quality assessment. For example, calculations can determine that the text offset between the compressed image and the preview image is greater than an offset threshold, i.e., it does not meet the condition of not exceeding the offset threshold. Therefore, the image quality of the image can be considered poor and does not meet the set quality conditions. Alternatively, in this case, the image quality of the image to be processed can be evaluated based on both the degree of text offset and the degree of image blur to improve the accuracy of the image quality assessment results. For example, although calculations can determine that the image blur of the compressed image is not greater than the blur threshold, the text offset between the compressed image and the preview image is greater than the offset threshold, i.e., it does not meet the condition of not exceeding the offset threshold. Therefore, the image quality of the image can be considered poor and does not meet the set quality conditions.

[0046] For example, using a reference image as... Figure 3A The preview image shown is the compressed image to be processed. Figure 3B Taking the compressed image as an example, the text in the preview image has a very small (or almost negligible) offset (or displacement) relative to the text in the compressed image, but the text in the compressed image is quite blurry. In this case, image quality can be evaluated based on the degree of blur to improve the efficiency of image quality assessment. For example, calculations can determine that the image blur is greater than a blur threshold, meaning it does not meet the condition of not exceeding the blur threshold, thus the image quality of the image can be considered poor and does not meet the set quality conditions. Alternatively, in this case, the image quality of the image to be processed can be evaluated based on both the text offset and the image blur, to improve the accuracy of the image quality assessment results. For example, although calculations can determine that the text offset between the compressed image and the preview image is not greater than the offset threshold, the image blur of the compressed image is determined to be greater than the blur threshold, meaning it does not meet the condition of not exceeding the blur threshold, thus the image quality of the image can be considered poor and does not meet the set quality conditions.

[0047] According to some embodiments, determining the image quality of the image to be processed based on the compressed image to be processed may include: determining the text offset between the compressed image to be processed and the reference image; determining the image blur of the compressed image to be processed in response to the text offset not being greater than an offset threshold; and determining that the image quality of the image to be processed meets a set quality condition in response to the image blur not being greater than a blur threshold. In other words, when evaluating the image quality of the image to be processed using both text offset and image blur as indicators, the text offset determination operation can be performed before the image blur determination operation. Through such processing, if it is determined that the text offset is greater than the offset threshold, it is not necessary to perform the image blur determination and judgment operation again, but it can be directly determined that the image quality of the image to be processed does not meet the set quality condition (for example, it can be directly determined that the text offset of the image to be processed is too large), thereby reducing the computational load of image processing and improving the efficiency of image processing. For example, in Figure 2A as well as Figure 2B In the example shown, since it can be determined that the text offset between the compressed image and the preview image is greater than the offset threshold, it can be directly concluded that the image quality of the image to be processed does not meet the set quality conditions, without having to perform the image blur determination and judgment operation.

[0048] According to other embodiments, determining the image quality of the image to be processed based on the compressed image to be processed may include: determining the image blur of the compressed image to be processed; determining the text offset between the compressed image to be processed and the reference image in response to the image blur not being greater than a blur threshold; and determining that the image quality of the image to be processed meets a set quality condition in response to the text offset not being greater than an offset threshold. In other words, when evaluating the image quality of the image to be processed using both text offset and image blur as indicators, the image blur determination operation can be performed before the text offset determination operation. Through such processing, if the image blur is determined to be greater than a blur threshold, it is not necessary to perform the text offset determination and judgment operation again. Instead, it can be directly determined that the image quality of the image to be processed does not meet the set quality condition (e.g., it can be directly determined that the image to be processed is unclear or too blurry), thereby reducing the computational load of image processing and improving the efficiency of image processing. For example, in Figure 3A as well as Figure 3B In the example shown, since it can be determined that the image blur of the compressed image is greater than the blur threshold, it can be directly concluded that the image quality of the image to be processed does not meet the set quality conditions, without having to perform the text offset determination and judgment operation.

[0049] As an alternative implementation, the text offset determination operation and the image blur determination operation can be performed in parallel. Then, the image quality of the image to be processed is comprehensively judged based on the corresponding text offset determination results and image blur determination results. For example, as mentioned above, if it is determined that the text offset is not greater than an offset threshold and the image blur is not greater than a blur threshold, then it can be determined that the image quality of the image to be processed meets the set quality conditions. If it is determined that the text offset is greater than an offset threshold and / or the image blur is greater than a blur threshold, then it can be determined that the image quality of the image to be processed does not meet the set quality conditions.

[0050] Furthermore, as can be seen from the above embodiments of this disclosure, during the image sensor (e.g., a camera, camcorder, etc.) performing the image capture operation, the displacement or blur caused by lens or text object shaking during the exposure to imaging period (typically 1-2 seconds) can be reflected in the preview image before taking the picture and the captured image after taking the picture. This makes good use of the image capture time without any additional time consumption.

[0051] According to some embodiments, a text offset detection algorithm can be used to determine the text offset between the compressed image to be processed and the reference image. The text offset detection algorithm may, for example, include a grayscale projection method. The grayscale projection method is a simplified extraction method for image distribution features. It uses the pixel rows and columns of a two-dimensional image as units, transforming image features into curves along row and column coordinates to facilitate the calculation of image distribution features.

[0052] Figure 4 A flowchart illustrating the determination of the text offset between the compressed image to be processed and the reference image using a grayscale projection method according to an exemplary embodiment of the present disclosure is shown. Figure 4 As shown, determining the text offset using the grayscale projection method may include the following steps.

[0053] Step S401: Calculate the sum of all pixels in each pixel row and each pixel column of the compressed image to be processed and the reference image.

[0054] According to some embodiments, the compressed image to be processed and the reference image may be converted to grayscale before calculating the sum of all pixels in each pixel row and each pixel column of the compressed image to be processed and the reference image. Alternatively, if the compressed image to be processed and the reference image are grayscale images, this grayscale conversion step can be omitted.

[0055] Step S402: Based on the sum of all pixels in each pixel row and each pixel column of the compressed image to be processed and the reference image, determine the row grayscale projection and column grayscale projection of the compressed image to be processed and the reference image.

[0056] According to some embodiments, for any image in the compressed image to be processed and the reference image, the row grayscale projection of the image can be obtained by calculating the difference between the sum of all pixels in each row of the image and the mean of the row pixel sum of the image; similarly, the column grayscale projection of the image can be obtained by calculating the difference between the sum of all pixels in each column of the image and the mean of the column pixel sum of the image. For any image, the mean of the row pixel sum refers to the mean of the sum of pixels in all rows of the image, and the mean of the column pixel sum refers to the mean of the sum of pixels in all columns of the image.

[0057] Step S403: Based on the row grayscale projection and column grayscale projection of the compressed image to be processed and the reference image, determine the vertical displacement of the matching rows and the horizontal displacement of the matching columns in the compressed image to be processed and the reference image.

[0058] According to some embodiments, the displacement of matching rows (e.g., the row with the highest cross-correlation) in the vertical direction (i.e., the column direction) and the displacement of matching columns (e.g., the column with the highest cross-correlation) in the horizontal direction (i.e., the row direction) can be obtained by calculating the cross-correlation of the row grayscale projections of the compressed image to be processed and the reference image, respectively. The vertical displacement of the matching rows and the horizontal displacement of the matching columns in the two images are the vertical and horizontal displacements of the object to be identified in the two images.

[0059] According to some embodiments, the cross-correlation between the row grayscale projections of the compressed image to be processed and the reference image can be represented by parameters that reflect the cross-correlation between row grayscale projections, such as the difference between the row grayscale projections of the compressed image to be processed and the reference image, or the sum of squares of the differences. A smaller difference or sum of squares indicates a higher cross-correlation between the row grayscale projections. Similarly, the cross-correlation between the column grayscale projections of the compressed image to be processed and the reference image can be represented by parameters that reflect the cross-correlation between column grayscale projections, such as the difference between the column grayscale projections of the compressed image to be processed and the reference image, or the sum of squares of the differences. A smaller difference or sum of squares indicates a higher cross-correlation between the column grayscale projections.

[0060] According to some embodiments, the difference or sum of squares between the row grayscale projections of the compressed image to be processed and the reference image can be obtained by directly calculating the difference or sum of squares between the row grayscale projections of the compressed image to be processed and the reference image. Alternatively, it can be obtained by calculating the difference or sum of squares between the row grayscale projections of the compressed image to be processed after displacement in the row direction (the displacement value can be flexibly set according to the actual situation) and the row grayscale projections of the reference image. Similarly, the difference or sum of squares between the column grayscale projections of the compressed image to be processed and the reference image can be obtained by directly calculating the difference or sum of squares between the column grayscale projections of the compressed image to be processed and the column grayscale projections of the reference image. Alternatively, it can be obtained by calculating the difference or sum of squares between the column grayscale projections of the compressed image to be processed after displacement in the column direction (the displacement value can be flexibly set according to the actual situation) and the column grayscale projections of the reference image.

[0061] According to some embodiments, taking the calculation of the cross-correlation of line grayscale projections by calculating the sum of squares of the differences between the line grayscale projections as an example, the minimum value of the sum of squares of the differences between the line grayscale projections of the compressed image to be processed and the line grayscale projections of the reference image can be obtained, and the vertical displacement of the matching lines in the two images can be determined based on the displacement corresponding to the obtained minimum value. Furthermore, the vertical displacement of the matching lines in the two images is the vertical text offset of the matching lines in the two images.

[0062] For example, suppose the sum of squares Ry(w) of the differences between the line grayscale projection of the compressed image to be processed and the line grayscale projection of the reference image can be expressed as shown in the following formula (1), where w is the independent variable, m is a fixed value (which can be flexibly set according to the actual situation), and the value of M is 2m+1, Col cur (j+w-1) is the row grayscale projection of the (j+w-1)th row of the compressed image to be processed, Col ref (m+j) is the row grayscale projection of the (m+j)th row of the reference image:

[0063]

[0064] Substitute 1, 2, ..., 2m+1 into formula (1) to calculate the value of Ry(w), and then obtain the value of w when Ry(w) reaches its minimum value. min The vertical displacement dy of the two images can then be expressed as: dy = m + 1 - w min .

[0065] The principle of the above method is as follows: by continuously calculating the square of the difference between the (j+w-1)th row of the compressed image to be processed and the (m+j)th row of the reference image, where the independent variable is w (i.e., within the range of w, the interval between the two rows can be calculated as w+m-1), the square of the gray-scale projection difference of all rows with an interval of w+m-1 in the entire M rows of the image is calculated iteratively, where m is a fixed value, that is, the row interval changes with w. Thus, when a certain w... min To minimize Ry(w), it means that the projections between rows are similar, and therefore the interval between rows is the vertical displacement of the two images.

[0066] Similarly, taking the calculation of the cross-correlation of column grayscale projections by calculating the sum of squares of the differences between the column grayscale projections as an example, the minimum value of the sum of squares of the differences between the column grayscale projections of the compressed image to be processed and the column grayscale projections of the reference image can be obtained. Based on the displacement corresponding to the obtained minimum value, the horizontal displacement of the matching columns in the two images is determined. Furthermore, the horizontal displacement of the matching columns in the two images is the horizontal text offset of the matching columns in the two images.

[0067] It should be noted that, Figure 4 The text offset calculation method shown is only one example; other algorithms can also be used to calculate text offsets.

[0068] As an alternative embodiment, the text offset detection algorithm may also include other text offset detection algorithms such as block matching and feature point matching. In other words, the embodiments of this disclosure can flexibly utilize various text offset detection algorithms in related technologies to calculate the required text offset according to actual needs, thereby improving the applicability of the image processing method.

[0069] According to some embodiments, an image blur detection algorithm can be used to determine the image blur of the compressed image to be processed. This image blur detection algorithm may include, for example, the Laplacian variance algorithm. The Laplacian operator measures the second derivative of an image and can highlight areas of rapidly changing density in the image; therefore, it can be used for boundary detection. In normal, well-focused images, the boundaries are relatively clear, meaning the pixels have a large degree of dispersion, and the pixel variance (or pixel standard deviation) calculated after the Laplacian transform will be relatively large. Conversely, blurred images contain very little boundary information, meaning the pixels have a low degree of dispersion, and the pixel variance (or pixel standard deviation) calculated after the Laplacian transform will be relatively small. This pixel variance (or pixel standard deviation) can reflect the image blur.

[0070] According to some implementation methods, the step of determining the image blur of the compressed image to be processed using the Laplacian variance algorithm may include: converting the compressed image to grayscale (this step can be omitted if the compressed image to be processed is a grayscale image); performing a Laplacian transform on the grayscale image to be processed; and calculating the pixel variance of the transformed image to obtain the corresponding image blur.

[0071] In this embodiment of the disclosure, image blur (the opposite of sharpness) can be represented by a floating-point number. The smaller the floating-point number, the greater the image blur (the less sharpness), and the larger the floating-point number, the less the image blur (the greater the sharpness).

[0072] According to some implementation methods, in order to improve the applicability of the Laplacian variance algorithm (e.g., its applicability to images with many solid color regions) and the accuracy of using the Laplacian variance algorithm to determine image blur, Figure 5 A schematic flowchart illustrating another method for determining the image blur of a compressed image according to an exemplary embodiment of this disclosure using the Laplacian variance algorithm is shown. Figure 5 As shown, the Laplacian variance algorithm is used to determine the image blur of the compressed image to be processed, which may include the following steps.

[0073] Step S501: Divide the compressed image into multiple image blocks, and calculate the Laplacian standard deviation of each image block based on the pixel values ​​of the pixels in each image block.

[0074] According to some embodiments, the compressed image to be processed can be divided into multiple image blocks of the same size (e.g., 8x8) using an average segmentation method. Of course, the sizes of the multiple image blocks can also be different. Furthermore, before dividing the compressed image to be processed into multiple image blocks, the compressed image to be processed can be converted to grayscale. If the compressed image to be processed is a grayscale image, this grayscale conversion step can be omitted.

[0075] According to some embodiments, similar to related technologies, Laplacian convolution filtering can be performed on each image block based on the pixel values ​​of the pixels in each image block to obtain the Laplacian standard deviation of each image block.

[0076] Step S502: Compare the calculated Laplacian standard deviation of each image block with the standard deviation threshold, and select the first image block whose corresponding Laplacian standard deviation is greater than the standard deviation threshold.

[0077] The standard deviation threshold is a threshold used to detect solid color regions in an image. It can have a default, small initial value, such as 10 to 15 (inclusive). Furthermore, the standard deviation threshold can be obtained from the server and stored in the storage device or storage medium of a local device (such as a terminal device) for subsequent reading and use. The server can be a remote device different from the local device, a sub-device set within the local device, or even the local device itself. Through step S502, one or more first image blocks can be determined.

[0078] Step S503: Determine the image blur of the compressed image based on the Laplacian standard deviation of the selected first image block.

[0079] According to some embodiments, the average value of the Laplacian standard deviation of each selected first image block can be calculated, and the average value can be used as the image blur of the compressed image to be processed.

[0080] Based on the above embodiments, it can be seen that in this disclosure, the following methods can be used: Figure 5 The method shown filters out solid-color regions with excessively small Laplacian standard deviations by segmenting the image and calculating the Laplacian standard deviation block by block, thus obtaining more accurate image blur detection results. This solves the problem in related technologies where the extremely small dispersion of solid-color regions lowers the overall Laplacian standard deviation calculation result, causing the Laplacian variance algorithm to fail for solid-color regions. Therefore, even for images with large blank areas and little text, the accuracy of image blur detection can be improved. Furthermore, from... Figure 5 It can be seen that, Figure 5 The method shown, which uses the Laplacian variance algorithm to determine image blur, is a method that can calculate the image blur of the image to be processed without considering the reference image. Therefore, it can further improve the efficiency of image processing and save image processing time.

[0081] According to some embodiments, the standard deviation threshold can be dynamically adjusted based on actual conditions. For example, the standard deviation threshold can be fine-tuned by a server or local device based on the acquired image material, such as the corresponding Laplacian standard deviation for each second image patch (i.e., solid color or near-solid color image patch in the compressed image to be processed) near the standard deviation threshold. The fine-tuned standard deviation threshold can take effect the next time the image blur is determined.

[0082] In other words, in the embodiments described in this disclosure, the method may further include the following steps: selecting a second image block from the plurality of image blocks of the compressed image to be processed, wherein the absolute value of the difference between the corresponding Laplacian standard deviation and the standard deviation threshold is not greater than the difference threshold; performing target detection processing on the second image block based on a first target detection neural network and / or a second target detection neural network, and adjusting the standard deviation threshold according to the target detection results of the second image block by the first target detection neural network and / or the second target detection neural network.

[0083] The first target that the first object detection neural network can detect is different from the second target that the second object detection neural network can detect. For example, the first target that the first object detection neural network can detect can be text, and the second target that the second object detection neural network can detect can be texture, or the first target that the first object detection neural network can detect can be texture, and the second target that the second object detection neural network can detect can be text, and so on.

[0084] The difference threshold can be set and dynamically adjusted according to the actual situation, as long as it ensures that each selected second image block is a second image block whose corresponding Laplacian standard deviation is near the standard deviation threshold. Furthermore, through the above steps, one or more second image blocks can be determined.

[0085] According to some embodiments, the second image block can be processed for target detection based on a first target detection neural network; and in response to the detection of a first target in the second image block, the standard deviation threshold can be down-adjusted.

[0086] For example, a second image patch can be input into a first target detection neural network that can be used to detect text. If it is determined that text can be detected in the second image patch, then it can be determined that the second image patch is not a solid color or a near-solid color image patch. That is, it can be determined that the standard deviation threshold is set too high, and therefore the standard deviation threshold can be lowered. According to some implementations, the degree of lowering of the standard deviation threshold can be flexibly set according to the actual situation. For example, the standard deviation threshold can be adjusted in steps of 1 or 2, etc., based on the standard deviation of the second image patch that can detect the first target.

[0087] In addition, in response to the inability to detect the first target in the second image block, it is not necessary to adjust the standard deviation threshold; instead, the standard deviation threshold can be kept unchanged.

[0088] According to other embodiments, in order to improve the accuracy of the standard deviation threshold adjustment, the standard deviation threshold can also be adjusted in the following ways: performing target detection processing on the second image block based on a first target detection neural network; performing target detection processing on the second image block based on a second target detection neural network in response to the inability to detect a first target in the second image block; and increasing the standard deviation threshold in response to the detection of a second target in the second image block and the Laplacian standard deviation of the second image block being greater than the standard deviation threshold.

[0089] For example, a second image patch can be input into a first object detection neural network that can detect text. If it is determined that no text can be detected in the second image patch, then the second image patch can be determined to be a solid color or near-solid color image patch. To improve the accuracy of threshold adjustment, the second image patch can be further input into a second object detection neural network that can detect other objects, such as textures. If it is determined that texture can be detected in the second image patch, and the Laplacian standard deviation of the second image patch is greater than the standard deviation threshold, then the standard deviation threshold can be increased. This is because some blank areas of text on a page may have small textures, so its solid color Laplacian standard deviation will be higher than that of a blank page without texture.

[0090] According to some implementation methods, the degree to which the standard deviation threshold is increased can be flexibly set according to the actual situation. For example, the standard deviation threshold can be adjusted in steps of 1 or 2, etc., based on the standard deviation of the second image patch that can detect the second target.

[0091] Additionally, in response to the inability to detect a first target in the second image block and the inability to detect a second target in the second image block; or, in response to the inability to detect a first target in the second image block and the ability to detect a second target in the second image block but the Laplacian standard deviation of the second image block is not greater than the standard deviation threshold, it is not necessary to adjust the standard deviation threshold.

[0092] After adjusting the standard deviation threshold, the corresponding standard deviation threshold stored in the local storage device or storage medium can also be updated synchronously so that the adjusted standard deviation threshold can be used the next time the image blur is determined.

[0093] According to some embodiments, Figure 6 A schematic diagram of the overall process for adjusting the standard deviation threshold according to an exemplary embodiment of this disclosure is shown. Figure 6 As shown, the standard deviation threshold can be adjusted through the following steps.

[0094] Step S601: Perform target detection processing on the second image block based on the first target detection neural network.

[0095] The second image block is a second image block in which the absolute value of the difference between the corresponding Laplacian standard deviation and the standard deviation threshold is not greater than the difference threshold. The first target that the first target detection neural network can detect may include, for example, text.

[0096] Step S602: Determine whether the first target can be detected in the second image block. If yes, proceed to step S603; otherwise, proceed to step S604.

[0097] Step S603: In response to detecting the first target in the second image block, the standard deviation threshold is down-adjusted.

[0098] As mentioned earlier, the degree to which the standard deviation threshold is lowered can be flexibly set according to the actual situation. For example, the standard deviation threshold can be adjusted by a step size of 1 or 2, etc., based on the standard deviation of the second image block that can detect the first target.

[0099] Step S604: In response to the inability to detect the first target in the second image block, target detection processing is performed on the second image block based on the second target detection neural network.

[0100] The second target detection neural network can detect, for example, a texture.

[0101] Step S605: Determine whether a second target can be detected in the second image block. If yes, proceed to step S606; otherwise, determine that no adjustment is needed to the standard deviation threshold.

[0102] Step S606: In response to detecting the second target in the second image block, determine whether the Laplacian standard deviation of the second image block is greater than the standard deviation threshold. If yes, proceed to step S607; otherwise, determine that no adjustment to the standard deviation threshold is required.

[0103] Step S607: In response to the fact that the Laplacian standard deviation of the second image block is greater than the standard deviation threshold, the standard deviation threshold is increased.

[0104] As mentioned earlier, the degree to which the standard deviation threshold is increased can be flexibly set according to the actual situation. For example, the standard deviation threshold can be adjusted with a step size of 1 or 2, etc., based on the standard deviation of the second image patch that can detect the second target.

[0105] in addition, Figure 6The method shown for adjusting the standard deviation threshold is only one possible example, and other methods can also be used to adjust the standard deviation threshold. For example, the second image patch can be processed in parallel for target detection based on a first target detection neural network and a second target detection neural network, and the adjustment method of the standard deviation threshold can be determined based on the corresponding detection results, etc.

[0106] Alternatively, as an alternative embodiment, the image blur detection algorithm may also include other image blur detection algorithms such as FFT (Fast Fourier Transform Algorithm) and Brenner gradient function algorithm. In other words, the embodiments of this disclosure can flexibly utilize various image blur detection algorithms in related technologies to calculate the required image blur according to actual needs, thereby improving the applicability of the image processing method.

[0107] It should be noted that the Laplacian variance algorithm, Fast Fourier Transform algorithm, and Brenner gradient function algorithm mentioned above are more suitable for determining the image blur of the image to be processed based on the image itself (without considering the reference image). For cases where the image blur of the image to be processed is determined in conjunction with a reference image, image blur detection algorithms such as the mean square error method or the peak signal-to-noise ratio method can also be used. For example, a corresponding mathematical model can be established based on the mean square error method or the peak signal-to-noise ratio method to compare the differences between the image to be processed and the reference image, and then the image blur of the image to be processed can be determined based on the differences.

[0108] Additionally, it should be noted that Figure 5 The image blur determination scheme shown is applicable to any image. In other words, in addition to being applicable to the compressed image to be processed mentioned in the embodiments of this disclosure, it is also applicable to the uncompressed image to be processed, or the cropped image to be processed mentioned below, or any other image to be processed. Furthermore, Figure 5 The image blur determination scheme shown can be combined with the text offset calculation scheme mentioned in some embodiments of this disclosure to determine the image quality of the image to be processed, or it can be applied independently to each image to be processed.

[0109] For example, for any acquired image to be processed (which may include the compressed image to be processed as described in the embodiments of this disclosure), the image blur of the image to be processed can be determined in the following way so that the image quality of the image to be processed can be determined based on the determined image blur: dividing the image to be processed into multiple image blocks (the image to be processed may also be grayscaled before image segmentation); calculating the Laplacian standard deviation of each image block according to the pixel values ​​of the pixels in each image block; comparing the calculated Laplacian standard deviation of each image block with a standard deviation threshold, selecting the first image block whose corresponding Laplacian standard deviation is greater than the standard deviation threshold; and determining the image blur of the image to be processed based on the Laplacian standard deviation of the selected first image block.

[0110] Accordingly, in response to determining that the image blur of the image to be processed is not greater than a blur threshold, it can be determined that the image quality of the image to be processed meets the set quality conditions. Additionally, as mentioned above, the image quality of the image to be processed can also be determined by combining the text offset between the image to be processed and a reference image of the image to be processed. For example, in response to determining that the image blur of the image to be processed is not greater than a blur threshold and that the text offset is not greater than an offset threshold, it can be determined that the image quality of the image to be processed meets the set quality conditions.

[0111] According to some embodiments, before determining the image quality of the image to be processed based on the compressed image to be processed, the method may further include: obtaining the position coordinate information of the text region in the reference image; and cropping the reference image and the compressed image to be processed based on the position coordinate information of the text region in the reference image.

[0112] In other words, the position coordinate information of the text region detected in the reference image can be used to crop the reference image (such as a preview image) and the compressed image to be processed (such as a compressed photographed image) to further reduce the size of the image to be processed and improve the efficiency of image processing.

[0113] According to some embodiments, the position coordinate information of the text region in the reference image can be stored (e.g., cached) in a storage device or storage medium. For example, the position coordinate information of the text region in the reference image can be stored simultaneously with the storage of the reference image (original and / or copy). This allows the stored position coordinate information of the text region in the reference image to be directly retrieved from the storage device or storage medium, improving image processing efficiency. According to other embodiments, if the position coordinate information of the text region in the reference image is not stored in the storage device or storage medium, a text box detection operation can be performed on the original or copy of the reference image stored in the storage device or storage medium to obtain the position coordinate information of the text region in the reference image.

[0114] According to some embodiments, the position coordinate information of the text region in the reference image can be the position coordinate information of the rectangle containing the text region (e.g., the smallest rectangle containing the text region, or a rectangle obtained by expanding the smallest rectangle containing the text region upwards, downwards, leftwards, and / or rightwards by a certain factor). The position coordinate information of the rectangle in the reference image may, for example, include the coordinate information of the four vertices of the rectangle, or it may include the coordinate information of any vertex of the rectangle, as well as the height and length information of the rectangle. However, the definition of the position coordinate information of the text region in the reference image is not limited to this, as long as it can represent the spatial position and size occupied by the text region in the reference image.

[0115] According to some embodiments, the cropped reference image can be composed of a rectangle containing the text region (e.g., the smallest rectangle containing the text region, or a rectangle obtained by expanding the smallest rectangle containing the text region upwards, downwards, leftwards, and / or rightwards by a certain factor). The cropped image to be processed can be composed of regions in the compressed image to be processed that correspond to the position coordinates of the rectangle containing the text region (e.g., the smallest rectangle containing the text region, or a rectangle obtained by expanding the smallest rectangle containing the text region upwards, downwards, leftwards, and / or rightwards by a certain factor) in the reference image. In this way, all or most of the regions of both the cropped reference image and the cropped image to be processed are regions where text can be detected, facilitating subsequent image quality determination operations and improving image processing efficiency.

[0116] According to some embodiments, determining the image quality of the image to be processed based on the compressed image may include: determining a text offset between the cropped image to be processed and a cropped reference image, and determining the image quality of the image to be processed based on the text offset; and / or determining the image blur of the cropped image to be processed, and determining the image quality of the image to be processed based on the image blur. Thus, since the determination of the text offset and / or image blur, etc., can be based on a smaller cropped image, the complexity of image processing can be further reduced, and the efficiency of image processing can be improved.

[0117] According to some embodiments, the method may further include: performing text recognition on the image to be processed in response to determining that the image quality of the image to be processed meets a set quality condition. For example, performing text recognition on the image to be processed in response to determining that the text offset between the compressed image to be processed and the reference image is not greater than a set offset threshold, and / or that the image blur of the compressed image to be processed is not greater than a set blur threshold. According to some implementations, various text recognition technologies such as OCR technology in related art can be used in this step. These text recognition technologies can recognize each character in an image and obtain the coordinate information of the character, and can also determine the rectangle containing the text line and its coordinate information.

[0118] According to some embodiments, the method may further include: in response to determining that the image quality of the image to be processed does not meet a set quality condition, determining that text recognition is not required for the image to be processed, and prompting the user to obtain a new image to be processed (or a new image to be processed and a reference image) so as to re-execute the image processing operation described in the embodiments of this disclosure. For example, in response to determining that the text offset between the compressed image to be processed and the reference image is greater than a set offset threshold, and / or that the image blur of the compressed image to be processed is greater than a set blur threshold, determining that text recognition is not required for the image to be processed, and prompting the user to obtain a new image to be processed. This reduces the computational cost of subsequent unnecessary OCR and other text recognition operations, saving image processing time and power consumption.

[0119] According to some embodiments, in response to determining that the image quality of the image to be processed meets a set quality condition, text recognition can be performed on the original image of the image to be processed. Since the original image of the image to be processed is large, has a high number of pixels, and does not have content deviations that may occur due to operations such as copying, the accuracy of text recognition can be improved.

[0120] Although not shown, the text recognition results can be provided to relevant applications for further processing, such as voice playback, text processing, etc., which can be used in applications related to audiobooks and visually impaired assistance applications.

[0121] The above has been combined Figures 1-6 The image processing method of this disclosure has been described by way of example. The following will be combined with... Figure 7 as well as Figures 8A-8E The image processing method of this disclosure will be described in further detail, wherein, Figure 7 A possible overall flowchart of an image processing method according to an exemplary embodiment of the present disclosure is shown. It should be noted that references to the preceding text are... Figures 1-6 The various definitions, embodiments, implementation methods, and examples described herein are also applicable to or can be combined with the exemplary embodiments described below.

[0122] According to some embodiments, such as Figure 7 As shown, the image processing method disclosed herein may include the following steps.

[0123] Step S701: Perform an image preview operation to obtain a preview image.

[0124] According to some implementations, for a given scenario (such as a scenario involving a user holding a book, or other scenarios requiring text recognition), an image sensor (such as a camera or video camera) can perform an image preview operation to obtain a preview image. According to some implementations, the preview image is relatively small, for example, with a relatively low number of pixels, such as 320x240 (in pixels).

[0125] Step S702: Perform text box detection on the preview image and determine whether a text box (i.e., text area) can be detected in the preview image. If a text box can be detected in the preview image, proceed to step S703. Otherwise, the current image processing operation can be terminated or the user can be prompted to reacquire the preview image.

[0126] Figure 8A The illustration shows a preview image in which a text box (i.e., a text region) can be detected according to an exemplary embodiment of the present disclosure, wherein the text region in the preview image may, for example, be presented as the smallest rectangle containing the text region (e.g., ...). Figure 8A The text box is a black rectangle (or the rectangle obtained by expanding the smallest rectangle containing the text area upwards, downwards, leftwards, and / or rightwards by a certain multiple).

[0127] In addition, as mentioned above, the text box detection algorithm used when performing text box detection can be a non-deep learning method, such as a method based on extreme regions (e.g., MSER) ​​or connected regions, to reduce the complexity of text box detection and improve the efficiency of image processing.

[0128] Step S703: Cache the preview image and the position coordinate information of the text box (i.e., the text area) in the preview image.

[0129] According to some implementations, the preview image (original image, or original image and copy thereof) and the position coordinate information of the text box (i.e. text area) in the preview image can be cached in a storage device or storage medium. In this way, the stored preview image and the position coordinate information of the text area in the reference image can be directly obtained from the storage device or storage medium to improve the efficiency of image processing.

[0130] According to some implementations, the position coordinate information of the text region in the reference image may be the position coordinate information of the rectangle containing the text region (e.g., the smallest rectangle containing the text region, or a rectangle obtained by expanding the smallest rectangle containing the text region upwards, downwards, leftwards, and / or rightwards by a certain factor). The position coordinate information of the rectangle in the reference image may, for example, include the coordinate information of the four vertices of the rectangle, or it may include the coordinate information of any vertex of the rectangle, as well as the height and length information of the rectangle, etc.

[0131] Step S704: Perform an image acquisition operation (taking a picture) to obtain the corresponding image to be processed (pictured image) and cache the image to be processed.

[0132] According to some implementation methods, an image sensor can perform image acquisition operations for the aforementioned defined scenario to obtain an image to be processed, and cache the original image or a copy of the original image and the image to be processed in a storage device or storage medium to improve the efficiency of subsequent image processing. Figure 8B This diagram illustrates an image to be processed (photographed image) acquired through a photographing operation according to an exemplary embodiment of the present disclosure. Figure 8B As can be seen, the images to be processed are usually large, for example, the number of pixels is usually high, such as 3264x2448 (in pixels).

[0133] In addition, in comparison Figure 8A as well as Figure 8BAs can be seen, during the image sensor's photo-taking process, the displacement or blur caused by lens or text object shake within the time from exposure to image formation (typically 1-2 seconds) can be reflected in both the preview image before taking the picture and the final image afterward. For example, during the photo-taking process, due to lens or text object shake, the text object... Figure 8B The position in the photograph shown is greater than in Figure 8A The preview image shown has shifted to the left, and the imaged text is also somewhat blurry.

[0134] Step S705: Based on the size of the preview image, compress the image copy of the image to be processed into an image with the same size as the preview image.

[0135] For example, it can be based on Figure 8A The preview image shown is 320x240 pixels. Figure 8B A copy of the image to be processed (3264x2448 pixels) is compressed to 320x240 pixels to obtain the compressed image to be processed. The compressed image to be processed can be as follows: Figure 8C As shown.

[0136] Alternatively, the image copy of the image to be processed may be selected from at least one copy of the image to be processed stored in a storage device or storage medium; or, a copy operation may be performed on the original image of the image to be processed stored in a storage device or storage medium to obtain a copy of the image to be processed.

[0137] Step S706: Based on the cached position coordinate information of the text box (i.e., text area) in the preview image, crop the preview image and the compressed image to be processed.

[0138] According to some implementations, the cropped preview image can be composed of a rectangle containing the text region (e.g., the smallest rectangle containing the text region, or a rectangle obtained by expanding the smallest rectangle containing the text region upwards, downwards, leftwards, and / or rightwards by a certain factor). The cropped image to be processed can be composed of a region in the compressed image to be processed that corresponds to the position coordinate information of the rectangle containing the text region (e.g., the smallest rectangle containing the text region, or a rectangle obtained by expanding the smallest rectangle containing the text region upwards, downwards, leftwards, and / or rightwards by a certain factor) in the reference image. In this way, the preview image and the image to be processed are further reduced in size, and all or most of the regions of the cropped preview image and the cropped image to be processed are regions where text can be detected, which facilitates the subsequent image quality determination operation and improves the efficiency of image processing.

[0139] According to some implementation methods, the position coordinate information of the text box (i.e., the text area) cached in step S703 can be used to respectively... Figure 8A The preview image shown and a copy of the image to be processed after compression in step S705 are cropped to obtain... Figure 8D The cropped preview image shown and Figure 8E The image shown is the cropped image to be processed.

[0140] Step S707: Compare the cropped image to be processed with the cropped preview image to determine the text offset between the two, and determine whether the text offset is not greater than the offset threshold. If so, proceed to step S708; otherwise, it can be determined that there is no need to perform text recognition on the image to be processed, and the user can be prompted to obtain a new image to be processed.

[0141] According to some implementation methods, the following can be adopted: Figure 4 The grayscale projection method and other text offset detection algorithms shown determine the text offset between the cropped image to be processed and the cropped preview image. If the text offset is greater than the offset threshold, it can be determined that the image displacement is too large (i.e., the text offset is too large), and therefore text recognition does not need to be performed on the image to be processed, and it can be displayed to the user; otherwise, the process can proceed to the next step. In this way, for images that do not pass the quality pre-assessment, OCR and other text recognition operations will not be performed, thereby reducing the computational cost of unnecessary OCR and other text recognition operations and saving image processing time and power consumption.

[0142] For example, for Figure 8D and Figure 8E In the example shown, if calculations show that the text offset between the cropped image to be processed and the cropped preview image is greater than the offset threshold, then the image displacement is considered too large, and text recognition is unnecessary for the image to be processed. Alternatively, the user can be prompted to acquire a new image to be processed, and the above steps can be repeated based on the new image. Of course, the image processing operation can also be terminated directly after determining that the text offset is greater than the offset threshold.

[0143] Step S708: Perform blur calculation on the cropped image to be processed to determine the image blur of the cropped image to be processed, and determine whether the image blur is not greater than the blur threshold. If so, proceed to step S709. Otherwise, it can be determined that there is no need to perform text recognition on the image to be processed, and the user can be prompted to obtain a new image to be processed.

[0144] As described in the foregoing embodiments, the following methods can be used: Figure 5The Laplacian variance algorithm and other image blur detection algorithms shown determine the image blur of the cropped image to be processed. Using this Laplacian variance algorithm to calculate image blur can filter out pure color regions with excessively small Laplacian standard deviations, thus obtaining more accurate image blur detection results. This solves the problem in related technologies where the extremely small dispersion of pure color regions in an image lowers the calculated Laplacian standard deviation for the entire image, causing the Laplacian variance algorithm to fail for pure color regions. Furthermore, Figure 5 The method shown, which uses the Laplacian variance algorithm to determine image blur, is a method that can calculate the image blur of the image to be processed without considering the reference image. Therefore, it can further improve the efficiency of image processing and save image processing time.

[0145] According to some implementation methods, if the image blurriness is greater than a blurriness threshold, it can be determined that the image is blurry (i.e., the clarity is very low), and therefore there is no need to perform text recognition on the image to be processed, and it can be displayed to the user; otherwise, the next step can be performed. In this way, for images that do not pass the quality pre-assessment, text recognition operations such as OCR will not be performed, thereby reducing the computational consumption caused by unnecessary text recognition operations such as OCR, and saving image processing time and power consumption.

[0146] For example, for Figure 8D and Figure 8E In the example shown, if calculations show that the blurriness of the cropped image is greater than the blurriness threshold, the image can be determined to be blurry, and text recognition is not required. Alternatively, the user can be prompted to acquire a new image and repeat the above steps based on the new image. Of course, the image processing operation can also be terminated directly after determining that the blurriness of the cropped image is greater than the blurriness threshold.

[0147] Step S709: Perform text recognition on the image to be processed.

[0148] According to some implementation methods, text recognition can be performed on the original image of the image to be processed. Since the original image of the image to be processed is relatively large, for example, has a high number of pixels, and there is no content deviation that may occur due to operations such as copying, the accuracy of text recognition can be improved.

[0149] Although not shown, the text recognition results can be provided to relevant applications for further processing, such as voice playback, text processing, etc., which can be used in applications related to audiobooks and visually impaired assistance applications.

[0150] As described above, in the embodiments of this disclosure, a smaller preview image can be used as a reference image. A relatively large captured image is compressed to the same size as the preview image. The preview image and the compressed captured image are cropped based on the position coordinates of the text boxes detected in the preview image. The difference between the two after cropping is compared to determine the degree of text offset. Furthermore, the image blur is calculated using the cropped captured image, and the quality of the captured image is evaluated by combining the degree of text offset and the image blur. This significantly reduces the computational load required for image processing while ensuring the accuracy of image quality assessment, achieving a relatively accurate OCR image quality pre-assessment at a relatively low cost. Testing shows that the image quality assessment process for captured images can reach the millisecond level, and the accuracy of image quality assessment can reach over 90%.

[0151] Exemplary methods for image processing according to the present disclosure have been described above with reference to the accompanying drawings. Exemplary apparatuses for image processing according to the present disclosure, as well as exemplary embodiments of electronic devices, will now be further described with reference to the accompanying drawings.

[0152] Figure 9 This is a block diagram illustrating an example of an electronic device according to exemplary embodiments of the present disclosure. It should be noted that... Figure 9 The structure shown is merely an example; depending on the specific implementation, the electronic device disclosed herein may include only [specific components]. Figure 9 One or more of the components shown.

[0153] Electronic device 2000 may be, for example, a general-purpose computer (such as a laptop computer, tablet computer, and various other computers), a mobile phone, or a personal digital assistant. According to some embodiments, electronic device 2000 may be an assistive device for the visually impaired.

[0154] The electronic device 2000 may be configured to capture an image, process the captured image, and provide an audio prompt in response to the data obtained from the processing. For example, the electronic device 2000 may be configured to capture an image, perform text detection and / or recognition on the image to obtain text data, convert the text data into audio data, and output the audio data for the user to listen to.

[0155] According to some embodiments, the electronic device 2000 may be configured to include an eyeglass frame or be configured to be detachably mounted to an eyeglass frame (e.g., the frame of the eyeglass frame, a connector connecting two frames, temples, or any other part) so as to capture an image that approximately includes the user's field of vision.

[0156] According to some embodiments, the electronic device 2000 can also be mounted on or integrated with other wearable devices. The wearable device may be, for example, a head-mounted device (e.g., a helmet or hat), or a device that can be worn on the ears. According to some embodiments, the electronic device can be implemented as an accessory that can be attached to a wearable device, for example, as an accessory that can be attached to a helmet or hat.

[0157] According to some embodiments, the electronic device 2000 may also have other forms. For example, the electronic device 2000 may be a mobile phone, a general-purpose computing device (e.g., a laptop computer, a tablet computer, etc.), a personal digital assistant, etc. The electronic device 2000 may also have a stand, so that it can be placed on a desktop.

[0158] According to some embodiments, the electronic device 2000 can be used as an assistive device for visually impaired individuals to aid reading. In this case, the electronic device 2000 is sometimes also referred to as an "electronic reader" or a "reading assistance device." With the aid of the electronic device 2000, users who cannot read independently (such as visually impaired individuals, individuals with dyslexia, etc.) can "read" conventional reading materials (such as books, magazines, etc.) by adopting a posture similar to reading postures. During the "reading" process, the electronic device 2000 can capture images according to the methods described in this disclosure, and process the captured images according to the methods described in this disclosure, including compression processing, image quality assessment processing, and character recognition (e.g., using optical character recognition (OCR) methods). In addition, the electronic device 2000 can also convert the character recognition results into sound data and output the sound data for the user to listen to through a sound output device such as a speaker or headphones.

[0159] Electronic device 2000 may include camera 2004 for acquiring images. Camera 2004 may include, but is not limited to, a webcam or camera. Electronic device 2000 may also include text recognition circuit 2005, configured to perform text detection and / or recognition (e.g., OCR processing) on ​​text contained in the image to obtain text data. The text recognition circuit 2005 may be implemented, for example, using a dedicated chip. Electronic device 2000 may also include sound conversion circuit 2006, configured to convert the text data into sound data. The sound conversion circuit 2006 may be implemented, for example, using a dedicated chip. Electronic device 2000 may also include sound output circuit 2007, configured to output the sound data. The sound output circuit 2007 may include, but is not limited to, headphones, speakers, or vibrators, and their corresponding driving circuits. Electronic device 2000 may also include electronic circuit 2100, which includes steps configured to perform the image processing method as described above (e.g., Figure 1 , Figure 4 , Figure 5 , Figure 6 ,or Figure 7 The circuit (as shown in the flowchart).

[0160] According to some embodiments, the electronic device 2000 may further include an image processing circuit 2008, which may include circuitry configured to perform various image processing operations on the image. The image processing circuit 2008 may include, for example, one or more of the following: circuitry configured to denoise the image; circuitry configured to deblur the image; circuitry configured to perform geometric correction on the image; circuitry configured to extract features from the image; circuitry configured to perform object detection and / or recognition on objects in the image; circuitry configured to perform text detection on text contained in the image; circuitry configured to extract text lines from the image; circuitry configured to extract text coordinates from the image; circuitry configured to extract object boxes from the image; circuitry configured to extract text boxes from the image; circuitry configured to perform layout analysis (e.g., paragraph segmentation) based on the image; and so on.

[0161] According to some embodiments, the electronic device 2000 may also include a text processing circuit 2009, which may be configured to perform various processing based on extracted text-related information (such as text data, text boxes, paragraph coordinates, text line coordinates, text coordinates, etc.) to obtain processing results such as paragraph sorting, text semantic analysis, and layout analysis results.

[0162] One or more of the various circuits described above (e.g., character recognition circuit 2005, voice conversion circuit 2006, voice output circuit 2007, image processing circuit 2008, character processing circuit 2009, electronic circuit 2100) may be implemented using custom hardware, and / or using hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. For example, one or more of the various circuits described above may be implemented by programming hardware (e.g., programmable logic circuits including field-programmable gate arrays (FPGAs) and / or programmable logic arrays (PLAs)) using assembly language or hardware programming languages ​​(such as Verilog, VHDL, C++) according to the logic and algorithms of this disclosure.

[0163] According to some embodiments, the electronic device 2000 may also include a communication circuit 2010, which may be any type of device or system that enables communication with external devices and / or with a network, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication devices and / or chipsets, such as Bluetooth devices, 1302.11 devices, WiFi devices, WiMax devices, cellular communication devices and / or the like.

[0164] According to some embodiments, the electronic device 2000 may also include an input device 2011, which may be any type of device capable of inputting information to the electronic device 2000, and may include, but is not limited to, various sensors, mice, keyboards, touch screens, buttons, joysticks, microphones and / or remote controls, etc.

[0165] According to some embodiments, the electronic device 2000 may also include an output device 2012, which may be any type of device capable of presenting information, and may include, but is not limited to, a display, a visual output terminal, a vibrator, and / or a printer. Although the electronic device 2000 is used in some embodiments for assistive devices for the visually impaired, the vision-based output device can facilitate the user's family members or maintenance personnel to obtain output information from the electronic device 2000.

[0166] According to some embodiments, the electronic device 2000 may further include a processor 2001. The processor 2001 may be any type of processor and may include, but is not limited to, one or more general-purpose processors and / or one or more dedicated processors (e.g., special-purpose chips). The processor 2001 may be, for example, but not limited to, a central processing unit (CPU) or a microprocessor (MPU), etc. The electronic device 2000 may also include a working memory 2002, which may store programs (including instructions) and / or data (e.g., images, text, sound, and other intermediate data) useful to the operation of the processor 2001, and may include, but is not limited to, random access memory and / or read-only memory devices. Electronic device 2000 may further include storage device 2003, which may include any non-transitory storage device. A non-transitory storage device can be any storage device that is non-transitory and capable of storing data, and may include, but is not limited to, disk drives, optical storage devices, solid-state storage, floppy disks, flexible disks, hard disks, magnetic tapes or any other magnetic media, optical discs or any other optical media, ROM (read-only memory), RAM (random access memory), cache memory and / or any other memory chip or cartridge, and / or any other medium from which a computer can read data, instructions, and / or code. Working memory 2002 and storage device 2003 may be collectively referred to as "memory" and in some cases may be used interchangeably.

[0167] According to some embodiments, the processor 2001 can control and schedule at least one of the camera 2004, the character recognition circuit 2005, the voice conversion circuit 2006, the voice output circuit 2007, the image processing circuit 2008, the character processing circuit 2009, the communication circuit 2010, the electronic circuit 2100, and other various devices and circuits included in the electronic device 2000. According to some embodiments, Figure 9 At least some of the components described herein may be interconnected and / or communicate with each other via bus 2013.

[0168] Software elements (programs) may be located in the working memory 2002, including but not limited to operating system 2002a, one or more application programs 2002b, drivers and / or other data and code.

[0169] According to some implementations, the instructions for performing the aforementioned control and scheduling may be included in the operating system 2002a or one or more applications 2002b.

[0170] According to some implementations, the steps of the method described in this disclosure are performed (e.g.) Figure 1 , Figure 4 , Figure 5 , Figure 6 ,or Figure 7 The instructions for the method steps shown in the flowcharts (e.g., etc.) can be included in one or more application programs 2002b, and the various modules of the electronic device 2000 described above can be implemented by the processor 2001 reading and executing the instructions of one or more application programs 2002b. In other words, the electronic device 2000 may include a processor 2001 and a memory storing a program (e.g., working memory 2002 and / or storage device 2003), the program including instructions that, when executed by the processor 2001, cause the processor 2001 to perform the methods described in the various embodiments of this disclosure.

[0171] According to some implementations, some or all of the operations performed by at least one of the character recognition circuit 2005, voice conversion circuit 2006, image processing circuit 2008, character processing circuit 2009, and electronic circuit 2100 can be implemented by the processor 2001 reading and executing instructions from one or more application programs 2002.

[0172] The executable code or source code of the instructions of the software element (program) may be stored in a non-transitory computer-readable storage medium (e.g., the storage device 2003) and may be stored in the working memory 2001 during execution (possibly compiled and / or installed). Therefore, this disclosure provides a computer-readable storage medium storing a program comprising instructions that, when executed by a processor of an electronic device (e.g., a visually impaired assistive device), cause the electronic device to perform the methods described in various embodiments of this disclosure. According to another embodiment, the executable code or source code of the instructions of the software element (program) may also be downloaded from a remote location.

[0173] It should also be understood that various modifications can be made depending on specific requirements. For example, custom hardware can be used, and / or the individual circuits, units, modules, or elements can be implemented using hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. For example, some or all of the circuits, units, modules, or elements included in the disclosed methods and apparatus can be implemented by programming the hardware (e.g., programmable logic circuits including field-programmable gate arrays (FPGAs) and / or programmable logic arrays (PLAs)) using the logic and algorithms according to this disclosure in assembly language or hardware programming languages ​​(such as Verilog, VHDL, C++).

[0174] According to some implementations, the processor 2001 in the electronic device 2000 can be distributed across a network. For example, one processor can perform some processing, while another processor located far away can perform other processing simultaneously. Other modules of the electronic device 2001 can also be distributed similarly. In this way, the electronic device 2001 can be interpreted as a distributed computing system that performs processing in multiple locations.

[0175] The following describes some exemplary aspects of this disclosure.

[0176] Aspect 1. An image processing method, comprising:

[0177] Obtain the image to be processed and a reference image of the image to be processed, wherein the size of the reference image is smaller than the size of the image to be processed;

[0178] The image to be processed is compressed according to the size of the reference image; and

[0179] The image quality of the image to be processed is determined based on the compressed image.

[0180] Aspect 2. The image processing method as described in Aspect 1, wherein the reference image and the image to be processed are images of the same scene acquired at different times, and a text region can be detected in the reference image.

[0181] Aspect 3. The image processing method as described in Aspect 2, wherein the reference image and the image to be processed are a preview image and a captured image of the same scene, respectively, acquired in the image preview stage and the image capture stage.

[0182] Aspect 4. The image processing method as described in Aspect 2, wherein determining the image quality of the image to be processed based on the compressed image to be processed includes any one or a combination of the following steps:

[0183] Determine the text offset between the compressed image to be processed and the reference image, and determine the image quality of the image to be processed based on the text offset;

[0184] The image blur of the compressed image to be processed is determined, and the image quality of the image to be processed is determined based on the image blur.

[0185] Aspect 5. The image processing method as described in aspect 4, wherein,

[0186] Determining the image quality of the image to be processed based on the text offset includes:

[0187] In response to the text offset not being greater than the offset threshold, it is determined that the image quality of the image to be processed meets the set quality conditions;

[0188] Determining the image quality of the image to be processed based on the image blurriness includes:

[0189] In response to the image blurring not being greater than a blurring threshold, it is determined that the image quality of the image to be processed meets the set quality conditions.

[0190] Aspect 6. The image processing method as described in Aspect 2, wherein determining the image quality of the image to be processed based on the compressed image to be processed includes:

[0191] Determine the text offset between the compressed image to be processed and the reference image;

[0192] In response to the text offset not being greater than an offset threshold, the image blur of the compressed image to be processed is determined; and

[0193] In response to the image blurring not being greater than a blurring threshold, it is determined that the image quality of the image to be processed meets the set quality conditions.

[0194] Aspect 7. The image processing method as described in Aspect 2, wherein determining the image quality of the image to be processed based on the compressed image to be processed includes:

[0195] Determine the image blur of the compressed image to be processed;

[0196] In response to the image blurring not exceeding a blurring threshold, a text offset between the compressed image to be processed and the reference image is determined; and

[0197] In response to the text offset not being greater than the offset threshold, it is determined that the image quality of the image to be processed meets the set quality conditions.

[0198] Aspect 8. The image processing method as described in Aspect 4, wherein determining the text offset between the compressed image to be processed and the reference image includes:

[0199] Calculate the sum of all pixels in each pixel row and each pixel column of the compressed image to be processed and the reference image;

[0200] Based on the calculated sum of all pixels in each row and each column of the compressed image to be processed and the reference image, the row grayscale projection and column grayscale projection of the compressed image to be processed and the reference image are determined; and

[0201] Based on the row and column grayscale projections of the compressed image to be processed and the reference image, determine the vertical displacement of the matching rows and the horizontal displacement of the matching columns in the compressed image to be processed and the reference image.

[0202] Aspect 9. The image processing method as described in Aspect 4, wherein determining the image blur of the compressed image to be processed includes:

[0203] The compressed image to be processed is divided into multiple image blocks, and the Laplacian standard deviation of each image block is calculated based on the pixel values ​​of the pixels in each image block.

[0204] The calculated Laplacian standard deviation of each image patch is compared with a standard deviation threshold, and the first image patch whose Laplacian standard deviation is greater than the standard deviation threshold is selected; and

[0205] The image blur of the compressed image to be processed is determined based on the Laplacian standard deviation of the selected first image patch.

[0206] Aspect 10. The image processing method as described in aspect 9 further includes:

[0207] Select a second image patch whose absolute value of the difference between the corresponding Laplacian standard deviation and the standard deviation threshold is not greater than the difference threshold; and

[0208] The second image patch is subjected to target detection processing based on a first target detection neural network and / or a second target detection neural network, and the standard deviation threshold is adjusted according to the target detection results of the second image patch by the first target detection neural network and / or the second target detection neural network.

[0209] The first target that the first target detection neural network can detect is different from the second target that the second target detection neural network can detect.

[0210] Aspect 11. The image processing method as described in aspect 9 further includes:

[0211] Select a second image patch whose absolute value of the difference between the corresponding Laplacian standard deviation and the standard deviation threshold is not greater than the difference threshold;

[0212] The second image patch is processed for target detection based on a first target detection neural network; and

[0213] In response to the detection of a first target in the second image block, the standard deviation threshold is down-adjusted.

[0214] Aspect 12. The image processing method as described in aspect 9 further includes:

[0215] Select a second image patch whose absolute value of the difference between the corresponding Laplacian standard deviation and the standard deviation threshold is not greater than the difference threshold;

[0216] The second image block is processed for target detection based on the first target detection neural network.

[0217] In response to the inability to detect the first target in the second image patch, target detection processing is performed on the second image patch based on a second target detection neural network; and

[0218] In response to the detection of a second target in the second image patch and the Laplacian standard deviation of the second image patch being greater than the standard deviation threshold, the standard deviation threshold is increased.

[0219] The first target that the first target detection neural network can detect is different from the second target that the second target detection neural network can detect.

[0220] Aspect 13. The image processing method as described in Aspect 2, wherein, before determining the image quality of the image to be processed based on the compressed image to be processed, the method further comprises:

[0221] Obtain the position coordinate information of the text region in the reference image; and

[0222] Based on the position coordinates of the text region in the reference image, the reference image and the compressed image to be processed are cropped.

[0223] Aspect 14. The image processing method as described in aspect 13, wherein the position coordinate information of the text region in the reference image is the position coordinate information of the rectangle containing the text region in the reference image, the position coordinate information of the rectangle in the reference image including the coordinate information of the four vertices of the rectangle, or the coordinate information of any vertex of the rectangle, as well as the height information and length information of the rectangle.

[0224] Aspect 15. The image processing method as described in aspect 1 further includes:

[0225] In response to determining that the image quality of the image to be processed meets the set quality conditions based on the compressed image to be processed, text recognition is performed on the image to be processed.

[0226] Aspect 16. The image processing method as described in aspect 1 further includes:

[0227] In response to determining that the image quality of the image to be processed does not meet the set quality conditions based on the compressed image to be processed, it is determined that there is no need to perform text recognition on the image to be processed, and the user is prompted to obtain a new image to be processed.

[0228] Aspect 17. An image processing method, comprising:

[0229] The image to be processed is acquired and divided into multiple image blocks;

[0230] Calculate the Laplacian standard deviation of each image block based on the pixel values ​​of the pixels in each image block;

[0231] The calculated Laplacian standard deviation of each image patch is compared with a standard deviation threshold, and the first image patch whose Laplacian standard deviation is greater than the standard deviation threshold is selected; and

[0232] The image blur of the image to be processed is determined based on the Laplacian standard deviation of the selected first image patch.

[0233] Aspect 18. The image processing method as described in aspect 17 further includes:

[0234] Select a second image patch whose absolute value of the difference between the corresponding Laplacian standard deviation and the standard deviation threshold is not greater than the difference threshold; and

[0235] The second image patch is subjected to target detection processing based on a first target detection neural network and / or a second target detection neural network, and the standard deviation threshold is adjusted according to the target detection results of the second image patch by the first target detection neural network and / or the second target detection neural network.

[0236] The first target that the first target detection neural network can detect is different from the second target that the second target detection neural network can detect.

[0237] Aspect 19. The image processing method as described in aspect 17 further includes:

[0238] Select a second image patch whose absolute value of the difference between the corresponding Laplacian standard deviation and the standard deviation threshold is not greater than the difference threshold;

[0239] The second image patch is processed for target detection based on a first target detection neural network; and

[0240] In response to the detection of a first target in the second image block, the standard deviation threshold is down-adjusted.

[0241] Aspect 20. The image processing method as described in aspect 17 further includes:

[0242] Select a second image patch whose absolute value of the difference between the corresponding Laplacian standard deviation and the standard deviation threshold is not greater than the difference threshold;

[0243] The second image block is processed for target detection based on the first target detection neural network.

[0244] In response to the inability to detect the first target in the second image patch, target detection processing is performed on the second image patch based on a second target detection neural network; and

[0245] In response to the detection of a second target in the second image patch and the Laplacian standard deviation of the second image patch being greater than the standard deviation threshold, the standard deviation threshold is increased.

[0246] The first target that the first target detection neural network can detect is different from the second target that the second target detection neural network can detect.

[0247] Aspect 21. The image processing method as described in aspect 17, wherein the image to be processed is an image obtained by compressing the first image according to the size of a reference image of the first image, wherein the size of the reference image is smaller than the size of the first image.

[0248] Aspect 22. The image processing method as described in aspect 21, wherein the reference image and the first image are images of the same scene acquired at different times, and a text region can be detected in the reference image.

[0249] Aspect 23. The image processing method as described in aspect 22, wherein the reference image and the first image are a preview image and a captured image of the same scene, respectively, acquired in the image preview stage and the image capture stage.

[0250] Aspect 24. The image processing method as described in aspect 17 further includes:

[0251] In response to determining that the image quality of the image to be processed meets the set quality conditions based on the image blur of the image to be processed, text recognition is performed on the image to be processed.

[0252] Aspect 25. The image processing method as described in aspect 17 further includes:

[0253] Determine the text offset between the image to be processed and a reference image of the image to be processed; and

[0254] In response to determining that the image quality of the image to be processed meets a set quality condition based on the image blur of the image to be processed and the text offset between the image to be processed and the reference image, text recognition is performed on the image to be processed.

[0255] Aspect 26. An electronic circuit, comprising:

[0256] A circuit configured to perform the steps of the method described according to any one of aspects 1-16, or a circuit configured to perform the steps of the method described according to any one of aspects 17-25.

[0257] Section 27. An assistive device for the visually impaired, comprising:

[0258] The camera is configured to acquire images; and

[0259] The electronic circuit as described in aspect 26.

[0260] Aspect 28. The assistive device for the visually impaired as described in aspect 27 further includes:

[0261] A circuit configured to perform text detection and / or recognition on text contained in the image to obtain text data;

[0262] Circuitry configured to convert the text data into sound data; and

[0263] The circuit is configured to output the sound data.

[0264] Aspect 29. An electronic device comprising:

[0265] Processor; and

[0266] A memory storing a program, the program comprising instructions that, when executed by the processor, cause the processor to perform the method according to any one of aspects 1-16, or cause the processor to perform the method according to any one of aspects 17-25.

[0267] Aspect 30. A computer-readable storage medium storing a program, the program comprising instructions that, when executed by a processor of an electronic device, cause the electronic device to perform the method according to any one of aspects 1-16, or cause the electronic device to perform the method according to any one of aspects 17-25.

[0268] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the methods, systems, and devices described above are merely exemplary embodiments or examples, and the scope of the invention is not limited by these embodiments or examples, but only by the granted claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as the technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure.

Claims

1. An image processing method comprising: obtaining a to-be-processed image, and dividing the to-be-processed image into a plurality of image blocks; calculating Laplacian standard deviations of the image blocks according to pixel values of pixel points in the image blocks; comparing the calculated Laplacian standard deviations of the image blocks with a standard deviation threshold, and selecting first image blocks corresponding to the Laplacian standard deviations greater than the standard deviation threshold; and determining image blurriness of the to-be-processed image based on the Laplacian standard deviations of the selected first image blocks; determining a text offset between the to-be-processed image and a reference image of the to-be-processed image; and in response to determining that image quality of the to-be-processed image meets a set quality condition according to the image blurriness of the to-be-processed image and the text offset between the to-be-processed image and the reference image, performing character recognition on the to-be-processed image.

2. The image processing method of claim 1, further comprising: selecting second image blocks corresponding to absolute values of differences between the Laplacian standard deviations and the standard deviation threshold not greater than a difference threshold; and performing target detection processing on the second image blocks based on a first target detection neural network and / or a second target detection neural network, and adjusting the standard deviation threshold according to target detection results of the second image blocks by the first target detection neural network and / or the second target detection neural network, wherein a first target detectable by the first target detection neural network and a second target detectable by the second target detection neural network are different from each other.

3. The image processing method of claim 1, further comprising: selecting second image blocks corresponding to absolute values of differences between the Laplacian standard deviations and the standard deviation threshold not greater than a difference threshold; performing target detection processing on the second image blocks based on a first target detection neural network; and in response to detecting a first target in the second image blocks, performing down-regulation processing on the standard deviation threshold.

4. The image processing method of claim 1, further comprising: selecting second image blocks corresponding to absolute values of differences between the Laplacian standard deviations and the standard deviation threshold not greater than a difference threshold; performing target detection processing on the second image blocks based on a first target detection neural network; in response to failing to detect a first target in the second image blocks, performing target detection processing on the second image blocks based on a second target detection neural network; and in response to detecting a second target in the second image blocks and the Laplacian standard deviation of the second image blocks being greater than the standard deviation threshold, performing up-regulation processing on the standard deviation threshold, wherein the first target detectable by the first target detection neural network and the second target detectable by the second target detection neural network are different from each other. The to-be-processed image is an image obtained by performing compression processing on a first image according to a size of a reference image of the first image, and the size of the reference image is smaller than the size of the first image. ​ ​ ​ 5. The image processing method of claim 1, wherein, ​ 6. The image processing method of claim 5, wherein, The reference image and the first image are images of the same scene acquired at different time instants, the reference image being capable of detecting a text region therein, and wherein the reference image and the first image are respectively a preview image and a shot image of the same scene acquired at an image preview stage and an image shot stage.

7. The image processing method of claim 1, further comprising: performing text recognition on the to-be-processed image in response to determining that the image quality of the to-be-processed image satisfies a set quality condition according to the image blurriness of the to-be-processed image.

8. An electronic circuit comprising: circuitry configured to perform the steps of the method of any one of claims 1-7.

9. A visual impairment assistance device comprising: a camera configured to acquire an image; and the electronic circuit of claim 8.

10. The visual impairment assistance device of claim 9, further comprising: circuitry configured to perform text detection and / or recognition on text contained in the image to obtain text data; circuitry configured to convert the text data into sound data; and circuitry configured to output the sound data.

11. An electronic device comprising: a processor; and a memory storing a program, the program comprising instructions that, when executed by the processor, cause the processor to perform the method of any one of claims 1-7.

12. A computer-readable storage medium storing a program, the program comprising instructions that, when executed by a processor of an electronic device, cause the electronic device to perform the method of any one of claims 1-7. ​ ​

Images