A mobile terminal text recognition method and system based on neural network

Abstract

The present application relates to the field of character recognition technology, and specifically to a method and system for mobile terminal text recognition based on a neural network. The method comprises: obtaining a grayscale image of a nameplate of an electric power facility; determining a brightness adjustment factor for each pixel in each area; determining a size adjustment factor for each pixel in each area based on the number of all pixels in the neighborhood of each pixel in each area and the distance from each pixel to all pixels in its neighborhood, combined with the size of the grayscale image of the nameplate in each area; determining an adjustment factor for each pixel in each area based on the brightness adjustment factor and size adjustment factor of each pixel in each area, so as to obtain a text image and recognize the text therein. The present application adjusts the segmentation threshold based on the nameplate image features to obtain accurate segmentation results, thereby improving the accuracy of text recognition by the neural network.

Description

Technical Field

[0001] The present application relates to the field of character recognition technology, and in particular to a method and system for mobile terminal text recognition based on a neural network. Background Art

[0002] In the power industry, safety inspections and equipment maintenance are critical to ensuring stable grid operation. During inspections, equipment information on nameplates of power facilities must be entered. Manual identification and entry is not only cumbersome but also prone to errors. An existing solution involves having workers use mobile devices to capture images of nameplates, then using a neural network for visual inspection to recognize the text in the images and enter the results, significantly improving efficiency and accuracy.

[0003] When workers use mobile devices to capture images outdoors or in complex indoor environments, they often face suboptimal lighting conditions. This lighting variation can negatively impact image quality, causing changes in the brightness of text pixels within the image. This can lead to inaccurate pixel clustering and segmentation, causing errors in neural network text recognition and compromising accuracy. Summary of the Invention

[0004] In order to solve the above technical problems, the purpose of this application is to provide a method and system for mobile terminal text recognition based on neural network. The technical solutions adopted are as follows:

[0005] In a first aspect, an embodiment of the present application provides a method for text recognition in a mobile terminal based on a neural network, the method comprising the following steps:

[0006] Obtain grayscale images of nameplates of power facilities under different light intensities;

[0007] The grayscale image of each nameplate is segmented into multiple regions. Based on the degree of change in the grayscale value of each pixel in each region and the degree of dispersion of the grayscale values ​​of all pixels in the neighborhood of each pixel, the grayscale value of each pixel in each region and the average distribution of the grayscale values ​​of all pixels in each region, the brightness adjustment factor of each pixel in each region is determined.

[0008] Based on the number of all pixels in the neighborhood of each pixel in each area and the distance between each pixel and all pixels in its neighborhood, and the size of the grayscale image of the nameplate in each area, the size adjustment factor of each pixel in each area is determined;

[0009] An adjustment factor for each pixel in each area is determined based on a brightness adjustment factor and a size adjustment factor for each pixel in each area, so as to recognize text in a grayscale image of the nameplate.

[0010] Preferably, the brightness adjustment factor of each pixel in each area is determined by:

[0011] Determine the grayscale change weight of each pixel in each area based on the grayscale value change degree of each pixel in each area and the discrete degree of the grayscale values ​​of all pixels in the neighborhood of each pixel;

[0012] Based on the average distribution of the grayscale values ​​of all pixels in each area and the grayscale change weights of all pixels, the average grayscale weight of each area is determined;

[0013] The brightness adjustment factor of each pixel in each area is the result of fusing the grayscale value of each pixel in each area with the average grayscale weight.

[0014] Preferably, the grayscale change weight of each pixel in each region is a result of fusion of the gradient of each pixel in each region and the variance of the grayscale values ​​of all pixels in its neighborhood.

[0015] Preferably, the method for determining the average grayscale weight of each region is:

[0016] Analyze the product of the grayscale change weight of each pixel in each area and the mean grayscale value of all pixels in the corresponding area, and record it as the grayscale weight of each pixel in each area;

[0017] The average grayscale weight of each pixel in each region is the mean of the grayscale weights of all pixels in each region.

[0018] Preferably, the method for determining the size adjustment factor of each pixel in each area is:

[0019] Determine the distance between each pixel in each region based on the number of all pixels in the neighborhood of each pixel and the distance between each pixel and all pixels in its neighborhood.

[0020] Determine the resolution weight of the grayscale image of the nameplate of each area based on the size of the grayscale image of the nameplate of each area;

[0021] A size adjustment factor for each pixel in each area is determined based on the distance size and the resolution weight.

[0022] Preferably, the method for determining the distance between the pixels in each area is:

[0023] Analyze the cumulative sum of the distances from each pixel in each region to all pixels in its neighborhood, and record it as the first distance of each pixel in each region;

[0024] The distance size between the pixels in each area is the ratio of the number of all pixels in the neighborhood of the pixel in each area to the first distance.

[0025] Preferably, the expression for the resolution weight of the grayscale image of the nameplate in each area is: Where k i represents the resolution weight of the grayscale image of the nameplate in the i-th area; h i represents the height of the grayscale image of the nameplate in the i-th area; w i It represents the width of the grayscale image of the nameplate in the i-th area; norm() represents the normalization function; min() represents the minimum function.

[0026] Preferably, the method for determining the size adjustment factor of each pixel in each area is:

[0027] The threshold segmentation algorithm is used to perform binary processing on the grayscale image of the nameplate. The pixels with a grayscale value of 1 are recorded as candidate text pixels, and the pixels with a grayscale value of 0 are recorded as non-candidate text pixels.

[0028] The expression for the size adjustment factor of each pixel in each area is: Where λ i,j k represents the size adjustment factor of the jth pixel in the i-th region; i represents the resolution weight of the grayscale image of the nameplate in the i-th area; l i,j Indicates the distance size of the jth pixel in the i-th region.

[0029] Preferably, determining the adjustment factor of each pixel in each area to recognize the text in the grayscale image of the nameplate includes:

[0030] The adjustment factor of each pixel in each area is the normalized product of the brightness adjustment factor and the size adjustment factor of each pixel in each area;

[0031] The product of the adjustment factor of each pixel point in each area and the threshold value in the binarization process of the nameplate grayscale image is used as the segmentation threshold value of each pixel point in each area;

[0032] The pixels in each area whose grayscale value is less than the corresponding segmentation threshold are text pixels. The coordinates and grayscale values ​​of each text pixel are combined into a vector. The vectors of all text pixel points in the grayscale image of the nameplate are clustered to obtain clusters. Each cluster corresponds to a text, and the text is recognized by combining the neural network.

[0033] In a second aspect, an embodiment of the present application also provides a mobile terminal text recognition system based on a neural network, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of any one of the above methods when executing the computer program.

[0034] This application has at least the following beneficial effects:

[0035] This application determines the brightness adjustment factor for each pixel in each area based on the degree of change in the grayscale value of each pixel in each area and the degree of dispersion of the grayscale values ​​of all pixels in the neighborhood of each pixel, combined with the grayscale value of each pixel in each area and the average distribution of the grayscale values ​​of all pixels. This reflects the degree of change in light intensity and also adjusts the pixels in the nameplate image, so that the nameplate image can be analyzed and processed more accurately under different lighting conditions.

[0036] This application determines the size adjustment factor of each pixel in each area based on the number of all pixels in the neighborhood of each pixel in each area and the distance between each pixel and all pixels in its neighborhood, combined with the size of the grayscale image of the nameplate in each area, adjusts the segmentation threshold to distinguish between text pixels and non-text pixels, uses a clustering algorithm to cluster the text pixels to complete the segmentation of the image, and combines a neural network to recognize the segmented text image.

[0037] This application adjusts the segmentation threshold based on the nameplate image features to obtain more accurate segmentation results, reduces the impact of strong light on text segmentation of power facility information nameplate images, and significantly improves the accuracy of neural network text recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0039] Figure 1 A flowchart of a method for text recognition in a mobile terminal based on a neural network;

[0040] Figure 2 Schematic diagram of the brightness adjustment factor extraction process;

[0041] Figure 3 Schematic diagram for adjusting factor extraction;

[0042] Figure 4 This is a flowchart of the text segmentation and recognition process;

[0043] Figure 5 Schematic diagram of the text recognition process based on neural network. DETAILED DESCRIPTION

[0044] In order to further illustrate the technical means and effects adopted by this application to achieve the predetermined invention purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail a method and system for mobile terminal text recognition based on a neural network proposed in this application, its specific implementation, structure, features and effects. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0045] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0046] The following describes in detail a specific scheme of a mobile terminal text recognition method and system based on a neural network provided by the present application with reference to the accompanying drawings.

[0047] See also Figure 1 , which shows a flowchart of a method for text recognition in a mobile terminal based on a neural network according to an embodiment of the present application, the method comprising the following steps:

[0048] Step S1: Use a mobile terminal to collect images of power facility information nameplates and perform preprocessing.

[0049] Staff used mobile terminals to take photos of power facility information nameplates under different light intensities to obtain multiple nameplate images, and then denoised each nameplate image to obtain a preprocessed nameplate image.

[0050] It should be noted that there are many commonly used methods for image denoising. In this embodiment, Gaussian filtering is used to reduce the noise of the nameplate image. The implementer can also use image denoising algorithms such as median filtering, mean filtering, bilateral filtering or BM3D algorithm to reduce the noise of the nameplate image. This embodiment does not impose any special restrictions.

[0051] Among them, Gaussian filtering is a well-known technology, and its specific process of image denoising is not described in detail here.

[0052] The preprocessed nameplate image is grayscaled to obtain a nameplate grayscale image.

[0053] Step S2: Based on the degree of change of the grayscale value of each pixel in each area and the degree of dispersion of the grayscale values ​​of all pixels in the neighborhood of each pixel, combined with the grayscale value of each pixel in each area and the average distribution of the grayscale values ​​of all pixels, determine the brightness adjustment factor of each pixel in each area.

[0054] The material of power facility information nameplates is usually stainless steel or galvanized aluminum plate. Under strong light conditions, light spots will usually appear in the nameplate images collected by reflective mobile terminals, thereby blocking the text on the information nameplate, which is not conducive to neural network recognition and may even cause errors in the recognition results.

[0055] When using the Shenjiang network to recognize text on nameplate images, a fixed-size image is required as input. However, the size, angle, and position of the text in the nameplate image may vary. If the entire nameplate image is directly fed into the neural network, the network may not be able to correctly recognize the text. Therefore, the nameplate image must first be segmented to obtain individual text images; these individual text images are then used as input to the neural network to obtain recognition results.

[0056] Traditional text segmentation uses the relationship between text pixels and non-text pixels to cluster text pixels and thus separate individual characters from the image. However, when uneven lighting occurs in the nameplate image, the contrast between the text and the background may decrease. The traditional text segmentation method cannot directly distinguish between text pixels and non-text pixels, resulting in broken or overlapping segmentation results, which in turn affects the recognition effect of the neural network.

[0057] Traditional text segmentation methods typically select a segmentation threshold based on pixel statistics or a fixed global threshold based on empirical values. However, due to lighting conditions, images often contain bright spots with high intensity and shadows with low intensity. Using the same segmentation threshold across these areas makes it difficult to distinguish between text pixels and non-text pixels, resulting in incomplete or overlapping text segmentation.

[0058] S201: Segment the grayscale image of each nameplate to obtain various regions.

[0059] Different superpixel blocks may contain different local features, such as text size, shape, and font. Image segmentation allows for more efficient extraction and utilization of these features. Directly analyzing the entire nameplate image is relatively complex. To simplify image processing and improve the efficiency of subsequent operations, each nameplate grayscale image is segmented into a preset number of superpixel blocks. Each superpixel block represents a region, and there is no overlap between regions.

[0060] It should be understood that the value of the preset number is set manually. In this embodiment, the value of the preset number is 20. The implementer can also set it according to the specific situation. This embodiment does not impose any special restrictions.

[0061] It should be noted that there are many commonly used image segmentation algorithms. In this embodiment, the SLIC superpixel segmentation algorithm is used to segment the nameplate grayscale image into multiple regions. The implementer can also use image segmentation algorithms such as graph-based algorithms to block the nameplate image according to specific circumstances. This embodiment does not impose any special restrictions.

[0062] The SLIC superpixel segmentation algorithm is a well-known technology in the field of image segmentation, and the specific process of image segmentation is not described in detail here.

[0063] S202: Determine the grayscale change weight of each pixel in each region based on the grayscale value change degree of each pixel in each region and the discrete degree of the grayscale values ​​of all pixels in the neighborhood of each pixel.

[0064] The light intensity at the edge of the light spot varies greatly. If the segmentation threshold is adjusted only considering the difference in light intensity in different areas, the detection of pixels at the edge of the light spot in the nameplate image, i.e., the boundary between light and dark, will be inaccurate. Therefore, the degree of interference of changes in light intensity in the same area on text recognition must also be considered to adjust the segmentation threshold more accurately.

[0065] The more dramatic the change in light intensity, the larger the light spot area in the nameplate image may be, and a larger segmentation threshold needs to be set to eliminate the interference of the light spot on the text pixels; if the light intensity changes smoothly, the segmentation threshold should be reduced.

[0066] Therefore, the grayscale change weight of each pixel in each area is determined based on the discrete degree of the grayscale value of each pixel and all the surrounding pixels in each area and the degree of change of the grayscale value of the pixel, so as to reflect the degree of change of the light intensity in the same area, thereby adjusting the segmentation threshold in a targeted manner. Specifically,

[0067] In each area, a W neighborhood is set with each pixel as the center. The gradient of each pixel in each area and the variance of the grayscale values ​​of all pixels in its neighborhood are calculated respectively. If there are not enough pixels in the neighborhood, the average grayscale value of all pixels in the window is used to fill it.

[0068] It should be understood that the value of the neighborhood W is artificially set. In this embodiment, the value of the neighborhood W is 8. The implementer can also set it by himself according to the specific situation. This embodiment does not impose any special restrictions.

[0069] It should be noted that there are many methods for measuring the degree of dispersion. In this embodiment, variance is used to measure the degree of dispersion of the grayscale values ​​of all pixels in the neighborhood of each pixel. In other embodiments, variance can be replaced by standard deviation; in still other embodiments, the average difference in the grayscale values ​​of each pixel and all pixels in its neighborhood can be used as a measure of its degree of dispersion.

[0070] The grayscale change weight of each pixel in each region is the result of the fusion of the gradient of each pixel in each region and the variance of the grayscale values ​​of all pixels in its neighborhood.

[0071] It should be understood that fusion refers to combining two or more indicators through addition or multiplication to obtain a comprehensive indicator, thereby more comprehensively and accurately evaluating a phenomenon or problem. This fusion method is not limited to simple arithmetic operations and can also include more complex statistical models and analysis methods. Implementers can choose according to their specific circumstances and this embodiment does not impose any special restrictions.

[0072] Preferably, as an embodiment of the present application, the grayscale change weight of each pixel in each region is the product of the gradient of each pixel in each region and the variance of the grayscale values ​​of all pixels in its neighborhood.

[0073] Preferably, as another embodiment of the present application, the grayscale change weight of each pixel in each area is an exponential function value with a natural constant as the base and the sum of the gradient of each pixel in each area and the variance of the grayscale values ​​of all pixels in its neighborhood as the independent variable.

[0074] The variance of the grayscale values ​​of all pixels in the neighborhood of each pixel in each region can reflect the degree of illumination edge in the neighborhood of each pixel, and the gradient of each pixel in each region can reflect the speed of illumination change. The larger the variance and the gradient, the greater the grayscale change weight of each pixel in each region, and the faster the illumination change speed; conversely, if the variance and the gradient are smaller, the grayscale change weight of each pixel in each region is smaller, and the illumination change speed is slower.

[0075] S203: Determine a brightness adjustment factor for each pixel in each region based on the average distribution of the grayscale values ​​of all pixels in each region and the grayscale change weights of all pixels, combined with the grayscale values ​​of each pixel in each region.

[0076] Due to the reflective nature of the nameplates of power facilities, there are light spots with high light intensity and shadow areas in the nameplate images. A larger segmentation threshold is used in areas with high light intensity because the high light intensity will increase the brightness of all pixels in these areas. If the segmentation threshold is too small, some text pixels may be misjudged as non-text pixels due to their high brightness. Similarly, a smaller segmentation threshold is used in areas with less light to avoid some non-text pixels being misjudged as text pixels, which will affect the clustering effect.

[0077] Therefore, in order to quantify the illumination changes in the nameplate image, the brightness adjustment factor of each pixel in each area is determined based on the average distribution of the grayscale values ​​of all pixels in each area and the grayscale change weight of all pixels, combined with the grayscale value of each pixel in each area. This reflects the degree of change in light intensity and also adjusts the pixels in the nameplate image, so that the nameplate image can be more accurately analyzed and processed under different lighting conditions.

[0078] The specific process of determining the brightness adjustment factor of each pixel in each area is as follows:

[0079] Analyze the product of the grayscale change weight of each pixel in each area and the mean grayscale value of all pixels in the corresponding area, and record it as the grayscale weight of each pixel in each area;

[0080] Determine the mean of the grayscale weights of all pixels in each area, and record it as the average grayscale weight of each area;

[0081] The brightness adjustment factor of each pixel in each area is the product of the grayscale value of each pixel in each area and the average grayscale weight.

[0082] The larger the value of the brightness adjustment factor, the brighter the pixel is, the greater the illumination intensity is, and the more dramatic the illumination change is. The brightness adjustment factor takes into account both the illumination intensity and the illumination intensity change within the pixel and its neighborhood. The larger its value, the larger the segmentation threshold should be set at the pixel; conversely, the smaller the segmentation threshold should be set at the pixel.

[0083] Preferably, as an embodiment of the present application, the brightness adjustment factor extraction process diagram is as follows: Figure 2 shown.

[0084] Step S3: Based on the number of all pixels in the neighborhood of each pixel in each area and the distance from each pixel to all pixels in its neighborhood, and in combination with the size of each nameplate grayscale image, determine the size adjustment factor of each pixel in each area.

[0085] In step S2, the segmentation threshold is adjusted by analyzing the illumination intensity. However, the effect of illumination on the clustering of text pixels in the nameplate image is not considered. Spectral clustering is based on similarity. Since illumination changes the brightness of some pixels in the nameplate image, the similarity between the original text pixels will also change. To achieve a better clustering effect, the size adjustment factor can be calculated based on the distance and position relationship between the text pixels to further adjust the segmentation threshold to achieve a better clustering effect.

[0086] The resolution of the image also has a certain impact on the segmentation threshold setting. The noise is more obvious in low-resolution images. Since the brightness change of each pixel has a great impact on the overall nameplate image, it is necessary to increase the segmentation threshold to reduce the error caused by noise. Conversely, the segmentation threshold needs to be reduced.

[0087] S301: Determine the distance size of each pixel point in each area based on the number of all pixels in the neighborhood of each pixel point in each area and the distance between each pixel point and all pixels in its neighborhood.

[0088] The text in the nameplate image is composed of multiple pixels. The turning points of the text are places with obvious features, which are reflected in the position of pixels where the distance between pixels is close and the number of pixels is large. The distance size of each pixel in each area can be determined based on the number of all pixels in the neighborhood of each pixel in each area and the distance from each pixel to all pixels in its neighborhood, so as to identify local features in the text image, especially the turning points of the text, so as to more accurately segment the text characters and improve the accuracy of text detection and recognition.

[0089] By analyzing the local features of the text to better achieve text segmentation, the distance size of each pixel in each area is determined based on the number of all pixels in the neighborhood of each pixel and the distance between each pixel and all pixels in its neighborhood. Specifically,

[0090] Analyze the cumulative sum of the distances from each pixel in each region to all pixels in its neighborhood, and record it as the first distance of each pixel in each region;

[0091] The distance size between the pixels in each area is the ratio of the number of all pixels in the neighborhood of the pixel in each area to the first distance.

[0092] The larger the number of pixels, the more likely it is that the pixel is located in the text area, and a larger segmentation threshold should be set, because the probability that the pixels in the neighborhood of the pixel are text pixels is relatively high; similarly, the smaller the first distance, the closer the association between the pixel and other pixels in the neighborhood, the more likely the pixel is at the turning point of the text, and a larger segmentation threshold should be set; conversely, the smaller the number of pixels and the larger the first distance, the less likely the pixel is to be a text pixel, and a smaller segmentation threshold can be set.

[0093] S302: Determine a resolution weight of the grayscale image of the nameplate of each area based on the size of the grayscale image of the nameplate of each area.

[0094] Resolution reflects the level of detail in an image. In images of different resolutions, changes in pixel brightness have different impacts on the overall image. Low-resolution images have more noticeable noise, so a higher segmentation threshold is needed to reduce noise-induced errors. High-resolution images can use a smaller segmentation threshold.

[0095] Therefore, in order to quantify the impact of resolution on the segmentation threshold, the resolution weight of the grayscale image of the nameplate in each area is determined based on the size of the grayscale image of the nameplate in each area, and the value range of the segmentation threshold is further limited to achieve better segmentation effect and thus realize accurate text recognition. The expression of the resolution weight of the grayscale image of the nameplate in each area is: Where k i represents the resolution weight of the grayscale image of the nameplate in the i-th area; h i represents the height of the grayscale image of the nameplate in the i-th area; w i It represents the width of the grayscale image of the nameplate in the i-th area; norm() represents the normalization function; min() represents the minimum function.

[0096] The larger the resolution of the nameplate grayscale image, the smaller the resolution weight, which means that the noise in the nameplate image is not very obvious, and a smaller segmentation threshold can be set; on the contrary, if the resolution of the nameplate grayscale image is small, the larger the resolution weight, which means that the noise in the nameplate image is obvious, which may cause the grayscale value of each pixel to change greatly, thereby affecting the details of the overall image, so a larger segmentation threshold should be set.

[0097] S303: Determine a size adjustment factor for each pixel in each area based on the distance size and the resolution weight.

[0098] In order to comprehensively analyze the impact of local features of text and the resolution of nameplate images on text segmentation, the size adjustment factor of each pixel in each area is determined based on the size distance and the resolution weight. This helps to adjust the segmentation threshold according to different image features, thereby ensuring recognition accuracy while reducing misjudgments. Specifically:

[0099] In order to preliminarily determine the text pixels in the nameplate image and obtain the positional relationship between them, the threshold segmentation algorithm is used to binarize the grayscale images of each nameplate. Due to the large difference between the text and the background in the nameplate image, the pixels with a pixel value of 1 in the nameplate image after binarization are recorded as candidate text pixels, and the pixels with a pixel value of 0 are recorded as non-candidate text pixels.

[0100] It should be noted that there are many commonly used threshold segmentation algorithms. In this embodiment, the maximum inter-class variance algorithm is used to segment the nameplate image. The implementer may also use other threshold segmentation algorithms based on specific circumstances. This embodiment does not impose any special restrictions.

[0101] The expression for the size adjustment factor of each pixel in each area is: Where λ i,j k represents the size adjustment factor of the jth pixel in the i-th region; i represents the resolution weight of the grayscale image of the nameplate in the i-th area; l i,j Indicates the distance size of the jth pixel in the i-th region.

[0102] The resizing factor combines the image resolution and the positional relationship between candidate text pixels and non-candidate text pixels in the image. The larger the resizing factor, the larger the segmentation threshold should be; conversely, the smaller the resizing factor, the smaller the segmentation threshold should be.

[0103] Step S4: determining an adjustment factor for each pixel in each area based on the brightness adjustment factor and the size adjustment factor, so as to obtain a text image and recognize the text therein.

[0104] In the case of uneven lighting, relying solely on the threshold segmentation algorithm for binarization cannot accurately detect text pixels and non-text pixels in the image. If there is no uneven lighting, the traditional threshold segmentation algorithm can better detect text pixels and non-text pixels. However, in scenes with uneven lighting, the segmentation threshold needs to be adjusted. Specifically:

[0105] (1) The normalized product of the brightness adjustment factor and the size adjustment factor of each pixel in each region is used as the adjustment factor of each pixel in each region;

[0106] The product of the adjustment factor of each pixel point in each area and the threshold value in the binarization process of the nameplate grayscale image is used as the segmentation threshold value of each pixel point in each area;

[0107] The pixels whose grayscale values ​​in each region are less than the corresponding segmentation threshold are text pixels; the pixels whose grayscale values ​​in each region are greater than or equal to the corresponding segmentation threshold are non-text pixels.

[0108] Preferably, as an embodiment of the present application, the adjustment factor extraction diagram is as follows: Figure 3 shown.

[0109] (2) The coordinates and grayscale values ​​of each character pixel point are combined into a vector, and the vectors of all the character pixel points in the grayscale image of the nameplate are clustered to obtain clusters, where each cluster corresponds to one character.

[0110] It should be noted that there are many commonly used clustering algorithms. In this embodiment, the mean shift clustering algorithm is used to cluster text pixels. The implementer can also use other clustering algorithms such as the k-means algorithm to achieve clustering. This embodiment does not impose any special restrictions.

[0111] The mean shift clustering algorithm is a well-known technology, and the specific process of clustering is not described in detail here.

[0112] (3) The minimum enclosing rectangle of the area where all pixels in each cluster are located is used as the bounding box of the text. The text is segmented from the nameplate image based on the bounding box, and the segmented text images are stored in an array in sequence.

[0113] (4) Use a pre-trained convolutional neural network to recognize text images, take the text images in the array as input, use the cross loss function as the loss function, use Adam as the optimization algorithm, and train 20 epochs. Output the text recognition results on the power facility information nameplate and store the recognition results in the mobile terminal.

[0114] At this point, the segmentation threshold is obtained based on the adjustment factor, and the text is segmented in combination with the clustering algorithm. The segmented text is input into the neural network to complete the text recognition.

[0115] Preferably, as an embodiment of the present application, the flowchart of the text segmentation and recognition process is as follows: Figure 4 shown.

[0116] Preferably, as an embodiment of the present application, the text recognition process diagram based on the neural network is as follows: Figure 5 shown.

[0117] Based on the same inventive concept as the above method, an embodiment of the present application also provides a mobile terminal text recognition system based on a neural network, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above-mentioned mobile terminal text recognition methods based on a neural network.

[0118] It should be noted that the order in which the embodiments of the present application are presented is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. Furthermore, the foregoing descriptions of specific embodiments of this specification are provided. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential sequence shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0119] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0120] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for mobile terminal text recognition based on neural network, characterized in that: The method comprises the following steps: Obtain grayscale images of nameplates of power facilities under different light intensities; The grayscale image of each nameplate is segmented into multiple regions. Based on the degree of change in the grayscale value of each pixel in each region and the degree of dispersion of the grayscale values ​​of all pixels in the neighborhood of each pixel, the grayscale value of each pixel in each region and the average distribution of the grayscale values ​​of all pixels in each region, the brightness adjustment factor of each pixel in each region is determined. Based on the number of all pixels in the neighborhood of each pixel in each area and the distance between each pixel and all pixels in its neighborhood, and the size of the grayscale image of the nameplate in each area, the size adjustment factor of each pixel in each area is determined; determining an adjustment factor for each pixel in each area based on a brightness adjustment factor and a size adjustment factor for each pixel in each area to recognize text in a grayscale image of the nameplate; Determining the adjustment factor of each pixel in each area to recognize the text in the grayscale image of the nameplate includes: The product of the adjustment factor of each pixel in each area and the threshold value in the binarization process of the nameplate grayscale image is used as the segmentation threshold of each pixel in each area. The adjustment factor of each pixel in each area is the normalized result of the product of the brightness adjustment factor and the size adjustment factor of each pixel in each area; The coordinates and grayscale values ​​of each text pixel point are combined into a vector, and the vectors of all text pixel points in the grayscale image of the nameplate are clustered to obtain clusters. Each cluster corresponds to a text, and the neural network is used to recognize each text; the text pixel point is the pixel point whose grayscale value in each area is less than the corresponding segmentation threshold.

2. A method for mobile terminal text recognition based on a neural network as claimed in claim 1, characterized in that: The method for determining the brightness adjustment factor of each pixel in each area is as follows: Determine the grayscale change weight of each pixel in each area based on the grayscale value change degree of each pixel in each area and the discrete degree of the grayscale values ​​of all pixels in the neighborhood of each pixel; Based on the average distribution of the grayscale values ​​of all pixels in each area and the grayscale change weights of all pixels, the average grayscale weight of each area is determined; The brightness adjustment factor of each pixel in each area is the result of fusing the grayscale value of each pixel in each area with the average grayscale weight.

3. The method for mobile terminal text recognition based on a neural network as claimed in claim 2, characterized in that: The grayscale change weight of each pixel point in each region is the result of the fusion of the gradient of each pixel point in each region and the variance of the grayscale values ​​of all pixels in its neighborhood.

4. The method for mobile terminal text recognition based on neural network according to claim 2, characterized in that: The method for determining the average grayscale weight of each region is: Analyze the product of the grayscale change weight of each pixel in each area and the mean grayscale value of all pixels in the corresponding area, and record it as the grayscale weight of each pixel in each area; The average grayscale weight of each pixel in each region is the mean of the grayscale weights of all pixels in each region.

5. The method for mobile terminal text recognition based on neural network according to claim 1, characterized in that: The method for determining the size adjustment factor of each pixel in each area is as follows: Determine the distance between each pixel in each region based on the number of all pixels in the neighborhood of each pixel and the distance between each pixel and all pixels in its neighborhood. Determine the resolution weight of the grayscale image of the nameplate of each area based on the size of the grayscale image of the nameplate of each area; A size adjustment factor for each pixel in each area is determined based on the distance size and the resolution weight.

6. A method for mobile terminal text recognition based on a neural network as claimed in claim 5, characterized in that: The method for determining the distance between the pixels in each area is as follows: Analyze the cumulative sum of the distances from each pixel in each region to all pixels in its neighborhood, and record it as the first distance of each pixel in each region; The distance size between the pixels in each area is the ratio of the number of all pixels in the neighborhood of the pixel in each area to the first distance.

7. The method for mobile terminal text recognition based on neural network according to claim 5, characterized in that: The expression of the resolution weight of the grayscale image of the nameplate in each area is: Where k i represents the resolution weight of the grayscale image of the nameplate in the i-th area; h i represents the height of the grayscale image of the nameplate in the i-th area; w i It represents the width of the grayscale image of the nameplate in the i-th area; norm() represents the normalization function; min() represents the minimum function.

8. The method for mobile terminal text recognition based on neural network according to claim 5, characterized in that: The method for determining the size adjustment factor of each pixel in each area is as follows: The threshold segmentation algorithm is used to perform binary processing on the grayscale image of the nameplate. The pixels with a grayscale value of 1 are recorded as candidate text pixels, and the pixels with a grayscale value of 0 are recorded as non-candidate text pixels. The expression for the size adjustment factor of each pixel in each area is: Where λ i,j k represents the size adjustment factor of the jth pixel in the i-th region; i represents the resolution weight of the grayscale image of the nameplate in the i-th area; l i,j Indicates the distance size of the jth pixel in the i-th region.

9. A mobile terminal text recognition system based on a neural network, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the mobile terminal text recognition method based on neural network as described in any one of claims 1 to 8 are implemented.

Images