Mobile terminal character recognition method and system based on neural network

Abstract

The invention relates to the technical field of character recognition, in particular to a mobile terminal character recognition method and system based on a neural network, and the method comprises the steps: obtaining a nameplate gray-scale map of an electric power facility; determining a brightness adjustment factor of each pixel point in each region; based on the number of all pixel points in the neighborhood of each pixel point in each region and the distance from each pixel point to all pixel points in the neighborhood of each pixel point in each region, determining a size adjustment factor of each pixel point in each region in combination with the size of the nameplate grey-scale map where each region is located; and determining an adjustment factor of each pixel point in each region based on the brightness adjustment factor and the size adjustment factor of each pixel point in each region so as to obtain a character picture and identify characters in the character picture. According to the method, the segmentation threshold is adjusted in combination with nameplate image features to obtain an accurate segmentation result, and the character recognition accuracy of the neural network is improved.

Description

Technical Field

[0001] The present application relates to the technical field of character recognition, and in particular to a method and system for mobile terminal text recognition based on a neural network. Background Art

[0002] In the power industry, safety inspections and equipment maintenance are key links to ensure the stable operation of the power grid. During the inspection process, the equipment information on the nameplate of the power facility needs to be entered. If it is manually identified and entered, the process is not only cumbersome, but also prone to errors. The existing solution is for staff to use mobile terminals to take pictures of the nameplates, use neural networks for visual inspection, recognize the text on the pictures, and enter the recognition results, which greatly improves efficiency and accuracy.

[0003] When workers use mobile terminals to collect pictures outdoors or in complex indoor light environments, they often face the problem of unsatisfactory lighting conditions. Such changes in lighting may have a negative impact on the quality of the picture, causing the brightness of the text pixels in the picture to change, resulting in inaccurate text pixel clustering and segmentation results, causing errors in the neural network during the text recognition process, affecting the accuracy of the recognition results. Summary of the invention

[0004] In order to solve the above technical problems, the purpose of this application is to provide a method and system for mobile terminal text recognition based on neural network. The technical solutions adopted are as follows:

[0005] In a first aspect, an embodiment of the present application provides a method for text recognition in a mobile terminal based on a neural network, the method comprising the following steps:

[0006] Get grayscale images of nameplates of power facilities under different light intensities;

[0007] The grayscale images of each nameplate are segmented to obtain multiple regions, and the brightness adjustment factor of each pixel in each region is determined based on the variation degree of the grayscale value of each pixel in each region and the discrete degree of the grayscale values ​​of all pixels in the neighborhood of each pixel, combined with the grayscale value of each pixel in each region and the average distribution of the grayscale values ​​of all pixels;

[0008] Based on the number of all pixels in the neighborhood of each pixel in each area and the distance from each pixel to all pixels in its neighborhood, combined with the size of the grayscale image of the nameplate in each area, determine the size adjustment factor of each pixel in each area;

[0009] The adjustment factor of each pixel in each area is determined based on the brightness adjustment factor and the size adjustment factor of each pixel in each area to recognize the text in the grayscale image of the nameplate.

[0010] Preferably, the method for determining the brightness adjustment factor of each pixel in each area is:

[0011] Based on the change degree of the grayscale value of each pixel in each area and the discrete degree of the grayscale values ​​of all pixels in the neighborhood of each pixel, the grayscale change weight of each pixel in each area is determined;

[0012] Based on the average distribution of the grayscale values ​​of all pixels in each area and the grayscale change weights of all pixels, the average grayscale weight of each area is determined;

[0013] The brightness adjustment factor of each pixel in each region is the result of merging the grayscale value of each pixel in each region with the average grayscale weight.

[0014] Preferably, the grayscale change weight of each pixel in each region is a result of fusion of the gradient of each pixel in each region and the variance of the grayscale values ​​of all pixels in its neighborhood.

[0015] Preferably, the method for determining the average grayscale weight of each region is:

[0016] Analyze the product of the grayscale change weight of each pixel in each area and the mean grayscale value of all pixels in the corresponding area, and record it as the grayscale weight of each pixel in each area;

[0017] The average grayscale weight of each pixel in each region is the mean of the grayscale weights of all pixels in each region.

[0018] Preferably, the method for determining the size adjustment factor of each pixel in each area is:

[0019] Determine the distance size of each pixel in each area based on the number of all pixels in the neighborhood of each pixel in each area and the distance from each pixel to all pixels in its neighborhood;

[0020] Determine the resolution weight of the grayscale image of the nameplate in each area based on the size of the grayscale image of the nameplate in each area;

[0021] A size adjustment factor for each pixel in each region is determined based on the distance size and the resolution weight.

[0022] Preferably, the method for determining the distance size of each pixel point in each area is:

[0023] Analyze the cumulative sum of the distances from each pixel in each region to all pixels in its neighborhood, and record it as the first distance of each pixel in each region;

[0024] The distance size of each pixel point in each area is the ratio of the number of all pixel points in the neighborhood of each pixel point in each area to the first distance.

[0025] Preferably, the expression of the resolution weight of the grayscale image of the nameplate in each area is: In the formula, k i represents the resolution weight of the grayscale image of the nameplate in the ith area; h i represents the height of the grayscale image of the nameplate in the ith area; w i It represents the width of the grayscale image of the nameplate in the ith area; norm() represents the normalization function; min() represents the minimum function.

[0026] Preferably, the method for determining the size adjustment factor of each pixel in each area is:

[0027] The grayscale image of the nameplate is binarized using a threshold segmentation algorithm. Pixels with a grayscale value of 1 are recorded as candidate text pixels, and pixels with a grayscale value of 0 are recorded as non-candidate text pixels.

[0028] The expression for the size adjustment factor of each pixel in each area is: In the formula, λ i,j represents the size adjustment factor of the jth pixel in the i-th region; k i represents the resolution weight of the grayscale image of the nameplate in the ith area; l i,j Indicates the distance size of the jth pixel in the i-th region.

[0029] Preferably, determining the adjustment factor of each pixel in each area to identify the text in the grayscale image of the nameplate includes:

[0030] The adjustment factor of each pixel in each region is the normalized result of the product of the brightness adjustment factor and the size adjustment factor of each pixel in each region;

[0031] The product of the adjustment factor of each pixel point in each area and the threshold value in the binarization process of the nameplate grayscale image is used as the segmentation threshold value of each pixel point in each area;

[0032] The pixels in each area whose grayscale value is less than the corresponding segmentation threshold are text pixels. The coordinates and grayscale values ​​of each text pixel are combined into a vector. The vectors of all text pixels in the grayscale image of the nameplate are clustered to obtain clusters, where each cluster corresponds to a text. The neural network is used to recognize each text.

[0033] In a second aspect, an embodiment of the present application also provides a mobile terminal text recognition system based on a neural network, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of any one of the above methods when executing the computer program.

[0034] This application has at least the following beneficial effects:

[0035] This application determines the brightness adjustment factor of each pixel in each area based on the degree of change of the grayscale value of each pixel in each area and the degree of discreteness of the grayscale values ​​of all pixels in the neighborhood of each pixel, combined with the grayscale value of each pixel in each area and the average distribution of the grayscale values ​​of all pixels, reflecting the degree of change of light intensity while also adjusting the pixels in the nameplate image, so that the nameplate image can be more accurately analyzed and processed under different lighting conditions;

[0036] The present application determines the size adjustment factor of each pixel in each area based on the number of all pixels in the neighborhood of each pixel in each area and the distance between each pixel and all pixels in its neighborhood, combined with the size of the grayscale image of the nameplate in each area, adjusts the segmentation threshold to distinguish between text pixels and non-text pixels, uses a clustering algorithm to cluster the text pixels to complete the segmentation of the image, and uses a neural network to recognize the segmented text image.

[0037] This application adjusts the segmentation threshold in combination with the nameplate image features to obtain more accurate segmentation results, reduces the impact of strong light on text segmentation of power facility information nameplate images, and significantly improves the accuracy of neural network text recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 A flowchart of a method for text recognition in a mobile terminal based on a neural network;

[0040] Figure 2 This is a schematic diagram of the brightness adjustment factor extraction process;

[0041] Figure 3 Extraction diagram for adjustment factors;

[0042] Figure 4 It is a flowchart of the text segmentation and recognition process;

[0043] Figure 5 Schematic diagram of the text recognition process based on neural network. DETAILED DESCRIPTION

[0044] In order to further explain the technical means and effects adopted by the present application to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, features and effects of a mobile terminal text recognition method and system based on a neural network proposed in the present application in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0045] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0046] The following is a detailed description of a specific scheme of a mobile terminal text recognition method and system based on a neural network provided by the present application in conjunction with the accompanying drawings.

[0047] See also Figure 1 , which shows a flowchart of a method for text recognition in a mobile terminal based on a neural network provided by an embodiment of the present application, the method comprising the following steps:

[0048] Step S1: Use a mobile terminal to collect images of power facility information nameplates and perform preprocessing.

[0049] The staff used a mobile terminal to take photos of the information nameplates of power facilities under different light intensities to obtain multiple nameplate images, and denoised each nameplate image to obtain a preprocessed nameplate image.

[0050] It should be noted that there are many commonly used methods for image denoising. In this embodiment, Gaussian filtering is used to reduce the noise of the nameplate image. The implementer may also use image denoising algorithms such as median filtering, mean filtering, bilateral filtering or BM3D algorithm to reduce the noise of the nameplate image. This embodiment does not impose any special restrictions.

[0051] Among them, Gaussian filtering is a well-known technology, and the specific process of image denoising is not described in detail.

[0052] The preprocessed nameplate image is grayed out to obtain a nameplate grayscale image.

[0053] Step S2: Based on the degree of change of the grayscale value of each pixel in each area and the degree of discreteness of the grayscale values ​​of all pixels in the neighborhood of each pixel, combined with the grayscale value of each pixel in each area and the average distribution of the grayscale values ​​of all pixels, determine the brightness adjustment factor of each pixel in each area.

[0054] The material of the information nameplate of power facilities is usually stainless steel or galvanized aluminum plate. Under strong light conditions, light spots will usually appear in the nameplate image collected by the reflective mobile terminal, thereby blocking the text on the information nameplate, which is not conducive to neural network recognition and may even cause errors in the recognition results.

[0055] When using the Shenjiang network to recognize the text on the nameplate image, a fixed-size image needs to be input. The size, angle, and position of the text in the nameplate image may vary. If the entire nameplate image is directly input into the neural network, the network may not be able to correctly recognize the text. Therefore, the nameplate image must first be segmented to obtain individual text images; then each text image is used as the input of the neural network for recognition to obtain the recognition result.

[0056] Traditional text segmentation uses the relationship between text pixels and non-text pixels to cluster text pixels and separate individual characters from the image. However, when uneven lighting occurs in the nameplate image, the contrast between the text and the background may be reduced. The traditional text segmentation method cannot directly distinguish between text pixels and non-text pixels, resulting in breaks or overlaps in the segmentation results, which in turn affects the recognition effect of the neural network.

[0057] The selection of segmentation threshold in traditional text segmentation methods is usually based on the statistical characteristics of pixels or based on empirical values. However, due to the influence of lighting, there are bright spots with high light intensity and shadow areas with low light intensity in the image. It is difficult to distinguish text pixels from non-text pixels using the same segmentation threshold in different areas, resulting in incomplete or overlapping text segmentation in the image.

[0058] S201: Segment the grayscale images of each nameplate to obtain various regions.

[0059] Different superpixel blocks may contain different local features, such as the size, shape, font, etc. of the text. These features can be extracted and utilized more effectively by segmenting the image. At the same time, directly analyzing the entire nameplate image is relatively complicated. In order to simplify the image processing task and improve the efficiency of subsequent operations, each nameplate grayscale image is segmented into a preset number of superpixel blocks. Each superpixel block represents an area, and there is no overlap between different areas.

[0060] It should be understood that the value of the preset number is artificially set. In this embodiment, the value of the preset number is 20. The implementer can also set it according to the specific situation. This embodiment does not impose any special restrictions.

[0061] It should be noted that there are many commonly used image segmentation algorithms. In this embodiment, the SLIC superpixel segmentation algorithm is used to segment the nameplate grayscale image into multiple regions. The implementer can also use image segmentation algorithms such as Graph-based algorithms to divide the nameplate image into blocks according to specific circumstances. This embodiment does not impose any special restrictions.

[0062] Among them, the SLIC superpixel segmentation algorithm is a well-known technology in the field of image segmentation, and its specific process of segmenting the image is not repeated here.

[0063] S202: Determine the grayscale change weight of each pixel in each region based on the change degree of the grayscale value of each pixel in each region and the discrete degree of the grayscale values ​​of all pixels in the neighborhood of each pixel.

[0064] The light intensity at the edge of the light spot varies greatly. If we only consider the difference in light intensity in different areas to adjust the segmentation threshold, it will lead to inaccurate pixel detection at the edge of the light spot in the nameplate image, i.e., the boundary between light and dark. Therefore, we must also consider the degree of interference of changes in light intensity in the same area on text recognition, so as to adjust the segmentation threshold more accurately.

[0065] The more dramatic the change in light intensity, the larger the light spot area in the nameplate image may be, and a larger segmentation threshold needs to be set to eliminate the interference of the light spot on the text pixels; if the light intensity changes smoothly, the segmentation threshold should be reduced.

[0066] Therefore, the grayscale change weight of each pixel in each area is determined based on the discrete degree of the grayscale value of each pixel and all the surrounding pixels in each area and the degree of change of the grayscale value of the pixel, so as to reflect the degree of change of the light intensity in the same area, so as to adjust the segmentation threshold in a targeted manner, specifically:

[0067] In each area, a W neighborhood is set with each pixel as the center, and the gradient of each pixel in each area and the variance of the grayscale values ​​of all pixels in its neighborhood are calculated respectively. If there are not enough pixels in the neighborhood, the average grayscale value of all pixels in the window is used to fill it.

[0068] It should be understood that the value of the neighborhood W is artificially set. In this embodiment, the value of the neighborhood W is 8. The implementer can also set it by himself according to the specific situation. This embodiment does not impose any special restrictions.

[0069] It should be noted that there are many methods for measuring the degree of discreteness. In this embodiment, variance is used to measure the degree of discreteness of the grayscale values ​​of all pixels in the neighborhood of each pixel. In other embodiments, variance can be replaced by standard deviation; in some other embodiments, the average difference of the grayscale values ​​of each pixel and all pixels in its neighborhood can be used as a measure of its discreteness.

[0070] The grayscale change weight of each pixel in each region is the result of the fusion of the gradient of each pixel in each region and the variance of the grayscale values ​​of all pixels in its neighborhood.

[0071] It should be understood that fusion refers to combining two or more indicators by adding or multiplying them together in order to obtain a comprehensive indicator, thereby more comprehensively and accurately evaluating a phenomenon or problem. This fusion method is not limited to simple arithmetic operations, but can also include more complex statistical models and analysis methods. The implementer can choose according to the specific situation, and this embodiment does not impose any special restrictions.

[0072] Preferably, as an embodiment of the present application, the grayscale change weight of each pixel in each region is the result of multiplying the gradient of each pixel in each region and the variance of the grayscale values ​​of all pixels in its neighborhood.

[0073] Preferably, as another embodiment of the present application, the grayscale change weight of each pixel in each region is an exponential function value with a natural constant as the base and the sum of the gradient of each pixel in each region and the variance of the grayscale values ​​of all pixels in its neighborhood as the independent variable.

[0074] The variance of the grayscale values ​​of all pixels in the neighborhood of each pixel in each region can reflect the degree of illumination edge in the neighborhood of each pixel, and the gradient of each pixel in each region can reflect the speed of illumination change. The larger the variance and the gradient, the greater the grayscale change weight of each pixel in each region, and the faster the illumination changes. Conversely, if the variance and the gradient are smaller, the grayscale change weight of each pixel in each region is smaller, and the illumination changes slower.

[0075] S203: Determine a brightness adjustment factor for each pixel in each region based on the average distribution of the grayscale values ​​of all pixels in each region and the grayscale change weights of all pixels, combined with the grayscale value of each pixel in each region.

[0076] Due to the reflection of the information nameplate of the power facility, there are light spots with high light intensity and shadow areas in the nameplate image. A larger segmentation threshold is used in the area with high light intensity because the high light intensity will increase the brightness of all pixels in these areas. If the segmentation threshold is too small, some text pixels may be misjudged as non-text pixels due to their high brightness. Similarly, a smaller segmentation threshold is used in the area with less light to avoid some non-text pixels being misjudged as text pixels, which will affect the clustering effect.

[0077] Therefore, in order to quantify the illumination changes in the nameplate image, the brightness adjustment factor of each pixel in each area is determined based on the average distribution of the grayscale values ​​of all pixels in each area and the grayscale change weight of all pixels, combined with the grayscale value of each pixel in each area. This reflects the degree of change in light intensity and also adjusts the pixels in the nameplate image, so that the nameplate image can be more accurately analyzed and processed under different lighting conditions.

[0078] The specific process of determining the brightness adjustment factor of each pixel in each area is as follows:

[0079] Analyze the product of the grayscale change weight of each pixel in each area and the mean grayscale value of all pixels in the corresponding area, and record it as the grayscale weight of each pixel in each area;

[0080] Determine the mean grayscale weight of all pixels in each region, and record it as the average grayscale weight of each region;

[0081] The brightness adjustment factor of each pixel in each area is the product of the grayscale value of each pixel in each area and the average grayscale weight.

[0082] The larger the value of the brightness adjustment factor, the greater the brightness of the pixel, the greater the light intensity, and the more dramatic the pixel light change. The brightness adjustment factor takes into account both the light intensity and the light intensity change within the pixel and its neighborhood. The larger its value, the larger the segmentation threshold should be set at the pixel; conversely, a smaller segmentation threshold should be set at the pixel.

[0083] Preferably, as an embodiment of the present application, the schematic diagram of the brightness adjustment factor extraction process is as follows: Figure 2 shown.

[0084] Step S3: Based on the number of all pixels in the neighborhood of each pixel in each area and the distance from each pixel to all pixels in its neighborhood, combined with the size of each nameplate grayscale image, determine the size adjustment factor of each pixel in each area.

[0085] In step S2, the segmentation threshold is adjusted by analyzing the illumination intensity, but the effect of illumination on the clustering of text pixels in the nameplate image is not taken into account. Spectral clustering is clustered based on similarity. Since illumination changes the brightness of some pixels in the nameplate image, the similarity between the original text pixels will also change. In order to achieve a better clustering effect, the size adjustment factor can be calculated based on the distance and position relationship between the text pixels to further adjust the segmentation threshold to achieve a better clustering effect.

[0086] The resolution of the image also has a certain impact on the segmentation threshold setting. The noise is more obvious in low-resolution images. Since the brightness changes of each pixel have a great impact on the overall nameplate image, the segmentation threshold needs to be increased to reduce the error caused by noise. Conversely, the segmentation threshold needs to be reduced.

[0087] S301: Determine the distance size of each pixel point in each region based on the number of all pixel points in the neighborhood of each pixel point in each region and the distance from each pixel point to all pixel points in its neighborhood.

[0088] The text in the nameplate image is composed of multiple pixels. The turning points of the text are places with obvious features, which are reflected in the position of pixels as the distance between pixels is short and the number of pixels is large. The distance size of each pixel in each area can be determined according to the number of all pixels in the neighborhood of each pixel in each area and the distance from each pixel to all pixels in its neighborhood, so as to identify local features in the text image, especially the turning points of the text, so as to more accurately segment the text characters and improve the accuracy of text detection and recognition.

[0089] By analyzing the local features of the text to better achieve text segmentation, the distance size of each pixel in each area is determined based on the number of all pixels in the neighborhood of each pixel in each area and the distance from each pixel to all pixels in its neighborhood. Specifically,

[0090] Analyze the cumulative sum of the distances from each pixel in each region to all pixels in its neighborhood, and record it as the first distance of each pixel in each region;

[0091] The distance size of each pixel point in each area is the ratio of the number of all pixel points in the neighborhood of each pixel point in each area to the first distance.

[0092] The larger the number of pixels, the more likely it is that the pixel is located in the text area, and a larger segmentation threshold should be set, because the probability that the pixels in the neighborhood of the pixel are text pixels is relatively high; similarly, the smaller the first distance, the closer the association between the pixel and other pixels in the neighborhood, the more likely the pixel is at the turning point of the text, and a larger segmentation threshold should be set; conversely, the smaller the number of pixels and the larger the first distance, the less likely the pixel is to be a text pixel, and a smaller segmentation threshold can be set.

[0093] S302: Determine a resolution weight of the grayscale image of the nameplate where each area is located based on the size of the grayscale image of the nameplate where each area is located.

[0094] Resolution reflects the level of detail in an image. In images of different resolutions, the brightness change of pixels has different effects on the overall image. In low-resolution images, noise is more obvious, and the segmentation threshold needs to be increased to reduce the error caused by noise; in high-resolution images, a smaller segmentation threshold can be set.

[0095] Therefore, in order to quantify the impact of resolution on the segmentation threshold, the resolution weight of the grayscale image of the nameplate in each area is determined based on the size of the grayscale image of the nameplate in each area, and the value range of the segmentation threshold is further limited to achieve better segmentation effect and thus realize accurate recognition of text. The expression of the resolution weight of the grayscale image of the nameplate in each area is: In the formula, k i represents the resolution weight of the grayscale image of the nameplate in the ith area; h i represents the height of the grayscale image of the nameplate in the ith area; w i It represents the width of the grayscale image of the nameplate in the ith area; norm() represents the normalization function; min() represents the minimum function.

[0096] The larger the resolution of the nameplate grayscale image, the smaller the resolution weight, which means that the noise in the nameplate image is not very obvious, and a smaller segmentation threshold can be set; on the contrary, if the resolution of the nameplate grayscale image is small, the larger the resolution weight, which means that the noise in the nameplate image is obvious, which may cause the grayscale value of each pixel to change greatly, thus affecting the details of the overall image, so a larger segmentation threshold should be set.

[0097] S303: Determine a size adjustment factor for each pixel in each area based on the distance size and the resolution weight.

[0098] In order to comprehensively analyze the influence of local features of text and resolution of nameplate image on text segmentation, the size adjustment factor of each pixel in each area is determined based on the size distance and the resolution weight, which helps to adjust the segmentation threshold according to different image features, thereby ensuring recognition accuracy while reducing misjudgment, specifically:

[0099] In order to preliminarily determine the text pixels in the nameplate image and obtain the positional relationship between them, the threshold segmentation algorithm is used to binarize the grayscale images of each nameplate. Due to the large difference between the text and the background in the nameplate image, the pixels with a pixel value of 1 in the nameplate image after binarization are recorded as candidate text pixels, and the pixels with a pixel value of 0 are recorded as non-candidate text pixels.

[0100] It should be noted that there are many commonly used threshold segmentation algorithms. In this embodiment, the maximum inter-class variance algorithm is used to segment the nameplate image. The implementer may also use other threshold segmentation algorithms according to the specific situation. This embodiment does not impose any special restrictions.

[0101] The expression for the size adjustment factor of each pixel in each area is: In the formula, λ i,j represents the size adjustment factor of the jth pixel in the i-th region; k i represents the resolution weight of the grayscale image of the nameplate in the ith area; l i,j Indicates the distance size of the jth pixel in the i-th region.

[0102] The resizing factor combines the image resolution and the positional relationship between candidate text pixels and non-candidate text pixels in the image. The larger the value of the resizing factor, the larger the segmentation threshold should be; conversely, the smaller the resizing factor, the smaller the segmentation threshold should be.

[0103] Step S4: determining the adjustment factor of each pixel in each area based on the brightness adjustment factor and the size adjustment factor to obtain a text image and recognize the text therein.

[0104] In the case of uneven lighting, relying solely on the threshold segmentation algorithm for binarization cannot accurately detect text pixels and non-text pixels in the image. If there is no uneven lighting, the traditional threshold segmentation algorithm can better detect text pixels and non-text pixels. However, in the case of uneven lighting, the segmentation threshold needs to be adjusted, specifically:

[0105] (1) The normalized result of the product of the brightness adjustment factor and the size adjustment factor of each pixel in each region is used as the adjustment factor of each pixel in each region;

[0106] The product of the adjustment factor of each pixel point in each area and the threshold value in the binarization process of the nameplate grayscale image is used as the segmentation threshold value of each pixel point in each area;

[0107] The pixels in each region whose grayscale values ​​are less than the corresponding segmentation threshold are text pixels; the pixels in each region whose grayscale values ​​are greater than or equal to the corresponding segmentation threshold are non-text pixels.

[0108] Preferably, as an embodiment of the present application, the adjustment factor extraction schematic diagram is as follows: Figure 3 shown.

[0109] (2) The coordinates and grayscale values ​​of each character pixel point are combined into a vector, and the vectors of all the character pixel points in the grayscale image of the nameplate are clustered to obtain clusters, where each cluster corresponds to one character.

[0110] It should be noted that there are many commonly used clustering algorithms. In this embodiment, the mean shift clustering algorithm is used to cluster text pixels. The implementer may also use other clustering algorithms such as the k-means algorithm to achieve clustering, and this embodiment does not impose any special restrictions.

[0111] Among them, the mean shift clustering algorithm is a well-known technology, and the specific process of its clustering is not repeated here.

[0112] (3) The minimum enclosing rectangle of the area where all pixels in each cluster are located is used as the bounding box of the text. The text is segmented from the nameplate image according to the bounding box, and the segmented text images are stored in an array in sequence.

[0113] (4) Use a pre-trained convolutional neural network to recognize text images. Take the text images in the array as input, use the cross loss function as the loss function, use Adam as the optimization algorithm, and train 20 epochs. Output the text recognition results on the power facility information plate and store the recognition results in the mobile terminal.

[0114] At this point, the segmentation threshold is obtained based on the adjustment factor, and the text is segmented in combination with the clustering algorithm. The segmented text is input into the neural network to complete the text recognition.

[0115] Preferably, as an embodiment of the present application, the flowchart of the text segmentation and recognition process is as follows: Figure 4 shown.

[0116] Preferably, as an embodiment of the present application, a schematic diagram of a text recognition process based on a neural network is shown in FIG. Figure 5 shown.

[0117] Based on the same inventive concept as the above method, an embodiment of the present application also provides a mobile terminal text recognition system based on a neural network, including a memory, a processor, and a computer program stored in the memory and running on the processor, and when the processor executes the computer program, it implements the steps of any one of the above-mentioned mobile terminal text recognition methods based on a neural network.

[0118] It should be noted that the above sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above is a description of a specific embodiment of this specification. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0119] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0120] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for mobile terminal text recognition based on neural network, characterized in that: The method comprises the following steps: Get grayscale images of nameplates of power facilities under different light intensities; The grayscale images of each nameplate are segmented to obtain multiple regions, and the brightness adjustment factor of each pixel in each region is determined based on the variation degree of the grayscale value of each pixel in each region and the discrete degree of the grayscale values ​​of all pixels in the neighborhood of each pixel, combined with the grayscale value of each pixel in each region and the average distribution of the grayscale values ​​of all pixels; Based on the number of all pixels in the neighborhood of each pixel in each area and the distance from each pixel to all pixels in its neighborhood, combined with the size of the grayscale image of the nameplate in each area, determine the size adjustment factor of each pixel in each area; The adjustment factor of each pixel in each area is determined based on the brightness adjustment factor and the size adjustment factor of each pixel in each area to recognize the text in the grayscale image of the nameplate.

2. A method for mobile terminal text recognition based on neural network as claimed in claim 1, characterized in that: The method for determining the brightness adjustment factor of each pixel in each area is as follows: Based on the change degree of the grayscale value of each pixel in each area and the discrete degree of the grayscale values ​​of all pixels in the neighborhood of each pixel, the grayscale change weight of each pixel in each area is determined; Based on the average distribution of the grayscale values ​​of all pixels in each area and the grayscale change weights of all pixels, the average grayscale weight of each area is determined; The brightness adjustment factor of each pixel in each region is the result of merging the grayscale value of each pixel in each region with the average grayscale weight.

3. A method for mobile terminal text recognition based on neural network as claimed in claim 2, characterized in that: The grayscale change weight of each pixel point in each region is the result of the fusion of the gradient of each pixel point in each region and the variance of the grayscale values ​​of all pixels in its neighborhood.

4. A method for mobile terminal text recognition based on neural network as claimed in claim 2, characterized in that: The method for determining the average grayscale weight of each region is: Analyze the product of the grayscale change weight of each pixel in each area and the mean grayscale value of all pixels in the corresponding area, and record it as the grayscale weight of each pixel in each area; The average grayscale weight of each pixel in each region is the mean of the grayscale weights of all pixels in each region.

5. The method for mobile terminal text recognition based on neural network as claimed in claim 1, characterized in that: The method for determining the size adjustment factor of each pixel in each area is as follows: Determine the distance size of each pixel in each area based on the number of all pixels in the neighborhood of each pixel in each area and the distance from each pixel to all pixels in its neighborhood; Determine the resolution weight of the grayscale image of the nameplate in each area based on the size of the grayscale image of the nameplate in each area; A size adjustment factor for each pixel in each region is determined based on the distance size and the resolution weight.

6. A method for mobile terminal text recognition based on neural network as claimed in claim 5, characterized in that: The method for determining the distance size of each pixel point in each area is as follows: Analyze the cumulative sum of the distances from each pixel in each region to all pixels in its neighborhood, and record it as the first distance of each pixel in each region; The distance size of each pixel point in each area is the ratio of the number of all pixel points in the neighborhood of each pixel point in each area to the first distance.

7. A method for mobile terminal text recognition based on neural network as claimed in claim 5, characterized in that: The expression of the resolution weight of the grayscale image of the nameplate in each area is: In the formula, k i represents the resolution weight of the grayscale image of the nameplate in the ith area; h i represents the height of the grayscale image of the nameplate in the ith area; w i It represents the width of the grayscale image of the nameplate in the ith area; norm() represents the normalization function; min() represents the minimum function.

8. The method for mobile terminal text recognition based on neural network as claimed in claim 5, characterized in that: The method for determining the size adjustment factor of each pixel in each area is as follows: The grayscale image of the nameplate is binarized using a threshold segmentation algorithm. Pixels with a grayscale value of 1 are recorded as candidate text pixels, and pixels with a grayscale value of 0 are recorded as non-candidate text pixels. The expression for the size adjustment factor of each pixel in each area is: In the formula, λ i,j represents the size adjustment factor of the jth pixel in the i-th region; k i represents the resolution weight of the grayscale image of the nameplate in the ith area; l i,j Indicates the distance size of the jth pixel in the i-th region.

9. A method for mobile terminal text recognition based on neural network as claimed in claim 8, characterized in that: The step of determining the adjustment factor of each pixel in each area to recognize the text in the grayscale image of the nameplate includes: The adjustment factor of each pixel in each region is the normalized result of the product of the brightness adjustment factor and the size adjustment factor of each pixel in each region; The product of the adjustment factor of each pixel point in each area and the threshold value in the binarization process of the nameplate grayscale image is used as the segmentation threshold value of each pixel point in each area; The pixels in each area whose grayscale value is less than the corresponding segmentation threshold are text pixels. The coordinates and grayscale values ​​of each text pixel are combined into a vector. The vectors of all text pixels in the grayscale image of the nameplate are clustered to obtain clusters, where each cluster corresponds to a text. The neural network is used to recognize each text.

10. A mobile terminal text recognition system based on a neural network, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of a method for text recognition in a mobile terminal based on a neural network as described in any one of claims 1 to 9 are implemented.

Images