Image processing method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to an image processing method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a target image obtained by shooting a reading screen of the power equipment, and intercepting a reading screen image from the target image; segmenting the reading screen image into a plurality of sub-images; for any sub-image, performing gain processing on each pixel point in the corresponding sub-image to obtain an enhanced sub-image; and splicing the enhanced sub-images to obtain an enhanced reading screen image, filtering the enhanced reading screen image, and performing character recognition on the filtered image to obtain the reading of the reading screen. By adopting the method, the digital detection process can be realized, the reading information can be identified more quickly and more accurately, and the readable precision is higher.

Description

Technical Field

[0001] The present application relates to the field of machine vision technology, and in particular to an image processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] The power communication system often involves many power equipment, such as AC / DC rectifiers, transformers, gas turbines, etc. When the power communication system is working, it is often necessary to read and analyze the test readings of the power equipment. Traditionally, when reading the numerical values ​​of the test results of the power equipment, the power communication operation and maintenance personnel often manually read the readings in the panel cabinet of the power equipment and then record them on paper work orders, keep the paper documents and then scan and enter them into the system. This method has the problem of low data reading efficiency. Summary of the invention

[0003] Based on this, it is necessary to provide an image processing method, apparatus, computer device, computer-readable storage medium and computer program product that can improve data reading efficiency in response to the above technical problems.

[0004] In a first aspect, the present application provides an image processing method, comprising:

[0005] Acquire a target image obtained by photographing a digital display screen of the electric power equipment, and cut out a digital display screen image from the target image;

[0006] Splitting the number display screen image into a plurality of sub-images;

[0007] For any sub-image, after performing gain processing on each pixel point in the sub-image, an enhanced sub-image is obtained;

[0008] The enhanced sub-images are spliced ​​to obtain an enhanced number display screen image, the enhanced number display screen image is filtered, and text recognition is performed on the filtered image to obtain the reading of the number display screen.

[0009] In one embodiment, the step of acquiring a target image obtained by photographing a display screen of the electric power equipment includes:

[0010] Acquire a photographic image of a digital display screen of an electric power device;

[0011] Determine a perspective transformation matrix; the perspective transformation matrix represents the conversion relationship between the space coordinate system and the image coordinate system;

[0012] The captured image is perspective transformed according to the perspective transformation matrix to obtain a target image.

[0013] In one embodiment, determining the perspective transformation matrix includes:

[0014] Acquire multiple environmental images of power equipment;

[0015] Constructing the spatial position information of the power equipment according to the plurality of environmental images; the spatial position information includes the spatial coordinates of the frame of the display screen of the power equipment;

[0016] Detecting the frame position of the digital display screen in the captured image of the digital display screen of the electric power equipment;

[0017] A perspective transformation matrix is ​​determined according to the spatial coordinates of the frame of the digital display screen of the electric power equipment and the frame position of the digital display screen in the captured image.

[0018] In one embodiment, the gain processing step of each sub-image includes:

[0019] determining an enhancement mask;

[0020] Performing logarithmic transformation and filtering processing on the sub-image to obtain a first image;

[0021] Determine the gain corresponding to each pixel point in the first image according to the enhancement mask, and enhance the pixel value of each pixel point in the first image according to the corresponding gain to obtain a second image;

[0022] Performing an inverse logarithmic transformation on the second image to obtain an enhanced sub-image.

[0023] In one embodiment, determining the gain corresponding to each pixel point in the first image according to the enhancement mask includes:

[0024] Traversing each pixel in the first image;

[0025] For a pixel point currently traversed, the center of the enhancement mask is placed on the pixel point currently traversed, and the gain of the pixel point currently traversed is determined according to the pixel value of the pixel point in the first image within the area of ​​the enhancement mask;

[0026] After the traversal is completed, the gain corresponding to each pixel point in the first image is obtained.

[0027] In one embodiment, filtering the enhanced number display screen image includes:

[0028] The enhanced display screen image is expanded and eroded.

[0029] In a second aspect, the present application further provides an image processing device, comprising:

[0030] An acquisition module, used for acquiring a target image obtained by photographing the display screen of the electric power equipment, and cutting out the display screen image from the target image;

[0031] A segmentation module, used for segmenting the number display screen image into a plurality of sub-images;

[0032] A processing module, for any sub-image, performs gain processing on each pixel point in the sub-image to obtain an enhanced sub-image;

[0033] The reading module is used for splicing the enhanced sub-images to obtain an enhanced number display screen image, filtering the enhanced number display screen image, and performing text recognition on the filtered image to obtain the reading of the number display screen.

[0034] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0035] Acquire a target image obtained by photographing a digital display screen of the electric power equipment, and cut out a digital display screen image from the target image;

[0036] Splitting the number display screen image into a plurality of sub-images;

[0037] For any sub-image, after performing gain processing on each pixel point in the sub-image, an enhanced sub-image is obtained;

[0038] The enhanced sub-images are spliced ​​to obtain an enhanced number display screen image, the enhanced number display screen image is filtered, and text recognition is performed on the filtered image to obtain the reading of the number display screen.

[0039] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0040] Acquire a target image obtained by photographing a digital display screen of the electric power equipment, and cut out a digital display screen image from the target image;

[0041] Splitting the number display screen image into a plurality of sub-images;

[0042] For any sub-image, after performing gain processing on each pixel point in the sub-image, an enhanced sub-image is obtained;

[0043] The enhanced sub-images are spliced ​​to obtain an enhanced number display screen image, the enhanced number display screen image is filtered, and text recognition is performed on the filtered image to obtain the reading of the number display screen.

[0044] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0045] Acquire a target image obtained by photographing a digital display screen of the electric power equipment, and cut out a digital display screen image from the target image;

[0046] Splitting the number display screen image into a plurality of sub-images;

[0047] For any sub-image, after performing gain processing on each pixel point in the sub-image, an enhanced sub-image is obtained;

[0048] The enhanced sub-images are spliced ​​to obtain an enhanced number display screen image, the enhanced number display screen image is filtered, and text recognition is performed on the filtered image to obtain the reading of the number display screen.

[0049] The above-mentioned image processing method, device, computer equipment, computer-readable storage medium and computer program product obtain a target image obtained by photographing the number screen of the power equipment, and cut out the number screen image from the target image, thereby reducing the influence of the background environment on the number screen reading. The number screen image is divided into multiple sub-images, and gain processing is performed on each pixel point of any sub-image. By performing gain processing on each pixel point of each sub-image, the influence of light in the display image can be eliminated, so that the processing effect of the entire number screen image is better. The enhanced sub-images are spliced ​​to obtain an enhanced number screen image, and the enhanced number screen image is filtered to filter out the noise in the enhanced number screen image, eliminate interference factors, and facilitate the improvement of the accuracy of text recognition. In this way, through the digital detection process, the recognition of the reading information can be made faster and more accurate, and the readability accuracy is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0051] FIG1 is a schematic diagram of a flow chart of an image processing method in one embodiment;

[0052] FIG2 is a Gaussian filter kernel in one embodiment;

[0053] FIG3 is a schematic flow chart of an image processing method in another embodiment;

[0054] FIG4 is a block diagram of an image processing device in one embodiment;

[0055] FIG. 5 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0057] In one embodiment, as shown in FIG1 , an image processing method is provided. This embodiment is described by taking the method applied to a terminal as an example. It is understandable that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps 102 to 108:

[0058] Step 102: acquiring a target image obtained by photographing a digital display screen of the electric power equipment, and cutting out a digital display screen image from the target image.

[0059] The power equipment may be any power equipment with a display screen, which is not limited in the present embodiment. The power equipment may be an AC / DC rectifier, a transformer, a gas turbine, etc. The display screen may display parameter information related to the power equipment.

[0060] In some embodiments, the terminal may obtain a target image captured by an image acquisition device or a camera, or the target image may be captured by the terminal itself. Before acquiring the target image, the terminal may process the captured image to obtain the target image, and then extract the digital screen image from the target image.

[0061] In some embodiments, the processing of the captured image may be to correct the image, or to filter the captured image and then correct it. The method of correcting the image may be to eliminate the skew of the captured image through a perspective matrix.

[0062] Specifically, the terminal can shoot the digital display screen of the power equipment to obtain the captured image of the digital display screen of the power equipment, and detect the border of the digital display screen in the captured image through the edge detection algorithm, and combine the spatial coordinates of the digital display screen border in the three-dimensional world and the plane coordinates of the digital display screen in the captured image to determine the perspective transformation matrix between the two. The captured image is perspective transformed according to the perspective transformation matrix to obtain the target image. The digital display screen image is cut out from the target image along the digital display screen border. Among them, the edge detection algorithm can also be other algorithms that can detect the border, and the embodiment of the present application does not limit this.

[0063] Step 104, dividing the number display screen image into a plurality of sub-images.

[0064] In some embodiments, the terminal may divide the number display screen image into a preset number of sub-images of the same size, and record the numbers and positions of the divided sub-images.

[0065] Step 106: for any sub-image, perform gain processing on each pixel in the sub-image to obtain an enhanced sub-image.

[0066] In some embodiments, the terminal may perform enhancement processing on the sub-image, wherein the enhancement processing method may be filtering processing, histogram equalization, gain control algorithm, etc.

[0067] Specifically, the terminal determines the size of the enhancement mask according to the size of the display screen image. The terminal performs logarithmic transformation and filtering on each sub-image to obtain a first image. Then, the gain corresponding to each pixel in the first image is calculated according to the enhancement mask, and each pixel in the first image is multiplied by the corresponding gain to obtain a second image. The second image is inversely logarithmically transformed to obtain an enhanced sub-image.

[0068] Among them, the filtering process can be Gaussian filtering or other filtering methods, such as adaptive median filtering, bilateral filtering, etc., which is not limited in the embodiments of the present application.

[0069] In some embodiments, the length of the enhanced mask is the length of the numbering screen image divided by a preset value a, and the width of the enhanced mask is the width of the numbering screen image divided by a preset value b. Where a and b are specifically determined by the aspect ratio of the numbering screen area.

[0070] Step 108, stitching the enhanced sub-images to obtain an enhanced number display screen image, filtering the enhanced number display screen image, and performing text recognition on the filtered image to obtain the reading of the number display screen.

[0071] The filtering process is to dilate and corrode the image. The text recognition method may be OCR (Optical Character Recognition) text recognition, which is not limited in the present embodiment.

[0072] In some embodiments, the terminal will splice the enhanced sub-images according to the numbers and positions during the above segmentation to obtain an enhanced digital display screen image, dilate and erode the enhanced digital display screen image to obtain a filtered image, and perform text recognition on the filtered image to obtain the reading of the digital display screen.

[0073] In some embodiments, dilating the enhanced digital screen image includes: defining a convolution kernel and sliding it on the enhanced digital screen image, when the area covered by the convolution kernel meets a specific condition (there is at least one point with a pixel value of 1), the corresponding pixel of the output image is set to 1, otherwise it remains at 0; thereby obtaining the dilated digital screen image.

[0074] In some embodiments, the erosion processing of the expanded image includes: defining a convolution kernel and sliding it on the expanded digital image. When the convolution kernel is completely contained in the target area of ​​the image, if the pixel values ​​of all positions are 1, the corresponding pixels of the output image are set to 1, otherwise they remain 0; thereby obtaining the eroded image.

[0075] In some embodiments, performing text recognition on the filtered image includes: the terminal performs character segmentation on the filtered image, extracts the feature representation of the characters, and then uses a general classifier to classify the character features, and finally combines the recognized characters into a final text for output to obtain the reading of the display screen. The general classifier is a model that is trained to predict the category of a given input data point.

[0076] In the above-mentioned image processing method, a target image is obtained by photographing the number screen of the electric equipment, and the number screen image is intercepted from the target image, thereby reducing the influence of the background environment on the number screen reading. The number screen image is divided into multiple sub-images, and each pixel point of any sub-image is subjected to gain processing. By performing gain processing on each pixel point of each sub-image, the influence of light in the image can be eliminated, so that the processing effect of the entire number screen image is better. The enhanced sub-images are spliced ​​to obtain an enhanced number screen image, and the enhanced number screen image is filtered to filter out the noise in the enhanced number screen image, eliminate interference factors, and facilitate the improvement of the accuracy of text recognition. In this way, through the digital detection process, the recognition of the reading information is faster, more accurate, and the readability is higher.

[0077] Next, the acquisition of the target image obtained by photographing the display screen of the electric power equipment and the specific implementation process are described.

[0078] In some exemplary embodiments, acquiring a target image obtained by photographing a number display screen of an electric power device includes: acquiring a photographed image of the number display screen of the electric power device; and determining a perspective transformation matrix; the perspective transformation matrix represents a conversion relationship between a space coordinate system and an image coordinate system.

[0079] In the actual implementation process, the terminal or image acquisition device shoots the digital display screen of the power equipment to obtain the captured image of the digital display screen of the power equipment, and detects the frame of the digital display screen in the captured image through the edge detection algorithm, and combines the spatial coordinates of the digital display screen frame in the three-dimensional world and the plane coordinates of the digital display screen in the captured image to determine the perspective transformation matrix between the two, that is, the conversion relationship between the spatial coordinate system and the image coordinate system of the two. The captured image is perspective transformed according to the perspective transformation matrix, that is, each pixel in the captured image is multiplied by the perspective transformation matrix to obtain the target image. Among them, the edge detection algorithm can also be other algorithms that can detect the frame, and the embodiment of the present application does not limit this.

[0080] Specifically, the method of detecting the border of the digital display screen in the captured image by the edge detection algorithm is as follows: the terminal first obtains the horizontal and vertical gradients in the captured image, calculates the first-order derivative values ​​of the horizontal and vertical gradients based on the gradients, and determines the gradient size and direction of the pixel point, where the position with the largest gradient and which can be connected into a line is the border.

[0081] In the above embodiment, the target image is obtained by performing perspective transformation on the captured image, thereby eliminating the image distortion generated during the image capturing process of the terminal, so that the final text recognition is more accurate.

[0082] In some exemplary embodiments, determining a perspective transformation matrix includes: acquiring multiple environmental images of the power equipment; constructing spatial position information of the power equipment based on the multiple environmental images; the spatial position information includes the spatial coordinates of a border of a number screen of the power equipment; detecting a border position of the number screen in a captured image of the number screen of the power equipment; and determining the perspective transformation matrix based on the spatial coordinates of the border of the number screen of the power equipment and the border position of the number screen in the captured image.

[0083] In actual implementation, the terminal captures multiple environmental images of the power equipment, and constructs the spatial position information of the power equipment based on the environmental images of the power equipment. The spatial position information of the power equipment includes the spatial coordinates of the digital display on the power equipment. The terminal calculates the mapping relationship between the spatial coordinates of the digital display frame of the power equipment and the position of the digital display frame in the captured image, that is, the perspective transformation matrix.

[0084] Specifically, the formula for obtaining the perspective transformation matrix is ​​shown in (1).

[0085]

[0086] in, is the spatial coordinate of the border of the power equipment display screen, is the coordinate of the digital screen border in the captured image, This is the desired perspective transformation matrix.

[0087] Specifically, the method for constructing the spatial position information of the power equipment based on the environmental image of the power equipment is as follows: the terminal obtains the position of the device that takes multiple environmental images of the power equipment, and establishes a mapping relationship between the three-dimensional world and the two-dimensional pixel plane based on the above position information and the spatial coordinates of each point of the power equipment on the power equipment environmental image and the two-dimensional coordinates on the taken image; and reconstructs the three-dimensional spatial model of the power equipment environment based on the mapping relationship, wherein the three-dimensional spatial model includes the spatial position information of the power equipment.

[0088] In the above embodiment, the mapping relationship between the spatial coordinates of the display screen of the electric power equipment and the two-dimensional coordinates in the image, that is, the perspective transformation matrix, is calculated based on the spatial coordinates of the frame of the display screen and the coordinates in the captured image. This makes preliminary preparations for eliminating image skew, reduces the amount of calculation in the skew elimination process, and improves the efficiency of text recognition.

[0089] In some exemplary embodiments, the gain processing step of each sub-image includes: determining an enhancement mask; performing logarithmic transformation and filtering on the sub-image to obtain a first image; determining the gain corresponding to each pixel point in the first image according to the enhancement mask, and enhancing the pixel value of each pixel point of the first image according to the corresponding gain to obtain a second image; and performing an inverse logarithmic transformation on the second image to obtain an enhanced sub-image.

[0090] In actual implementation, the terminal will determine an enhancement mask of size m*n according to the size of the digital screen image. The length of the enhancement mask is the length of the digital screen image divided by the preset value a. The specific calculation formula is shown in (2). The width of the enhancement mask is the width of the digital screen image divided by the preset value b. The specific calculation formula is shown in (3).

[0091]

[0092]

[0093] Wherein, width is the length of the digital screen image, height is the width of the digital screen image, and a and b are determined by the aspect ratio of the digital screen area.

[0094] The terminal performs logarithmic transformation and filtering processing on each sub-image to obtain a first image.

[0095] In a specific embodiment, the filtering process may be Gaussian filtering, and specific formulas of logarithmic transformation and Gaussian filtering are shown in (4) and (5).

[0096]

[0097]

[0098] in, is the sub-image after logarithmic transformation, is the d-th sub-image, G is the sub-image after logarithmic transformation and Gaussian filtering, is the Gaussian filter kernel, as shown in Figure 2.

[0099] Determining the gain corresponding to each pixel point in the first image according to the enhancement mask, including: traversing each pixel point in the first image; for the pixel point currently traversed, placing the center of the enhancement mask on the pixel point currently traversed, and determining the gain of the pixel point currently traversed according to the pixel value of the pixel point in the first image within the area of ​​the enhancement mask; after the traversal is completed, obtaining the gain corresponding to each pixel point in the first image.

[0100] In actual implementation, the terminal may traverse each pixel point in the first image. For the currently traversed pixel point, the terminal will calculate the gain of each traversed pixel point according to the pixel points around the enhancement mask.

[0101] In some embodiments, the terminal places the center of the enhancement mask on the pixel currently traversed, translates it in steps of 1 pixel, traverses all the pixels in this row from left to right, and after completing the traversal of this row, moves the center of the mask down by 1 pixel, and continues to traverse until all the pixels in the first image are traversed.

[0102] In a specific embodiment, the gain of each pixel in the first image is calculated, and the specific formula is shown in (6).

[0103]

[0104] in, represents the gain of pixel (i, j), It represents the standard deviation of the pixel values ​​of all pixels in the mask when the center point of the mask is located at (i, j). The calculation formula is shown in (7). k is the gain control function, which is obtained based on experience and is used to control the gain value not to be too large.

[0105]

[0106] in, are all pixels in the mask, x is the average value of all pixels in the mask, and n is the number of all pixels in the mask.

[0107] In the above embodiment, by calculating the gain of each pixel in the first image through a mask, each first image can be effectively enhanced, so that the final result is more accurate.

[0108] Multiply each pixel of the first image by its corresponding gain to obtain the second image. The specific formula is shown in (8).

[0109]

[0110] in, is the enhanced pixel value of point (i, j), is the original pixel value of point (i, j).

[0111] After the terminal enhances the first image, it obtains the second image. Then, the second image is inversely logarithmically transformed to obtain an enhanced sub-image. The specific formula is shown in (9).

[0112]

[0113] in, is the enhanced sub-image, is a set of pixels of the second image.

[0114] In the above embodiment, through a series of processing of the image, the adaptability of the reading of the digital display screen to various environments is enhanced, the interference of the background information of the image on the reading result is reduced, and the accuracy of the final result is improved.

[0115] In some exemplary embodiments, filtering the enhanced number display screen image includes: dilating and corroding the enhanced number display screen image.

[0116] In actual implementation, the terminal first obtains the enhanced image, defines a convolution kernel, and slides it on the enhanced numerical image. When the area covered by the convolution kernel meets specific conditions (there is at least one point with a pixel value of 1), the corresponding pixel of the output image is set to 1, otherwise it remains at 0; thereby obtaining the expanded numerical image.

[0117] Furthermore, a convolution kernel is defined and slid on the expanded digital screen image. When the convolution kernel is completely contained in the target area of ​​the image, if the pixel values ​​of all positions are 1, the corresponding pixels of the output image are set to 1, otherwise they remain 0; thus, the eroded image is obtained. The filtering process of the enhanced digital screen image is completed.

[0118] In the above embodiment, the background noise of the image is eliminated by dilating and corroding the enhanced number display screen image, so that the image for text recognition is clearer.

[0119] In a specific embodiment, the image processing method comprises the following steps:

[0120] The terminal obtains the locations where multiple images of the power equipment environment are taken, and establishes a mapping relationship between the three-dimensional world and the two-dimensional pixel plane based on the above-mentioned location information and the spatial coordinates of each point of the power equipment on the power equipment environment image and the two-dimensional coordinates on the image, and reconstructs a three-dimensional space model of the power equipment environment based on the mapping relationship, and the three-dimensional space model includes the spatial location information of the power equipment.

[0121] The terminal can shoot the digital display screen of the power equipment to obtain the photographed image of the digital display screen of the power equipment, and detect the frame of the digital display screen in the photographed image through the edge detection algorithm, and combine the spatial coordinates of the digital display screen frame in the three-dimensional world and the plane coordinates of the digital display screen in the photographed image to determine the perspective transformation matrix between the two. The photographed image is perspectively transformed according to the perspective transformation matrix to obtain the target image. The digital display screen image is cut out from the target image along the digital display screen frame. The digital display screen image is divided into a preset number of sub-images of the same size.

[0122] The size of the enhancement mask is determined according to the size of the digital screen image. The terminal performs logarithmic transformation and filtering on each sub-image to obtain the first image. Then, the gain corresponding to each pixel in the first image is calculated according to the enhancement mask, and each pixel in the first image is multiplied by the corresponding gain to obtain the second image. The second image is inversely logarithmically transformed to obtain the enhanced sub-image.

[0123] The dilation processing of the enhanced digital screen image includes: defining a convolution kernel and sliding it on the enhanced digital screen image, when the area covered by the convolution kernel meets a specific condition (there is at least one point with a pixel value of 1), the corresponding pixel of the output image is set to 1, otherwise it remains to 0; thereby obtaining the dilated digital screen image.

[0124] Define a convolution kernel and slide it on the expanded image. When the convolution kernel is completely contained in the target area of ​​the image, if the pixel values ​​at all positions are 1, the corresponding pixels of the output image are set to 1, otherwise they remain 0; thus obtaining the eroded image.

[0125] The filtered image is segmented and the feature representation of the characters is extracted. The character features are then classified using a general classifier. Finally, the recognized characters are combined into the final text for output to obtain the readings on the display screen.

[0126] To explain the image processing method in the present application in detail, an embodiment is used for explanation below. The specific flow chart is shown in FIG3 . By way of example, the present application is explained with an image processing method under a digital display of a specific AC / DC rectifier panel cabinet.

[0127] First, the terminal obtains the location of the device that takes multiple images of the AC / DC rectifier environment, and establishes a mapping relationship between a three-dimensional world and a two-dimensional pixel plane based on the above location information and the spatial coordinates of each point of the AC / DC rectifier on the AC / DC rectifier environment image and the two-dimensional coordinates on the image. According to the mapping relationship, a three-dimensional spatial model of the AC / DC rectifier environment is reconstructed, and the three-dimensional spatial model includes the spatial location information of the AC / DC rectifier.

[0128] Obtain the horizontal and vertical gradients in the captured image, calculate the first-order derivative values ​​in the horizontal and vertical directions based on the gradients, determine the gradient size and direction of the pixel point, and the position where the gradient is the largest and can be connected into a line is the border. According to the spatial coordinates of the display screen border of the AC / DC rectifier and the position of the display screen border in the captured image, calculate the mapping relationship between the two, that is, the perspective transformation matrix. The formula of the perspective transformation matrix is ​​shown in (1). Perform a perspective transformation on the captured image according to the perspective transformation matrix, that is, multiply each pixel in the captured image by the perspective transformation matrix to obtain the target image. The display screen image is cut out from the target image along the display screen border, and the display screen image is divided into a preset number of sub-images of the same size, and the numbers and positions of the divided sub-images are recorded.

[0129] In actual implementation, the terminal determines an enhancement mask of size m*n according to the size of the digital screen image. The length of the enhancement mask is the length of the digital screen image divided by a preset value a, and the width of the enhancement mask is the width of the digital screen image divided by a preset value b. The specific calculation formulas are shown in (2) and (3). Then, each sub-image is subjected to logarithmic transformation and Gaussian filtering to obtain the first image. The specific formulas of logarithmic transformation and Gaussian filtering are shown in (4) and (5).

[0130] The enhancement mask is traversed over each pixel in the first image. For the pixel currently traversed, the terminal places the center of the enhancement mask on the pixel currently traversed, performs a translation of 1 pixel step, and traverses all the pixels in this row from left to right. After completing the traversal of this row, the center of the mask is moved down by 1 pixel, and the traversal is continued until all the pixels in the first image are traversed. The gain of each pixel in the first image is calculated, and the specific formula is shown in (6). After the traversal in the first image is completed, if the enhancement of the first image is not completed, the center of the enhancement mask is moved to the right / down by 1 pixel, and the pixels of the first image are continued to be traversed until the enhancement of the first image is completed. Each pixel of the first image is multiplied by its corresponding gain to obtain the second image, and the specific formula is shown in (8). Then the second image is inversely logarithmically transformed to obtain the enhanced sub-image, and the specific formula is shown in (9).

[0131] The enhanced sub-images are spliced ​​according to the numbers and positions during the above segmentation to obtain the enhanced digital screen image. A convolution kernel is defined and slid on the enhanced digital screen image. When the area covered by the convolution kernel meets a specific condition (there is at least one point with a pixel value of 1), the corresponding pixel of the output image is set to 1, otherwise it remains at 0; thus, the expanded digital screen image is obtained.

[0132] Define a convolution kernel and slide it on the expanded digital image. When the convolution kernel is completely contained in the target area of ​​the image, if the pixel values ​​at all positions are 1, the corresponding pixels of the output image are set to 1, otherwise they remain 0; thus obtaining the filtered image.

[0133] The filtered image is segmented and the feature representation of the characters is extracted. The character features are then classified using a general classifier. Finally, the recognized characters are combined into the final text for output to obtain the readings on the display screen.

[0134] The present application filters out the noise in the image and enhances the brightness of the image by performing a series of processing on the display screen before reading, so that the final automated reading result is more accurate. The present application provides an automated reading method, realizes a digital detection process, reduces the waste of manpower and material resources in the reading process, and improves the efficiency of reading.

[0135] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0136] Based on the same inventive concept, the embodiment of the present application also provides an image processing device for implementing the above-mentioned image processing method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more image processing device embodiments provided below can refer to the limitations on the image processing method above, and will not be repeated here.

[0137] In an exemplary embodiment, as shown in FIG4 , an image processing device is provided, comprising: an acquisition module 401 , a segmentation module 402 , a processing module 403 and a reading module 404 , wherein:

[0138] The acquisition module is used to acquire a target image obtained by photographing the display screen of the electric power equipment, and to cut out the display screen image from the target image.

[0139] The segmentation module is used to segment the number display screen image into multiple sub-images.

[0140] The processing module performs gain processing on each pixel point in any sub-image to obtain an enhanced sub-image.

[0141] The reading module is used for splicing the enhanced sub-images to obtain an enhanced number display screen image, filtering the enhanced number display screen image, and performing text recognition on the filtered image to obtain the reading of the number display screen.

[0142] In some embodiments, the acquisition module is also used to acquire a captured image of the display screen of the power equipment; determine the perspective transformation matrix; the perspective transformation matrix represents the conversion relationship between the space coordinate system and the image coordinate system; perform perspective transformation on the captured image according to the perspective transformation matrix to obtain the target image.

[0143] In some embodiments, the device also includes a determination module, which is used to obtain multiple environmental images of the power equipment; construct spatial position information of the power equipment based on the multiple environmental images; the spatial position information includes the spatial coordinates of the border of the number screen of the power equipment; detect the border position of the number screen in the captured image of the number screen of the power equipment; determine the perspective transformation matrix based on the spatial coordinates of the border of the number screen of the power equipment and the border position of the number screen in the captured image.

[0144] In some embodiments, the above-mentioned processing module is also used to determine an enhancement mask; perform logarithmic transformation and filtering on the sub-image to obtain a first image; determine the gain corresponding to each pixel point in the first image according to the enhancement mask, and enhance the pixel value of each pixel point of the first image according to the corresponding gain to obtain a second image; perform inverse logarithmic transformation on the second image to obtain an enhanced sub-image.

[0145] In some embodiments, the above-mentioned processing module is also used to traverse each pixel point in the first image; for the pixel point currently traversed, the center of the enhancement mask is placed on the pixel point currently traversed, and the gain of the pixel point currently traversed is determined based on the pixel value of the pixel point in the first image within the area of ​​the enhancement mask; after the traversal is completed, the gain corresponding to each pixel point in the first image is obtained.

[0146] In some embodiments, the processing module is further used to dilate and erode the enhanced display screen image. Each module in the image processing device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0147] In an exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG5. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, an image processing method is implemented. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0148] Those skilled in the art will understand that the structure shown in FIG. 5 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0149] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the following steps are implemented when the processor executes the computer program.

[0150] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented.

[0151] In one embodiment, a computer program product is provided, comprising a computer program, which implements the following steps when executed by a processor.

[0152] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0153] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0154] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0155] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a target image obtained by photographing a digital display screen of the electric power equipment, and cut out a digital display screen image from the target image; Splitting the number display screen image into a plurality of sub-images; For any sub-image, after performing gain processing on each pixel point in the sub-image, an enhanced sub-image is obtained; The enhanced sub-images are spliced ​​to obtain an enhanced number display screen image, the enhanced number display screen image is filtered, and text recognition is performed on the filtered image to obtain the reading of the number display screen.

2. The method according to claim 1, characterized in that The step of acquiring a target image obtained by photographing a display screen of the electric power equipment comprises: Acquire a photographic image of a digital display screen of an electric power device; Determine a perspective transformation matrix; the perspective transformation matrix represents the conversion relationship between the space coordinate system and the image coordinate system; The captured image is perspective transformed according to the perspective transformation matrix to obtain a target image.

3. The method according to claim 2, characterized in that The determining of the perspective transformation matrix comprises: Acquire multiple environmental images of power equipment; Constructing the spatial position information of the power equipment according to the plurality of environmental images; the spatial position information includes the spatial coordinates of the frame of the display screen of the power equipment; Detecting the frame position of the digital display screen in the captured image of the digital display screen of the electric power equipment; A perspective transformation matrix is ​​determined according to the spatial coordinates of the frame of the digital display screen of the electric power equipment and the frame position of the digital display screen in the captured image.

4. The method according to claim 1, characterized in that: The gain processing steps for each sub-image include: determining an enhancement mask; Performing logarithmic transformation and filtering processing on the sub-image to obtain a first image; Determine the gain corresponding to each pixel point in the first image according to the enhancement mask, and enhance the pixel value of each pixel point in the first image according to the corresponding gain to obtain a second image; Performing an inverse logarithmic transformation on the second image to obtain an enhanced sub-image.

5. The method according to claim 4, characterized in that The determining, according to the enhancement mask, a gain corresponding to each pixel point in the first image, comprises: Traversing each pixel in the first image; For a pixel point currently traversed, the center of the enhancement mask is placed on the pixel point currently traversed, and the gain of the pixel point currently traversed is determined according to the pixel value of the pixel point in the first image within the area of ​​the enhancement mask; After the traversal is completed, the gain corresponding to each pixel point in the first image is obtained.

6. The method according to claim 1, characterized in that The filtering process of the enhanced display screen image comprises: The enhanced display screen image is expanded and eroded.

7. An image processing device, characterized in that: The device comprises: An acquisition module, used for acquiring a target image obtained by photographing the display screen of the electric power equipment, and cutting out the display screen image from the target image; A segmentation module, used for segmenting the number display screen image into a plurality of sub-images; A processing module, for any sub-image, performs gain processing on each pixel point in the sub-image to obtain an enhanced sub-image; The reading module is used for splicing the enhanced sub-images to obtain an enhanced number display screen image, filtering the enhanced number display screen image, and performing text recognition on the filtered image to obtain the reading of the number display screen.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images