Meter image extraction method and device

Abstract

The invention discloses a meter image extraction method and device, and the method comprises the steps: extracting a plurality of to-be-processed images from an image list corresponding to a target meter according to a preset time interval, obtaining a geometric figure of a dial plate through employing each to-be-processed image for shape analysis, and further selecting a time period through employing a plurality of dial plate shape analysis results, and obtaining a geometric figure of the dial plate according to the selected time period. And performing image extraction on the image list again according to the selected at least one target time period, calculating a definition score for the dial reading area in each extracted candidate image, selecting the candidate image corresponding to the highest definition score as a target image, and displaying the selected target image on the dial reading area. The method realizes staged image selection under the constraint of the geometrical shape and definition of the part with the reading in the meter corresponding to the image, helps to improve the extraction probability of the meter image with the highest reading definition, and helps to better support accurate identification of the reading of the meter.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a meter image extraction method and device. Background Art

[0002] In the actual substation inspection process, due to various factors (such as three-dimensional model errors, inspection positioning errors, etc.), it is difficult to stably obtain images that meet quality requirements if only one photo is taken at the planned point. Therefore, it is necessary to extract the highest quality image from multiple images or video frames for subsequent use. That is, when automatically identifying meter indications, the extracted images should better support accurate identification of meter indications (such as pointers or numbers, etc.).

[0003] However, in the image extraction process, the existing methods simply consider the overall clarity of each image, but do not consider the specific situation of the parts with indications in the meter corresponding to the image. The probability of extracting the best image reflecting the meter indication area is low, and there is a high possibility that the extracted image will have large errors or even fail in recognition when used for meter indication recognition. Summary of the invention

[0004] The present invention aims to solve the problems in the related art to at least a certain extent, and provides a meter image extraction method and device.

[0005] In one aspect, the present invention provides a meter image extraction method, comprising:

[0006] Extracting a number of images to be processed from an image list corresponding to a target meter at a preset time interval;

[0007] Obtaining a dial geometry figure from each of the to-be-processed images, and performing shape analysis on the dial geometry figure; wherein the dial geometry figure is suitable for characterizing the shape of the dial in the target meter;

[0008] Selecting a time period based on a number of dial shape analysis results corresponding to a number of the images to be processed, and extracting a number of candidate images from the image list according to at least one selected target time period;

[0009] Calculating a clarity score for a dial indication area in each candidate image; wherein the dial indication area is suitable for indicating the indication on the dial;

[0010] The candidate image corresponding to the highest definition score is selected as the target image.

[0011] Preferably, obtaining a dial geometry from each of the images to be processed comprises:

[0012] For any of the images to be processed, determining an original size and setting a target size; wherein the target size is larger than a preset meter imaging size and smaller than the original size;

[0013] A meter area is extracted from the image to be processed according to the target size, and then the dial geometry is detected from the meter area.

[0014] Preferably, the detecting the dial geometry from the meter area comprises:

[0015] Performing line detection on the meter area to obtain meter line data;

[0016] Performing ellipse detection on the meter line data;

[0017] If only one ellipse is detected, the ellipse is directly used as the dial geometry;

[0018] If multiple ellipses are detected, obtain the major axis radius and the minor axis radius of each ellipse respectively; determine whether the major axis radius of each ellipse is equal. If so, take the ellipse with the smallest difference between the minor axis radius and the preset radius as the dial geometry; if not, take the ellipse with the smallest difference between the major axis radius and the preset radius as the dial geometry.

[0019] Preferably, the dial geometric figure is an ellipse, and the dial shape analysis result includes the major axis distance and the minor axis ratio of the ellipse;

[0020] The performing shape analysis on the dial geometric figure comprises:

[0021] Measuring the major and minor wheelbases of the ellipse respectively;

[0022] The ratio between the short wheelbase and the long wheelbase is determined as the short-to-long wheelbase ratio.

[0023] Preferably, measuring the major wheelbase and the minor wheelbase of the ellipse respectively comprises:

[0024] Generating a minimum outer bounding rectangle for the ellipse;

[0025] Calculate the length and width of the minimum outer enclosing rectangle;

[0026] The length is determined as the major wheelbase, and the width is determined as the minor wheelbase.

[0027] Preferably, each of the dial shape analysis results includes the minor-major axis ratio and the major-axis distance of the ellipse;

[0028] The selecting of time periods based on a number of dial shape analysis results corresponding one-to-one to a number of the images to be processed comprises:

[0029] According to a number of moments matched one by one with the number of images to be processed, all the short-to-long axis ratios in the number of dial shape analysis results are sorted into a first sequence, and all the long-axis distances in the number of dial shape analysis results are sorted into a second sequence;

[0030] Selecting the moment corresponding to the minor-major axis ratio with the largest value in the first sequence as the first reference moment, and selecting each moment adjacent to the first reference moment before and after as the first adjacent moment;

[0031] Determine two long wheelbases corresponding to the two first adjacent moments in the second sequence, and use the larger one of the two long wheelbases corresponding to the two first adjacent moments as the first long wheelbase;

[0032] A target time period is determined according to the first reference time and the first adjacent time corresponding to the first long wheelbase.

[0033] Preferably, the selecting of time periods based on a number of dial shape analysis results corresponding one-to-one to a number of the images to be processed further comprises:

[0034] Taking the period between the two first adjacent moments as a processed period, and partially filtering the first sequence according to the processed period to obtain a corresponding third sequence;

[0035] Selecting the moment corresponding to the minor-major axis ratio with the largest value in the third sequence as the second reference moment, and selecting each moment adjacent to the second reference moment before and after as the second adjacent moment;

[0036] Determine two long wheelbases corresponding to the two second adjacent moments in the second sequence, and use the larger one of the two long wheelbases corresponding to the two second adjacent moments as the second long wheelbase;

[0037] Another target period is determined according to the second reference time and the second adjacent time corresponding to the second major wheelbase.

[0038] Preferably, calculating the clarity score for the dial indication area in each candidate image comprises:

[0039] Extracting the dial indication area from any of the candidate images;

[0040] Calculate the gradient average value for the edge in the dial indication area;

[0041] The clarity score matching the meter reading area is determined based on the gradient average.

[0042] Preferably, extracting the dial indication area from any of the candidate images comprises:

[0043] Extracting at least one pending area from the candidate image according to a preset dial indication color;

[0044] If only one undetermined area is extracted, directly determine the undetermined area as the dial indication area;

[0045] If multiple pending regions are extracted, performing numerical structure judgment on the multiple pending regions;

[0046] When it is determined that the plurality of pending areas are characterized by digital structures, the pending area closest to the center of the dial is selected as the dial indication area;

[0047] When it is determined that a plurality of the to-be-determined areas are all characterized as pointer structures, the to-be-determined area with the largest area is selected as the dial indication area.

[0048] Based on the above method embodiments, the present invention provides corresponding device embodiments.

[0049] Another aspect of the present invention provides a meter image extraction device, comprising: an image extraction module, a shape analysis result generation module, an image screening module, an image clarity calculation module and an image extraction module;

[0050] The image extraction module is used to extract a number of images to be processed from the image list corresponding to the target meter at a preset time interval;

[0051] The shape analysis result generating module is used to obtain a dial geometry figure from each of the to-be-processed images, and perform shape analysis on the dial geometry figure; wherein the dial geometry figure is suitable for characterizing the shape of the dial in the target meter;

[0052] The image screening module is used to select a time period based on a number of dial shape analysis results corresponding to a number of the images to be processed, and extract a number of candidate images from the image list according to at least one selected target time period;

[0053] The image clarity calculation module is used to calculate a clarity score for a dial indication area in each candidate image; wherein the dial indication area is suitable for indicating the indication on the dial;

[0054] The image extraction module is used to select the candidate image corresponding to the highest clarity score as the target image.

[0055] Using the meter image extraction method and device, firstly, a number of images to be processed are extracted from the image list at a preset time interval. For the image list, some images are appropriately filtered out and the uniformity of the images to be processed in the time dimension is ensured. Then, the dial geometry is obtained by using each image to be processed, and the dial geometry is analyzed to obtain the dial shape analysis result. The dial geometry represents the geometric shape of the dial with indication in the target meter corresponding to the image, which can take into account the efficiency and accuracy of the specific situation analysis of the geometric shapes of the dial corresponding to different images. Further, using a number of The dial shape analysis result selection period will use at least one selected target period as the basis for re-extracting images from the image list, so that several candidate images can be selected quickly and accurately, and the clarity score is calculated for the dial indication area in each candidate image, and the candidate image corresponding to the highest clarity score is selected as the target image, which is the meter image with good indication. This realizes the staged selection of images under the constraints of the geometric shape and clarity of the parts with indication in the meter corresponding to the image, which helps to increase the probability of extracting the meter image with the best indication, and helps to better support the accurate identification of meter indications. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 The figure is a flow chart of a meter image extraction method provided by one embodiment of the present invention.

[0057] Figure 2 It is a schematic diagram of the curve distribution of the short-to-long axis ratio and the long wheelbase provided by another embodiment of the present invention.

[0058] Figure 3 It is a flowchart of key frame image extraction provided by another embodiment of the present invention.

[0059] Figure 4 It is a structural schematic diagram of a meter image extraction device provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] Meters usually have components with indication functions (for example, dials). Affected by factors such as camera viewing angle and imaging distortion, the geometric shape characteristics and clarity of the aforementioned components in the image may become important information on the quality of the indication. In order to better obtain an image that helps to accurately identify the meter indication from a series of images corresponding to the target meter, it is necessary to take into account the geometric shape and clarity of the aforementioned components in the image during the meter image selection process.

[0062] like Figure 1 FIG. 1 is a flow chart of a meter image extraction method provided by an embodiment of the present invention. The meter image extraction method comprises:

[0063] Step S1: extracting a number of images to be processed from an image list corresponding to a target meter at a preset time interval;

[0064] Step S2: obtaining a dial geometry figure from each of the to-be-processed images, and performing shape analysis on the dial geometry figure; wherein the dial geometry figure is suitable for characterizing the shape of the dial in the target meter;

[0065] Step S3: performing time period selection based on a number of dial shape analysis results corresponding one-to-one to a number of the images to be processed, and extracting a number of candidate images from the image list according to at least one selected target time period.

[0066] Step S4: calculating a clarity score for a dial indication area in each candidate image; wherein the dial indication area is suitable for indicating the indication on the dial;

[0067] Step S5: Select the candidate image corresponding to the highest clarity score as the target image.

[0068] For step S1, the present invention can extract an image suitable for meter readings from images collected during substation meter inspections. In order to ensure the clarity and readability of the meter area of ​​the final extracted image in the image, the present invention can generate the final image to be read through two screening methods: dial geometry extraction and clarity comparison.

[0069] Based on the acquired image list, several images to be processed can be extracted from the image list at preset time intervals; it can be understood that the present invention can sort the serial images / videos of the patrol points according to time, and extract a frame of images as images to be processed at fixed time intervals, such as 3 seconds.

[0070] Since a large amount of image data may be generated during the inspection of power station meters, the embodiment of the present invention can significantly reduce the number of images that need to be processed by extracting a number of images to be processed from the image list at preset time intervals, thereby improving the overall processing efficiency and speed.

[0071] After the image to be processed is extracted, subsequent processing and analysis can be performed more conveniently and quickly on the image to be processed, such as ellipse extraction, calculation of the minor-major axis ratio and the major-axis ratio, so as to quickly respond to the image processing speed and extraction rate.

[0072] For step S2, in a preferred embodiment, obtaining a dial geometry from each of the images to be processed comprises:

[0073] For any of the images to be processed, determine the original size and set the target size; wherein the target size is larger than a preset meter imaging size and smaller than the original size;

[0074] A meter area is extracted from the image to be processed according to the target size, and then the dial geometry is detected from the meter area.

[0075] In a preferred embodiment, for images taken at a closer distance, that is, images in which the meter occupies a larger space, the center area of ​​the entire photo can be directly used as the approximate position. For example, if the image size is 10*10, then based on the calculation result between the preset ratio and the original image size of 10*10, the imaging size of the area used to represent the meter area is 6*6, and then the dial geometry used to represent the meter area is extracted based on the graphic detection algorithm.

[0076] In a preferred embodiment, for images taken at a long distance, that is, when the meter occupies a small space in the image, or when the meter cannot be found by directly cropping the central area, a lightweight deep learning method (such as YOLO) can be used to extract the meter in the image. If it still cannot be found, it is considered that there is no meter in the image, and the image is deleted from the second image list, and the analysis continues for the next image.

[0077] Illustratively, if the distance at which the image is captured is greater than a preset threshold, the previous embodiment is adopted, otherwise, the next embodiment is adopted, and the distance at which each image is captured can be considered as a known image attribute.

[0078] In a preferred embodiment, if the image is taken near the patrol point of the meter, that is, the image of the object at the distance of the meter is clearer and the background is blurred, the background can be removed by retaining the high frequency, thereby obtaining an elliptical area used to characterize the dial area.

[0079] By extracting the dial geometry, the embodiment of the present invention can make subsequent processing and analysis only for the area without processing the entire image, thereby improving processing efficiency. In addition, the present invention can use different methods to extract dial geometry for images taken at different distances and angles. For images taken at a closer distance, the center area is cropped and a graphic detection algorithm is used; for images taken at a longer distance, a lightweight deep learning method is used. Therefore, the present invention can adapt to a variety of complex scenes.

[0080] Furthermore, the dial geometric figure is an ellipse, and detecting the dial geometric figure from the meter area includes:

[0081] Performing line detection on the meter area to obtain meter line data;

[0082] Performing ellipse detection on the meter line data;

[0083] If only one ellipse is detected, the ellipse is directly used as the dial geometry;

[0084] If multiple ellipses are detected, obtain the major axis radius and the minor axis radius of each ellipse respectively; determine whether the major axis radius of each ellipse is equal. If so, take the ellipse with the smallest difference between the minor axis radius and the preset radius as the dial geometry; if not, take the ellipse with the smallest difference between the major axis radius and the preset radius as the dial geometry.

[0085] It is understandable that an ellipse detection algorithm based on Hough transform may be used to extract a number of target ellipses from the meter area;

[0086] When it is determined that the number of the target ellipses is one, the target ellipse is used as an ellipse for representing the dial area;

[0087] When it is determined that there are multiple target ellipses, the major axis radius and the minor axis radius of each target ellipse are obtained; it is determined whether each target ellipse has the same major axis radius, and if so, the target ellipse corresponding to the minor axis radius with the smallest difference from the preset radius is used as the ellipse for representing the dial area; if not, the target ellipse corresponding to the major axis radius with the smallest difference from the preset radius is used as the ellipse for representing the dial area;

[0088] The preset radius is generated by the actual radius of the dial, the distance from the inspection station to the dial, and the focal length of the camera.

[0089] If multiple ellipses are identified, the ellipse with the longest axis radius closest to the ideal imaging radius r is selected; if the long axes are the same, the ellipse with the short axis radius close to the ideal imaging radius r is selected. Among them, the ideal imaging radius r = R / D×f; R is the actual radius of the dial, D is the distance from the inspection station to the dial, and f is the focal length of the camera.

[0090] Illustratively, when multiple ellipses are identified, the present invention selects an ellipse closest to the ideal imaging radius as the ellipse representing the dial area by comparing the difference between the major axis radius and the minor axis radius of the target ellipse and the preset radius, thereby ensuring that the finally selected ellipse can truly reflect the state of the dial, reducing misjudgments and missed judgments caused by factors such as image quality, shooting angle or distance, thereby helping to improve the accuracy and reliability of subsequent meter readings.

[0091] In a preferred embodiment, when the dial geometric figure is a rhombus, the process of detecting the dial geometric figure from the meter area specifically includes:

[0092] After converting the image to be processed into a grayscale image and applying a filtering algorithm (such as Gaussian filtering, median filtering, etc.) to remove the noise in the image, an edge detection algorithm (such as Canny edge detection, Sobel operator, etc.) can be used to detect the edge information in the grayscale image, and the edge detection result can be obtained;

[0093] Based on the edge detection results, a contour detection algorithm (such as a contour tracking algorithm) can be used to extract all contours in the image, and then shape analysis can be performed on each extracted contour. According to the shape characteristics, the contour that best meets the diamond characteristics is screened out as a candidate dial contour; the candidate dial contour is used as the dial geometry, and its position and size information is recorded as the dial shape analysis result for use in subsequent steps.

[0094] Further, in a preferred embodiment, when the dial geometric figure is an ellipse, the dial shape analysis result includes the major axis distance and the minor-major axis ratio of the ellipse;

[0095] Then the dial geometry is subjected to shape analysis, including:

[0096] Measuring the major and minor wheelbases of the ellipse respectively;

[0097] The ratio between the short wheelbase and the long wheelbase is determined as the short-to-long wheelbase ratio.

[0098] When measuring the long wheelbase and short wheelbase, specifically include:

[0099] Generating a minimum outer bounding rectangle for the ellipse;

[0100] Calculate the length and width of the minimum outer enclosing rectangle;

[0101] The length is determined as the major wheelbase, and the width is determined as the minor wheelbase.

[0102] It can be understood that an outer enclosing rectangle can be generated for the ellipse, so as to calculate the short-to-long axis ratio and the long-to-long axis ratio based on the length and width of the outer enclosing rectangle; schematically, the outer enclosing rectangle can be the minimum outer enclosing rectangle of the ellipse, then:

[0103] According to the geometric properties of the ellipse, the corresponding points when the minimum outer enclosing rectangle of the ellipse intersects the ellipse are: a, b, c, d; the diagonal lines of the minimum outer enclosing rectangle are perpendicular to each other, so the length and width of the minimum outer enclosing rectangle can be obtained as: length ac, width bd.

[0104] According to the ratio of ac to bd, the short-long axis distance representing the inclination angle of the dial is obtained, and then: short-long axis ratio The larger the ratio of the minor axis to the major axis, and the closer it is to 1, the closer the ellipse is to being shot perpendicular to the dial.

[0105] According to the length of the minimum outer enclosing rectangle, the major axis distance representing the size of the ellipse is obtained, and the major axis distance L=ac representing the size of the major axis of the distance between the camera and the dial. The larger the major axis distance is, the larger the major axis of the ellipse is at the same angle, which means that the distance between the camera and the ellipse (or dial) is closer.

[0106] The embodiment of the present invention can extract the key features of the ellipse shape by calculating the minor-major axis ratio and the major axis distance, which facilitates subsequent analysis and processing. That is, based on the minor-major axis ratio and the major axis distance of the ellipse corresponding to each image, the images can be screened and classified to screen out images that are closer to the front view in angle (through the minor-major axis ratio) and have a larger image area occupied by the dial area (through the major axis distance), thereby improving the clarity and readability of the dial information in the image.

[0107] For step S3, in a preferred embodiment, when each of the dial shape analysis results includes the minor-major axis ratio and the major-axis distance of the ellipse, the time period selection is performed based on a plurality of dial shape analysis results corresponding to a plurality of the images to be processed, including:

[0108] According to a number of moments matched one by one with the number of images to be processed, all the short-to-long axis ratios in the number of dial shape analysis results are sorted into a first sequence, and all the long-axis distances in the number of dial shape analysis results are sorted into a second sequence;

[0109] Selecting the moment corresponding to the minor-major axis ratio with the largest value in the first sequence as the first reference moment, and selecting each moment adjacent to the first reference moment before and after as the first adjacent moment;

[0110] Determine two long wheelbases corresponding to the two first adjacent moments in the second sequence, and use the larger one of the two long wheelbases corresponding to the two first adjacent moments as the first long wheelbase;

[0111] determining a target time period according to the first reference time period and the first adjacent time period corresponding to the first long wheelbase, and finally extracting a plurality of candidate images from the image list according to the target time period;

[0112] In another preferred embodiment, the present invention can extract candidate images based on two target time periods. After determining a target time period according to the first reference time and the first adjacent time corresponding to the first long wheelbase, the method further includes:

[0113] Taking the period between the two first adjacent moments as a processed period, and partially filtering the first sequence according to the processed period to obtain a corresponding third sequence;

[0114] Selecting the moment corresponding to the minor-major axis ratio with the largest value in the third sequence as the second reference moment, and selecting each moment adjacent to the second reference moment before and after as the second adjacent moment;

[0115] Determine two long wheelbases corresponding to the two second adjacent moments in the second sequence, and use the larger one of the two long wheelbases corresponding to the two second adjacent moments as the second long wheelbase;

[0116] Another target time period is determined according to the second reference moment and the second adjacent moment corresponding to the second long wheelbase. Finally, several candidate images can be extracted from the image list based on the target time period determined by the first reference moment and the first adjacent moment, and the second reference moment and another target time period determined by the second adjacent moment.

[0117] Specifically, the process of determining the first reference time and the target time period determined by the first adjacent time, and the second reference time and another target time period determined by the second adjacent time is as follows:

[0118] Step a: among the images to be processed, find the image with the largest short-to-long axis ratio, select the moment corresponding to the image as the first reference moment, select the moments before and after the first reference moment as the first adjacent moments, and the ellipse of the image corresponding to the first reference moment is closest to the vertical state;

[0119] Step b: Find the moments before and after the first reference moment, and mark them as first adjacent moments. Schematically, the two images corresponding to the two first adjacent moments can be called: left adjacent image and right adjacent image; then, the moment corresponding to the left adjacent image is T1, the moment of the first reference moment is T2, and the moment corresponding to the right adjacent image is T3; the time period between T1 and T2 is t1; the time period between T2 and T3 is t2; it can be understood that, if Figure 2 The curve distribution diagram shown is, in which the two sequential image / video frames on both sides of the highest point in the Ft curve represent the two sequential image / video frames between the image / video frame with the highest short-to-long axis ratio and the adjacent image / video frame.

[0120] Step c: Find the image with the largest long axis distance in the left adjacent image and the right adjacent image, take the long axis distance of the image as the first long axis distance, and obtain the first adjacent moment corresponding to the first long axis distance, indicating that in the first adjacent image, the ellipse size of the first adjacent image corresponding to the first adjacent moment of the first long axis distance is the largest; schematically, as Figure 2 As shown, the first adjacent image corresponding to the first long wheelbase can be the right adjacent image, and the corresponding moment is T3; that is, two sequential images / video frames on both sides of the highest point in the Lt curve can be taken, representing the two sequential images / video frames between the image / video frame with the highest long wheelbase and the adjacent image / video frame, thereby determining that the long wheelbase corresponding to the moment T3 of the right adjacent image is the first long wheelbase.

[0121] Step d: Based on the first reference moment (T2) and the moment (T3) corresponding to the first adjacent image corresponding to the first major axis distance (i.e., the image where the ellipse is closest to the vertical state), generate a first time period, which can be recorded as time period t2, i.e., the time period between moment T2 and moment T3.

[0122] Step e: Further, the time period between the two first adjacent moments is taken as a processed time period, and the first sequence is partially filtered according to the processed time period to obtain a corresponding third sequence; it can be understood that here, the images in the time period corresponding to the left adjacent image moment T1 and the right adjacent image moment T3 are marked as the processed images that have been compared, and then these marked processed images can be removed from the first sequence to obtain the third sequence, so that the image with the largest short-to-long axis ratio and the corresponding moment can be found from the third sequence later;

[0123] Step f: Repeat the process of steps ac in the second and third sequences of images to be processed, and this time mark the moment with the maximum minor-major axis ratio as the second reference moment, and mark the moments before and after the second reference moment as the second adjacent moments;

[0124] Find the second adjacent time corresponding to the maximum long wheelbase among the second adjacent times adjacent to the second reference time, mark the maximum long wheelbase as the second long wheelbase, and record the second adjacent time corresponding to the second long wheelbase as T4;

[0125] Step g: Generate a second time period, which may be recorded as time period t3, according to the second adjacent time T4 corresponding to the second long wheelbase and the time of the second reference time (recorded as T5).

[0126] Step h: both the first time period and the second time period are regarded as target time periods, that is, both the time period t2 and the time period t3 are regarded as target time periods, so that according to the time period t2 and the time period t3, the images corresponding to the time period t2 and the time period t3 are respectively extracted from the image list and marked as candidate images. Then, the images in the time period t2 and the time period t3 have a higher vertical degree and a larger ellipse size.

[0127] It can be understood that the maximum ratio of the short axis to the long axis reflects the maximum angle between the ellipse and the horizontal direction, that is, the dial is most vertical, and the time closer to the frontal shooting can be obtained; further querying the time corresponding to the adjacent image adjacent to the image to be processed with the maximum ratio of the short axis to the long axis, according to the time closer to the frontal shooting and the time adjacent to it, it can be ensured that the image in the selected target time period does not change much in angle;

[0128] Among the adjacent images determined above, the image with the largest long wheelbase is found. Since the long wheelbase reflects the largest size of the dial, it means that the content of the dial area in the image is clearer. According to the target time period (i.e., time period t2 and time period t3) obtained at the above moment, the corresponding image can include a dial image with a suitable angle (high verticality) and a large size (large long wheelbase). Therefore, according to time period t2 and time period t3, after extracting several candidate images from the image list, these candidate images are closer to the frontal shooting in terms of angle, so the dial is presented in the image more clearly and intuitively, which can reduce the reading difficulty or error caused by the angle problem. At the same time, since the target time period (time period t2 and time period t3) takes into account the image moment corresponding to the maximum long wheelbase, the image in the target time period is also relatively large in size, and the image area occupied by the dial area is larger, thereby improving the clarity and readability of the dial information in the image.

[0129] Therefore, the embodiment of the present invention can ensure that the selected image is ideal in verticality and size by calculating the minor-major axis ratio and the major-axis distance of the ellipse and selecting the image based on these ratios. Moreover, by generating the target time period, the number of images to be processed can be significantly reduced. That is, only the images within the target time period need to be processed, without analyzing the entire image sequence one by one, which saves computing resources and time and improves processing efficiency.

[0130] The embodiments of the present invention are not only applicable to the analysis of static image sequences, but can also be extended to the processing of video streams or dynamic image sequences. That is, it is only necessary to calculate the short-to-long axis ratio and the long-to-long axis distance of the image at each time point, and generate the target time period according to these ratios, so as to realize accurate recognition and analysis of dial images in various scenarios.

[0131] For step S4, in a preferred embodiment, calculating the clarity score for the dial indication area in each candidate image includes:

[0132] Extracting the dial indication area from any of the candidate images;

[0133] Calculate the gradient average value for the edge in the dial indication area;

[0134] The clarity score matching the meter reading area is determined based on the gradient average.

[0135] In a preferred embodiment, extracting the dial indication area from any of the candidate images specifically includes:

[0136] Extracting at least one pending area from the candidate image according to a preset dial indication color;

[0137] If only one undetermined area is extracted, directly determine the undetermined area as the dial indication area;

[0138] If multiple pending regions are extracted, performing numerical structure judgment on the multiple pending regions;

[0139] When it is determined that a plurality of the pending areas are characterized by digital structures, the pending area closest to the center of the dial is selected as the dial indication area; specifically, after extracting the meter area from the image to be processed according to the target size, a dial geometry of the meter area is obtained from the candidate image, and the center of the dial geometry is taken as the center of the dial, and the distance between each pending area and the aforementioned center of the dial is calculated, and the pending area with the smallest distance is taken as the dial indication area;

[0140] When it is determined that a plurality of the to-be-determined areas are all characterized as pointer structures, the to-be-determined area with the largest area is selected as the dial indication area.

[0141] Indicatively, the present invention can firstly preset the dial color and use the color threshold segmentation method to accurately extract the undetermined area with the same color as the dial in the candidate image, and then further obtain the dial indication area based on the undetermined area according to the dial structure (digital structure or pointer structure). For a dial with a digital structure, the dial indication area is extracted from the center of the undetermined area to ensure that the area containing all the numbers is extracted; for a dial with a pointer structure, the undetermined area with the largest area is directly used as the dial indication area to avoid the loss of pointer information due to excessive cropping.

[0142] The present invention is not only applicable to dials and backgrounds with obvious color contrast, but can also adapt to images under different lighting conditions by adjusting the color threshold, thereby enhancing the adaptability of the algorithm in different environments. Moreover, by processing different dial structures, two different types of dials, digital and pointer, can be flexibly processed, thereby improving versatility and achieving accurate readings of fast-response meters.

[0143] Furthermore, in a preferred embodiment, calculating the gradient average value for the edge in the dial indication area specifically includes:

[0144] For each meter indication area, extract the edge of the meter indication area;

[0145] According to the gradient of the edge in the horizontal direction and the gradient of the edge in the vertical direction, the gradient average value of the meter indication area is calculated; wherein, when calculating the gradient of the edge in the horizontal direction and the gradient of the edge in the vertical direction, the gradient of a single pixel in the horizontal direction and the gradient of the pixel in the vertical direction can be calculated,

[0146] A clarity score for the meter reading area is generated based on the gradient average.

[0147] It can be understood that the gradient value reflects the rate of change of the pixel value in the image. The larger the gradient, the more obvious the edge of the image in that direction, and thus the higher the clarity. By calculating the average value of the gradient of the reading area in the horizontal and vertical directions, the clarity of the area can be quantitatively characterized, so that in the subsequent dial reading recognition process, the reading area with a higher clarity score is preferentially selected for processing, which helps to improve the accuracy and reliability of recognition.

[0148] Therefore, the embodiment of the present invention not only takes into account the color information of the image (such as the color threshold segmentation method), but also combines the gradient information of the image, thereby enhancing the adaptability of the present invention to dial images of different types and qualities. Even in some cases of uneven lighting and high noise, the clarity of the image can be evaluated through gradient information.

[0149] In another preferred embodiment, Figure 3 The process diagram of key frame image extraction shown in FIG. 1 is as follows:

[0150] First, data preparation: obtain camera parameters, three-dimensional model of inspection site, inspection point information, and various images collected during the inspection; calculate the ideal imaging radius r = R / D×f;

[0151] According to the type of patrol point, obtain the structural information of the meter, for example, whether it displays digital or pointer scale, the color difference between the digital / pointer and the dial, etc. Obtain the pointer / digit color Sf and the background color Sd;

[0152] Next, the extraction interval is generated and the image of the extraction interval is extracted: the sequence images are sorted in time and divided according to fixed time intervals. For the intermediate images of the divided sequence segments, the two indicators of the short-to-long axis ratio F and the long axis L are calculated. Two extraction intervals are obtained based on the two indicators, so as to obtain the corresponding image based on the extraction interval;

[0153] Finally, for the images (i.e., images) extracted in the extraction interval, the clarity score of each image is calculated; further, when calculating the clarity score, it can be divided into two steps, one is the extraction of the display part, and the other is the calculation of the clarity score; specifically, each candidate image is first converted to the HSV space, and the color threshold segmentation method is used to extract the area with a color similar to that of the needle or the number in the elliptical area representing the dial, such as the digital part of the liquid crystal display or the needle part of the scale display;

[0154] For multiple color areas, if it is a digital structure, select the area closest to the center of the ellipse; if it is a pointer structure, select the area with the largest area as the meter indication area of ​​the indication part;

[0155] Use the Sobel operator to calculate the horizontal and vertical gradients of the edge of the reading area. The gradient size can be calculated using the Pythagorean theorem to obtain the gradient of the edge of the reading area. Calculate the average value of the edge gradient to obtain the clarity score of the reading area;

[0156] Finally, the image with the highest clarity score is taken as the key frame image, that is, it is marked as the target image for meter reading.

[0157] For step S5, in a preferred embodiment, the reading regions are first sorted according to the clarity scores of the reading regions calculated in the previous step.

[0158] In the sorted meter reading areas, the candidate image corresponding to the meter reading area with the highest clarity score is marked as the target image, so that in the subsequent processing flow, the meter reading will be mainly recognized and recorded based on this target image.

[0159] The embodiment of the present invention can significantly improve the recognition accuracy of meter readings by selecting the candidate image with the highest definition as the target image.

[0160] like Figure 4 As shown, based on the above-mentioned various embodiments of the meter image extraction method, the present invention provides a corresponding device embodiment;

[0161] An embodiment of the present invention provides a meter image extraction device, comprising: an image extraction module, a shape analysis result generation module, an image screening module, an image clarity calculation module and an image extraction module;

[0162] The image extraction module is used to extract a number of images to be processed from the image list corresponding to the target meter at a preset time interval;

[0163] The shape analysis result generating module is used to obtain a dial geometry figure from each of the to-be-processed images, and perform shape analysis on the dial geometry figure; wherein the dial geometry figure is suitable for characterizing the shape of the dial in the target meter;

[0164] The image screening module is used to select a time period based on a number of dial shape analysis results corresponding to a number of the images to be processed, and extract a number of candidate images from the image list according to at least one selected target time period;

[0165] The image clarity calculation module is used to calculate a clarity score for a dial indication area in each candidate image; wherein the dial indication area is suitable for indicating the indication on the dial;

[0166] The image extraction module is used to select the candidate image corresponding to the highest clarity score as the target image.

[0167] It should be noted that the device embodiments described above are merely schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, and may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. Those of ordinary skill in the art can understand and implement it without paying creative labor.

[0168] Those skilled in the art can clearly understand that, for the sake of convenience and simplicity, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0169] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A meter image extraction method, characterized in that: include: Extracting a number of images to be processed from an image list corresponding to a target meter at a preset time interval; Obtaining a dial geometry figure from each of the to-be-processed images, and performing shape analysis on the dial geometry figure; wherein the dial geometry figure is suitable for characterizing the shape of the dial in the target meter; Selecting a time period based on a number of dial shape analysis results corresponding to a number of the images to be processed, and extracting a number of candidate images from the image list according to at least one selected target time period; Calculating a clarity score for a dial indication area in each candidate image; wherein the dial indication area is suitable for indicating the indication on the dial; The candidate image corresponding to the highest definition score is selected as the target image.

2. The meter image extraction method according to claim 1, characterized in that: The obtaining of the dial geometry from each of the images to be processed comprises: For any of the images to be processed, determining an original size and setting a target size; wherein the target size is larger than a preset meter imaging size and smaller than the original size; A meter area is extracted from the image to be processed according to the target size, and then the dial geometry is detected from the meter area.

3. The meter image extraction method according to claim 2, characterized in that: The detecting the dial geometry from the meter area comprises: Performing line detection on the meter area to obtain meter line data; Performing ellipse detection on the meter line data; If only one ellipse is detected, the ellipse is directly used as the dial geometry; If multiple ellipses are detected, obtain the major axis radius and the minor axis radius of each ellipse respectively; determine whether the major axis radius of each ellipse is equal. If so, take the ellipse with the smallest difference between the minor axis radius and the preset radius as the dial geometry; if not, take the ellipse with the smallest difference between the major axis radius and the preset radius as the dial geometry.

4. The meter image extraction method according to claim 1, characterized in that: The dial geometry is an ellipse, and the dial shape analysis result includes the major axis distance and the minor-to-major axis ratio of the ellipse; The performing shape analysis on the dial geometric figure comprises: Measuring the major and minor wheelbases of the ellipse respectively; The ratio between the short wheelbase and the long wheelbase is determined as the short-to-long wheelbase ratio.

5. The meter image extraction method according to claim 4, characterized in that: The measuring of the major wheelbase and the minor wheelbase of the ellipse respectively comprises: Generating a minimum outer bounding rectangle for the ellipse; Calculate the length and width of the minimum outer enclosing rectangle; The length is determined as the major wheelbase, and the width is determined as the minor wheelbase.

6. The meter image extraction method according to claim 1, characterized in that: Each of the dial shape analysis results includes the minor-major axis ratio and the major-axis distance of the ellipse; The selecting of time periods based on a number of dial shape analysis results corresponding one-to-one to a number of the images to be processed comprises: According to a number of moments matched one by one with the number of images to be processed, all the short-to-long axis ratios in the number of dial shape analysis results are sorted into a first sequence, and all the long-axis distances in the number of dial shape analysis results are sorted into a second sequence; Selecting the moment corresponding to the minor-major axis ratio with the largest value in the first sequence as the first reference moment, and selecting each moment adjacent to the first reference moment before and after as the first adjacent moment; Determine two long wheelbases corresponding to the two first adjacent moments in the second sequence, and use the larger one of the two long wheelbases corresponding to the two first adjacent moments as the first long wheelbase; A target time period is determined according to the first reference time and the first adjacent time corresponding to the first long wheelbase.

7. The meter image extraction method according to claim 6, characterized in that: The selecting of time periods based on a number of dial shape analysis results corresponding one-to-one to a number of the images to be processed further comprises: Taking the period between the two first adjacent moments as a processed period, and partially filtering the first sequence according to the processed period to obtain a corresponding third sequence; Selecting the moment corresponding to the minor-major axis ratio with the largest value in the third sequence as the second reference moment, and selecting each moment adjacent to the second reference moment before and after as the second adjacent moment; Determine two long wheelbases corresponding to the two second adjacent moments in the second sequence, and use the larger one of the two long wheelbases corresponding to the two second adjacent moments as the second long wheelbase; Another target period is determined according to the second reference time and the second adjacent time corresponding to the second major wheelbase.

8. The meter image extraction method according to any one of claims 1 to 7, characterized in that: The calculating of the clarity score for the dial indication area in each candidate image comprises: Extracting the dial indication area from any of the candidate images; Calculate the gradient average value for the edge in the dial indication area; The clarity score matching the meter reading area is determined based on the gradient average.

9. The meter image extraction method according to claim 8, characterized in that: The step of extracting the dial indication area from any of the candidate images comprises: Extracting at least one pending area from the candidate image according to a preset dial indication color; If only one undetermined area is extracted, directly determine the undetermined area as the dial indication area; If multiple pending regions are extracted, performing numerical structure judgment on the multiple pending regions; When it is determined that the plurality of pending areas are characterized by digital structures, the pending area closest to the center of the dial is selected as the dial indication area; When it is determined that a plurality of the to-be-determined areas are all characterized as pointer structures, the to-be-determined area with the largest area is selected as the dial indication area.

10. A meter image extraction device, characterized in that: include: Image extraction module, shape analysis result generation module, image screening module, image clarity calculation module and image extraction module; The image extraction module is used to extract a number of images to be processed from the image list corresponding to the target meter at a preset time interval; The shape analysis result generating module is used to obtain a dial geometry figure from each of the to-be-processed images, and perform shape analysis on the dial geometry figure; wherein the dial geometry figure is suitable for characterizing the shape of the dial in the target meter; The image screening module is used to select a time period based on a number of dial shape analysis results corresponding to a number of the images to be processed, and extract a number of candidate images from the image list according to at least one selected target time period; The image clarity calculation module is used to calculate a clarity score for a dial indication area in each candidate image; wherein the dial indication area is suitable for indicating the indication on the dial; The image extraction module is used to select the candidate image corresponding to the highest clarity score as the target image.

Images