Instrument inspection image processing method, computer program product and equipment

By processing instrument inspection images using clustering algorithms, interfering instruments are filtered out, solving the problem of erroneous instrument readings in robot inspections and improving the accuracy and intelligence of instrument inspections.

CN118968108APending Publication Date: 2024-11-15SUZHOU GUANGGE EQUIP

Patent Information

Application Number
CN202411042823.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

In application scenarios where instruments are densely arranged, robot inspections are prone to errors in instrument readings, especially due to mismatched readings and multiple inspections caused by interference with instruments.

Method used

Clustering algorithms are used to process instrument inspection images. By obtaining the center point of the instrument and performing clustering, interfering instruments are screened out. After eliminating interfering instruments, the inspection is carried out, thereby improving the level of intelligence.

Benefits of technology

It effectively reduces instrument reading errors caused by interference, and improves the accuracy of instrument inspection and intelligent processing capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118968108A_ABST
    Figure CN118968108A_ABST
Patent Text Reader

Abstract

The invention relates to the field of instrument image recognition, and provides an instrument inspection image processing method, a computer program product and equipment, and the method comprises the steps: obtaining a first inspection image and center points corresponding to a plurality of instruments in the first inspection image, and enabling each center point to be used for marking the position of the corresponding instrument in the first inspection image; according to a preset clustering center area, a minimum sample number and a neighborhood radius, clustering the plurality of center points to obtain a clustering result, the clustering result comprising a plurality of clusters obtained by clustering the plurality of center points and clustering center points of the plurality of clusters; and judging whether the instruments in the first inspection image are interference instruments or not according to the clustering result. The method can be used for eliminating the interference instrument in the inspection image obtained by robot inspection, and is beneficial to reducing the problem that the inspection reading of the instrument is wrong.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of instrument image recognition, and more specifically, to a method, computer program product, and device for processing instrument inspection images. Background Art

[0002] With the development of science and technology, the inspection of instruments has gradually changed from traditional manual inspection to intelligent operation and maintenance mode dominated by automated power inspection robots. During the inspection process, the robot needs to accurately identify the readings and contents of a large number of instruments in the application scenario, such as identifying multiple instruments densely arranged in a distribution station.

[0003] Then, in application scenarios where instruments are densely arranged, there is a problem that when the robot reaches a specific inspection position, the robot's inspection readings of the instruments at the specific inspection position are erroneous. The errors mainly manifest themselves in the readings not corresponding to the instruments and multiple readings being inspected. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a method for processing instrument inspection images, so as to reduce the problem of errors in the inspection readings of instruments at specific inspection positions by robots in application scenarios where instruments are densely arranged.

[0005] In a first aspect, a method for processing an instrument inspection image comprises: obtaining a first inspection image and center points respectively corresponding to a plurality of instruments included in the first inspection image, wherein each center point is used to mark the position of a corresponding instrument in the first inspection image; clustering the plurality of center points according to a preset cluster center area, a minimum number of samples, and a neighborhood radius to obtain a clustering result, wherein the clustering result comprises a plurality of clusters obtained by clustering the plurality of center points and a plurality of cluster center points of the clusters; and judging whether each instrument in the first inspection image is an interference instrument according to the clustering result.

[0006] In the above scheme, multiple instruments contained in the first inspection image are represented by center points, and a clustering algorithm is applied to the inspection image of the processed instruments to screen out interfering instruments that have mistakenly entered the field of view of the inspection robot. Eliminating the interfering instruments is conducive to the smooth execution of instrument inspection and improves the intelligence level of processing instrument inspection images. That is, it is conducive to reducing the problem of erroneous inspection readings of instruments at specific inspection positions by the robot due to the presence of interference meters in application scenarios where instruments are densely arranged.

[0007] As an optional method, the multiple center points are clustered according to the pre-set cluster center area, minimum number of samples and neighborhood radius to obtain a clustering result, including: for each center point, calculating the number of center points contained in the neighborhood radius of the center point; if the number of center points is greater than or equal to the minimum number of samples, marking the center point as a core object; for each core object, all center points that can be reached from the core object density and the core object are determined as a cluster; calculating the average values ​​of the horizontal and vertical coordinates of all the center points contained in the cluster to obtain the cluster center point of the cluster.

[0008] In the above scheme, it is explained how to create a cluster with the center point of the instrument as the sample point and the center point of the cluster based on the idea of ​​the clustering algorithm, which is the basis for further judging the interference instrument according to the clustering results.

[0009] As an optional method, judging whether each instrument in the first inspection image is an interference instrument according to the clustering result includes: for each cluster center point, if the cluster center point is not within the range of the cluster center area, judging that the instruments corresponding to all the center points of the cluster where the cluster center point is located are the interference instruments. In the above scheme, judging whether each instrument is an interference instrument according to the relationship between the cluster center point and the cluster center area, simplifying the center points of most instruments to be represented by a few cluster center points, helps to reduce the complexity of data storage and processing, while retaining the main features and structure of the data.

[0010] As an optional method, the cluster center area and the first inspection image satisfy the relationship: λ·W≤w≤(1-λ)W; λ·H≤h≤(1-λ)H; wherein W and H are the width and height of the first inspection image, w and h are the width and height of the cluster center area, λ is a proportional coefficient and 0<λ<0.5. In the above scheme, the conditional formula followed by the pre-setting of the cluster center area is given, and the reasonable setting of the cluster center area is conducive to ensuring the reliability of the judgment result.

[0011] As an optional way, before clustering the multiple center points according to the preset cluster center area, minimum number of samples and neighborhood radius to obtain the clustering result, the method also includes: calculating the minimum value of the distance between each center point and the remaining center points, and sorting all the minimum values ​​in ascending order to obtain a distance set; calculating the distance average of the first preset number of distances in the distance set, and calculating the average size of all instruments in the first inspection image; determining the sum of the distance average and the average size as the neighborhood radius. In the above scheme, a specific implementation method for presetting the neighborhood radius is given. The neighborhood radius is mainly determined based on the average size of all instruments in the first inspection image. It can have stronger versatility and adaptability in scenarios with different instrument types and sizes. Reasonable setting of the neighborhood radius is conducive to ensuring the reliability of the judgment result.

[0012] As an optional manner, the calculating the average size of all instruments in the first inspection image includes: calculating the average size using the following formula: Wherein, s is the average size, n is the number of instruments in the first inspection image, (x 1i ,y 1i ) and (x 2i ,y 2i ) represent the coordinates of the diagonal vertices of the detection box corresponding to the ith instrument. In the above scheme, a specific formula for calculating the average size of the instrument is given. Since the instrument itself may have a ratio of length to width that is too large or too small, it is necessary to combine the length and width to calculate, which is conducive to setting a more reasonable neighborhood radius.

[0013] As an optional manner, after clustering the plurality of center points according to the pre-set cluster center area, the minimum number of samples and the neighborhood radius to obtain a clustering result, wherein the clustering result includes a plurality of clusters obtained by clustering the plurality of center points and a plurality of cluster center points of the clusters, the method further comprises: marking the center point that does not belong to any cluster as noise. In the above scheme, the noise that interferes with the clustering result is eliminated, which is conducive to improving the reliability and accuracy of the interference instrument judgment result.

[0014] In a second aspect, an embodiment of the present application further provides a computer program product, including computer program instructions, which, when read and executed by a processor, execute the method described in the first aspect.

[0015] In a third aspect, an embodiment of the present application further provides a device, comprising: a processor, a memory and a bus, wherein the processor and the memory communicate with each other through the bus; the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the method described in the first aspect above.

[0016] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, comprising: the computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the method described in the first aspect above.

[0017] Other features and advantages of the present application will be described in the subsequent description, and in part will become apparent from the description, or will be understood by implementing the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A schematic diagram of a process flow of a method for processing an instrument inspection image provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of the device structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] The following embodiments of the technical solution of the present application are described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present application, and are therefore only used as examples, and cannot be used to limit the scope of protection of the present application.

[0022] It should be noted that all technical and scientific terms used in this document have the same meanings as those generally understood by technicians in the technical field to which this application belongs; the terms used in this document are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions.

[0023] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0024] In the description of the embodiments of the present application, the term "and / or" is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0025] In the distribution station scenario, when the robot conducts instrument inspection, the type, number, and location of the corresponding instrument at each preset location are pre-configured in the system.

[0026] However, in the distribution station scenario, the substation cabinet that the robot needs to identify contains a large number of instruments and meters. Since the distribution boxes in the distribution station are often placed in rows, the instruments in each distribution box are distributed over a wide range. During the robot inspection process, when the robot arrives at a specific inspection position, there is a certain error between its actual position and the set inspection position due to sensor accuracy issues, resulting in the inspection robot's field of view may mistakenly capture the instruments of the inspection points on both sides of the inspection point. Other instruments that mistakenly enter the field of view will cause the number of instruments to not match the actual calibrated number. This will cause the reading of the instrument read by the robot inspection to be different from the instrument pre-set by the current position system, resulting in inspection interference and errors.

[0027] More specifically, through analysis, it was found that when the robot reaches a specific inspection position, the inspection readings at the preset position are read in order from left to right and from top to bottom. If an interference meter appears in the field of view, according to this rule, it is easy for the reading of this interference meter to be ranked before other normal meters, and the readings of other normal meters are considered to be interference meters, that is, the readings do not correspond to the instrument, resulting in inspection interference. From the inspection results, the error is manifested as the inspection readings of the instrument at the preset position are wrong during the robot inspection process, and the readings are checked multiple times.

[0028] Please refer to Figure 1 , Figure 1 A schematic diagram of a process flow of a method for processing an instrument inspection image provided in this application, the method comprising the following steps:

[0029] Step S10: Acquire the first inspection image and the center points corresponding to the multiple instruments included in the first inspection image, wherein each center point is used to mark the position of the corresponding instrument in the first inspection image.

[0030] The first inspection image is the image data obtained by the inspection personnel or the inspection system (such as drones, inspection robots, etc.) when shooting or recording the inspection object. The first inspection image can be data extracted from on-site shooting, or data in a directly accessible state, such as inspection images sent by external devices, read from the cloud or stored locally in advance, etc.

[0031] Each center point is used to mark the position of the corresponding instrument in the first inspection image and can be used to represent the instrument.

[0032] It should be noted that the embodiments of the present application do not specifically limit the specific implementation method for obtaining the center point of the instrument in the first inspection image, and those skilled in the art may make appropriate adjustments according to actual conditions. For example, a key point detection algorithm may be used to determine the center point of the instrument in the first inspection image; or an edge detection algorithm and a geometric center method may be used to determine the center point of the instrument in the first inspection image; or the center point of the instrument in the first inspection image sent by an external device may be received.

[0033] The following uses the key point detection algorithm as an example to introduce the specific implementation method of obtaining the center points corresponding to the multiple instruments included in the first inspection image. As an implementation method, before obtaining the center point, the key point detection algorithm can be used to detect the position of the instrument and the pointer position. When detecting the position of the instrument, one instrument corresponds to one detection frame, and the coordinates of the diagonal vertices of each detection frame are output. Among them, the embodiment of the present application does not specifically limit the specific implementation method of the above-mentioned key point detection algorithm. For example, the key point detection algorithm can be YOLO v8, Open Pose or DeepPose, etc.

[0034] Therefore, the center point can be determined based on the coordinates of the upper left and lower right boundary points of the instrument or the upper right and lower left boundary points, and these two pairs of boundary point coordinates are the coordinates of the diagonal vertices of the detection frame corresponding to each instrument. The center point is the midpoint of the coordinates of the diagonal vertices of the detection frame. For example, the coordinates of the upper left and lower right coordinates of the detection frame are (x1, y1) and (x2, y2) respectively, then the coordinates of the center point are

[0035] Step S20: Clustering multiple center points according to the preset cluster center area, minimum sample number and neighborhood radius to obtain clustering results, wherein the clustering results include multiple clusters obtained by clustering the multiple center points and cluster center points of the multiple clusters.

[0036] The cluster center area, the minimum number of samples, and the neighborhood radius are all parameters that need to be determined in advance before using the clustering algorithm. Clustering algorithm is a process used in data mining and statistics to group samples or observations in a data set into multiple classes or clusters. The purpose is to make the samples in the same cluster as similar as possible under some metric (such as distance, similarity, etc.), while the samples in different clusters are as different as possible.

[0037] In clustering algorithms, the neighborhood radius is used to define the neighborhood range of a point. For any point p in the data set, all points in the circular area (in two-dimensional space) or spherical area (in higher-dimensional space) with point p as the center and ε as the radius are called points in the neighborhood of point p, and the radius ε is the neighborhood radius.

[0038] The minimum number of samples refers to the minimum number of sample points within the neighborhood radius used to determine whether a point is a core object. If the number of sample points contained in the neighborhood radius of a point is greater than or equal to the minimum number of samples, the point is considered a core object.

[0039] It should be noted that the specific implementation method of clustering multiple center points in the embodiment of the present application is not specifically limited, and those skilled in the art can make appropriate adjustments according to actual conditions. For example, the DBSCAN clustering algorithm can be used to cluster the center point coordinates of the instrument; or the K-means clustering algorithm can be used to cluster the center point coordinates of the instrument; or the Gaussian mixture model can be used to cluster the center point coordinates of the instrument, etc.

[0040] Step S30: judging whether each instrument in the first inspection image is an interference instrument according to the clustering result.

[0041] The cluster center area divides a boundary for judging interference instruments. The instruments represented by the center points within the boundary are judged as non-interference instruments, and vice versa. The instruments corresponding to the center points of the same cluster in the clustering results have a higher similarity, while the instruments corresponding to the center points of different clusters have a lower similarity. The cluster center point of a cluster represents the center point of the entire cluster. Therefore, by judging whether the cluster center point is located in the cluster center area, it can be concluded whether all the instruments in the cluster where the cluster center point is located are interference instruments.

[0042] In the above scheme, multiple instruments contained in the first inspection image are represented by center points, and a clustering algorithm is applied to the inspection image of the processed instruments to screen out interfering instruments that have mistakenly entered the field of view of the inspection robot. Eliminating the interfering instruments is conducive to the smooth execution of instrument inspection and improves the intelligence level of processing instrument inspection images. That is, it is conducive to reducing the problem of erroneous inspection readings of instruments at specific inspection positions by the robot due to the presence of interference meters in application scenarios where instruments are densely arranged.

[0043] In some embodiments, step S20 includes:

[0044] Step S201: For each center point, calculate the number of center points contained in the neighborhood radius of the center point.

[0045] For each center point, the range covered by the circle with the center point as the center and the neighborhood radius as the radius is the neighborhood radius range.

[0046] Step S202: If the number of center points is greater than or equal to the minimum number of samples, the center point is marked as a core object.

[0047] Step S203: For each core object, all center points that are density-reachable from the core object and the core object are determined as a cluster.

[0048] Density reachability may mean that if, starting from a core object, another point can be reached through a series of points (each point is a direct density reachable point of the next point), then the point is said to be density reachable from the core object. Direct density reachability means that the distance between the two points is less than or equal to the neighborhood radius, and the starting point is a core object. In other embodiments, density reachability may also refer to all points within the neighborhood radius of a core object starting from a core object.

[0049] According to the cluster classification method in step S203, one center point can only belong to one cluster.

[0050] Step S204: Calculate the average values ​​of the horizontal and vertical coordinates of all the center points included in the cluster to obtain the cluster center point of the cluster.

[0051] The average value of the horizontal coordinates of all the center points contained in a cluster is used as the horizontal coordinate of the cluster center point, and the average value of the vertical coordinates of all the center points contained in a cluster is used as the vertical coordinate of the cluster center point.

[0052] For the case where a single instrument is clustered, the cluster center is the coordinate of the center point of the instrument.

[0053] In the above scheme, it is explained how to create a cluster with the center point of the instrument as the sample point and the center point of the cluster based on the idea of ​​the clustering algorithm, which is the basis for further judging the interference instrument according to the clustering results.

[0054] In some embodiments, step S30 includes: for each cluster center point, if the cluster center point is not within the range of the cluster center area, determining that the instruments corresponding to all the center points of the cluster where the cluster center point is located are interference instruments.

[0055] The cluster center area is a range pre-defined in the first inspection image, and those skilled in the art can adjust the size of the range according to actual conditions.

[0056] In the above scheme, whether each instrument is an interference instrument is determined based on the relationship between the cluster center point and the cluster center area, and the majority of instrument center points are simplified to be represented by a few cluster center points, which helps to reduce the complexity of data storage and processing while retaining the main features and structure of the data.

[0057] In some embodiments, the cluster center area and the first inspection image satisfy the relationship:

[0058] λ·W≤w≤(1-λ)W;

[0059] λ·H≤h≤(1-λ)H;

[0060] Wherein, W and H are the width and height of the first inspection image, w and h are the width and height of the cluster center area, λ is the scale factor and 0<λ<0.5.

[0061] The proportional coefficient λ is mainly determined by the accuracy error of the inspection robot itself. If the accuracy error of the inspection robot itself is small, the cluster center area can be appropriately enlarged, that is, λ can be set smaller, and vice versa.

[0062] Since the selection of cluster center areas is subjective and uncertain, we can find better clustering results by trying different parameter settings multiple times, and adjust the selection strategy of cluster center areas based on the feedback of clustering results.

[0063] In the above scheme, the conditional formula followed by presetting the cluster center area is given. Reasonable setting of the cluster center area is conducive to ensuring the reliability of the judgment result.

[0064] In some embodiments, before clustering the plurality of center points according to the preset cluster center area, the minimum number of samples, and the neighborhood radius in step S20 to obtain the clustering result, the method further includes:

[0065] Step 1): Calculate the minimum distance between each center point and the remaining center points, and sort all minimum values ​​in ascending order to obtain a distance set.

[0066] Calculate the minimum value d of the distance between each center point and the remaining center points, and sort all minimum values ​​d in ascending order to obtain the distance set D.

[0067] Step 2): Calculate the average distance of the first preset number of distances in the distance set, and calculate the average size of all instruments in the first inspection image.

[0068] The present application embodiment does not specifically limit the specific value of the preset number, and those skilled in the art can adjust it according to actual conditions. The preset number is set to select the smallest multiple distance values ​​from the distance set D and calculate the average value d of these multiple distance values. average The expression form of the preset number can be either a numerical value or a percentage. As an implementation method, the average value of the first 80% distance values ​​in the distance set D is calculated as d average .

[0069] The average size s is used to characterize the size of the instrument.

[0070] Step 3): The sum of the distance average and the average size is determined as the neighborhood radius.

[0071] Neighborhood radius = d average +s. It can be seen that the size of the neighborhood radius depends on the distance between the instruments and the size of the instruments, and the neighborhood radius changes adaptively in different scenarios.

[0072] In the above scheme, a specific implementation method for pre-setting the neighborhood radius is given. Reasonable setting of the neighborhood radius is conducive to ensuring the reliability of the judgment result. The neighborhood radius obtained by the above method has the ability to adaptively adjust according to different application scenarios, and has better versatility and adaptability for scenarios with different instrument types and sizes.

[0073] In some embodiments, the step 2) of calculating the average size of all instruments in the first inspection image includes:

[0074] The average size is calculated using the following formula:

[0075]

[0076] Among them, s is the average size, n is the number of instruments in the first inspection image, (x 1i ,y 1i ) and (x 2i ,y 2i ) represent the diagonal vertex coordinates of the detection box corresponding to the i-th instrument.

[0077] According to the average size s and the distance average d average After the neighborhood radius is obtained, the neighborhood radius is substituted into step S20 to obtain a clustering result, and step S30 is continued to be performed to intelligently exclude the interfering instruments in the first inspection image.

[0078] In the above scheme, a specific formula for calculating the average size of the instrument is given, which is conducive to setting a more reasonable neighborhood radius.

[0079] In some embodiments, after step S20, the method further includes: marking a center point that does not belong to any cluster as noise.

[0080] Different processing methods can be selected for the marked noise, for example, discarding the noise and focusing only on the center points that form the cluster, or storing the noise separately for further anomaly analysis.

[0081] In the above scheme, the noise that interferes with the clustering results is eliminated, which is beneficial to improving the reliability and accuracy of the interference instrument judgment results.

[0082] In some embodiments, after using a key point detection algorithm for the first inspection image, a second inspection image corresponding to each meter is extracted, and the boundary range of the second inspection image is equal to the range covered by the detection frame for detecting a single meter. The detection frame can be a square frame.

[0083] For the second inspection image, the shooting angle of the robot camera and the installation angle of the instrument are limited by the position of the current environment during the inspection process, which makes it impossible for the robot to obtain a front view of the instrument. If the inspection instrument is a pointer instrument, this will cause errors in the robot's identification readings. Therefore, the following method can be used to correct the deformity of the second inspection image and identify the readings.

[0084] Step 1): obtaining the center coordinates of each scale value of the instrument in the second inspection image, wherein the second inspection image is determined according to an image collected from the instrument during the inspection process, and the second inspection image includes an instrument.

[0085] Step 2): Determine a preset number of center coordinates as correction points according to the center coordinates of each scale value, wherein each correction point is not on a straight line and is used to perform deformity correction on the second inspection image.

[0086] In some embodiments, the preset number is 4, and the step 2) of determining the center coordinates of the preset number as the correction points according to the center coordinates of each scale value includes:

[0087] Step 2.1): Among the multiple initial convex quadrilaterals surrounded by four central coordinates, multiple candidate convex quadrilaterals are determined according to the diagonal angles of the multiple initial convex quadrilaterals, wherein the candidate convex quadrilateral is the initial convex quadrilateral with the smallest absolute value of the difference between the diagonal angle and the right angle.

[0088] Step 2.2): Determine a target convex quadrilateral from multiple candidate convex quadrilaterals, wherein the four center coordinates corresponding to the target convex quadrilateral are the correction points.

[0089] In some embodiments, determining a target convex quadrilateral from a plurality of candidate convex quadrilaterals in step 2.2) above includes:

[0090] Step 2.2.1): Calculate the areas of multiple candidate convex quadrilaterals.

[0091] Step 2.2.2): Determine the candidate convex quadrilateral with the largest area as the target convex quadrilateral.

[0092] In some embodiments, before determining multiple candidate convex quadrilaterals among the multiple initial convex quadrilaterals according to the diagonal angles of the multiple initial convex quadrilaterals in the above step 2.1), the method further includes:

[0093] Step 2.0.1): Convert the two diagonals of the initial convex quadrilateral into vectors, and calculate the trigonometric function value corresponding to the angle between the two vectors.

[0094] As an implementation method, the four vertices of the initial convex quadrilateral are denoted as A, B, C, and D, and the diagonals of the initial convex quadrilateral are denoted as AC and BD, and AC and BD are converted into vectors accordingly. and By calculating the vector and The cosine value cosθ can be used to obtain the angle between the two vectors.

[0095] Step 2.0.2): Determine the diagonal angle based on the trigonometric function value.

[0096] The formula for calculating cosθ is: When the absolute value of the cosine value is closer to 0, the difference between the diagonal angle of the initial convex quadrilateral and the right angle is smaller, that is, the diagonal angle of the initial convex quadrilateral is closer to a right angle, and thus the shape of the initial convex quadrilateral is closer to a rectangle.

[0097] The implementation method corresponding to the step of determining multiple candidate convex quadrilaterals from multiple initial convex quadrilaterals according to the diagonal angles of multiple initial convex quadrilaterals is: after calculating the cosine values ​​cosθ of all the initial convex quadrilaterals, arrange them in order from small to large according to the absolute values ​​of the cosine values, and select multiple convex quadrilaterals with the smallest absolute values ​​of the cosine values ​​as candidate quadrilaterals. The multiple number can be two, three, five, etc., and the number of the multiple number is not specifically limited here.

[0098] In the above scheme, vectors and trigonometric functions are introduced to calculate the diagonal angle of a convex quadrilateral, which is simpler, more efficient, more practical and more operable.

[0099] In some embodiments, after determining a preset number of center coordinates as correction points according to the center coordinates of each scale value in the above step 2), the method further includes:

[0100] Step 3): Perform deformity correction on the second inspection image according to the correction point to obtain a corrected inspection image.

[0101] After the correction point is determined, the second inspection image is subjected to deformity correction. As an implementation method, the second inspection image is subjected to deformity correction by using image perspective transformation, and the perspective transformation formula is as follows:

[0102]

[0103] In the above formula, [u, v, w] is the coordinate in the second inspection image before transformation, [x′, y′, w′] is the three-dimensional space coordinate in the corrected inspection image after transformation, is the transformation matrix. Since the second inspection image is a two-dimensional plane, w here is 1, and the final transformed coordinates are:

[0104]

[0105] Step 4): For each pointer in the corrected inspection image, the pointer angle between the pointer and the horizontal direction is calculated according to the coordinates of the two end points of the pointer in the corrected inspection image.

[0106] The coordinates of the two end points of the pointer in the corrected inspection image are P1′(P x ′1,P y ′1), P2′(P′ x2 , P′ y2 ), let the angle between the pointer and the horizontal direction be α, then the formula for calculating α is:

[0107]

[0108] Step 5): Determine the reading result of the instrument based on the scale value of the instrument in the second inspection image, the center coordinates of the scale value in the corrected inspection image, the pointer angle, and the prior knowledge of the position angle of each scale value of the instrument.

[0109] In some embodiments, before obtaining the center coordinates of each scale value in the second inspection image of the instrument in the above step 1), the method further includes:

[0110] Step 1.1): Detect the area of ​​each scale value.

[0111] Step 1.2): Get the coordinates of the upper left vertex, upper right vertex, lower right vertex, and lower left vertex of the area where each scale value is located.

[0112] Step 1.3): Determine the average value of the coordinates of the upper left vertex, the upper right vertex, the lower right vertex, and the lower left vertex as the center coordinate of the corresponding scale value.

[0113] As an implementation method, the point coordinates recognized by OCR are marked as [t, pt], where t is the specific number or character recognized, and pt is the position information of the number or character. For a scale value, use ((pt x1 ,pt y1 ),(pt x2 ,pt y2 ),(pt x3 ,pt y3 ),(pt x4 ,pt y4 )) represents the coordinates of the upper left vertex, upper right vertex, lower right vertex, and lower left vertex where the scale value t is located.

[0114] Thus, the center coordinates of each scale value (cpt x ,cpt y ) is as follows:

[0115]

[0116] Vertical axis cpt y The calculation formula is the same, thus obtaining the center coordinates of all scale values ​​of the instrument.

[0117] The correction points selected according to the method provided in the above embodiment are used to correct the deformity of the second inspection image, and the correction effect is good, thereby reducing the reading error of the instrument and improving the accuracy.

[0118] An embodiment of the present application further provides a computer program product, including computer program instructions, which, when read and executed by a processor, execute the method provided in any of the above embodiments.

[0119] Please refer to Figure 2 , Figure 2 The schematic diagram of the device structure provided for the embodiment of the present application includes: a processor 200, a memory 300 and a bus 400, wherein the processor 200 and the memory 300 communicate with each other through the bus 400; the memory 300 stores program instructions that can be executed by the processor 200, and the processor 200 calls the program instructions to execute the method described in any of the above embodiments. In some embodiments, the device can be a robot or other device, specifically a hanging rail, a drone, a legged robot or other device.

[0120] Among them, the processor 200 includes one or more (only one is shown in the figure), which can be an integrated circuit chip with signal processing capabilities. The above-mentioned processor 200 can be a general-purpose processor, including a central processing unit (CPU), a micro control unit (MCU), a network processor (NP) or other conventional processors; it can also be a dedicated processor, including a neural network processor (NPU), a graphics processor (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. In addition, when there are multiple processors 200, some of them can be general-purpose processors and the other part can be dedicated processors.

[0121] The memory 300 includes one or more (only one is shown in the figure), which may be, but not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The processor 200 and other possible components may access the memory 300 and read and / or write data therein.

[0122] An embodiment of the present application further provides a computer-readable storage medium, including: the computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the method described in any of the above embodiments.

[0123] This embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided by the above-mentioned method embodiments.

[0124] This embodiment provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions enable the computer to execute the methods provided by the above-mentioned method embodiments.

[0125] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0126] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0127] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0128] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for processing instrument inspection images, It is characterized in that include: Acquire a first inspection image and center points corresponding to a plurality of instruments included in the first inspection image, wherein each center point is used to mark a position of a corresponding instrument in the first inspection image; According to the preset cluster center area, the minimum number of samples and the neighborhood radius, the plurality of center points are clustered to obtain a clustering result, wherein the clustering result includes a plurality of clusters obtained by clustering the plurality of center points and a plurality of cluster center points of the clusters; It is determined whether each instrument in the first inspection image is an interference instrument according to the clustering result.

2. The method according to claim 1, It is characterized in that The clustering of the plurality of center points according to the preset cluster center area, the minimum number of samples and the neighborhood radius to obtain the clustering result includes: For each center point, calculate the number of center points contained in its neighborhood radius; If the number of the center points is greater than or equal to the minimum number of samples, the center point is marked as a core object; For each core object, all center points that are density-reachable from the core object and the core object are determined as a cluster; The average values ​​of the horizontal and vertical coordinates of all the center points contained in the cluster are calculated to obtain the cluster center point of the cluster.

3. The method according to claim 1, It is characterized in that The determining, according to the clustering result, whether each instrument in the first inspection image is an interference instrument includes: For each cluster center point, if the cluster center point is not within the range of the cluster center area, it is determined that the instruments corresponding to all the center points of the cluster where the cluster center point is located are the interference instruments.

4. The method according to claim 1, It is characterized in that The cluster center area and the first inspection image satisfy the relationship: λ·W≤w≤(1-λ)W; λ·H≤h≤(1-λ)H; Wherein, W and H are respectively the width and height of the first inspection image, w and h are respectively the width and height of the cluster center area, λ is the proportional coefficient and 0<λ<0.

5.

5. The method according to claim 1, It is characterized in that Before clustering the plurality of center points according to the preset cluster center area, the minimum number of samples and the neighborhood radius to obtain the clustering result, the method further includes: Calculate the minimum distance between each center point and the remaining center points, and sort all minimum values ​​in ascending order to obtain a distance set; Calculating the average distance of the first preset number of distances in the distance set, and calculating the average size of all instruments in the first inspection image; The sum of the distance average and the average size is determined as the neighborhood radius.

6. The method according to claim 5, It is characterized in that The calculating the average size of all instruments in the first inspection image includes: The average size is calculated using the following formula: Wherein, s is the average size, n is the number of instruments in the first inspection image, (x 1i ,y 1i ) and (x 2i ,x 2i ) represent the diagonal vertex coordinates of the detection box corresponding to the i-th instrument.

7. The method according to any one of claims 1 to 6, It is characterized in that After clustering the plurality of center points according to the preset cluster center area, the minimum number of samples, and the neighborhood radius to obtain a clustering result, wherein the clustering result includes a plurality of clusters obtained by clustering the plurality of center points and a plurality of cluster center points of the clusters, the method further includes: The center points that do not belong to any cluster are marked as noise.

8. A computer program product, It is characterized in that The method comprises computer program instructions, and when the computer program instructions are read and executed by a processor, the method according to any one of claims 1 to 7 is executed.

9. A device, It is characterized in that include: A processor, a memory and a bus, wherein the processor and the memory communicate with each other via the bus; The memory stores program instructions executable by the processor, and the processor can execute the method according to any one of claims 1 to 7 by calling the program instructions.

10. A computer-readable storage medium, It is characterized in that include: The computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the method according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Night traffic flow calculation method and system based on vehicle lamp detection

    CN114663664A

  • Power transmission line multi-fitting detection method and device

    CN115797806A

  • Equipment inspection method and device, equipment and storage medium

    CN117892110A

Cited By

  • 4D millimeter wave radar clustering method and device and electronic equipment

    CN119249177A