Method, apparatus, device, and storage medium for instrument recognition

Abstract

The present disclosure relates to the technical field of instrument recognition. According to an embodiment of the present disclosure, there is provided a method, apparatus, device, and storage medium for instrument recognition. The method includes: based on an image containing an instrument dial, determining feature information of the instrument, where the feature information includes information related to at least one of the following: an arc associated with the contour of the instrument scale, a line segment associated with the instrument pointer in the instrument dial, and the type of the instrument dial. Based on the starting point of the arc, the ending point of the arc, the center point of the arc, and the intersection point of the extension line of the line segment and the arc, determining a first deflection amount and a second deflection amount of the instrument pointer in the instrument scale. Based on the first deflection amount, the second deflection amount, and the instrument range associated with the type of the instrument dial, determining the value indicated by the instrument pointer. Thereby, the accuracy of instrument recognition can be improved, the error of instrument recognition can be reduced, and thus the recognition rate of the instrument can be improved.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the technical field of meter identification, and more particularly, to a method, apparatus, device, and computer-readable storage medium for meter identification. Background Art

[0002] With the digital transformation of industrial automation, smart grids and transportation systems, the demand for accurate readings of instruments as key monitoring equipment under complex working conditions has become increasingly prominent. Machine vision technology, with its non-contact measurement advantages, provides high-precision and high-reliability intelligent reading solutions for scenarios such as industrial equipment status monitoring, real-time monitoring of power parameters, and transportation facility operation evaluation, promoting the transformation and upgrading of traditional monitoring modes to digitalization and intelligence. In traditional solutions, visual recognition or deep learning is usually used to identify instrument dials. After identifying the instrument dial, the pointer is identified by visual recognition to obtain the value indicated by the pointer. However, in complex environments or when there is a lot of interference, the recognition accuracy of traditional solutions is low and the error is high. Summary of the invention

[0003] In a first aspect of the present disclosure, a method for instrument recognition is provided. The method includes: determining characteristic information of the instrument based on an image containing an instrument dial, the characteristic information including information related to at least one of the following: an arc associated with the outline of the instrument scale, a line segment associated with the instrument pointer in the instrument dial, and the type of the instrument dial; determining a first deflection amount and a second deflection amount of the instrument pointer in the instrument scale based on a starting point of the arc, an ending point of the arc, a center point of the arc, and an intersection point between an extended line of the line segment and the arc, the intersection point being a point close to the front end of the instrument pointer, the first deflection amount and the second deflection amount being determined in different ways; and determining a value indicated by the instrument pointer based on the first deflection amount, the second deflection amount, and an instrument range associated with the type of the instrument dial.

[0004] In a second aspect of the present disclosure, there is provided an apparatus for instrument recognition. The apparatus includes: a first determination module configured to determine characteristic information of the instrument based on an image including the instrument dial, where the characteristic information includes information related to at least one of the following: an arc associated with the contour of the instrument scale, a line segment associated with the instrument pointer in the instrument dial, and the type of the instrument dial; a second determination module configured to determine a first deflection amount and a second deflection amount of the instrument pointer in the instrument scale based on the starting point of the arc, the ending point of the arc, the center point of the arc, and the intersection point of the extension line of the line segment and the arc, where the intersection point is the point close to the front end of the instrument pointer, and the first deflection amount and the second deflection amount are determined in different ways; and a third determination module configured to determine the value indicated by the instrument pointer based on the first deflection amount, the second deflection amount, and the instrument range associated with the type of the instrument dial.

[0005] In a third aspect of the present disclosure, there is provided an electronic device. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, there is provided a computer-readable storage medium. A computer program is stored on the medium, and when executed by a processor, the computer program implements the method of the first aspect.

[0007] According to the embodiments of the present disclosure, the following beneficial effects can be achieved: First, based on an image including the instrument dial, the characteristic information of the instrument is determined. The characteristic information includes information related to at least one of the following: an arc associated with the contour of the instrument scale, a line segment associated with the instrument pointer in the instrument dial, and the type of the instrument dial. Then, based on the starting point of the arc, the ending point of the arc, the center point of the arc, and the intersection point of the extension line of the line segment and the arc, the first deflection amount and the second deflection amount of the instrument pointer in the instrument scale are determined. The intersection point is the point close to the front end of the instrument pointer, and the first deflection amount and the second deflection amount are determined in different ways. Subsequently, based on the first deflection amount, the second deflection amount, and the instrument range associated with the type of the instrument dial, the value indicated by the instrument pointer is determined. Thus, the accuracy of instrument recognition can be improved, the error of instrument recognition can be reduced, and the recognition rate of the instrument can be improved.

[0008] It should be understood that the content described in this part is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. Description of the Drawings

[0009] The above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent by referring to the following detailed description in conjunction with the accompanying drawings. In the drawings, the same or similar reference numerals denote the same or similar elements, where:

[0010] Figure 1 A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented;

[0011] Figure 2 A flowchart showing a process for instrument recognition according to some embodiments of the present disclosure;

[0012] Figure 3 A schematic diagram showing a first image segment according to some embodiments of the present disclosure;

[0013] Figure 4 A schematic diagram showing a second image segment according to some embodiments of the present disclosure;

[0014] Figure 5 A schematic diagram showing a third image segment according to some embodiments of the present disclosure;

[0015] Figure 6 A schematic diagram showing the outline of a fitted instrument scale according to some embodiments of the present disclosure;

[0016] Figure 7 A schematic diagram showing the intersection of a fitted line and the outline of an instrument scale according to some embodiments of the present disclosure;

[0017] Figure 8 A schematic diagram showing the corner points of an arc according to some embodiments of the present disclosure;

[0018] Figure 9 A schematic diagram showing a line segment according to some embodiments of the present disclosure;

[0019] Figure 10 A schematic diagram showing the intersection point of the extension line of a line segment and an arc according to some embodiments of the present disclosure;

[0020] Figure 11 A schematic diagram showing an example of determining a first deflection amount according to some embodiments of the present disclosure;

[0021] Figure 12 A schematic diagram showing an example of determining a second deflection amount according to some embodiments of the present disclosure;

[0022] Figure 13 A block diagram showing a device for instrument recognition according to some embodiments of the present disclosure; and

[0023] Figure 14An electronic device is shown in which one or more embodiments of the present disclosure can be implemented. Detailed implementation manners

[0024] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0025] In the description of the embodiments of the present disclosure, the term "including" and its like terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "an embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". There may also be other explicit and implicit definitions hereinafter.

[0026] In this article, unless otherwise specified, performing a step "in response to A" does not mean that the step is immediately performed after "A", but may include one or more intermediate steps.

[0027] It can be understood that the data involved in the technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the corresponding laws, regulations and related regulations.

[0028] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner according to the relevant laws and regulations.

[0029] For example, when receiving a user's active request, a prompt message is sent to the user to clearly prompt the user that the operation requested by the user will require the acquisition and use of the user's personal information, so that the user can autonomously choose whether to provide personal information to the software or hardware such as an electronic device, an application program, a server or a storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0030] As an optional but non-limiting implementation manner, the manner of sending a prompt message to the user in response to receiving the user's active request may be, for example, in the form of a pop-up window, and the prompt message may be presented in text in the pop-up window. In addition, the pop-up window may also carry a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0031] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not limit the implementation manner of the present disclosure. Other manners that comply with relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0032] As briefly mentioned above, as a key monitoring device, the demand for accurate readings of the instrument in complex working conditions is becoming increasingly prominent. The instrument has the advantages of dust-proof, waterproof, and strong anti-interference ability, and has been widely used in the power industry.

[0033] However, since the instrument is usually installed on different devices, the angle of collection or shooting is likely to be inclined when collecting the instrument images on different devices. At the same time, due to different lighting conditions, there are optical phenomena such as shadows or overexposure when collecting or shooting instrument images.

[0034] In response to the above phenomena, in traditional solutions, visual recognition methods or deep learning are usually used to recognize the instrument dial. After the instrument dial is recognized, the pointer is recognized by visual recognition methods to obtain the value indicated by the pointer.

[0035] For example, in some solutions, the regional image can be obtained through rough positioning first, and a series of operations such as noise reduction and Gaussian blur are performed on the regional image to obtain the feature vector representation of the instrument dial. Then, feature point matching is performed through the template image and the feature vector representation. Finally, the value indicated by the instrument pointer is obtained by using the Hough transform algorithm. However, these solutions require a stable collection or shooting environment. For complex environments, the recognition accuracy is relatively low and the error is relatively high.

[0036] For another example, in some other solutions, the area of the instrument dial in the original image can be detected by using a machine learning model first, so as to crop the image of the instrument dial in the corresponding area from the original image. Then, affine transformation is performed on the cropped image, and the instrument dial is located by random circle detection. Finally, a series of operations such as morphological operations and canny algorithms are used to extract the starting scale, ending scale, and pointer position to obtain the value indicated by the pointer. However, in the case of more interference, the recognition rate of these solutions is relatively low.

[0037] In view of this, embodiments of the present disclosure provide a solution for instrument recognition. According to this solution, first, based on an image containing an instrument dial, the characteristic information of the instrument is determined. The characteristic information includes information related to at least one of the following: an arc associated with the contour of the instrument scale, a line segment associated with the instrument pointer in the instrument dial, and the type of the instrument dial. Then, based on the starting point of the arc, the ending point of the arc, the center point of the arc, and the intersection point of the extension line of the line segment and the arc, the first deflection amount and the second deflection amount of the instrument pointer in the instrument scale are determined. The intersection point is the point close to the front end of the instrument pointer, and the first deflection amount and the second deflection amount are determined in different ways. Subsequently, based on the first deflection amount, the second deflection amount, and the instrument range associated with the type of the instrument dial, the value indicated by the instrument pointer is determined. Thus, the accuracy of instrument recognition can be improved, the error of instrument recognition can be reduced, and the recognition rate of the instrument can be improved.

[0038] Figure 1 FIG. shows a schematic diagram of an exemplary environment 100 in which embodiments of the present disclosure can be implemented. As Figure 1 shown, the environment 100 generally may include an electronic device 110. The electronic device 110 can obtain an image 120 containing an instrument dial as input and output the value 130 indicated by the instrument pointer.

[0039] In some embodiments, the electronic device 110 may have an image acquisition unit, so that the image 120 acquired by it can be processed to output the value 130 indicated by the instrument pointer.

[0040] In some embodiments, the electronic device 110 may also interact with an image acquisition device (not shown in the figure). For example, the electronic device 110 may receive an input message (such as the image 120) from the image acquisition device and output the value 130 indicated by the instrument pointer. In some embodiments, the image acquisition device may be a camera, a webcam, a video camera, an infrared / thermal imaging device, an inspection robot, etc. The communication connection between the electronic device 110 and the image acquisition device can be established by a wired or wireless method. The communication connection may include, but is not limited to, a Bluetooth connection, a mobile network connection, a Universal Serial Bus (USB) connection, a Wireless Fidelity (WiFi) connection, etc., and embodiments of the present disclosure are not limited in this regard.

[0041] In the environment 100, the electronic device 110 may be any type of device with computing capabilities, including a terminal device or a server device.

[0042] The terminal device can be any type of mobile terminal, fixed terminal or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, TV receivers, radio broadcast receivers, e-book devices, gaming devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. The terminal device can also be an inspection robot with image acquisition and processing functions.

[0043] The server device is an independent physical server, or can also be a server cluster or distributed system composed of multiple physical servers, and can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. The server device can, for example, include computing systems / servers, such as mainframes, edge computing nodes, computing devices in a cloud environment, and so on.

[0044] It should be understood that Figure 1 The illustrated environment 100 is merely exemplary and not restrictive. In other embodiments according to the present disclosure, the environment 100 may include one or more other devices, units, modules, etc., and the present disclosure places no restrictions thereon.

[0045] Figure 2 A flowchart of a process 200 for instrument recognition according to some embodiments of the present disclosure is shown. The process 200 can be implemented at the electronic device 110. For the purpose of discussion only, the following refers to Figure 1 Describe the process 200.

[0046] At 210, the electronic device 110 determines the characteristic information of the instrument based on the image 120 containing the instrument dial. The characteristic information of the instrument includes information related to at least one of the following: an arc associated with the contour of the instrument scale, a line segment associated with the instrument pointer in the instrument dial, and the type of the instrument dial. In the embodiments of the present disclosure, the instrument dial in the image 120 can be the dial of a pointer-type instrument. Pointer-type instruments can, for example, include but are not limited to: mechanical pointer instruments, electronic pointer instruments, electromechanical integrated pointer instruments, etc. It should be understood that these examples of pointer-type instruments are merely illustrative, and the embodiments of the present disclosure place no restrictions thereon.

[0047] In some embodiments, the electronic device 110 may utilize a model deployed locally or on other devices (such as other electronic devices or third-party platforms) to determine an image segment indicating the instrument dial from the image 120. For the purpose of discussion, such a model may be referred to as the "first machine learning model", and such an image segment may be referred to as the "first image segment". The first machine learning model may be a traditional machine learning model, such as K-Nearest Neighbors (KNN), Support Vector Machine (SVM), etc. The first machine learning model may also be a deep learning model, such as Convolutional Neural Network (CNN), YOLO (You Only Look Once) model, Vision Transformer (ViT), etc. It should be understood that these examples are merely illustrative, and the embodiments of the present disclosure are not limited thereto.

[0048] In some embodiments, the YOLOv8 model may be adopted to extract the first image segment from the image 120. Since this model introduces multi-scale object detection and non-linear activation functions, such as the ELU (Exponential Linear Unit) activation function, the performance and convergence speed of the model can be improved, and thus the first image segment can be quickly extracted from the image 120, enhancing the efficiency of determining the first image segment from the image 120.

[0049] For example, the electronic device 110 may use the image 120 as the input of the trained YOLOv8 model, thereby outputting the first image segment. In some embodiments, the images for training the YOLOv8 model may include, for example, images taken under different backgrounds, images taken under different lighting conditions, images of different types of instruments, etc. Before training the model, it is also necessary to annotate the above-mentioned images. Generally, the instrument dial may include an instrument range and an instrument pointer, and the instrument range may include main scales and sub-scales. In some embodiments, the main scales, pointers, and instrument types in the training images may be annotated, without annotating the sub-scales. Thereby, the training efficiency of the YOLOv8 model can be improved.

[0050] Further, the electronic device 110 can also use a model deployed locally or on other devices (such as other electronic devices or third-party platforms) to determine, from the first image segment, a first region indicating the contour of the instrument scale and a second region indicating the instrument pointer. For the purpose of discussion, such a model can be referred to as the "second machine learning model". Similar to the first machine learning model, the second machine learning model can be a traditional machine learning model, such as K-Nearest Neighbors (KNN), Mean-Shift, etc. The second machine learning model can also be a deep learning model, such as a model based on dilated convolution and multi-scale context (Pyramid Scene Parsing Network, PSPNet), U-Net, DeepLab, etc. It should be understood that these examples are merely illustrative, and the embodiments of the present disclosure are not limited thereto.

[0051] Since the DeepLab model is a model specifically designed for semantic segmentation tasks, it can accurately capture the boundary information in the image, effectively solve the problem of spatial resolution loss caused by downsampling in traditional convolutional neural networks, and can also perform pixel-level segmentation on the image. Therefore, the DeepLab model can be used to determine the first region and the second region from the first image segment. Specifically, the first image segment can be used as the input of the trained DeepLab model, so as to output the first region and the second region. Similar to the YOLOv8 model, when training the DeepLab model, it is also necessary to label the training data. Here, the training data can be image segments of the instrument dial output by the YOLOv8 model or obtained from other data sources (such as some open-source image sets). In some embodiments, when labeling the image segments of the instrument dial, the main scale and the pointer in the image can also be labeled, and the sub-scale is not labeled. Thereby, the training efficiency of the DeepLab model can be improved. For example, Figure 3 shows a schematic diagram of a first image segment 310 according to some embodiments of the present disclosure. In Figure 3 it, the region 301 can indicate the contour of the instrument scale, and the region 302 can indicate the instrument pointer.

[0052] Further, the electronic device 110 can determine information related to the arc and information related to the line segment from the first image segment based on the first region and the second region.

[0053] Specifically, the electronic device 110 may determine an image segment corresponding to the first region and an image segment corresponding to the second region from the first image segment. For the purpose of discussion, the image segment corresponding to the first region may be referred to as the "second image segment", and the image segment corresponding to the second region may be referred to as the "third image segment". For example, the electronic device 110 may extract the second image segment and the third image segment from the first image segment 310 by the RGB values of each pixel point in the first image segment 310 for the outline of the instrument scale. For example, Figure 4 FIG. shows a schematic diagram of the second image segment 320 according to some embodiments of the present disclosure, Figure 5 FIG. shows a schematic diagram of the third image segment 330 according to some embodiments of the present disclosure.

[0054] Next, the electronic device 110 may perform dilation on the second image segment 320 and the third image segment 330 respectively using a dilation structuring element. For example, a structuring element of a certain size (e.g., 5×5) may be used to perform dilation on the second image segment 320 and the third image segment 330 respectively. Thereby, the relative positions, proportions, etc. of the region 301 in the dilated second image segment 320 and the region 302 in the dilated third image segment 330 are kept consistent with those before dilation. Subsequently, the outline of the instrument scale may be fitted in the dilated second image segment 320 to determine the arc and information related to the arc, and the Hough line transform may be used in the dilated third image segment 330 to determine the line segment and information related to the line segment.

[0055] In some embodiments, the information related to the arc may include the starting point of the arc, the ending point of the arc, and the center point of the arc.

[0056] When determining the information related to the arc, the electronic device 110 may determine the outline of the scale in the dilated second image segment 320, and fit an ellipse or a circle through the determined outline, thereby obtaining a fitted line, and the center of the fitted line may be calculated through the fitted line as the center point of the arc. For example, Figure 6 FIG. shows a schematic diagram of the fitted outline 303 of the instrument scale according to some embodiments of the present disclosure.

[0057] When determining the arc and the starting point and ending point of the arc, the electronic device 110 may traverse the pixel values of each pixel point in the second image segment 320, and determine the intersection of the outline of the instrument scale and the fitted line through the RGB values of each pixel point. Alternatively or additionally, the pixel points with RGB values higher than a predetermined value (e.g., 0) may be used as the intersection of the outline of the instrument scale and the fitted line. For example, Figure 7 FIG. shows a schematic diagram of the intersection 304 of the fitted line and the outline of the instrument scale according to some embodiments of the present disclosure.

[0058] Further, the electronic device 110 may convert the second image segment 320 into a grayscale image, then perform binarization processing on the grayscale image, and subsequently, using the Thomas detection method, arcs and the starting and ending points of the arcs can be obtained. Specifically, the intersections 304 in the binarized image can be traversed to determine whether there are only identical elements above or in the same row in the 3-neighborhood of each pixel point in the intersections 304. If there are identical elements above or in the same row of a certain pixel point, then that pixel point is determined to be a corner point; if there are no identical elements above or in the same row of a certain pixel point, then that pixel point is determined not to be a corner point. For example, Figure 8 shows a schematic diagram of the corner points 305 of an arc according to some embodiments of the present disclosure. In Figure 8 , the corner point 305 on the left side may be the starting point of the arc, and the corner point 305 on the right side may be the ending point of the arc. The arc passing through the starting point and the ending point may be an arc associated with the contour of the instrument scale.

[0059] In some embodiments, the information related to the line segment may include the endpoints at both ends of the line segment and the intersection points of the extension line of the line segment and the arc.

[0060] Before determining the information related to the line segment, the line segment associated with the instrument pointer may be determined first. Specifically, the electronic device 110 may use the Hough line transform in the dilated third image segment 330 to determine multiple line segments, and the line segment with the longest length may be used as the line segment indicating the pointer. For example, Figure 9 shows a schematic diagram of the line segment 306 according to some embodiments of the present disclosure. After obtaining the line segment, the endpoints at both ends of the line segment and the intersection points of the extension line of the line segment and the arc can be determined. In some embodiments, the intersection point may be a point close to the front end of the instrument pointer. Since there may be two intersection points between the extension line of the line segment and the arc, when determining the intersection point close to the front end of the instrument pointer, each pixel value on the contour of the instrument scale can be traversed, the distances between the two endpoints of the line segment and each pixel value can be calculated, and the endpoint corresponding to the minimum distance is used as the intersection point 307. Figure 10 shows a schematic diagram of the intersection point 307 of the extension line of the line segment and the arc according to some embodiments of the present disclosure.

[0061] Continuing to refer to Figure 2 , at 220, the electronic device 110 determines a first deflection amount and a second deflection amount of the instrument pointer in the instrument scale based on the starting point of the arc, the ending point of the arc, the center point of the arc, and the intersection point of the extension line of the line segment and the arc. Here, the intersection point is the point close to the front end of the instrument pointer. The first deflection amount and the second deflection amount are determined in different ways. Next, the different ways of determining the first deflection amount and the second deflection amount of the instrument pointer in the instrument scale are introduced in detail.

[0062] In some embodiments, the electronic device 110 can determine a first angle based on a first distance between a starting point and a center point, a second distance between the starting point and an intersection point, and a third distance between the center point and the intersection point. The electronic device 110 can determine a second angle based on a fourth distance between an ending point and the center point, a fifth distance between the ending point and the intersection point, and the third distance. Then, the absolute value of the difference between the first angle and the second angle is determined as a third angle. Subsequently, based on the first angle, the third angle, a first offset angle, and a second offset angle, a first deflection amount is determined. The first offset angle indicates the offset amount of the starting point, and the second offset angle indicates the offset amount of the ending point.

[0063] In some embodiments, the first offset angle is determined based on the first distance and the size of the dilation structuring element, and the second offset angle is determined based on the fourth distance and the size of the dilation structuring element.

[0064] Taking the instrument pointer in the fourth quadrant as an example, how to determine the first deflection amount will be described below. Figure 11 A schematic diagram showing an example of determining the first deflection amount according to some embodiments of the present disclosure is shown.

[0065] As Figure 11 shown, the electronic device 110 can first calculate the coordinates of the center point 402 of the fitting line, the coordinates of the starting point 403, the coordinates of the ending point 404, and the coordinates of the intersection point 405. By calculating the first distance between the starting point 403 and the center point 402, the second distance between the starting point 403 and the intersection point 405, and the third distance between the center point 402 and the intersection point 405, the first angle 406 can be calculated using the cosine theorem. By calculating the fourth distance between the ending point 404 and the center point 402, the fifth distance between the ending point 404 and the intersection point 405, and the third distance between the center point 402 and the intersection point 405, the second angle 407 can be calculated using the cosine theorem. The absolute value of the difference between the first angle 406 and the second angle 407 is taken as the third angle 408.

[0066] Since the second image segment is dilated when obtaining the arc, the starting point 403 and the ending point 404 of the arc 401 are both offset. The offset amount of the starting point 403 can be referred to as the first offset angle, and the offset amount of the ending point 404 can be referred to as the second offset angle.

[0067] Taking the calculation of the first offset angle as an example, continue to refer to Figure 11, since the first offset angle is caused by dilation, the distance between the true position A of the starting point 403 and the current position can be determined to be 1 / 2 of the convolution kernel size of the dilation structuring element. Also, since the distance here is small enough, a right triangle can be formed with the true position A, the current position, and the center point 402. That is, the first offset angle can be calculated by the following formula (1):

[0068] (1)

[0069] where, is the first offset angle, d is the distance between the starting point and the center point, and s is 1 / 2 of the convolution kernel size of the dilation structuring element.

[0070] It should be understood that for the second offset angle of the termination point 404, the same method can be used for calculation.

[0071] Furthermore, the electronic device 110 can calculate the first deflection amount of the instrument pointer on the instrument scale based on the following formula (2):

[0072] (2)

[0073] where, is the first deflection amount, is the first angle, is the third angle, is the first offset angle, is the second offset angle.

[0074] In some embodiments, the electronic device 110 can determine the first slope of the first straight line passing through the starting point and the center point and the second slope of the second straight line passing through the termination point and the center point. Then, on the arc, a first point set related to the first slope is determined, a second point set related to the second slope is determined on the arc, and a third point set is determined on the arc using the third straight line passing through the intersection point and the center point. Subsequently, based on the intersection of the first point set, the second point set, and the third point set, the second deflection amount is determined.

[0075] Continuing with the example of the instrument pointer in the fourth quadrant, how to determine the second deflection amount will be described. Figure 12 FIG. shows a schematic diagram of an example of determining the second deflection amount according to some embodiments of the present disclosure.

[0076] As Figure 12As shown, the electronic device 110 can determine a first set of points related to the first slope k1 of the first straight line 409 on the arc 401, determine a second set of points related to the second slope k2 on the arc 401, and determine a third set of points on the arc 401 using the third straight line 411 passing through the intersection point 405 and the center point 402.

[0077] For example, when the first slope k1 is less than 0, the points on the arc 401 and below the first straight line 409 can be used as the first set of points 412. When the second slope k2 is less than 0, the points on the arc 401 and above the second straight line 410 can be used as the second set of points 413. The points on the arc 401 and below the third straight line 411 are used as the third set of points 414. Thus, the second deflection amount can be calculated by the following formula (3):

[0078] (3)

[0079] Wherein, is the second deflection amount, is the intersection of the first set of points, the second set of points, and the third set of points, is the set of all points on the arc.

[0080] It should be understood that the above methods for calculating the first deflection amount and the second deflection amount are only exemplary, and other methods in related technologies can also be used to calculate the first deflection amount and the second deflection amount, and the embodiments of the present disclosure do not limit this.

[0081] Continuing to refer to Figure 2 , at 230, the electronic device 110 determines the value 130 indicated by the instrument pointer based on the first deflection amount, the second deflection amount, and the instrument range associated with the type of the instrument dial.

[0082] In some embodiments, the instrument range can be determined based on the type of the instrument dial. For example, after determining the type of the instrument dial, the electronic device 110 can obtain the corresponding instrument range from local or other devices (such as other electronic devices or third-party platforms).

[0083] In some embodiments, the electronic device 110 can determine the product of the first deflection amount and the instrument range as the first value, and determine the product of the second deflection amount and the instrument range as the second value.

[0084] For example, the first value can be determined by the following formula (4):

[0085] (4)

[0086] Wherein, is the first value, is the first deflection amount, is the instrument range.

[0087] For example, the second value can be determined by the following formula (5):

[0088] (5)

[0089] where is the second value, is the second deflection amount, is the instrument range.

[0090] Furthermore, if both the first value and the second value satisfy the error range, the average value of the first value and the second value can be determined as the value indicated by the instrument pointer. If either the first value or the second value does not satisfy the error range, the first value can be determined as the value indicated by the instrument pointer, where the first value is greater than the second value.

[0091] In some embodiments, the error range can be determined based on the instrument range. For example, the error range can be no more than 2% of the instrument range.

[0092] In summary, according to the embodiments of the present disclosure, the first deflection amount and the second deflection amount of the instrument pointer in the instrument scale are determined by the starting point, the ending point, the center point of the arc indicating the instrument range, and the intersection point of the extension line of the line segment indicating the instrument pointer and the arc, and the value indicated by the instrument pointer is determined by the first deflection amount, the second deflection amount, and the instrument range associated with the type of the instrument dial. The accuracy of instrument recognition can be improved, the error of instrument recognition can be reduced, and thus the recognition rate of the instrument can be improved.

[0093] Figure 13 FIG. shows a block diagram of an apparatus 500 for instrument recognition according to some embodiments of the present disclosure. The apparatus 500 can be implemented as or included in an electronic device 110. Each module / component in the apparatus 500 can be implemented by hardware, software, firmware, or any combination thereof.

[0094] As Figure 13As shown, device 500 includes a first determination module 510 configured to determine characteristic information of the instrument based on an image including the instrument dial, where the characteristic information includes information related to at least one of the following: an arc associated with the contour of the instrument scale, a line segment associated with the instrument pointer in the instrument dial, and the type of the instrument dial. Device 500 further includes a second determination module 520 configured to determine a first deflection amount and a second deflection amount of the instrument pointer in the instrument scale based on the starting point of the arc, the ending point of the arc, the center point of the arc, and the intersection point of the extension line of the line segment and the arc, where the intersection point is the point close to the front end of the instrument pointer, and the first deflection amount and the second deflection amount are determined in different ways. Device 500 further includes a third determination module 530 configured to determine the value indicated by the instrument pointer based on the first deflection amount, the second deflection amount, and the instrument range associated with the type of the instrument dial.

[0095] In some embodiments, the third determination module 530 is further configured to determine the product of the first deflection amount and the instrument range as a first value; determine the product of the second deflection amount and the instrument range as a second value; if both the first value and the second value satisfy the error range, determine the average of the first value and the second value as the value indicated by the instrument pointer, where the error range is determined based on the instrument range; and if either the first value or the second value does not satisfy the error range, determine the first value as the value indicated by the instrument pointer, where the first value is greater than the second value.

[0096] In some embodiments, the second determination module 520 is further configured to determine a first angle through a first distance between the starting point and the center point, a second distance between the starting point and the intersection point, and a third distance between the center point and the intersection point; determine a second angle through a fourth distance between the ending point and the center point, a fifth distance between the ending point and the intersection point, and the third distance; determine the absolute value of the difference between the first angle and the second angle as a third angle; and determine the first deflection amount based on the first angle, the third angle, a first offset angle, and a second offset angle, where the first offset angle indicates the offset amount of the starting point, and the second offset angle indicates the offset amount of the ending point.

[0097] In some embodiments, the contour of the instrument scale is dilated via a dilation structuring element, and the first offset angle is determined based on the first distance and the size of the dilation structuring element, and the second offset angle is determined based on the fourth distance and the size of the dilation structuring element.

[0098] In some embodiments, the second determination module 520 is further configured to determine a first slope of a first straight line passing through the starting point and the center point and a second slope of a second straight line passing through the ending point and the center point; determine a first point set related to the first slope on the arc; determine a second point set related to the second slope on the arc; use a third straight line passing through the intersection point and the center point to determine a third point set on the arc; and determine a second deflection amount based on the intersection of the first point set, the second point set, and the third point set.

[0099] In some embodiments, the apparatus 500 further includes a fourth determination module configured to determine, using a first machine learning model, a first image segment indicating the instrument dial from the image; determine, using a second machine learning model, a first region indicating the contour of the instrument scale and a second region indicating the instrument pointer from the first image segment; and determine information related to the arc and information related to the line segment from the first image segment.

[0100] In some embodiments, the fourth determination module is further configured to determine a second image segment corresponding to the first region from the first image segment; dilate the second image segment using a dilation structuring element; and fit the contour of the instrument scale in the dilated second image segment to determine information related to the arc.

[0101] In some embodiments, the fourth determination module is further configured to determine the starting point and the ending point of the arc through the pixel values of the dilated second image segment.

[0102] In some embodiments, the fourth determination module is further configured to determine a third image segment corresponding to the second region from the first image segment; dilate the third image segment using a dilation structuring element; and use the Hough line transform in the dilated third image segment to determine information related to the line segment.

[0103] In some embodiments, the instrument range is determined based on the type of the instrument dial.

[0104] Figure 14 The block diagram of an electronic device 600 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that Figure 14 The illustrated electronic device 600 is merely exemplary and should not constitute any limitation to the functions and scopes of the embodiments described herein. Figure 14 The illustrated electronic device 600 may be used to implement Figure 13 the apparatus 500 for instrument recognition or Figure 1 the electronic device 110 in

[0105] As Figure 14As shown, the electronic device 600 is in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to, one or more processors or processing units 610, a memory 620, a storage device 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. The processing unit 610 may be an actual or virtual processor and be capable of performing various processes according to the programs stored in the memory 620. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing ability of the electronic device 600.

[0106] The electronic device 600 generally includes multiple computer storage media. Such media can be any accessible media that can be obtained by the electronic device 600, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 620 may be a volatile memory (such as registers, caches, random access memory (RAM)), a non-volatile memory (such as read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 630 may be a removable or non-removable medium and may include machine-readable media, such as a flash drive, a magnetic disk, or any other medium that can be used to store information and / or data and can be accessed within the electronic device 600.

[0107] The electronic device 600 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in Figure 14 a disk drive for reading from or writing to a removable, non-volatile magnetic disk (such as a "floppy disk") and an optical disk drive for reading from or writing to a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data media interfaces. The memory 620 may include a computer program product 625 having one or more program modules configured to execute the various methods or actions of the various embodiments of the present disclosure.

[0108] The communication unit 640 enables communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 600 may be implemented in a single computing cluster or multiple computer machines that are capable of communicating through a communication connection. Thus, the electronic device 600 may operate in a networked environment using a logical connection to one or more other servers, network personal computers (PCs), or another network node.

[0109] The input device 650 can be one or more input devices, such as a mouse, a keyboard, a trackball, etc. The output device 660 can be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 600 can also communicate with one or more external devices (not shown) via the communication unit 640 as needed. The external devices such as a storage device, a display device, etc., communicate with one or more devices that enable a user to interact with the electronic device 600, or communicate with any device (e.g., a network card, a modem, etc.) that enables the electronic device 600 to communicate with one or more other electronic devices. Such communication can be performed via an input / output (I / O) interface (not shown).

[0110] According to an exemplary implementation of the present disclosure, there is provided a computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, there is also provided a computer program product, the computer program product being tangibly stored on a non-transitory computer-readable medium and including computer-executable instructions, and the computer-executable instructions being executed by a processor to implement the method described above.

[0111] Aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0112] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is produced that implements the functions / acts specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, which instructions cause a computer, a programmable data processing device, and / or other devices to operate in a particular manner, so that the computer-readable medium storing the instructions includes a manufacture, which includes instructions for implementing various aspects of the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0113] The computer-readable program instructions can be loaded onto a computer, other programmable data processing device, or other device, such that a series of operation steps are performed on the computer, other programmable data processing device, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing device, or other device implement the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0114] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various implementations of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks may in fact be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or by a combination of dedicated hardware and computer instructions.

[0115] The various implementations of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art in the field of the present technology without departing from the scope and spirit of the described implementations. The choice of terms used herein is intended to explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other ordinary skilled persons in the field of the present technology to understand the various implementation manners disclosed herein.

Claims

1. A method for instrument recognition, characterized in that, Including: Based on an image including an instrument dial, determining characteristic information of the instrument, where the characteristic information includes information related to at least one of the following: an arc associated with the contour of the instrument scale, a line segment associated with an instrument pointer in the instrument dial, and the type of the instrument dial; Based on the starting point of the arc, the ending point of the arc, the center point of the arc, and the intersection point of the extension line of the line segment and the arc, determining a first deflection amount and a second deflection amount of the instrument pointer in the instrument scale, where the intersection point is the point close to the front end of the instrument pointer, and the first deflection amount and the second deflection amount are determined in different ways; And Based on the first deflection amount, the second deflection amount, and the instrument range associated with the type of the instrument dial, determining the value indicated by the instrument pointer; Where determining the second deflection amount includes: Determining a first slope of a first straight line passing through the starting point and the center point and a second slope of a second straight line passing through the ending point and the center point; Determining a first point set related to the first slope on the arc; Determining a second point set related to the second slope on the arc; Using a third straight line passing through the intersection point and the center point to determine a third point set on the arc; and Based on the intersection of the first point set, the second point set, and the third point set, determining the second deflection amount.

2. The method according to claim 1, characterized in that, Determining the value indicated by the instrument pointer includes: Determining the product of the first deflection amount and the instrument range as a first value; Determining the product of the second deflection amount and the instrument range as a second value; If both the first value and the second value satisfy the error range, determining the average value of the first value and the second value as the value indicated by the instrument pointer, where the error range is determined based on the instrument range; and If the first value or the second value does not satisfy the error range, determining the first value as the value indicated by the instrument pointer, where the first value is greater than the second value.

3. The method according to claim 1, wherein Determining the first deflection amount includes: Determining a first angle based on a first distance between the starting point and the center point, a second distance between the starting point and the intersection point, and a third distance between the center point and the intersection point; Determining a second angle based on a fourth distance between the ending point and the center point, a fifth distance between the ending point and the intersection point, and the third distance; Determining the absolute value of the difference between the first angle and the second angle as a third angle; and Based on the first angle, the third angle, a first offset angle, and a second offset angle, determining the first deflection amount, where the first offset angle indicates the offset amount of the starting point, and the second offset angle indicates the offset amount of the ending point.

4. The method according to claim 3, characterized in that, The contour of the instrument scale is dilated via a dilation structuring element, and the first offset angle is determined based on the first distance and the size of the dilation structuring element, and the second offset angle is determined based on the fourth distance and the size of the dilation structuring element.

5. The method according to claim 1, characterized in that Also including: Using a first machine learning model, determine a first image segment indicating the instrument dial from the image; Using a second machine learning model, determine a first region indicating the contour of the instrument scale and a second region indicating the instrument pointer from the first image segment; And Determine information related to the arc and information related to the line segment from the first image segment.

6. The method according to claim 5, characterized in that, Determining information related to the arc includes: Determine a second image segment corresponding to the first region from the first image segment; Dilate the second image segment using a dilation structuring element; and Fit the contour of the instrument scale in the dilated second image segment to determine information related to the arc.

7. The method according to claim 6, wherein Determining the starting point and the ending point includes: Determine the starting point and the ending point of the arc through the pixel values of the dilated second image segment.

8. The method according to claim 5, characterized in that, Determining information related to the line segment includes: Determine a third image segment corresponding to the second region from the first image segment; Dilate the third image segment using a dilation structuring element; and Use the Hough line transform in the dilated third image segment to determine information related to the line segment.

9. The method according to claim 1, wherein The instrument range is determined based on the type of the instrument dial.

10. A device for meter recognition, characterized in that, Includes: A first determination module, configured to determine the characteristic information of the instrument based on an image containing an instrument dial, where the characteristic information includes information related to at least one of the following: an arc associated with the contour of the instrument scale, a line segment associated with the instrument pointer in the instrument dial, and the type of the instrument dial; A second determination module, configured to determine a first deflection amount and a second deflection amount of the instrument pointer in the instrument scale based on the starting point of the arc, the ending point of the arc, the center point of the arc, and the intersection point of the extension line of the line segment and the arc, where the intersection point is a point close to the front end of the instrument pointer, and the first deflection amount and the second deflection amount are determined in different ways; And A third determination module, configured to determine the value indicated by the instrument pointer based on the first deflection amount, the second deflection amount, and the instrument range associated with the type of the instrument dial; Wherein the second determination module is further configured to: Determine a first slope of a first straight line passing through the starting point and the center point and a second slope of a second straight line passing through the ending point and the center point; Determine a first point set related to the first slope on the arc; Determine a second point set related to the second slope on the arc; Use a third straight line passing through the intersection point and the center point to determine a third point set on the arc; And Determine the second deflection amount based on the intersection of the first point set, the second point set, and the third point set.

11. An electronic device, characterized in that, Includes: At least one processing unit; And At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the device to perform the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images