Double-range dial plate automatic identification system and method based on RGB camera

Abstract

The invention belongs to the field of image recognition. In order to solve the problem that an existing double-range dial depends on manual reading, the invention provides a double-range dial automatic identification system and method based on an RGB camera, and the technical scheme can be summarized as follows: a double-range dial pointer angle-numerical value mapping relation database, an image acquisition module, a dial identification module and a pointer identification module, a circle center identification module, a character identification module, a pointer included angle proportion identification module and a result output module; the device has the beneficial effects that the device has an automatic recognition function, the labor intensity of workers is reduced, and meanwhile, the recognition accuracy is improved. The method is suitable for automatic identification of double-range dial plates.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition, and in particular relates to a dual-range dial automatic recognition system and method based on an RGB camera. Background Art

[0002] A dual-range dial is a dial that can switch or simultaneously display two different measurement ranges. It is widely used in measuring instruments such as ammeters and voltmeters.

[0003] Currently, the identification of dual-range dials generally relies on manual labor, that is, human intervention is required to determine the current range before reading. The low degree of automation increases the labor intensity of the staff. At the same time, in some cases, the uncertainty of human factors may reduce the accuracy of the reading.

[0004] In view of this, a dual-range dial automatic recognition system and method based on RGB camera is designed to solve the above problems. Summary of the Invention

[0005] To address the current problem of manual reading of dual-range dials, the present invention provides a dual-range dial automatic recognition system and method based on an RGB camera. To achieve the above objectives, the present invention provides the following technical solution: a dual-range dial automatic recognition system based on an RGB camera, comprising:

[0006] A dual-range dial pointer angle-value mapping relationship database is used to establish and save the mapping relationship between the pointer angle and the value of the dual-range dial based on the relationship between the actual pointer angle and the actual value of the dual-range dial;

[0007] Image acquisition module, which uses an RGB camera to capture images of the dual-range dial;

[0008] A dial recognition module, which identifies the position of the dual-range dial in the image using a dual-range dial recognition model and outputs an image containing only the dual-range dial;

[0009] A circle center recognition module, which uses a dual-range dial circle center recognition model to identify the center position of a circle in an image that only contains a dual-range dial;

[0010] a pointer recognition module that recognizes the pointer in the image containing only the dual-range dial using a dual-range dial pointer recognition model, and converts the pointer into a pointer line based on the center position of the circle in the image containing only the dual-range dial;

[0011] A text recognition module, which uses a dual-range dial text recognition model to identify text in an image containing only the dual-range dial to obtain the range text;

[0012] The pointer angle ratio recognition module draws an enclosing box for each recognized range text, calculates the coordinates of the center point of each enclosing box, calculates the vertical distance from the center of each enclosing box to the pointer line, and selects two enclosing boxes with the shortest vertical distance from the pointer line on both sides of the pointer line relative to the image of the dual-range dial according to the distance, then calculates the length of the line from the center of each enclosing box to the center of the circle, and selects two enclosing boxes with the same line length and distributed on both sides of the pointer line based on the lengths of the four lines to obtain a secondary screening enclosing box, extracts digital information from the secondary screening enclosing box, calculates the ratio of the angle formed by the pointer line and the line connecting any enclosing box to the angle formed by the lines connecting the two enclosing boxes based on the enclosing box line connecting the center of the secondary screening enclosing box and the center of the circle and the pointer line, and obtains the angle ratio;

[0013] The result output module calculates and outputs the actual value of the dual-range dial based on the range text and angle ratio corresponding to the secondary filtered bounding box and the stored mapping relationship between the pointer angle and the value of the dual-range dial.

[0014] Furthermore, the dual-range dial pointer angle-value mapping database also stores non-range text information on the dual-range dial;

[0015] After the text recognition module identifies the text in the image that only contains the dual-range dial through the dual-range dial text recognition model, it also eliminates the non-range text information in the identified text based on the non-range text information stored on the dual-range dial to obtain the range text.

[0016] Furthermore, it also includes:

[0017] The image preprocessing module is used to preprocess the image of the dual-range dial acquired by the image acquisition module.

[0018] Furthermore, in the pointer recognition module, the pointer in the image containing only the dual-range dial is recognized by the dual-range dial pointer recognition model, and the pointer is equivalent to a pointer line according to the center position of the circle in the image containing only the dual-range dial, including:

[0019] The dual-range dial pointer recognition model is used to identify the center point of the pixels occupied by the pointer needle in the image that only contains the dual-range dial, which is used as the pointer needle point. The pointer needle point is connected to the center position of the circle in the image that only contains the dual-range dial to obtain a pointer line.

[0020] Furthermore, the circle center recognition module identifies the center position of the circle in the image containing only the dual-range dial through the dual-range dial center recognition model, including:

[0021] The dual-range dial center recognition model is used to identify the numerical marking lines in the image that only contains the dual-range dial, and at least two of the numerical marking lines are extended and intersected. The intersection point is the center position of the circle in the image that only contains the dual-range dial.

[0022] Furthermore, the dual-range dial center recognition model in the center recognition module can also be obtained using the following steps:

[0023] Collect images of various types of dual-range dials with different backgrounds, lighting conditions, angles, and resolutions.

[0024] Mark the center position of the dual-range dial in each image;

[0025] Build a dual-range dial center recognition model based on YOLOv5;

[0026] The dual-range dial center recognition model based on YOLOv5 is trained using labeled images, and the model is tuned according to the results until convergence, thus obtaining the dual-range dial center recognition model.

[0027] Furthermore, in the pointer angle ratio recognition module, the coordinates of the center point of each bounding box are calculated, including:

[0028] For any bounding box, a side of the bounding box close to the numerical marking line is identified, and the center point coordinates of this side are calculated as the center point coordinates of the bounding box.

[0029] Furthermore, in the pointer angle ratio recognition module, the ratio of the angle formed by the pointer line and any bounding box line to the angle formed by the two bounding box lines is calculated based on the bounding box line and the circle center that are secondary screened out, to obtain the angle ratio, including:

[0030] Connect the center of the bounding box and the center of the circle according to the secondary screening to obtain two bounding box lines on both sides of the pointer line, and calculate the angle between the two bounding box lines, which is recorded as the first angle;

[0031] Calculate the angle between the pointer line and the line connecting the bounding box near the starting point of the range, and record it as the second angle; or calculate the angle between the pointer line and the line connecting the bounding box far from the starting point of the range, and record it as the third angle;

[0032] The angle ratio is obtained by dividing the second angle or the third angle by the first angle.

[0033] Furthermore, in the result output module, the actual value of the dual-range dial is calculated and output based on the range text and angle ratio corresponding to the secondarily screened bounding box and the stored mapping relationship between the pointer angle and the value of the dual-range dial, including:

[0034] Based on the mapping relationship between the range text corresponding to the two bounding boxes screened out twice and the pointer angle and value of the stored dual-range dial, the pointer angles corresponding to the two range texts are obtained and recorded as the first range angle and the second range angle respectively. The first range angle is closer to the range starting point than the second range angle.

[0035] Calculate the second actual angle corresponding to the second angle: subtract the first range angle from the second range angle to obtain the first actual angle; multiply the first actual angle by the angle ratio to obtain the second actual angle corresponding to the second angle or the third actual angle corresponding to the third angle; when the third actual angle is obtained, subtract the third actual angle from the first actual angle to obtain the second actual angle;

[0036] Add the second actual angle to the first range angle to get the actual angle of the pointer;

[0037] The stored mapping relationship between the pointer angle and the value of the dual-range dial is queried according to the actual pointer angle, and the actual value of the dual-range dial is obtained and output.

[0038] Furthermore, in the result output module, the actual value of the dual-range dial is calculated and output based on the range text and angle ratio corresponding to the secondarily screened bounding box and the stored mapping relationship between the pointer angle and the value of the dual-range dial, including:

[0039] According to the mapping relationship between the range text corresponding to the two bounding boxes screened out twice and the pointer angle and value of the stored dual-range dial, the values ​​corresponding to the two range texts and the corresponding ranges are obtained and recorded as the first range value and the second range value respectively, wherein the first range value is smaller than the second range value;

[0040] Calculate the difference: subtract the first range value from the second range value to get the difference;

[0041] Calculating a second actual value corresponding to the second angle: multiplying the calculated difference by the angle ratio to obtain the second actual value corresponding to the second angle or the third actual value corresponding to the third angle; if the third actual value is obtained, subtracting the third actual value from the difference to obtain the second actual value;

[0042] Add the second actual value to the first range value to obtain the actual value under the corresponding range;

[0043] According to the actual value under the corresponding range, the stored mapping relationship between the pointer angle and the value of the dual-range dial is queried to obtain the actual value under the other range of the dual-range dial, and the actual value under the corresponding range is output as the actual value of the dual-range dial.

[0044] Furthermore, the steps for obtaining the dual-range dial recognition model in the dial recognition module are as follows:

[0045] Collect images of various types of dual-range dials with different backgrounds, lighting conditions, angles, and resolutions.

[0046] Mark the location of the dual-range dial in each image;

[0047] Build a dual-range dial recognition model based on YOLOv5;

[0048] The dual-range dial recognition model based on YOLOv5 is trained using labeled images, and the model is tuned according to the results until convergence, thus obtaining the dual-range dial recognition model.

[0049] Furthermore, the steps for obtaining the dual-range dial pointer recognition model in the pointer recognition module are as follows:

[0050] Collect images of various types of dual-range dials with different backgrounds, lighting conditions, angles, and resolutions.

[0051] Mark the pointer position of the dual-range dial in each image;

[0052] Build a dual-range dial pointer recognition model based on YOLOv5;

[0053] The dual-range dial pointer recognition model based on YOLOv5 is trained using labeled images, and the model is tuned according to the results until convergence, thus obtaining the dual-range dial pointer recognition model.

[0054] Furthermore, the steps for obtaining the dual-range dial text recognition model in the text recognition module are as follows:

[0055] Collect images of various types of dual-range dials with different backgrounds, lighting conditions, angles, and resolutions.

[0056] Label the text on the dual-range dial in each image;

[0057] Build a dual-range dial text recognition model based on OCR;

[0058] The dual-range dial text model based on OCR is trained using labeled images, and the model is tuned according to the results until convergence, thus obtaining the dual-range dial text model.

[0059] The present invention provides a dual-range dial automatic recognition method based on an RGB camera, comprising the following steps:

[0060] A mapping relationship between the pointer angle and the value of the dual-range dial is established based on the relationship between the actual pointer angle and the actual value of the dual-range dial, and saved;

[0061] Capture images of the dual-range dial using an RGB camera;

[0062] Identify the position of the dual-range dial in the image using the dual-range dial recognition model, and output an image containing only the dual-range dial;

[0063] The center position of the circle in the image containing only the dual-range dial is identified by using the dual-range dial center recognition model;

[0064] Identifying a pointer in an image containing only the dual-range dial using a dual-range dial pointer recognition model, and equating the pointer to a pointer line based on a center position of a circle in the image containing only the dual-range dial;

[0065] Recognize the text in the image containing only the dual-range dial using the dual-range dial text recognition model to obtain the range text;

[0066] Draw an enclosing box for each recognized range text, calculate the coordinates of the center point of each enclosing box, calculate the vertical distance from the center of each enclosing box to the pointer line, and select two enclosing boxes with the shortest vertical distance from the pointer line on both sides of the pointer line relative to the image of the dual-range dial according to the distance, then calculate the length of the line from the center of each enclosing box to the center of the circle, and select two enclosing boxes with the same line length and distributed on both sides of the pointer line according to the lengths of the four lines to obtain a secondary screening enclosing box, extract digital information from the secondary screening enclosing box, calculate the proportion of the angle formed by the pointer line and the line connecting any enclosing box in the angle formed by the lines connecting the two enclosing boxes according to the enclosing box line connecting the center of the secondary screening enclosing box and the center of the circle and the pointer line, and obtain the angle ratio;

[0067] The actual value of the dual-range dial is calculated and output based on the range text and angle ratio corresponding to the secondary filtered enclosing frame and the stored mapping relationship between the pointer angle and the value of the dual-range dial.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] 1. The present invention first establishes a mapping relationship between the pointer angle and the value of the dual-range dial based on the relationship between the actual pointer angle and the actual value of the dual-range dial, then collects the image of the dual-range dial through an RGB camera, and then performs dial positioning, pointer positioning, circle center positioning and text recognition on the collected dual-range dial image, and then performs pointer angle recognition based on the pointer, circle center and text positions. Finally, based on the recognized text, pointer angle ratio and the mapping relationship between the pointer angle and the value of the dual-range dial, the actual value of the dual-range dial is recognized. Compared with the existing technology, the present invention has an automatic recognition function, reduces the labor intensity of the staff, and improves the recognition accuracy.

[0070] 2. The present invention can change the mapping relationship between the pointer angle and the numerical value of the range dial according to the actual situation of the dual-range dial, and then identify the range through the above method. It can adapt to various types of dual-range dials and has strong compatibility.

[0071] 3. The present invention recognizes the pointer angle based on the pointer, circle center and text position, and can recognize not only the actual value of the dual-range dial, but also the actual value of the single-range dial, and has strong adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 This is a system framework diagram of the dual-range dial automatic recognition system based on RGB camera of the present invention;

[0073] Figure 2 This is a flow chart of the automatic recognition method of a dual-range dial based on an RGB camera of the present invention. DETAILED DESCRIPTION

[0074] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0075] A dual-range dial generally has two ranges on its surface, sharing a pointer, and has one or two range indicator lines with the center of the pointer's rotation circle as the center (when there are two range indicator lines, one is close to the center and the other is far from the center). The range indicator line is equally divided by each numerical marking line, and the two ranges correspond to each numerical marking line, and are located on the side of the range indicator line close to the center and the side far from the center respectively. Since there are two ranges on it and there are many range texts, the simple deep learning model has poor training effect on the dual-range dial, and it is difficult to analyze the dual-range dial information, thus relying on manual reading of the dual-range dial.

[0076] See also Figure 1 The embodiment of the present invention provides a dual-range dial automatic recognition system based on an RGB camera, comprising:

[0077] A dual-range dial pointer angle-value mapping relationship database is used to establish and save the mapping relationship between the pointer angle and the value of the dual-range dial based on the relationship between the actual pointer angle and the actual value of the dual-range dial;

[0078] Image acquisition module, which uses an RGB camera to capture images of the dual-range dial;

[0079] A dial recognition module, which identifies the position of the dual-range dial in the image using a dual-range dial recognition model and outputs an image containing only the dual-range dial;

[0080] A circle center recognition module, which uses a dual-range dial circle center recognition model to identify the center position of a circle in an image that only contains a dual-range dial;

[0081] a pointer recognition module that recognizes the pointer in the image containing only the dual-range dial using a dual-range dial pointer recognition model, and converts the pointer into a pointer line based on the center position of the circle in the image containing only the dual-range dial;

[0082] A text recognition module, which uses a dual-range dial text recognition model to identify text in an image containing only the dual-range dial to obtain the range text;

[0083] The pointer angle ratio recognition module draws an enclosing box for each recognized range text, calculates the coordinates of the center point of each enclosing box, calculates the vertical distance from the center of each enclosing box to the pointer line, and selects two enclosing boxes with the shortest vertical distance from the pointer line on both sides of the pointer line relative to the image of the dual-range dial according to the distance, then calculates the length of the line from the center of each enclosing box to the center of the circle, and selects two enclosing boxes with the same line length and distributed on both sides of the pointer line based on the lengths of the four lines to obtain a secondary screening enclosing box, extracts digital information from the secondary screening enclosing box, calculates the ratio of the angle formed by the pointer line and the line connecting any enclosing box to the angle formed by the lines connecting the two enclosing boxes based on the enclosing box line connecting the center of the secondary screening enclosing box and the center of the circle and the pointer line, and obtains the angle ratio;

[0084] The result output module calculates and outputs the actual value of the dual-range dial based on the range text and angle ratio corresponding to the secondary filtered bounding box and the stored mapping relationship between the pointer angle and the value of the dual-range dial.

[0085] It can be understood that in the above scheme, a mapping relationship between the pointer angle and the value of the dual-range dial is first established based on the relationship between the actual pointer angle and the actual value of the dual-range dial, and then the image of the dual-range dial is collected through the RGB camera, and then the dial positioning, pointer positioning, circle center positioning and text recognition are performed on the collected dual-range dial image, and then the pointer angle ratio is identified based on the pointer, circle center and text positions, and finally the actual value of the dual-range dial is identified based on the identified text, pointer angle ratio and the mapping relationship between the pointer angle and the value of the dual-range dial.

[0086] Among them, because when the RGB camera captures the image of the dual-range dial, the text in the image of the dual-range dial may be blocked by the pointer or cannot be recognized due to reflections and other reasons. Therefore, in the above scheme, in the pointer angle ratio recognition module, four bounding boxes are first screened out (two on each side of the pointer), and then after a secondary screening, two bounding boxes with the same line connecting the center of the circle are screened out, and two bounding boxes with the same line connecting the center of the circle are selected. In this way, it can be determined that the ranges corresponding to the two selected bounding boxes are the same, overcoming the problem that the deep learning model is difficult to analyze the information of the dual-range dial.

[0087] At this time, assuming that the outer bounding box corresponds to the range away from the center of the circle, and the inner bounding box corresponds to the range close to the center of the circle, if the outer bounding box and the inner bounding box are not blocked by the pointer and there is no reflection, the pointer angle ratio recognition module will first screen out 4 bounding boxes, that is, two on each side of the pointer. For either side of the pointer, they are the shortest outer bounding box (the outer bounding box closest to the pointer) and the shortest inner bounding box (the inner bounding box closest to the pointer). As can be seen from the subsequent description, it is only necessary to select the bounding boxes of the same range to perform subsequent calculations to obtain relatively accurate readings.

[0088] If an outer bounding box (the shortest outer bounding box) closest to the pointer on either side of the pointer cannot be identified due to reflection, this side will select two inner bounding boxes (since the outer bounding box and the inner bounding box have different radii relative to the center of the circle, the distance from the center point of the second shortest outer bounding box to the pointer line must be greater than the distance from the center point of the second shortest inner bounding box to the pointer line). Therefore, at this time, at least three bounding boxes have the same distance to the center of the circle, and it is necessary to screen out the two bounding boxes on both sides of the pointer for the second time. According to the subsequent description, when there are two bounding boxes at the same distance from the center of the circle on the same side, you can choose any one of them and both can be read. However, for the sake of maximum accuracy, it is recommended that when there are two bounding boxes at the same distance from the center of the circle on the same side, when selecting the bounding box on this side for the second time, give priority to the bounding box closest to the pointer.

[0089] If the inner bounding box closest to the pointer on either side (the shortest inner bounding box) cannot be identified due to reflection, this side will select an inner bounding box (actually the second shortest inner bounding box) and an outer bounding box. In actual use, this will not affect the reading.

[0090] If an inner bounding box (shortest inner bounding box) closest to the pointer and an outer bounding box (shortest outer bounding box) closest to the pointer on either side cannot be identified due to reflection, this side will select the second shortest inner bounding box and the second shortest outer bounding box, and the subsequent processing is the same as when there is no reflection.

[0091] The same applies to other situations, which do not affect the reading. Thus, the above solution can also prevent the text in the image of the dual-range dial from being blocked by the pointer or being unable to be recognized due to reflections, etc., resulting in the situation where the reading cannot be read.

[0092] In some embodiments, the dual-range dial pointer angle-value mapping database may also store non-range text information on the dual-range dial;

[0093] The text recognition module, after identifying the text in the image that only contains the dual-range dial through the dual-range dial text recognition model, also eliminates the non-range text information in the identified text based on the non-range text information stored on the dual-range dial to obtain the range text.

[0094] It is understandable that on some dual-range dials, in addition to the range text, there will also be non-range text such as the manufacturer, the numerical unit in the middle, etc., which may cause reading errors. Therefore, these non-range texts need to be excluded. The above scheme adopts the method of pre-storing these non-range text information and excluding them after identification to eliminate these non-range text information to avoid reading interference.

[0095] In some embodiments, it may also include:

[0096] The image preprocessing module is used to preprocess the image of the dual-range dial acquired by the image acquisition module.

[0097] It is understandable that preprocessing may include operations such as denoising, enhancement, and perspective transformation to further improve the accuracy of subsequent recognition and calculation.

[0098] In some embodiments, in the pointer recognition module, identifying the pointer in an image containing only the dual-range dial using a dual-range dial pointer recognition model, and equating the pointer to a pointer line based on the center position of a circle in the image containing only the dual-range dial may include:

[0099] The dual-range dial pointer recognition model is used to identify the center point of the pixels occupied by the pointer needle in the image that only contains the dual-range dial, which is used as the pointer needle point. The pointer needle point is connected to the center position of the circle in the image that only contains the dual-range dial to obtain a pointer line.

[0100] It is understandable that the pointers in existing dual-range dials come in various forms and must have a certain width (some devices even design the pointers to be curved for the sake of aesthetics or to highlight individuality), but the point corresponding to the pointer needle must correspond to the value to be read. Therefore, the center point of the pixel occupied by the pointer needle is directly used as the pointer needle point, and then it is connected to the center position of the circle in the image that only contains the dual-range dial to obtain the pointer line.

[0101] In some embodiments, the circle center recognition module, which identifies the center position of the circle in the image containing only the dual-range dial using the dual-range dial circle center recognition model, may include:

[0102] The dual-range dial center recognition model is used to identify the numerical marking lines in the image that only contains the dual-range dial, and at least two of the numerical marking lines are extended and intersected. The intersection point is the center position of the circle in the image that only contains the dual-range dial.

[0103] It is understandable that since some dual-range dials do not directly mark the center of the circle, but their numerical markings generally point to the center of the circle, the center of the circle can be determined by extending the numerical markings. Generally speaking, the numerical markings corresponding to the range text are generally longer and easier to identify. Therefore, this type of numerical marking can be selected to extend and intersect to determine the center of the circle.

[0104] In order to avoid errors in the identification of numerical markings, you can select as many identified numerical markings as possible for extension. When there is no error, the extension lines of each numerical marking should only intersect at the center of the circle. If there is an error, the extension line of a numerical marking will not pass through the center of the circle. At this time, you can select a point that passes through the largest number of numerical marking extensions as the center of the circle, avoiding the situation where the center of the circle cannot be determined due to errors in a few numerical markings.

[0105] Since there are usually many numerical markings, identifying each one would consume a lot of computing resources. Therefore, we can simplify the process by only identifying the longer numerical markings and extending them to determine the center of the circle. This avoids the problem of consuming a lot of computing resources.

[0106] In some embodiments, the dual-range dial center recognition model in the center recognition module can also be obtained using the following steps:

[0107] Collect images of various types of dual-range dials with different backgrounds, lighting conditions, angles, and resolutions.

[0108] Mark the center position of the dual-range dial in each image;

[0109] Build a dual-range dial center recognition model based on YOLOv5;

[0110] The dual-range dial center recognition model based on YOLOv5 is trained using labeled images, and the model is tuned according to the results until convergence, thus obtaining the dual-range dial center recognition model.

[0111] It is understandable that the above solution uses a trained model to identify the center of the circle. This method can directly preset the trained model without the need to identify the numerical markings in real time to find the center of the circle.

[0112] This solution can replace the above-mentioned solution of finding the center of a circle by numerical markings. At the same time, it can also coexist with the solution of finding the center of a circle by numerical markings. There are many ways of coexistence: for example, when the type of dual-range dial to be identified is a type that does not exist in the training set of the above-mentioned dual-range dial center recognition model, it is obviously more accurate to use numerical markings to find the center of the circle. At this time, the solution of finding the center of the circle by numerical markings can be called; when the trained model cannot identify the center position of a dual-range dial, the solution of finding the center of the circle by numerical markings is called; when the solution of finding the center of the circle by numerical markings cannot select the corresponding center of the circle, the trained model is called for identification, etc. Adjustments can be made according to actual conditions during use or production.

[0113] In some embodiments, in the pointer angle ratio recognition module, calculating the coordinates of the center point of each bounding box may include:

[0114] For any bounding box, identify the side of the bounding box close to the numerical marking line, and calculate the center point coordinates of this side as the center point coordinates of the bounding box.

[0115] It can be understood that since each range text is at a certain distance from the numerical marking line, when generating the bounding box, each bounding box is also at a certain distance from the corresponding numerical marking line, and the farther the point is from the numerical marking line, the greater the error will be. Therefore, when selecting the center point coordinates of the bounding box, it is preferred to select the side of the bounding box close to the numerical marking line, and calculate the center point coordinates of this side as the center point coordinates of the bounding box.

[0116] In some embodiments, the pointer angle ratio recognition module calculates the ratio of the angle formed by the pointer line and any bounding box line to the angle formed by the two bounding box lines based on the bounding box line and the circle center that are secondary filtered out to obtain the angle ratio, which may include:

[0117] Connect the center of the bounding box and the center of the circle according to the secondary screening to obtain two bounding box lines on both sides of the pointer line, and calculate the angle between the two bounding box lines, which is recorded as the first angle;

[0118] Calculate the angle between the pointer line and the line connecting the bounding box near the starting point of the range, and record it as the second angle; or calculate the angle between the pointer line and the line connecting the bounding box far from the starting point of the range, and record it as the third angle;

[0119] The angle ratio is obtained by dividing the second angle or the third angle by the first angle.

[0120] It can be understood that the above solution proposes a specific calculation method for calculating the angle ratio.

[0121] In some embodiments, in the result output module, calculating and outputting the actual value of the dual-range dial based on the range text and angle ratio corresponding to the secondarily screened bounding box and the stored mapping relationship between the pointer angle and the value of the dual-range dial may include:

[0122] Based on the mapping relationship between the range text corresponding to the two bounding boxes screened out twice and the pointer angle and value of the stored dual-range dial, the pointer angles corresponding to the two range texts are obtained and recorded as the first range angle and the second range angle respectively. The first range angle is closer to the range starting point than the second range angle.

[0123] Calculate the second actual angle corresponding to the second angle: subtract the first range angle from the second range angle to obtain the first actual angle; multiply the first actual angle by the angle ratio to obtain the second actual angle corresponding to the second angle or the third actual angle corresponding to the third angle; when the third actual angle is obtained, subtract the third actual angle from the first actual angle to obtain the second actual angle;

[0124] Add the second actual angle to the first range angle to get the actual angle of the pointer;

[0125] The stored mapping relationship between the pointer angle and the value of the dual-range dial is queried according to the actual pointer angle, and the actual value of the dual-range dial is obtained and output.

[0126] It is understandable that the above solution directly calculates the actual pointer angle, through which the actual value of the dual-range dial can be obtained and output according to the pre-stored mapping relationship between the pointer angle and the value of the dual-range dial.

[0127] When calculating the second actual angle, the following calculation method can also be used:

[0128] When the angle ratio corresponds to the third angle, the second actual angle can be obtained by subtracting the angle ratio from 1 and multiplying the result by the first actual angle.

[0129] In some embodiments, in the result output module, calculating and outputting the actual value of the dual-range dial based on the range text and angle ratio corresponding to the secondarily screened bounding box and the stored mapping relationship between the pointer angle and the value of the dual-range dial may include:

[0130] According to the mapping relationship between the range text corresponding to the two bounding boxes screened out twice and the pointer angle and value of the stored dual-range dial, the values ​​corresponding to the two range texts and the corresponding ranges are obtained and recorded as the first range value and the second range value respectively, wherein the first range value is smaller than the second range value;

[0131] Calculate the difference: subtract the first range value from the second range value to get the difference;

[0132] Calculating a second actual value corresponding to the second angle: multiplying the calculated difference by the angle ratio to obtain the second actual value corresponding to the second angle or the third actual value corresponding to the third angle; if the third actual value is obtained, subtracting the third actual value from the difference to obtain the second actual value;

[0133] Add the second actual value to the first range value to obtain the actual value under the corresponding range;

[0134] According to the actual value under the corresponding range, the stored mapping relationship between the pointer angle and the value of the dual-range dial is queried to obtain the actual value under the other range of the dual-range dial, and the actual value under the corresponding range is output as the actual value of the dual-range dial.

[0135] It is understandable that the above solution does not calculate the actual angle of the pointer, but directly obtains the actual value. The principle is as follows:

[0136] Assume that the angle ratio is x, the first range value is recorded as a, the pointer angle corresponding to the first range value is α, the second range value is recorded as b, the pointer angle corresponding to the second range value is β, the actual angle of the pointer is φ, and the actual reading under this range is c;

[0137] Then we have:

[0138]

[0139] therefore,

[0140] Then there is: c=(ba)x+a

[0141] It can be seen that since the range value corresponds to the pointer angle and is linear, the actual reading under the corresponding range can be directly calculated.

[0142] In some embodiments, the steps for obtaining the dual-range dial recognition model in the dial recognition module may be as follows:

[0143] Collect images of various types of dual-range dials with different backgrounds, lighting conditions, angles, and resolutions.

[0144] Mark the location of the dual-range dial in each image;

[0145] Build a dual-range dial recognition model based on YOLOv5;

[0146] The dual-range dial recognition model based on YOLOv5 is trained using labeled images, and the model is tuned according to the results until convergence, thus obtaining the dual-range dial recognition model.

[0147] In some embodiments, the steps for obtaining the dual-range dial pointer recognition model in the pointer recognition module may be as follows:

[0148] Collect images of various types of dual-range dials with different backgrounds, lighting conditions, angles, and resolutions.

[0149] Mark the pointer position of the dual-range dial in each image;

[0150] Build a dual-range dial pointer recognition model based on YOLOv5;

[0151] The dual-range dial pointer recognition model based on YOLOv5 is trained using labeled images, and the model is tuned according to the results until convergence, thus obtaining the dual-range dial pointer recognition model.

[0152] In some embodiments, the steps for obtaining the dual-range dial text recognition model in the text recognition module may be as follows:

[0153] Collect images of various types of dual-range dials with different backgrounds, lighting conditions, angles, and resolutions.

[0154] Label the text on the dual-range dial in each image;

[0155] Build a dual-range dial text recognition model based on OCR;

[0156] The dual-range dial text model based on OCR is trained using labeled images, and the model is tuned according to the results until convergence, thus obtaining the dual-range dial text model.

[0157] It is understandable that each of the above-mentioned identifications can be performed using a corresponding deep learning recognition model or a machine learning model.

[0158] See also Figure 2 , which is a flow chart of a dual-range dial automatic recognition method based on an RGB camera provided in an embodiment of the present invention, comprising the following steps:

[0159] A mapping relationship between the pointer angle and the value of the dual-range dial is established based on the relationship between the actual pointer angle and the actual value of the dual-range dial, and saved;

[0160] Capture images of the dual-range dial using an RGB camera;

[0161] Identify the position of the dual-range dial in the image using the dual-range dial recognition model, and output an image containing only the dual-range dial;

[0162] The center position of the circle in the image containing only the dual-range dial is identified by using the dual-range dial center recognition model;

[0163] Identifying a pointer in an image containing only the dual-range dial using a dual-range dial pointer recognition model, and equating the pointer to a pointer line based on a center position of a circle in the image containing only the dual-range dial;

[0164] Recognize the text in the image containing only the dual-range dial using the dual-range dial text recognition model to obtain the range text;

[0165] Draw an enclosing box for each recognized range text, calculate the coordinates of the center point of each enclosing box, calculate the vertical distance from the center of each enclosing box to the pointer line, and select two enclosing boxes with the shortest vertical distance from the pointer line on both sides of the pointer line relative to the image of the dual-range dial according to the distance, then calculate the length of the line from the center of each enclosing box to the center of the circle, and select two enclosing boxes with the same line length and distributed on both sides of the pointer line according to the lengths of the four lines to obtain a secondary screening enclosing box, extract digital information from the secondary screening enclosing box, calculate the proportion of the angle formed by the pointer line and the line connecting any enclosing box in the angle formed by the lines connecting the two enclosing boxes according to the enclosing box line connecting the center of the secondary screening enclosing box and the center of the circle and the pointer line, and obtain the angle ratio;

[0166] The actual value of the dual-range dial is calculated and output based on the range text and angle ratio corresponding to the secondary filtered enclosing frame and the stored mapping relationship between the pointer angle and the value of the dual-range dial.

[0167] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. The dual-range dial automatic recognition system based on RGB camera is characterized by: include: A dual-range dial pointer angle-value mapping relationship database is used to establish and save the mapping relationship between the pointer angle and the value of the dual-range dial based on the relationship between the actual pointer angle and the actual value of the dual-range dial; Image acquisition module, which uses an RGB camera to capture images of the dual-range dial; A dial recognition module, which identifies the position of the dual-range dial in the image using a dual-range dial recognition model and outputs an image containing only the dual-range dial; A circle center recognition module, which uses a dual-range dial circle center recognition model to identify the center position of a circle in an image that only contains a dual-range dial; a pointer recognition module that recognizes the pointer in the image containing only the dual-range dial using a dual-range dial pointer recognition model, and converts the pointer into a pointer line based on the center position of the circle in the image containing only the dual-range dial; A text recognition module, which uses a dual-range dial text recognition model to identify text in an image containing only the dual-range dial to obtain the range text; The pointer angle ratio recognition module draws an enclosing box for each recognized range text, calculates the coordinates of the center point of each enclosing box, calculates the vertical distance from the center of each enclosing box to the pointer line, and selects two enclosing boxes with the shortest vertical distance from the pointer line on both sides of the pointer line relative to the image of the dual-range dial according to the distance, then calculates the length of the line from the center of each enclosing box to the center of the circle, and selects two enclosing boxes with the same line length and distributed on both sides of the pointer line based on the lengths of the four lines to obtain a secondary screening enclosing box, extracts digital information from the secondary screening enclosing box, calculates the ratio of the angle formed by the pointer line and the line connecting any enclosing box to the angle formed by the lines connecting the two enclosing boxes based on the enclosing box line connecting the center of the secondary screening enclosing box and the center of the circle and the pointer line, and obtains the angle ratio; The result output module calculates and outputs the actual value of the dual-range dial based on the range text and angle ratio corresponding to the secondary filtered bounding box and the stored mapping relationship between the pointer angle and the value of the dual-range dial.

2. The dual-range dial automatic recognition system based on RGB camera according to claim 1, characterized in that: The dual-range dial pointer angle-value mapping database also stores non-range text information on the dual-range dial; After the text recognition module identifies the text in the image that only contains the dual-range dial through the dual-range dial text recognition model, it also eliminates the non-range text information in the identified text based on the non-range text information stored on the dual-range dial to obtain the range text.

3. The dual-range dial automatic recognition system based on RGB camera according to claim 1, characterized in that: The pointer recognition module identifies the pointer in the image containing only the dual-range dial using the dual-range dial pointer recognition model, and equates the pointer to a pointer line based on the center position of the circle in the image containing only the dual-range dial, including: The dual-range dial pointer recognition model is used to identify the center point of the pixels occupied by the pointer needle in the image that only contains the dual-range dial, which is used as the pointer needle point. The pointer needle point is connected to the center position of the circle in the image that only contains the dual-range dial to obtain a pointer line.

4. The dual-range dial automatic recognition system based on an RGB camera according to claim 1, characterized in that: The circle center recognition module identifies the center position of the circle in the image containing only the dual-range dial through the dual-range dial center recognition model, including: The dual-range dial center recognition model is used to identify the numerical marking lines in the image that only contains the dual-range dial, and at least two of the numerical marking lines are extended and intersected. The intersection point is the center position of the circle in the image that only contains the dual-range dial.

5. The dual-range dial automatic recognition system based on RGB camera according to claim 4, characterized in that: In the pointer angle ratio recognition module, the coordinates of the center point of each bounding box are calculated, including: For any bounding box, a side of the bounding box close to the numerical marking line is identified, and the center point coordinates of this side are calculated as the center point coordinates of the bounding box.

6. The dual-range dial automatic recognition system based on RGB camera according to claim 1, characterized in that: In the pointer angle ratio recognition module, the ratio of the angle formed by the pointer line and any bounding box line to the angle formed by the two bounding box lines is calculated based on the bounding box line and the circle center that are secondary filtered out, and the angle ratio is obtained, including: Connect the center of the bounding box and the center of the circle according to the secondary screening to obtain two bounding box lines on both sides of the pointer line, and calculate the angle between the two bounding box lines, which is recorded as the first angle; Calculate the angle between the pointer line and the line connecting the bounding box near the starting point of the range, and record it as the second angle; or calculate the angle between the pointer line and the line connecting the bounding box far from the starting point of the range, and record it as the third angle; The angle ratio is obtained by dividing the second angle or the third angle by the first angle.

7. The dual-range dial automatic recognition system based on RGB camera according to claim 6, characterized in that: In the result output module, the actual value of the dual-range dial is calculated and output based on the range text and angle ratio corresponding to the secondarily screened bounding box and the stored mapping relationship between the pointer angle and the value of the dual-range dial, including: Based on the mapping relationship between the range text corresponding to the two bounding boxes screened out twice and the pointer angle and value of the stored dual-range dial, the pointer angles corresponding to the two range texts are obtained and recorded as the first range angle and the second range angle respectively. The first range angle is closer to the range starting point than the second range angle. Calculate the second actual angle corresponding to the second angle: subtract the first range angle from the second range angle to obtain the first actual angle; multiply the first actual angle by the angle ratio to obtain the second actual angle corresponding to the second angle or the third actual angle corresponding to the third angle; when the third actual angle is obtained, subtract the third actual angle from the first actual angle to obtain the second actual angle; Add the second actual angle to the first range angle to get the actual angle of the pointer; The stored mapping relationship between the pointer angle and the value of the dual-range dial is queried according to the actual pointer angle, and the actual value of the dual-range dial is obtained and output.

8. The dual-range dial automatic recognition system based on RGB camera according to claim 6, characterized in that: In the result output module, the actual value of the dual-range dial is calculated and output based on the range text and angle ratio corresponding to the secondarily screened bounding box and the stored mapping relationship between the pointer angle and the value of the dual-range dial, including: According to the mapping relationship between the range text corresponding to the two bounding boxes screened out twice and the pointer angle and value of the stored dual-range dial, the values ​​corresponding to the two range texts and the corresponding ranges are obtained and recorded as the first range value and the second range value respectively, wherein the first range value is smaller than the second range value; Calculate the difference: subtract the first range value from the second range value to get the difference; Calculating a second actual value corresponding to the second angle: multiplying the calculated difference by the angle ratio to obtain the second actual value corresponding to the second angle or the third actual value corresponding to the third angle; if the third actual value is obtained, subtracting the third actual value from the difference to obtain the second actual value; Add the second actual value to the first range value to obtain the actual value under the corresponding range; According to the actual value under the corresponding range, the stored mapping relationship between the pointer angle and the value of the dual-range dial is queried to obtain the actual value under the other range of the dual-range dial, and the actual value under the corresponding range is output as the actual value of the dual-range dial.

9. The dual-range dial automatic recognition system based on an RGB camera according to any one of claims 1 to 6, characterized in that: The steps for obtaining the dual-range dial recognition model in the dial recognition module are as follows: Collect images of various types of dual-range dials with different backgrounds, lighting conditions, angles, and resolutions. Mark the location of the dual-range dial in each image; Build a dual-range dial recognition model based on YOLOv5; The dual-range dial recognition model based on YOLOv5 is trained using the labeled images. The model is tuned according to the results until convergence, thus obtaining the dual-range dial recognition model. The steps for obtaining the dual-range dial pointer recognition model in the pointer recognition module are as follows: Collect images of various types of dual-range dials with different backgrounds, lighting conditions, angles, and resolutions. Mark the pointer position of the dual-range dial in each image; Build a dual-range dial pointer recognition model based on YOLOv5; The dual-range dial pointer recognition model based on YOLOv5 is trained using the labeled images. The model is tuned according to the results until convergence, thus obtaining the dual-range dial pointer recognition model. The steps for obtaining the dual-range dial center recognition model in the center recognition module are as follows: Collect images of various types of dual-range dials with different backgrounds, lighting conditions, angles, and resolutions. Mark the center position of the dual-range dial in each image; Build a dual-range dial center recognition model based on YOLOv5; The YOLOv5-based dual-range dial center recognition model is trained using labeled images. The model is optimized based on the results until convergence, resulting in a dual-range dial center recognition model. The steps for obtaining the dual-range dial text recognition model in the text recognition module are as follows: Collect images of various types of dual-range dials with different backgrounds, lighting conditions, angles, and resolutions. Label the text on the dual-range dial in each image; Build a dual-range dial text recognition model based on OCR; The dual-range dial text model based on OCR is trained using labeled images, and the model is tuned according to the results until convergence, thus obtaining the dual-range dial text model.

10. The automatic recognition method of dual-range dial based on RGB camera is characterized by: The following steps are involved: A mapping relationship between the pointer angle and the value of the dual-range dial is established based on the relationship between the actual pointer angle and the actual value of the dual-range dial, and saved; Capture images of the dual-range dial using an RGB camera; Identify the position of the dual-range dial in the image using the dual-range dial recognition model, and output an image containing only the dual-range dial; The center position of the circle in the image containing only the dual-range dial is identified by using the dual-range dial center recognition model; Identifying a pointer in an image containing only the dual-range dial using a dual-range dial pointer recognition model, and equating the pointer to a pointer line based on a center position of a circle in the image containing only the dual-range dial; Recognize the text in the image containing only the dual-range dial using the dual-range dial text recognition model to obtain the range text; Draw an enclosing box for each recognized range text, calculate the coordinates of the center point of each enclosing box, calculate the vertical distance from the center of each enclosing box to the pointer line, and select two enclosing boxes with the shortest vertical distance from the pointer line on both sides of the pointer line relative to the image of the dual-range dial according to the distance, then calculate the length of the line from the center of each enclosing box to the center of the circle, and select two enclosing boxes with the same line length and distributed on both sides of the pointer line according to the lengths of the four lines to obtain a secondary screening enclosing box, extract digital information from the secondary screening enclosing box, calculate the proportion of the angle formed by the pointer line and the line connecting any enclosing box in the angle formed by the lines connecting the two enclosing boxes according to the enclosing box line connecting the center of the secondary screening enclosing box and the center of the circle and the pointer line, and obtain the angle ratio; The actual value of the dual-range dial is calculated and output based on the range text and angle ratio corresponding to the secondary filtered enclosing frame and the stored mapping relationship between the pointer angle and the value of the dual-range dial.

Images