Pointer type instrument panel reading identification method, device and equipment and medium

Abstract

The invention provides a pointer type instrument panel reading identification method and device, equipment and a medium, and the method comprises the steps: obtaining an instrument panel image data set through determining the panel information of a target instrument panel, constructing a first instrument panel detection model, determining the position of the instrument panel, and carrying out the cutting, and constructing a second instrument panel detection model to determine the scale and pointer positions, and finally calculating the instrument panel reading based on the scale, the pointer contour coordinates and the panel surface information. The reading recognition efficiency of the pointer type instrument panel can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, equipment and medium for identifying readings of a pointer-type instrument panel. Background Art

[0002] In many fields such as electricity, machinery, and chemical industry, dashboards are often used frequently to clearly display various key data information. However, when actually reading and identifying the information on the dashboard, in most cases it still depends on manual viewing and judgment one by one, or based on traditional relatively old image algorithms to identify the dial information.

[0003] The former method often requires more manpower costs, including recruiting a large number of professionals, training, and paying continuous wages and salaries; the latter requires high-quality, clear, and interference-free image data to a large extent, such as high-resolution images, non-reflective and unobstructed dial photos, etc. Otherwise, it is difficult to accurately identify the information on the dashboard. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a pointer-type instrument panel reading recognition method, device, equipment and medium, which can improve the efficiency of pointer-type instrument panel reading recognition.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for identifying a pointer-type instrument panel reading, the method comprising:

[0007] Determine the panel information of the target dashboard;

[0008] Acquire a first instrument panel image dataset of the target instrument panel, construct a first instrument panel detection model for determining a position of the target instrument panel in an image based on the first instrument panel image dataset, and perform cropping processing on the first instrument panel image dataset based on the first instrument panel detection model to obtain a second instrument panel image dataset;

[0009] constructing a second instrument panel detection model for determining the positions of the scale and the pointer in the target instrument panel based on the second instrument panel image data set, and determining the scale outline coordinate data and the pointer outline coordinate data in the second instrument panel image data set by using the second instrument panel detection model;

[0010] The reading of the target instrument panel is calculated based on the scale outline coordinate data, the pointer outline coordinate data and the panel surface information.

[0011] In a second aspect, an embodiment of the present application further provides a pointer-type instrument panel reading recognition device, the device comprising:

[0012] A determination module, used to determine the panel information of the target instrument panel;

[0013] A first construction module is used to obtain a first dashboard image dataset of the target dashboard, construct a first dashboard detection model for determining the position of the target dashboard in the image based on the first dashboard image dataset, and perform cropping processing on the first dashboard image dataset based on the first dashboard detection model to obtain a second dashboard image dataset;

[0014] A second construction module is used to construct a second instrument panel detection model for determining the scale and pointer positions in the target instrument panel based on the second instrument panel image data set, and determine the scale outline coordinate data and the pointer outline coordinate data in the second instrument panel image data set by using the second instrument panel detection model;

[0015] A calculation module is used to calculate the reading of the target instrument panel based on the scale outline coordinate data, the pointer outline coordinate data and the panel surface information.

[0016] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to execute the pointer instrument panel reading recognition method described in any one of the first aspects.

[0017] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for identifying a pointer instrument panel reading described in any one of the first aspects is executed.

[0018] The embodiments of the present application have the following beneficial effects:

[0019] (1) The embodiments of the present application can be widely used in multiple fields such as electricity, machinery, and chemical industry. Through the efficient remote monitoring function, the dial readings can be grasped in real time regardless of the distance, providing timely and accurate information support for decision-making.

[0020] (2) The embodiments of the present application can realize automatic inspection without manual inspection one by one, which greatly saves manpower and time costs, improves work efficiency, and reduces the risk of human error, bringing greater economic benefits to enterprises and individuals.

[0021] (3) The embodiments of the present application can read a large number of dial readings at the same time, thereby extracting valuable information therefrom, providing strong support for decision-making, helping enterprises or individuals make more informed decisions, and promoting business development and innovation.

[0022] (4) The embodiments of the present application can timely discover potential problems and risks, improve safety, and avoid safety accidents caused by human negligence or errors. At the same time, its advanced technical concepts and innovative design ideas are favored by the market and users, and have broad market application prospects and huge commercial value. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 It is a flowchart of steps S101-S104 provided in an embodiment of the present application;

[0025] Figure 2 It is a flowchart of steps S201-S202 provided in an embodiment of the present application;

[0026] Figure 3 It is a flowchart of steps S301-S302 provided in an embodiment of the present application;

[0027] Figure 4 It is a flowchart of steps S401-S404 provided in an embodiment of the present application;

[0028] Figure 5 It is a structural schematic diagram of a pointer-type instrument panel reading recognition device provided in an embodiment of the present application;

[0029] Figure 6 It is a schematic diagram of the composition structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0030] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0031] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0032] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0033] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0034] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0036] See also Figure 1 , Figure 1is a flow chart of steps S101-S104 of the pointer instrument panel reading recognition method provided in the embodiment of the present application, which will be combined with Figure 1 Steps S101-S104 are shown for explanation.

[0037] In step S101, the panel information of the target instrument panel is determined.

[0038] This step is the preparation stage, which requires collecting basic information about the target dashboard, such as the dashboard model, size, scale range, scale value, etc. This information will be used for readings.

[0039] In step S102, a first dashboard image dataset of the target dashboard is obtained, a first dashboard detection model for determining the position of the target dashboard in the image is constructed based on the first dashboard image dataset, and the first dashboard image dataset is cropped based on the first dashboard detection model to obtain a second dashboard image dataset.

[0040] Here, we first collect a large amount of image data containing the target dashboard to form a first dashboard image dataset. Then, we use these image data to build a detection model that can identify and locate the dashboard in the image. After training, we use the first dashboard detection model to crop each image in the image dataset, retaining only the dashboard part, to form a second dashboard image dataset.

[0041] In step S103, a second instrument panel detection model for determining the scale and pointer positions in the target instrument panel is constructed based on the second instrument panel image data set, and the scale outline coordinate data and pointer outline coordinate data in the second instrument panel image data set are determined by the second instrument panel detection model.

[0042] Next, a more sophisticated detection model is built using the second dashboard image dataset, which can recognize the scale and pointer on the dashboard. Through this model, the scale outline coordinate data and the pointer outline coordinate data can be extracted, which are used to calculate the reading.

[0043] In step S104, the reading of the target instrument panel is calculated based on the scale outline coordinate data, the pointer outline coordinate data and the panel surface information.

[0044] With the scale outline coordinate data, pointer outline coordinate data and disk information, we can start calculating the reading. First, determine the scale range of the instrument panel based on the scale outline coordinate data. Then, determine the current position of the pointer based on the pointer outline coordinate data. Finally, combine the disk information (such as the minimum scale value) and the pointer position to calculate the precise reading of the instrument panel.

[0045] In some embodiments, see Figure 2 , Figure 2 It is a flow chart of steps S201-S202 provided in an embodiment of the present application. The first dashboard image data set is obtained by collecting the target dashboard in multiple directions, angles, and distances through an image acquisition device. The first dashboard detection model for determining the position of the target dashboard in the image based on the first dashboard image data set can be implemented through steps S201-S202, which will be explained in combination with each step.

[0046] In step S201, a first annotation process is performed on each first instrument image in the first instrument image data set to obtain a first instrument panel training data set; wherein the first annotation process is used to determine a bounding box where the target instrument panel is located in each first instrument image.

[0047] Here, a camera is used to collect images of the target dashboard from multiple directions, angles, and distances to obtain a dashboard image dataset d1, which is manually annotated. The width and height of the image are w and h, respectively. The bounding box where the dashboard is located is framed, and the coordinate positions of each bounding box are recorded. The coordinate position includes the horizontal and vertical coordinates of the center point of the bounding box (x a ,y a ), the width w of the bounding box a , the height h of the bounding box a , and they are all normalized values. Let the coordinates of the upper left corner of the bounding box be (x1, y1) and the coordinates of the lower right corner of the bounding box be (x2, y2), then Thus, the labeled dashboard training data set t1 is obtained.

[0048] In step S202, the first instrument panel training data set is trained to obtain the first instrument panel detection model.

[0049] Here, the target detection model yolov5 is used to train the training data set t1 to obtain the dashboard detection model M1, which is the first dashboard detection model.

[0050] In some embodiments, the cropping of the first instrument panel image dataset based on the first instrument panel detection model to obtain a second instrument panel image dataset includes:

[0051] Each of the first instrument images is cropped based on the first coordinate information of the bounding box, image content outside the bounding box is deleted, and the instrument image within the bounding box is retained as the second instrument panel image data set.

[0052] Here, the instrument panel detection model M1 is used to detect the collected instrument panel dataset d1, and the coordinate position of the instrument panel bounding box in the instrument panel image dataset d1 is obtained. The dataset d1 is cropped according to the coordinate position of the instrument panel bounding box, the instrument panel image content is retained, and the image content outside the dial bounding box is deleted. The cropped dataset is d2.

[0053] In some embodiments, see Figure 3 , Figure 3 It is a flow chart of steps S301-S302 provided in an embodiment of the present application. The second instrument panel detection model for determining the scale and pointer position in the target instrument panel based on the second instrument panel image data set can be implemented through steps S301-S302, which will be explained in combination with each step.

[0054] In step S301, a second annotation process is performed on each second instrument image in the second instrument image data set to obtain a second instrument panel training data set; wherein the second annotation process is used to determine a scale outline and a pointer outline in each second instrument image.

[0055] Manually annotate the dashboard image dataset d2, where the width and height of the image are w d2 and h d2 . Draw the outline of the scale and pointer, and record the coordinates of each outline. Taking the quadrilateral as an example, the coordinates of the pointer outline include the normalized coordinates of the vertices of the polygon ABCD, which is {A(x A′ ,y A′ ),B(x B′ ,y B′ ),C(x C′ ,y C′ ),D(x D′ ,y D′ )}, let the coordinates of polygon ABCD in the image be {A(x A ,y A ), B(x B ,y B ), C(x C ,y C ), D(x D ,y D )},but The same goes for the rest. Thus, the labeled dashboard training data set t2 is obtained.

[0056] In step S302, the second dashboard training data set is trained to obtain the second dashboard detection model.

[0057] Here, the YOLOv5-segment model training data set t2 is used for training to obtain the dashboard detection model M2, which is the second dashboard detection model.

[0058] In some embodiments, see Figure 4 , Figure 4 It is a flow chart of steps S401-S404 provided in an embodiment of the present application. The calculation of the reading of the target instrument panel based on the scale contour coordinate data, the pointer contour coordinate data and the disk surface information can be achieved through steps S401-S404, which will be explained in combination with each step.

[0059] In step S401, the scale position and the pointer position are calculated based on the scale outline coordinate data and the pointer outline coordinate data, respectively.

[0060] Here, the scale coordinates and the pointer coordinates can be obtained based on the idea of ​​averaging the coordinate values ​​of all pixels in the contour, and the pointer coordinates can also be obtained based on the pixel farthest from the center point of the dial among all the pixels in the contour.

[0061] The width of the dashboard image is w, the height is h, and the coordinates of the center point of the dashboard image are O(x c ,y c ). The instrument panel contains multiple scales and a pointer. The scales are represented as scale 1, scale 2, scale 3, and scale n. The outline coordinates of each scale and pointer are represented by the vertex coordinates of the polygon. Let the outline coordinates of the pointer be {(x h1 ,y h1 ), (x h2 ,y h2 ), (x h3 ,y h3 ), (x h4 ,y h4 )......}, the contour coordinates of scale 1 are {(x 1s1 ,y 1s1 ), (x 1s2 ,y 1s2 ), (x 1s3 ,y 1s3 ), (x 1s4 ,y 1s4 )......}, the contour coordinates of scale 2 are {(x 2s1 ,y 2s1 ), (x 2s2 ,y 2s2 ), (x 2s3 ,y 2s3 ), (x 2s4 ,y 2s4 )......}......The contour coordinates of scale n are {(xns1 ,y ns1 ), (x ns2 ,y ns2 ), (x ns3 ,y ns3 ), (x ns4 ,y ns4 )......}. You can use the contourArea function of OpenCV to get all the pixel points inside the polygonal outline of the pointer, set to {(x1, y1), (x2, y2), (x3, y3), (x4, y4)......(x m ,y m )}, the position coordinates of the pointer are set to (x h ,y h ). If we obtain it based on the idea of ​​averaging the coordinate values ​​of all pixels in the contour, then If the position coordinates of the pointer are obtained based on the pixel farthest from the center point of the dial among all the pixels in the outline, first calculate the distance between all points inside the polygonal outline of the pointer and the center point O, and set them as d1, d2...d m ,available The same goes for other points. Then get d1, d2, ... m The maximum value among them is set as d i , d i The corresponding polygon outline interior (x i ,y i ) is the position coordinate of the pointer. The position coordinate of the scale should be obtained by averaging the coordinate values ​​of all pixels in the contour. Referring to the calculation of the position coordinate of the pointer, the position coordinate of scale 1 can be obtained as (x 1s ,y 1s ), the position coordinate of scale 2 is (x 2s ,y 2s )......The position coordinate of scale n is (x ns ,y ns ).

[0062] In step S402, the slope and angle of the first line connecting the scale position and the center point of the target instrument panel are calculated; wherein the center point of the target instrument panel is the coordinate origin, the horizontal axis is directed to the right, the vertical axis is directed downward, and the positive direction of the vertical axis is 0 degrees.

[0063] Calculate the slopes of scale 1, scale 2, scale 3, scale n and the center point O respectively, which are k1, k2, k3, k n , the corresponding angles are w1, w2, w3...w n, the coordinate system takes the center point of the target instrument panel as the coordinate origin, the horizontal axis is to the right, the vertical axis is downward, the positive direction of the vertical axis is 0 degrees, and the angle is calculated clockwise. The corresponding angle is w1, if x 1s >x c ,y 1s >y c , then w1=360-atank1*180 / π. If x 1s <x c ,y 1s >y c , then w1=atank1*180 / π. If x 1s <x c ,y 1s <y c , then w1=180-atank1*180 / π. If x 1s >x c ,y 1s <y c , then w1=180+atank1*180 / π, and other scales can be calculated in the same way.

[0064] In step S403, a second line connecting the pointer position and the center point of the target instrument panel is determined, and two scale positions adjacent to the second line are determined from the scale positions based on the angle of the second line.

[0065] Here, calculate the straight line between the pointer and the center point, and find two adjacent scales so that the two scales are on both sides of the straight line.

[0066] Sort the angle values ​​corresponding to all scales in ascending order. After sorting, the position coordinates of all scales are as follows: The position coordinates of scale 1 (x 1st ,y 1st ), the position coordinate of scale 2 (x 2st ,y 2st )......The position coordinate of scale n (x nst ,y nst ). The straight line between the pointer and the center point l n The equation is ax+by+c=0, so it is easy to get a=y h -y c , b = x c -x h , c = x h ×y c -x c ×y h. Traverse all scales in order from small to large, that is, first select scale 1 and scale 2, then select scale 2 and scale 3, ... and finally select scale n-1 and scale n. The currently selected scale is scale u (x ust ,y ust ) and the scale v(x vst ,y vst ), the equation of the straight line formed by the two scales is o1x+p1y+q1=0, and it is easy to get o1=y vst -y ust , p1=x ust -x vst ,q1=x vst ×y ust -x ust ×y vst The intersection of the straight line ax+by+c=0 and the straight line o1x+p1y+q1=0 is (x in1 ,y in1 ), easy to obtain Determine whether the intersection point position coordinates, scale u position coordinates, and scale v position coordinates satisfy x 1st <x in1 <x 2st or x 1st >x in1 >x 2st If the condition is not met, continue to traverse. If the condition is met, stop traversing and select scale u and scale v to calculate the reading.

[0067] However, in actual measurement, there is a situation where the pointer is between the scales u, v, and w, and the v scale is not recognized because the pointer partially covers the v scale. In this case, the u and w scales should be used for subsequent calculations. How to determine whether the v scale in the image is not recognized can be determined based on the angle values ​​of each scale in the disk information and the angle measured between the current scales.

[0068] In step S404, a reading is calculated based on the distance between the second connecting line and two adjacent scale positions.

[0069] Scale u(x ust ,y ust ) and the straight line l between the center point O u The equation is o u x+p u y+q u =0, scale v(x vst ,y vst ) and the straight line l between the center point O v The equation is o v x+p v y+q v =0, easy to get o u=y c -y ust , p u =x ust -x c ,q u =x c ×y ust -x ust ×y c , o v =y c -y vst , p v =x vst -x c ,q v =x c ×y vst -x vst ×y c Straight line l u With straight line l n The angle is θ u , straight line l v With straight line l n The angle is θ v , easy to get The initial scale value of the dial is w0, and the absolute value of the difference between adjacent scales is w1. The current dial reading is

[0070] In some embodiments, the disk information includes at least the number of scales and the actual value of each scale, the identification mode of the pointer, and the angle value of each scale in the target instrument panel when it is placed upright.

[0071] The dashboard information is necessary information for the dashboard. The data value and format may be different for different dashboards. It should at least include:

[0072] (1) The number of scales and the actual value of each scale;

[0073] (2) Pointer recognition mode; (involving the calculation of the pointer position based on the contour, including average mode, farthest distance mode, etc.);

[0074] (3) The angular value of each scale on the instrument panel when it is placed upright; this facilitates the one-to-one correspondence between the scale outline and the actual scale in some complex dials.

[0075] In some embodiments, the disk information further includes interval information, wherein the interval information represents state information corresponding to the reading interval under specific requirements, and the method further includes:

[0076] Determining a state interval corresponding to the reading based on the reading of the target instrument panel;

[0077] The state information corresponding to the reading is fed back based on the state interval.

[0078] Here, the disk information may also include interval information such as a normal pointer interval, an abnormal pointer interval, and a faulty pointer interval, so as to facilitate feedback of the category corresponding to the reading after reading.

[0079] As an example, there can be 11 scales, namely {-0.1, 0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9}. The pointer is not connected to the center of the dial in the screen, and the average mode can be used; the actual measurement (or obtained through the instruction manual) of the angle value of each scale; normal interval [0.58, 0.9], abnormal interval [0.55, 0.58), fault interval [-0.1, 0.55).

[0080] In summary, the embodiments of the present application have the following beneficial effects:

[0081] (1) The embodiments of the present application can be widely used in multiple fields such as electricity, machinery, and chemical industry. Through the efficient remote monitoring function, the dial readings can be grasped in real time regardless of the distance, providing timely and accurate information support for decision-making.

[0082] (2) The embodiments of the present application can realize automatic inspection without manual inspection one by one, which greatly saves manpower and time costs, improves work efficiency, and reduces the risk of human error, bringing greater economic benefits to enterprises and individuals.

[0083] (3) The embodiments of the present application can read a large number of dial readings at the same time, thereby extracting valuable information therefrom, providing strong support for decision-making, helping enterprises or individuals make more informed decisions, and promoting business development and innovation.

[0084] (4) The embodiments of the present application can timely discover potential problems and risks, improve safety, and avoid safety accidents caused by human negligence or errors. At the same time, its advanced technical concepts and innovative design ideas are favored by the market and users, and have broad market application prospects and huge commercial value.

[0085] Based on the same inventive concept, the embodiment of the present application also provides a pointer-type instrument dial reading recognition device corresponding to the pointer-type instrument dial reading recognition method in the first embodiment. Since the principle of solving the problem by the device in the embodiment of the present application is similar to that of the above-mentioned pointer-type instrument dial reading recognition method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0086] like Figure 5 As shown, Figure 5: is a schematic diagram of the structure of a pointer-type instrument panel reading recognition device 500 provided in an embodiment of the present application. The pointer-type instrument panel reading recognition device 500 comprises:

[0087] A determination module 501 is used to determine the panel information of the target instrument panel;

[0088] A first construction module 502 is used to obtain a first dashboard image dataset of the target dashboard, construct a first dashboard detection model for determining the position of the target dashboard in the image based on the first dashboard image dataset, and perform cropping processing on the first dashboard image dataset based on the first dashboard detection model to obtain a second dashboard image dataset;

[0089] A second construction module 503 is used to construct a second instrument panel detection model for determining the scale and pointer positions in the target instrument panel based on the second instrument panel image data set, and determine the scale outline coordinate data and the pointer outline coordinate data in the second instrument panel image data set by using the second instrument panel detection model;

[0090] The calculation module 504 is used to calculate the reading of the target instrument panel based on the scale outline coordinate data, the pointer outline coordinate data and the panel surface information.

[0091] Those skilled in the art should understand that Figure 5 The functions implemented by each unit in the pointer instrument panel reading recognition device 500 shown can be understood by referring to the relevant description of the aforementioned pointer instrument panel reading recognition method. Figure 5 The functions of each unit in the pointer-type instrument panel reading recognition device 500 shown can be implemented by a program running on a processor, or by a specific logic circuit.

[0092] In a possible implementation manner, the first dashboard image data set is obtained by collecting the target dashboard in multiple directions, angles, and distances through an image acquisition device, and the first construction module 502 constructs a first dashboard detection model for determining the position of the target dashboard in the image based on the first dashboard image data set, including:

[0093] Performing a first annotation process on each first instrument image in the first instrument image data set to obtain a first instrument panel training data set; wherein the first annotation process is used to determine a bounding box where the target instrument panel is located in each first instrument image;

[0094] The first instrument panel training data set is trained to obtain the first instrument panel detection model.

[0095] In a possible implementation manner, the first construction module 502 performs a first annotation process on each first instrument image in the first instrument image dataset to obtain a first instrument panel training dataset, including:

[0096] Assume that the width of each first instrument image is w and the height is h;

[0097] A bounding box is drawn in each first instrument image where the target instrument panel is located, and first coordinate information of the bounding box is recorded; wherein the first coordinate information includes the horizontal and vertical coordinates (x a ,y a ), the width w of the bounding box a , the height h of the bounding box a , let the coordinates of the upper left corner of the bounding box be (x1, y1), and the coordinates of the lower right corner of the bounding box be (x2, y2), then

[0098] In a possible implementation manner, the first construction module 502 performs cropping processing on the first dashboard image dataset based on the first dashboard detection model to obtain a second dashboard image dataset, including:

[0099] Each of the first instrument images is cropped based on the first coordinate information of the bounding box, image content outside the bounding box is deleted, and the instrument image within the bounding box is retained as the second instrument panel image data set.

[0100] In a possible implementation, the second construction module 503 constructs a second instrument panel detection model for determining the scale and pointer positions in the target instrument panel based on the second instrument panel image data set, including:

[0101] Performing a second annotation process on each second instrument image in the second instrument image data set to obtain a second instrument panel training data set; wherein the second annotation process is used to determine a scale outline and a pointer outline in each second instrument image;

[0102] The second instrument panel training data set is trained to obtain the second instrument panel detection model.

[0103] In a possible implementation manner, the second construction module 503 performs a second annotation process on each second instrument image in the second instrument image data set to obtain a second instrument panel training data set, including:

[0104] Assume that the width of each second instrument image is w d2 , height is h d2 ;

[0105] Describe the scale outline and the pointer outline, and record the second coordinate information of the scale outline and the pointer outline; wherein the scale outline or the pointer outline is a polygon, and the normalized coordinate of any vertex of the polygon is (x A′ ,y A′ ), let the coordinates of any vertex of the polygon be (x A ,y A ),but

[0106] In a possible implementation, the calculation module 504 calculates the reading of the target instrument panel based on the scale outline coordinate data, the pointer outline coordinate data and the panel surface information, including:

[0107] Calculate the scale position and the pointer position respectively based on the scale outline coordinate data and the pointer outline coordinate data;

[0108] Calculate the slope and angle of the first line connecting the scale position and the center point of the target instrument panel; wherein the center point of the target instrument panel is the coordinate origin, the horizontal axis direction is rightward, the vertical axis direction is downward, and the positive direction of the vertical axis is 0 degrees;

[0109] Determine a second connecting line between the pointer position and the center point of the target instrument panel, and determine two scale positions adjacent to the second connecting line from the scale positions based on an angle of the second connecting line;

[0110] The reading is calculated based on the distance between the second connecting line and two adjacent scale positions.

[0111] In a possible implementation manner, the panel information includes at least the number of scales and the actual value of each scale, a pointer recognition mode, and an angle value of each scale in the upright target instrument panel.

[0112] In a possible implementation manner, the disk information further includes interval information, where the interval information represents state information corresponding to a reading interval under specific requirements, and the calculation module 504 further includes:

[0113] Determining a state interval corresponding to the reading based on the reading of the target instrument panel;

[0114] The state information corresponding to the reading is fed back based on the state interval.

[0115] The above-mentioned pointer-type instrument panel reading recognition device has the following beneficial effects:

[0116] (1) The embodiments of the present application can be widely used in multiple fields such as electricity, machinery, and chemical industry. Through the efficient remote monitoring function, the dial readings can be grasped in real time regardless of the distance, providing timely and accurate information support for decision-making.

[0117] (2) The embodiments of the present application can realize automatic inspection without manual inspection one by one, which greatly saves manpower and time costs, improves work efficiency, and reduces the risk of human error, bringing greater economic benefits to enterprises and individuals.

[0118] (3) The embodiments of the present application can read a large number of dial readings at the same time, thereby extracting valuable information therefrom, providing strong support for decision-making, helping enterprises or individuals make more informed decisions, and promoting business development and innovation.

[0119] (4) The embodiments of the present application can timely discover potential problems and risks, improve safety, and avoid safety accidents caused by human negligence or errors. At the same time, its advanced technical concepts and innovative design ideas are favored by the market and users, and have broad market application prospects and huge commercial value.

[0120] like Figure 6 As shown, Figure 6 The present invention provides a schematic diagram of the structure of an electronic device 600, wherein the electronic device 600 includes:

[0121] A processor 601, a storage medium 602 and a bus 603, wherein the storage medium 602 stores machine-readable instructions executable by the processor 601. When the electronic device 600 is running, the processor 601 communicates with the storage medium 602 via the bus 603, and the processor 601 executes the machine-readable instructions to perform the steps of the pointer instrument panel reading recognition method described in the embodiment of the present application.

[0122] In actual application, the components in the electronic device 600 are coupled together via the bus 603. It is understood that the bus 603 is used to realize the connection and communication between these components. In addition to the data bus, the bus 603 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 6 Various buses are labeled as bus 603.

[0123] The above electronic device has the following beneficial effects:

[0124] (1) The embodiments of the present application can be widely used in multiple fields such as electricity, machinery, and chemical industry. Through the efficient remote monitoring function, the dial readings can be grasped in real time regardless of the distance, providing timely and accurate information support for decision-making.

[0125] (2) The embodiments of the present application can realize automatic inspection without manual inspection one by one, which greatly saves manpower and time costs, improves work efficiency, and reduces the risk of human error, bringing greater economic benefits to enterprises and individuals.

[0126] (3) The embodiments of the present application can read a large number of dial readings at the same time, thereby extracting valuable information therefrom, providing strong support for decision-making, helping enterprises or individuals make more informed decisions, and promoting business development and innovation.

[0127] (4) The embodiments of the present application can timely discover potential problems and risks, improve safety, and avoid safety accidents caused by human negligence or errors. At the same time, its advanced technical concepts and innovative design ideas are favored by the market and users, and have broad market application prospects and huge commercial value.

[0128] The embodiment of the present application also provides a computer-readable storage medium, which stores executable instructions. When the executable instructions are executed by at least one processor 601, the pointer instrument panel reading recognition method described in the embodiment of the present application is implemented.

[0129] In some embodiments, the storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface storage, an optical disk, or a compact disc read-only memory (CD ROM); it can also be various devices including one or any combination of the above memories.

[0130] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0131] As an example, executable instructions may, but do not necessarily, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (for example, files storing one or more modules, subroutines, or code portions).

[0132] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0133] The computer-readable storage medium has the following advantages:

[0134] (1) The embodiments of the present application can be widely used in multiple fields such as electricity, machinery, and chemical industry. Through the efficient remote monitoring function, the dial readings can be grasped in real time regardless of the distance, providing timely and accurate information support for decision-making.

[0135] (2) The embodiments of the present application can realize automatic inspection without manual inspection one by one, which greatly saves manpower and time costs, improves work efficiency, and reduces the risk of human error, bringing greater economic benefits to enterprises and individuals.

[0136] (3) The embodiments of the present application can read a large number of dial readings at the same time, thereby extracting valuable information therefrom, providing strong support for decision-making, helping enterprises or individuals make more informed decisions, and promoting business development and innovation.

[0137] (4) The embodiments of the present application can timely discover potential problems and risks, improve safety, and avoid safety accidents caused by human negligence or errors. At the same time, its advanced technical concepts and innovative design ideas are favored by the market and users, and have broad market application prospects and huge commercial value.

[0138] In the several embodiments provided in the present application, it should be understood that the disclosed methods and electronic devices can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0139] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0140] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0141] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a platform server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0142] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for identifying readings of a pointer-type instrument panel, characterized in that: The method comprises: Determine the panel information of the target dashboard; Acquire a first instrument panel image dataset of the target instrument panel, construct a first instrument panel detection model for determining a position of the target instrument panel in an image based on the first instrument panel image dataset, and perform cropping processing on the first instrument panel image dataset based on the first instrument panel detection model to obtain a second instrument panel image dataset; Building a second instrument panel detection model for determining the scale and pointer positions in the target instrument panel based on the second instrument panel image data set, and determining the scale outline coordinate data and the pointer outline coordinate data in the second instrument panel image data set by using the second instrument panel detection model; The reading of the target instrument panel is calculated based on the scale outline coordinate data, the pointer outline coordinate data and the panel surface information.

2. The method according to claim 1, characterized in that The first instrument panel image data set is obtained by collecting the target instrument panel in multiple directions, angles, and distances through an image acquisition device. The first instrument panel detection model for determining the position of the target instrument panel in the image is constructed based on the first instrument panel image data set, including: Performing a first annotation process on each first instrument image in the first instrument image data set to obtain a first instrument panel training data set; wherein the first annotation process is used to determine a bounding box where the target instrument panel is located in each first instrument image; The first instrument panel training data set is trained to obtain the first instrument panel detection model.

3. The method according to claim 2, characterized in that The step of performing a first annotation process on each first instrument image in the first instrument image data set to obtain a first instrument panel training data set includes: Assume that the width of each first instrument image is w and the height is h; A bounding box is drawn in each first instrument image where the target instrument panel is located, and first coordinate information of the bounding box is recorded; wherein the first coordinate information includes the horizontal and vertical coordinates (x a ,y a ), the width w of the bounding box a , the height h of the bounding box a , let the coordinates of the upper left corner of the bounding box be (x1, y1), and the coordinates of the lower right corner of the bounding box be (x2, y2), then 4. The method according to claim 3, characterized in that The step of performing cropping processing on the first instrument panel image dataset based on the first instrument panel detection model to obtain a second instrument panel image dataset includes: Each of the first instrument images is cropped based on the first coordinate information of the bounding box, image content outside the bounding box is deleted, and the instrument image within the bounding box is retained as the second instrument panel image data set.

5. The method according to claim 1, characterized in that: The constructing a second instrument panel detection model for determining the scale and pointer positions in the target instrument panel based on the second instrument panel image data set includes: Performing a second annotation process on each second instrument image in the second instrument image data set to obtain a second instrument panel training data set; wherein the second annotation process is used to determine a scale outline and a pointer outline in each second instrument image; The second instrument panel training data set is trained to obtain the second instrument panel detection model.

6. The method according to claim 5, characterized in that The performing a second annotation process on each second instrument image in the second instrument image data set to obtain a second instrument panel training data set includes: Assume that the width of each second instrument image is w d2 , height is h d2 ; Describe the scale outline and the pointer outline, and record the second coordinate information of the scale outline and the pointer outline; wherein the scale outline or the pointer outline is a polygon, and the normalized coordinate of any vertex of the polygon is (x A′ ,y A′ ), let the coordinates of any vertex of the polygon be (x A ,y A ),but 7. The method according to claim 1, characterized in that The calculating the reading of the target instrument panel based on the scale outline coordinate data, the pointer outline coordinate data and the panel surface information includes: Calculate the scale position and the pointer position respectively based on the scale outline coordinate data and the pointer outline coordinate data; Calculate the slope and angle of the first line connecting the scale position and the center point of the target instrument panel; wherein the center point of the target instrument panel is the coordinate origin, the horizontal axis direction is rightward, the vertical axis direction is downward, and the positive direction of the vertical axis is 0 degrees; Determine a second connecting line between the pointer position and the center point of the target instrument panel, and determine two scale positions adjacent to the second connecting line from the scale positions based on an angle of the second connecting line; The reading is calculated based on the distance between the second connecting line and two adjacent scale positions.

8. The method according to claim 1, characterized in that The panel information at least includes the number of scales and the actual value of each scale, the identification mode of the pointer, and the angle value of each scale in the upright target instrument panel.

9. The method according to claim 1, characterized in that: The disk information also includes interval information, and the interval information represents the state information corresponding to the reading interval under specific requirements. The method also includes: Determining a state interval corresponding to the reading based on the reading of the target instrument panel; The state information corresponding to the reading is fed back based on the state interval.

10. A pointer-type instrument panel reading recognition device, characterized in that: The device comprises: A determination module, used to determine the panel information of the target instrument panel; A first construction module is used to obtain a first dashboard image dataset of the target dashboard, construct a first dashboard detection model for determining the position of the target dashboard in the image based on the first dashboard image dataset, and perform cropping processing on the first dashboard image dataset based on the first dashboard detection model to obtain a second dashboard image dataset; A second construction module is used to construct a second instrument panel detection model for determining the scale and pointer positions in the target instrument panel based on the second instrument panel image data set, and determine the scale outline coordinate data and the pointer outline coordinate data in the second instrument panel image data set by using the second instrument panel detection model; A calculation module is used to calculate the reading of the target instrument panel based on the scale outline coordinate data, the pointer outline coordinate data and the panel surface information.

Images