HRNet-based pointer instrument identification method, device, and medium

Abstract

This invention discloses a method, device, and medium for identifying pointer-type instruments based on HRNet, relating to the field of intelligent rail transit technology. The method includes: training an instrument recognition model using an HRNet network based on an instrument dataset; acquiring real-time images of the instruments; cropping the real-time images according to a preset region of interest; and inputting the cropped images into the instrument recognition model to obtain instrument values. The method provided by this invention solves the problem that existing surveillance videos lack automatic analysis, cannot promptly identify important information contained in the videos, and cannot detect abnormal readings of pointer-type instruments in a timely manner, posing significant safety hazards.

Description

Technical Field

[0001] This invention relates to the field of intelligent technology for rail transit, and more specifically, to a pointer-type instrument identification method, device, and medium based on HRNet. Background Technology

[0002] The pump rooms and other facilities in the high-speed railway station area provide flood control, drainage, fire-fighting water, and production and domestic water services for the lines, stations, and production units. They are directly related to the safety of the lines, the safety of train operation, and the quality of passenger services at the stations, and bear great responsibility.

[0003] The current water supply and drainage maintenance of high-speed railways relies primarily on extensive manual labor, with a large workforce for manual operation of pump stations, regular maintenance, and line patrols. Emergency repairs are required in case of unforeseen incidents, necessitating the deployment of personnel at every station to manually monitor equipment operation, resulting in a significant workload. While existing pump stations are generally equipped with high-definition cameras to monitor the operating environment, the massive amounts of real-time video footage lack automated analysis. This prevents timely identification of critical information within the videos and fails to detect abnormal readings on pointer-type instruments, posing a significant safety hazard. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a pointer instrument identification method, device, and medium based on HRNet, which solves the problems of existing surveillance videos lacking automatic analysis, failing to identify important information contained in the video in a timely manner, failing to detect abnormal readings of pointer instruments in a timely manner, and posing significant security risks.

[0005] In a first aspect, the present invention provides a pointer-type instrument identification method based on HRNet, the method comprising:

[0006] Based on the instrument dataset, an instrument recognition model was obtained by training the HRNet network.

[0007] A real-time image of the instrument is acquired, and after cropping the real-time image according to a preset region of interest, it is input into the instrument recognition model to obtain the instrument value.

[0008] Furthermore, the instrument dataset includes an instrument image and corresponding pointer scale labels; the pointer scale labels include the pointer base point coordinates, the pointer tail point coordinates, and the coordinates from the start scale to the end scale.

[0009] Furthermore, based on the meter dataset, an meter recognition model is trained using the HRNet network, including:

[0010] The instrument images and corresponding pointer scale labels in the instrument dataset are imported into the HRNet network for training to obtain the instrument recognition model.

[0011] Furthermore, the HRNet network includes multiple convolutional layers; wherein the outputs of at least two convolutional layers are downsampled using a max-pooling layer to halve the feature map size.

[0012] Furthermore, the HRNet network comprises 20 convolutional layers; wherein:

[0013] The first and second layers are convolutional layers with a kernel size of 3×3, a stride of 2, and an output channel size of 64.

[0014] The third layer is a maximum pooling layer with a kernel size of 2×2 and a step size of 2;

[0015] Layers 4 through 7 each have two convolutional layers with a kernel size of 3×3, a stride of 2, and an output channel size of 64. Each convolutional layer is followed by normalization and activation.

[0016] Layers 8 to 11 each have two convolutional layers with a kernel size of 3×3, a stride of 2, and an output channel size of 128.

[0017] Layers 12 to 15 each have two convolutional layers with a kernel size of 3×3, a stride of 2, and an output channel size of 256.

[0018] Layers 16 to 19 each have two convolutional layers with a kernel size of 3×3, a stride of 2, and an output channel size of 512.

[0019] The results of the 11th, 15th, and 19th layers are downsampled using a 2×2 max pooling layer to halve the feature map size; the output of each stage is fused with the output of the previous stage.

[0020] Layer 20 is a fully connected layer with a 1×1 core and 3×c output channels, where c represents the number of keypoints.

[0021] Further, a real-time image of the instrument is acquired, and after cropping the real-time image according to a preset region of interest, it is input into the instrument recognition model to obtain the instrument value, including:

[0022] Real-time images are acquired, images are cropped according to preset regions of interest, and then fed into the instrument recognition model;

[0023] Obtain the output of the instrument recognition model to get n vectors; where each vector represents the detection result of an instrument panel; each vector is composed of [x -1 ,y -1 ,x0,y0,x1,y1,x2,y2,…,x m ,y m], representing the x and y coordinates of the pointer base point, the pointer tail point, and the x and y coordinates of the first to m-th large scales, respectively; the pointer base point R and tail point T are extracted, with coordinates [x, y, y], ... -1 ,y -1 [x0, y0]; extract the scale coordinates P. i The coordinates are [x i ,y i ], where i = 1, 2, ..., m, and m is the number of scale divisions;

[0024] Calculate the tick index k such that arg(RP) k ) < arg(RT) < arg(RP) k+1 ); RPk, RT, and RPk+1 are the vectors from the pointer base point to the kth scale, the pointer tail point, and the (k+1)th scale, respectively, and arg represents the angle between the vector and the vertical axis; calculate the scale value a based on the angle between the vector corresponding to the scale index k and the vertical axis; calculate the instrument value based on the scale value a and the preset range L.

[0025] Furthermore, based on the angle between the vector corresponding to the scale index k and the vertical axis, the scale value a is calculated using the following formula:

[0026]

[0027] Furthermore, based on the proportional value 'a' and the preset range 'L', the instrument value is calculated using the following formula:

[0028]

[0029] Secondly, the present invention provides a pointer-type instrument identification device based on HRNet, the device comprising:

[0030] The model training unit is configured to train an instrument recognition model using the HRNet network based on the instrument dataset.

[0031] The numerical recognition unit is configured to acquire a real-time image of the instrument, crop the real-time image of the instrument according to a preset region of interest, and then input it into the instrument recognition model to obtain the instrument value.

[0032] Thirdly, the present invention provides a readable storage medium storing one or more programs that can be executed by one or more processors to implement the method described above.

[0033] The present invention has at least the following beneficial effects:

[0034] This invention can identify the readings of pointer instruments in the instrument images captured by high-definition cameras in real time, enabling timely acquisition of pointer instrument readings. Based on a set threshold, once the threshold is exceeded, an alarm message is immediately reported, thereby improving security. Attached Figure Description

[0035] Figure 1 A schematic diagram of an application scenario based on existing technology is shown.

[0036] Figure 2 A flowchart of a pointer-type meter identification method based on HRNet according to an embodiment of the present invention is shown.

[0037] Figure 3 The output result diagram of the instrument recognition model according to an embodiment of the present invention is shown.

[0038] Figure 4 A structural diagram of a pointer-type meter identification device based on HRNet according to an embodiment of the present invention is shown. Detailed Implementation

[0039] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and specific examples, but this is not intended to limit the present invention. If there is no necessary sequential relationship between the various steps described herein, the order in which they are described as examples should not be considered a limitation. Those skilled in the art should understand that the order can be adjusted, as long as it does not disrupt the logical consistency between them and render the entire process impossible.

[0040] Figure 1 A schematic diagram of an application scenario based on existing technology is shown. For example... Figure 1 As shown, this application scenario includes a camera component 101 and a data processing terminal 102. The camera component 101 can be an existing high-definition camera, which is deployed in the pump room of the high-speed railway to monitor the operating environment. The video data stream acquired by the camera is fed to the data processing terminal 102 in real time. The data processing terminal 102 is equipped with an instrument recognition model. Based on the video data stream fed by the camera component 101 in real time, the data processing terminal 102 extracts the real-time image of the instrument, crops the real-time image of the instrument according to the preset region of interest, and inputs it into the instrument recognition model to obtain the instrument value. In this way, the real-time reading of the pointer instrument can be performed. Once the threshold is exceeded, the alarm information is reported immediately, so as to identify important information contained in the video in time and detect abnormal readings of the pointer instrument in time, eliminating potential major safety hazards.

[0041] This invention provides a pointer-type meter recognition method based on HRNet. The main purpose of this method is to construct a meter recognition model, and the constructed recognition model can be configured in, for example... Figure 1 In the data processing terminal 102 of the application scenario shown, the data processing terminal 102 identifies the real-time image of the instrument based on the instrument recognition model and obtains the instrument value.

[0042] Specifically, such as Figure 2 The diagram shows a flowchart of a pointer-type meter recognition method based on HRNet. This HRNet-based pointer-type meter recognition method includes steps S10 to S20, which are described in detail below.

[0043] S10: Based on the instrument dataset, an instrument recognition model is obtained by training the HRNet network.

[0044] In some embodiments, the instrument dataset includes an instrument image and corresponding pointer scale labels; the pointer scale labels include pointer base point coordinates, pointer tail point coordinates, and starting scale coordinates to ending scale coordinates.

[0045] In some embodiments, the meter recognition model can be constructed using the following method:

[0046] The instrument dataset includes pointer scale labels, which contain the coordinates of the pointer's base point, tail point, start scale, and end scale. The dataset labels are imported into an HRNet network for training to obtain an instrument recognition model. The input image size of the HRNet network is 3×256×192, and its specific structure includes:

[0047] The first and second layers are convolutional layers with a kernel size of 3×3, a stride of 2, and an output channel size of 64.

[0048] The third layer is a maximum pooling layer with a kernel size of 2×2 and a step size of 2;

[0049] Layers 4 through 7 each have two convolutional layers with a kernel size of 3×3, a stride of 2, and an output channel size of 64. Each convolutional layer is followed by normalization and activation.

[0050] Layers 8 to 11 each have two convolutional layers with a kernel size of 3×3, a stride of 2, and an output channel size of 128.

[0051] Layers 12 to 15 each have two convolutional layers with a kernel size of 3×3, a stride of 2, and an output channel size of 256.

[0052] Layers 16 to 19 each have two convolutional layers with a kernel size of 3×3, a stride of 2, and an output channel size of 512.

[0053] The results of the 11th, 15th, and 19th layers are downsampled using a 2×2 max pooling layer to halve the feature map size; the output of each stage is fused with the output of the previous stage.

[0054] Layer 20 is a fully connected layer with a 1×1 core and 3×c output channels, where c represents the number of keypoints.

[0055] S20: Acquire a real-time image of the instrument, crop the real-time image of the instrument according to a preset region of interest, and input it into the instrument recognition model to obtain the instrument value.

[0056] In some embodiments, step S20 may be implemented by the following steps:

[0057] S21: Acquire real-time images, crop the images according to preset regions of interest, and send them into the instrument recognition model.

[0058] S22: Obtain the output of the instrument recognition model to get n vectors; where each vector represents the detection result of an instrument panel; each vector is composed of [x -1 ,y -1 ,x0,y0,x1,y1,x2,y2,…,x m ,y m ], representing the x and y coordinates of the pointer base point, the pointer tail point, and the x and y coordinates of the first to m-th large scales, respectively; the pointer base point R and tail point T are extracted, with coordinates [x, y, y], ... -1 ,y -1 [x0, y0]; extract the scale coordinates P. i The coordinates are [x i ,y i ], where i = 1, 2, ..., m, and m is the number of scale divisions.

[0059] S23: Calculate the tick index k, satisfying arg(RP) k ) < arg(RT) < arg(RP) k+1 ); RPk, RT, RP k+1 The vectors are the pointer base point to the kth scale, the pointer tail point, and the (k+1)th scale, respectively, and arg represents the angle between the vector and the vertical axis. The scale value a is calculated based on the angle between the vector corresponding to the scale index k and the vertical axis. The instrument value is calculated based on the scale value a and the preset range L.

[0060] In this embodiment, the scale value a is calculated using the following formula based on the angle between the vector corresponding to the scale index k and the vertical axis:

[0061]

[0062] The instrument value is calculated using the following formula based on the proportional value 'a' and the preset range 'L':

[0063]

[0064] To further illustrate the feasibility of the present invention, a specific example is provided to describe the pointer-type instrument recognition method based on HRNet.

[0065] Specifically, the instrument dataset mentioned in step S10 is a pointer scale label, which includes the pointer base point coordinates, pointer tail point coordinates, starting scale coordinates to ending scale coordinates, a total of 7 coordinates, each coordinate including a horizontal coordinate value and a vertical coordinate value; the HRNet network structure includes a 20th layer with 3×14 output channels.

[0066] Specifically, step S21 uses the TensorRT graphics card inference computing framework to perform model inference computing. The output of the instrument recognition model is as follows: Figure 3 As shown.

[0067] Specifically, the instrument recognition model in step S22 outputs a vector; the pointer base point coordinates are [296,263], the pointer tail point coordinates are [174,305], and the five scale coordinates are [161,312], [207,168], [342,145], [413,262], and [363,356].

[0068] Specifically, in step S23, the scale index k is 1; the calculated proportional value a is 0.027; the preset range is 1.5, and the instrument value v = 1.5 / (5-1)×(1-1+0.027) = 0.01 is obtained.

[0069] This invention also provides a pointer-type instrument recognition device based on HRNet, such as... Figure 4 As shown, the device includes:

[0070] Model training unit 401 is configured to train an instrument recognition model based on the instrument dataset using the HRNet network.

[0071] The numerical recognition unit 402 is configured to acquire a real-time image of the instrument, crop the real-time image of the instrument according to a preset region of interest, and input it into the instrument recognition model to obtain the instrument value.

[0072] In some embodiments, the instrument dataset includes an instrument image and corresponding pointer scale labels; the pointer scale labels include pointer base point coordinates, pointer tail point coordinates, and starting scale coordinates to ending scale coordinates.

[0073] In some embodiments, the model training unit is further configured to:

[0074] The instrument images and corresponding pointer scale labels in the instrument dataset are imported into the HRNet network for training to obtain the instrument recognition model.

[0075] In some embodiments, the HRNet network includes multiple convolutional layers; wherein the outputs of at least two convolutional layers are downsampled using a max-pooling layer to halve the feature map size.

[0076] In some embodiments, the HRNet network includes 20 convolutional layers; wherein:

[0077] The first and second layers are convolutional layers with a kernel size of 3×3, a stride of 2, and an output channel size of 64.

[0078] The third layer is a maximum pooling layer with a kernel size of 2×2 and a step size of 2;

[0079] Layers 4 through 7 each have two convolutional layers with a kernel size of 3×3, a stride of 2, and an output channel size of 64. Each convolutional layer is followed by normalization and activation.

[0080] Layers 8 to 11 each have two convolutional layers with a kernel size of 3×3, a stride of 2, and an output channel size of 128.

[0081] Layers 12 to 15 each have two convolutional layers with a kernel size of 3×3, a stride of 2, and an output channel size of 256.

[0082] Layers 16 to 19 each have two convolutional layers with a kernel size of 3×3, a stride of 2, and an output channel size of 512.

[0083] The results of the 11th, 15th, and 19th layers are downsampled using a 2×2 max pooling layer to halve the feature map size; the output of each stage is fused with the output of the previous stage.

[0084] Layer 20 is a fully connected layer with a 1×1 core and 3×c output channels, where c represents the number of keypoints.

[0085] In some embodiments, the numerical recognition unit is further configured to:

[0086] Real-time images are acquired, images are cropped according to preset regions of interest, and then fed into the instrument recognition model;

[0087] Obtain the output of the instrument recognition model to get n vectors; where each vector represents the detection result of an instrument panel; each vector is composed of [x -1 ,y -1 ,x0,y0,x1,y1,x2,y2,…,x m ,ym ], representing the x and y coordinates of the pointer base point, the pointer tail point, and the x and y coordinates of the first to m-th large scales, respectively; the pointer base point R and tail point T are extracted, with coordinates [x, y, y], ... -1 ,y -1 [x0, y0]; extract the scale coordinates P. i The coordinates are [x i ,y i ], where i = 1, 2, ..., m, and m is the number of scale divisions;

[0088] Calculate the tick index k such that arg(RP) k ) < arg(RT) < arg(RP) k+1 ); RPk, RT, and RPk+1 are the vectors from the pointer base point to the kth scale, the pointer tail point, and the (k+1)th scale, respectively, and arg represents the angle between the vector and the vertical axis; calculate the scale value a based on the angle between the vector corresponding to the scale index k and the vertical axis; calculate the instrument value based on the scale value a and the preset range L.

[0089] In some embodiments, the numerical recognition unit is further configured to calculate the scale value a based on the angle between the vector corresponding to the scale index k and the vertical axis using the following formula:

[0090]

[0091] In some embodiments, the numerical identification unit is further configured to calculate the instrument value based on the proportional value a and the preset range L using the following formula:

[0092]

[0093] It should be noted that the structures of the various HRNet-based pointer instrument recognition devices described in this embodiment belong to the same technical concept as the previously described HRNet-based pointer instrument recognition method, achieving the same beneficial effects through the same principle, and will not be repeated here.

[0094] This invention also provides a readable storage medium storing one or more programs that can be executed by one or more processors to implement the methods described in any of the above embodiments.

[0095] Furthermore, although exemplary embodiments have been described herein, their scope includes any and all embodiments based on the invention that have equivalent elements, modifications, omissions, combinations (e.g., schemes involving intersections of various embodiments), adaptations, or alterations. Elements in the claims will be interpreted broadly based on the language used in the claims and are not limited to the examples described in this specification or during the implementation of this application, and such examples will be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered illustrative only, and the true scope and spirit are indicated by the following claims and the full scope of their equivalents.

[0096] The above description is intended to be illustrative and not restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. Other embodiments can be used by those skilled in the art when reading the above description. Furthermore, in the above detailed description, various features may be grouped together to simplify the invention. This should not be construed as an intention that a feature of an unclaimed invention is necessary for any claim. Rather, the subject matter of the invention may be less than all the features of a particular embodiment of the invention. Thus, the following claims are incorporated herein by reference as examples or embodiments, wherein each claim is an independent, separate embodiment, and these embodiments are contemplated to be combined with each other in various combinations or arrangements. The scope of the invention should be determined by reference to the appended claims and the full scope of their equivalents.

Claims

1. A pointer-type instrument recognition method based on HRNet, characterized in that, The method includes: Based on the instrument dataset, an instrument recognition model was obtained by training the HRNet network. Acquire a real-time image of the instrument, crop the real-time image of the instrument according to a preset region of interest, and input it into the instrument recognition model to obtain the instrument value; Acquire a real-time image of the instrument, crop the image according to a preset region of interest, and input it into the instrument recognition model to obtain instrument values, including: Real-time images are acquired, images are cropped according to preset regions of interest, and then fed into the instrument recognition model; Obtain the output of the instrument recognition model to get n vectors; where each vector represents the detection result of an instrument panel; each vector is composed of [x -1 ,y -1 ,x0,y0,x1,y1,x2,y2,…,x m ,y m ], representing the x and y coordinates of the pointer base point, the pointer tail point, and the x and y coordinates of the first to m-th large scales, respectively; the pointer base point R and tail point T are extracted, with coordinates [x, y, y], ... -1 ,y -1 [x0, y0]; extract the scale coordinates P. i The coordinates are [x i ,y i ], where i = 1, 2, ..., m, and m is the number of scale divisions; Calculate the tick index k such that arg(RP) k ) < arg(RT) < arg(RP) k+1 ); RP k RT, RP k+1 The vectors from the pointer base point to the kth scale, the pointer tail point, and the (k+1)th scale are respectively, and arg represents the angle between the vector and the vertical axis; the scale value a is calculated based on the angle between the vector corresponding to the scale index k and the vertical axis; the instrument value is calculated based on the scale value a and the preset range L. The scale value a is calculated using the following formula, based on the angle between the vector corresponding to the scale index k and the vertical axis:

2. The pointer-type instrument recognition method based on HRNet according to claim 1, characterized in that, The instrument dataset includes instrument images and corresponding pointer scale labels; the pointer scale labels include pointer base point coordinates, pointer tail point coordinates, and starting scale coordinates to ending scale coordinates.

3. The pointer-type instrument recognition method based on HRNet according to claim 2, characterized in that, Based on the meter dataset, a meter recognition model was trained using the HRNet network, including: The instrument images and corresponding pointer scale labels in the instrument dataset are imported into the HRNet network for training to obtain the instrument recognition model.

4. The pointer-type instrument recognition method based on HRNet according to claim 1, characterized in that, The HRNet network includes multiple convolutional layers; wherein the outputs of at least two convolutional layers are downsampled using a max pooling layer to halve the feature map size.

5. The pointer-type instrument recognition method based on HRNet according to claim 4, characterized in that, The HRNet network comprises 20 convolutional layers; wherein: The first and second layers are convolutional layers with a kernel size of 3×3, a stride of 2, and an output channel size of 64. The third layer is a maximum pooling layer with a kernel size of 2×2 and a step size of 2; Layers 4 through 7 each have two convolutional layers with a kernel size of 3×3, a stride of 2, and an output channel size of 64. Each convolutional layer is followed by normalization and activation. Layers 8 to 11 each have two convolutional layers with a kernel size of 3×3, a stride of 2, and an output channel size of 128. Layers 12 to 15 each have two convolutional layers with a kernel size of 3×3, a stride of 2, and an output channel size of 256. Layers 16 to 19 each have two convolutional layers with a kernel size of 3×3, a stride of 2, and an output channel size of 512. The results of the 11th, 15th, and 19th layers are downsampled using a 2×2 max pooling layer to halve the feature map size; the output of each stage is fused with the output of the previous stage. Layer 20 is a fully connected layer with a 1×1 core and 3×c output channels, where c represents the number of keypoints.

6. The pointer-type instrument recognition method based on HRNet according to claim 1, characterized in that, The instrument value is calculated using the following formula based on the proportional value 'a' and the preset range 'L':

7. A pointer-type instrument identification device based on HRNet, used to implement the method as described in any one of claims 1 to 6, characterized in that, The device includes: The model training unit is configured to train an instrument recognition model using the HRNet network based on the instrument dataset. The numerical recognition unit is configured to acquire a real-time image of the instrument, crop the real-time image of the instrument according to a preset region of interest, and then input it into the instrument recognition model to obtain the instrument value.

8. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, perform the method according to any one of claims 1 to 6.

Images