A post - processing method, system, device and medium for obtaining pointer water meter readings

Abstract

The present invention discloses a post - processing method, system, device and medium for obtaining the readings of a pointer water meter. The method includes: acquiring an image of the pointer water meter, detecting each pointer in the pointer water meter image, and cropping to obtain pictures of each pointer, and sorting all the pointer pictures according to the relative position relationship of the pointers; extracting visual features and text features from each pointer picture by using a preset image encoder; performing context modeling on the extracted visual features and text features based on a Transformer encoder to obtain enhanced multi - modal features; and autoregressively outputting the final reading based on the multi - modal features by using a Transformer decoder. The present invention makes full use of the context information between pointers, thereby realizing the correct recognition of the readings at the over - position and being able to correctly recognize the readings of the pointers with parallax. The present invention can be widely applied to the fields of pattern recognition and artificial intelligence technology.

Description

Technical Field

[0001] The present invention relates to the technical field of pattern recognition and artificial intelligence, and particularly to a post-processing method, system, device and medium for obtaining readings of a pointer water meter. Background Art

[0002] Mechanical pointer water meters are widely used in existing water supply systems. Automating the reading of these water meters can greatly improve production efficiency and reduce labor costs. A common approach is to use computer vision technology, taking the image of a pointer water meter as input and outputting the reading result. However, existing methods usually identify the readings of each pointer separately and then directly combine the reading results to obtain the final reading of the water meter. This approach ignores the context information contained between the pointers, resulting in inaccurate final readings in many cases. For example, when the current pointer exactly points to the carry position, it is difficult to determine whether the current pointer needs to carry without the assistance of the lower-order pointer. In addition, parallax may be introduced when the water meter image is captured and imaged. To eliminate the influence of parallax, the assistance of the lower-order pointer is also required. Summary of the Invention

[0003] To at least to some extent solve one of the technical problems existing in the prior art, an object of the present invention is to provide a post-processing method, system, device and medium for obtaining readings of a pointer water meter.

[0004] The technical solution adopted by the present invention is as follows:

[0005] A post-processing method for obtaining readings of a pointer water meter includes the following steps:

[0006] Obtain an image of a pointer water meter, detect each pointer in the pointer water meter image, crop to obtain each pointer picture, and sort all pointer pictures according to the relative position relationship of the pointers;

[0007] Extract visual features and text features from each pointer picture using a preset image encoder;

[0008] Perform context modeling on the extracted visual features and text features based on a Transformer encoder to obtain enhanced multi-modal features; [[ID=2......]]

[0009] Output the final reading autoregressively based on the multi-modal features using a Transformer decoder.

[0010] Further, the step of obtaining an image of a pointer water meter, detecting each pointer in the pointer water meter image, cropping to obtain each pointer picture, and sorting all pointer pictures according to the relative position relationship of the pointers includes:

[0011] Obtain the pointer water meter image, input the obtained pointer water meter image into a preset object detection network for pointer detection, and obtain the detection frames of all pointers;

[0012] Crop and obtain each pointer picture according to the detection frame;

[0013] Sort all the pointer pictures according to the relative position relationship of each pointer detection frame.

[0014] Further, the preset image encoder is implemented by a pre-trained deep neural network, which includes a convolutional feature extraction module and a fully connected classification module. The output of the convolutional feature extraction module is used as visual features, and the probability distribution output by the fully connected classification module is used as text features.

[0015] Further, the deep neural network adopts the ResNet18 network structure.

[0016] Further, the input of the Transformer encoder consists of the visual features, text features, one-dimensional position embeddings, and label embeddings of all pointers; among them, the one-dimensional position embeddings are used to indicate the order information of the pointers, and the label embeddings are used to indicate the modality information;

[0017] The output of the Transformer encoder includes multi-modal features of context information.

[0018] Further, based on the multi-modal features, autoregressively output the final reading based on the Transformer decoder, including:

[0019] Take the obtained enhanced multi-modal features as the input and autoregressively output the final reading; where the autoregressive process starts from the lowest digit of the water meter reading.

[0020] Further, the training processes of the image encoder, Transformer encoder, and Transformer decoder are trained using a manually annotated pointer water meter data set; where the data annotation process includes: pointer water meter image acquisition, single pointer detection frame annotation, single pointer reading annotation, and complete reading annotation.

[0021] Another technical solution adopted by the present invention is:

[0022] A post-processing system for obtaining the reading of a pointer water meter, including:

[0023] A pointer detection module, configured to obtain a pointer water meter image, detect each pointer in the pointer water meter image, crop and obtain each pointer picture, and sort all the pointer pictures according to the relative position relationship of the pointers;

[0024] A feature extraction module, configured to extract visual features and text features from each pointer picture by using a preset image encoder;

[0025] A feature enhancement module, configured to perform context modeling on the extracted visual features and text features based on a Transformer encoder to obtain enhanced multi-modal features;

[0026] A degree output module, configured to autoregressively output a final reading based on the multi-modal features by using a Transformer decoder.

[0027] Another technical solution adopted by the present invention is:

[0028] A post-processing device for obtaining readings of a pointer water meter, including:

[0029] At least one processor;

[0030] At least one memory, configured to store at least one program;

[0031] When the at least one program is executed by the at least one processor, the at least one processor implements the method as described above.

[0032] Another technical solution adopted by the present invention is:

[0033] A computer-readable storage medium, in which a program executable by a processor is stored, and the program executable by the processor is used to execute the method as described above when executed by the processor.

[0034] The beneficial effects of the present invention are as follows: By using the context information between the pointers, the present invention can correctly determine the correct reading when the pointer exactly points to the transition position; furthermore, it can obtain the correct reading by combining the context information when parallax occurs. Description of the Drawings

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following introduces the accompanying drawings related to the technical solutions in the embodiments of the present invention or the prior art. It should be understood that the accompanying drawings introduced below are only for conveniently and clearly presenting some embodiments of the technical solutions in the present invention, and those skilled in the art can also obtain other accompanying drawings based on these drawings without creative efforts.

[0036] Figure 1 It is a comparison diagram of whether to read the final reading of the pointer water meter according to the context information in the embodiment of the present invention;

[0037] Figure 2 It is a schematic diagram of the post-processing method for obtaining readings of a pointer water meter in the embodiment of the present invention;

[0038] Figure 3 It is a schematic structural diagram of an image encoder in an embodiment of the present invention;

[0039] Figure 4 It is a step flowchart of a post - processing method for obtaining the reading of a pointer water meter in an embodiment of the present invention. Detailed implementation manners

[0040] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as a limitation to the present invention. For the step numbers in the following embodiments, they are only set for the convenience of elaboration and explanation, and no limitation is imposed on the order between steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0041] In the description of the present invention, it should be understood that for the orientation description, such as the orientation or positional relationship indicated by up, down, front, back, left, right, etc., is based on the orientation or positional relationship shown in the accompanying drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation to the present invention.

[0042] In the description of the present invention, the meaning of several is one or more, the meaning of multiple is two or more, greater than, less than, exceeding, etc. are understood as not including the present number, above, below, within, etc. are understood as including the present number. If the first and second are described only for the purpose of distinguishing technical features, they should not be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features or implicitly indicating the sequence of the indicated technical features.

[0043] In addition, in the description of the present invention, unless otherwise specified, "multiple" means two or more. "And / or" describes the association relationship of associated objects and indicates that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0044] In the description of the present invention, unless otherwise clearly defined, words such as setting, installation, connection, etc. should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above words in the present invention in combination with the specific content of the technical solution.

[0045] As Figure 4 shown, this embodiment provides a post - processing method for obtaining the reading of a pointer water meter, which specifically includes the following steps:

[0046] A1. Obtain the pointer water meter image, detect each pointer in the pointer water meter image, crop to obtain each pointer picture, and sort all the pointer pictures according to the relative position relationship of the pointers;

[0047] A2. Use a preset image encoder to extract visual features and text features from each pointer picture;

[0048] A3. Perform context modeling on the extracted visual features and text features based on the Transformer encoder to obtain enhanced multi-modal features;

[0049] A4. Based on the multi-modal features, autoregressively output the final reading based on the Transformer decoder.

[0050] The above method will be explained in detail below in combination with specific embodiments and the accompanying drawings.

[0051] As Figure 1 shown, this embodiment provides a post-processing method for obtaining the final reading of a pointer water meter, which is used to obtain the final reading of the pointer water meter. The method specifically includes the following steps:

[0052] S1. Input the pointer water meter image into the YOLOv5 object detection network. The network outputs the detection frame of a single pointer, and the pointer is cropped into a small picture (64*64*3) according to the detection frame. Determine the pointer arrangement order according to the horizontal and vertical coordinate size relationships of the midpoint positions of each detection frame. Taking Figure 2 the shown meter type as an example: Determination of the highest and lowest position pointers: Select the two pointers with the largest vertical coordinates of the center point positions. Among them, the pointer with the smaller horizontal coordinate of the center point position is the highest position pointer, and the larger one is the lowest position pointer. Among the remaining pointers, select the two pointers with the largest vertical coordinates of the center position again. Among them, the pointer with the smaller horizontal coordinate of the center point position is the second highest position pointer, and the larger one is the second lowest position pointer. And so on, all the pointers can be sorted.

[0053] In some optional embodiments, in step S1, YOLOv5 is used to predict the detection frame of a single pointer, and the parameters of the network are trained and optimized using a dataset manually collected and labeled. The specific optimization of the network parameters includes:

[0054] (1) Data acquisition: Collect 4000 pointer water meter images, and mark the detection frames of single pointers using the Labelme annotation tool;

[0055] (2) Network training:

[0056] (2-1) Construct a deep neural network: Use the YOLOv5 object detection model as the network structure.

[0057] (2-2) Training method: The training uses the gradient descent algorithm. By calculating the gradient from the last layer and propagating it layer by layer, all parameters are updated to achieve the purpose of training the network. The loss function during training is the YOLOv5 model loss function.

[0058] (2-3) Setting of training parameters:

[0059] Number of iterations: 100 epochs

[0060] Optimizer: Adam

[0061] Learning rate: 0.0001 (Learning rate update strategy: After every 30 iterations, the learning rate decays to 1 / 10 of the original.)

[0062] Weight decay: 0.0005

[0063] (2-4) Start training the deep neural network with randomly initialized parameters.

[0064] (2-5) Optimization objective: Minimize the YOLOv5 model loss function.

[0065] S2. Extract visual and text features from each pointer map using an image encoder; ResNet18 is used as the image encoder, and the number of output head categories is set to 30. As Figure 3 shown, the image encoder takes the pointer sub-image (64*64*3) obtained in S1 as the input. After passing through the convolutional feature extraction module, a visual feature of 1*1*512 is obtained; the visual feature passes through the fully connected classification module to obtain a probability distribution of 1*1*30, which is used as the text feature.

[0066] In some optional embodiments, ResNet18 is used as the image encoder in step S2, and the parameters of the network are trained and optimized using a dataset manually collected and labeled. The specific optimization of the network parameters includes:

[0067] (1) Data acquisition: 4000 pointer water meter images are collected. After using the Labelme annotation tool to mark the detection boxes of individual pointers and then cropping the individual pointers, all pointer sub-images are divided into 30 categories according to the pointer directions, as shown in Table 1 below:

[0068] Table 1

[0069] Angle between the pointer and the 0 scale line Category [0°,7°) 0 [7°,29°) 1 [29°,36°) 2 [36°,43°) 3 [43°,65°) 4 [65°,72°) 5 … … [353°,360°) 29

[0070] (2) Network training:

[0071] (2-1) Construct a deep neural network: Use ResNet18 as the network structure, and the number of categories in the classification layer is set to 30.

[0072] (2-2) Training method: The gradient descent algorithm is used for training. By calculating the gradients from the last layer and propagating them layer by layer, all parameters are updated to achieve the purpose of training the network. The loss function during training is the cross-entropy loss.

[0073] (2-3) Setting of training parameters:

[0074] Number of iterations: 100 epochs

[0075] Optimizer: Adam

[0076] Learning rate: 0.0001

[0077] Weight decay: 0.0005

[0078] (2-4) Start training the deep neural network with randomly initialized parameters.

[0079] (2-5) Optimization objective: Minimize the cross-entropy loss.

[0080] Step S3, Transformer encoder: The visual and text features of all pointers, one-dimensional position embeddings, and label embeddings constitute its input: Denote the pointer as p i , i ∈ {1, 2, …, N} (N is the total number of pointers, for the lowest-bit pointer, i = 1), and the visual and text features obtained in step S2 corresponding to it are denoted as and The dimension-unified visual and text embeddings are obtained through a linear mapping layer:

[0081] The one-dimensional position embedding is used to indicate the order information of the pointers, and the original Sinusoidal position encoding is adopted: PosEmb1D(i).

[0082] The label embedding is used to indicate the modality information: SegEmb(V), SegEmb(T). It is a learnable embedding layer.

[0083] The final input token is obtained by adding the visual and text embeddings, one-dimensional position embedding, and label embedding:

[0084] Proj v (v i ) + PosEmb1D(i) + SegEmb(V)

[0085] Proj t (t i ) + PosEmb1D(i) + SegEmb(T)

[0086] After splicing visual and text tokens together, it forms the input of the Transformer encoder. After being modeled by the multi-head attention mechanism, enhanced features are obtained.

[0087] Step S4: Taking the enhanced features obtained in step S3 as the input, autoregressively output the final reading, which is output from the low bit to the high bit of the pointer.

[0088] In some optional embodiments, for the Transformer encoder and decoder used in step S3, the network parameters are trained and optimized using a dataset manually collected and labeled. The specific network parameter optimization includes:

[0089] (1) Data acquisition: 4000 pointer water meter images are collected. After using the Labelme annotation tool to mark the detection frames of individual pointers and then cutting out the individual pointers, the complete readings of each pointer water meter are labeled.

[0090] (2) Network training:

[0091] (2-1) Constructing a deep neural network: Using standard Transformer encoder and decoder as the network structure, the number of channels in the middle layer is set to 512, the number of encoder layers is set to 6, and the number of decoder layers is set to 6. The image encoder uses the trained ResNet18 in step S2, and its parameters are frozen.

[0092] (2-2) Training method: The gradient descent algorithm is used for training. By calculating the gradient from the last layer and propagating layer by layer, all parameters are updated to achieve the purpose of training the network. The loss function during training is the cross-entropy loss

[0093] (2-3) Setting of training parameters:

[0094] Number of iterations: 500 epochs

[0095] Optimizer: AdamW

[0096] Learning rate: 0.0001 (the learning rate drops to 0.00001 after the 400th epoch)

[0097] Weight decay: 0.0005

[0098] (2-4) Start training the deep neural network with randomly initialized parameters.

[0099] (2-5) Optimization objective: Minimize the cross-entropy loss.

[0100] This embodiment also provides a post-processing system for obtaining the reading of a pointer water meter, including:

[0101] A pointer detection module, which is used to obtain an image of a pointer water meter, detect each pointer in the pointer water meter image, crop and obtain pictures of each pointer, and sort all the pointer pictures according to the relative position relationship of the pointers;

[0102] A feature extraction module, which is used to extract visual features and text features from each pointer picture by using a preset image encoder;

[0103] A feature enhancement module, which is used to perform context modeling on the extracted visual features and text features based on a Transformer encoder to obtain enhanced multi-modal features;

[0104] A reading output module, which is used to autoregressively output the final reading based on the multi-modal features by using a Transformer decoder.

[0105] A post-processing system for obtaining the reading of a pointer water meter according to this embodiment can execute a post-processing method for obtaining the reading of a pointer water meter provided by an embodiment of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0106] This embodiment further provides a post-processing device for obtaining the reading of a pointer water meter, including:

[0107] At least one processor;

[0108] At least one memory, which is used to store at least one program;

[0109] When the at least one program is executed by the at least one processor, the at least one processor is caused to implement the method as Figure 4 shown.

[0110] A post-processing device for obtaining the reading of a pointer water meter according to this embodiment can execute a post-processing method for obtaining the reading of a pointer water meter provided by an embodiment of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0111] An embodiment of the present application also discloses a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes [[ID= the method shown.

[0112] This embodiment also provides a storage medium storing instructions or programs that can execute a post - processing method for obtaining pointer water meter readings provided by the method embodiment of the present invention. When the instructions or programs are run, any combination of implementation steps of the method embodiment can be executed, and the corresponding functions and beneficial effects of the method are achieved.

[0113] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order mentioned in the operation diagrams. For example, depending on the functions / operations involved, two consecutive blocks shown may actually be executed substantially simultaneously or the blocks can sometimes be executed in the reverse order. In addition, the embodiments presented and described in the flowcharts of the present invention are provided by way of example for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operations and logical flows presented herein. Alternative embodiments are contemplated where the order of various operations is changed and where sub - operations described as part of a larger operation are executed independently.

[0114] Furthermore, although the present invention has been described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the described functions and / or features may be integrated in a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It can also be understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. Rather, considering the attributes, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the modules will be understood within the ordinary skills of an engineer. Thus, those skilled in the art can implement the present invention as set forth in the claims without undue experimentation. It can also be understood that the specific concepts disclosed are illustrative only and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0115] If the described functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0116] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a defined sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch instructions from the instruction execution system, apparatus, or device and execute the instructions), or used in combination with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in combination with an instruction execution system, apparatus, or device.

[0117] More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection part with one or more wirings (electronic device), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), fiber optic device, and portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, then editing, interpreting, or otherwise processing it as necessary, and then storing it in a computer memory.

[0118] It should be understood that various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0119] In the above description of this specification, the description referring to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples", etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0120] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that: various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the claims and their equivalents.

[0121] The above has specifically described the preferred embodiments of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art can also make various equivalent deformations or substitutions without departing from the spirit of the present invention, and these equivalent deformations or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A post-processing method for obtaining pointer water meter readings, characterized in that: The following steps are involved: Obtaining a pointer water meter image, detecting each pointer in the pointer water meter image, cropping each pointer image, and sorting all pointer images according to the relative position relationship of the pointers; A preset image encoder is used to extract visual features and text features from each pointer image; the text features are obtained by: the image encoder takes the pointer image as input, passes it through a convolutional feature extraction module and a fully connected classification module to obtain a probability distribution, and uses the probability distribution as the text feature; The extracted visual features and text features are contextually modeled based on the Transformer encoder to obtain enhanced multimodal features. According to the multimodal features, the final reading is output based on the Transformer decoder autoregression.

2. A post-processing method for obtaining pointer water meter readings according to claim 1, characterized in that: The step of obtaining a pointer water meter image, detecting each pointer in the pointer water meter image, cropping each pointer image, and sorting all pointer images according to the relative position relationship of the pointers includes: Obtain a pointer water meter image, input the obtained pointer water meter image into a preset target detection network for pointer detection, and obtain detection frames of all pointers; Get each pointer image by cropping according to the detection frame; All pointer images are sorted according to the relative position relationship of each pointer detection frame.

3. A post-processing method for obtaining pointer water meter readings according to claim 1, characterized in that: The preset image encoder is implemented using a pre-trained deep neural network, which includes a convolutional feature extraction module and a fully connected classification module, wherein the output of the convolutional feature extraction module is used as a visual feature, and the probability distribution of the output of the fully connected classification module is used as a text feature.

4. A post-processing method for obtaining pointer water meter readings according to claim 3, characterized in that: The deep neural network adopts the ResNet18 network structure.

5. A post-processing method for obtaining pointer water meter readings according to claim 1, characterized in that: The input of the Transformer encoder consists of the visual features, text features, one-dimensional position embeddings and label embeddings of all pointers; the one-dimensional position embeddings are used to indicate the order information of the pointers, and the label embeddings are used to indicate the modality information; The output of the Transformer encoder includes multimodal features of contextual information.

6. A post-processing method for obtaining pointer water meter readings according to claim 1, characterized in that: The final reading is output based on the Transformer decoder autoregression according to the multimodal features, including: The obtained enhanced multimodal features are used as input, and the autoregressive process outputs the final reading; the autoregressive process starts from the lowest bit of the water meter reading.

7. A post-processing method for obtaining pointer water meter readings according to claim 1, characterized in that: The training process of the image encoder, Transformer encoder, and Transformer decoder is performed using a manually labeled pointer water meter dataset; The data annotation process includes: pointer water meter image acquisition, single pointer detection frame annotation, single pointer reading annotation, and complete reading annotation.

8. A post-processing system for obtaining pointer water meter readings, characterized in that: include: The pointer detection module is used to obtain the pointer water meter image, detect each pointer in the pointer water meter image, crop and obtain each pointer image, and sort all pointer images according to the relative position relationship of the pointers; A feature extraction module is configured to extract visual features and text features from each pointer image using a preset image encoder; the text features are obtained by: the image encoder takes the pointer image as input, passes it through a convolutional feature extraction module and a fully connected classification module to obtain a probability distribution, and uses the probability distribution as the text feature; The feature enhancement module is used to perform context modeling on the extracted visual features and text features based on the Transformer encoder to obtain enhanced multimodal features; The degree output module is used to output the final reading based on the Transformer decoder autoregression according to the multimodal features.

9. A post-processing device for obtaining pointer water meter readings, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The processor-executable program is used to perform the method according to any one of claims 1 to 7 when executed by the processor.

Images