A Multimodal Recognition Method and System for Water Meter Based on AI Vision

Abstract

This invention relates to the field of water meter dial recognition technology, specifically to a multimodal recognition method and system for water meter dials based on AI vision. In this invention, the AI ​​vision analysis and recognition module automatically locates the water meter number area, digital flow rate area, barcode area, and pointer dial area in the acquired water meter image based on a constructed AI vision analysis model. Based on image recognition technology, it obtains the corresponding water meter number, digital flow rate, pointer flow rate, and the confidence level of each recognition result in the acquired water meter image, and sends the obtained image recognition results to the production testing terminal. This invention adopts overall image recognition and AI vision algorithms for the dial, is insensitive to the internal layout of the dial, and greatly improves the universality of recognizing various water meter dials. Before recognition, this invention performs zeroing detection on the sub-dial, indirectly checking the completion quality of the previous process, while ensuring the accuracy of subsequent recognition.

Description

Technical Field

[0001] This invention relates to the field of water meter dial recognition technology, specifically to a multimodal recognition method and system for water meter dials based on AI vision. Background Technology

[0002] Currently, there are two main types of OCR recognition technology used in water meter dials (hereinafter referred to as "dials"): one is applied to the recognition of the text area of ​​the dial, directly recognizing the numbers on the dial wheel without recognizing the sub-dial pointer; the other is to directly recognize the swivel pointer, setting up a speed recognition device above the swivel pointer, using laser scanning to calculate the number of rotations of the swivel pointer, and then calculating the final value based on the speed ratio between the swivel pointer and the sub-dial and dial wheel.

[0003] In the first existing technology, OCR recognition technology is mainly applied to text area recognition, without using OCR recognition or similar technology for sub-dials. This is because the randomness of the pointer installation on the sub-dials makes it impossible to directly recognize the pointer position based on the scanned pointer position, resulting in an inaccurate determination of the sub-dials pointer position. Furthermore, the display of digits on the text wheel is related to the sub-dials pointer position. When the digits on the wheel are incomplete, missing strokes, or have other abnormalities, the sub-dials pointer position cannot be used for auxiliary recognition, leading to a certain flaw in the overall text area OCR recognition rate, which is below 95%. The second existing technology is highly dependent on the position of the pendulum pointer on the dial, lacking versatility. The pendulum pointer recognition scheme requires the recognition sensor to be precisely positioned above the pendulum pointer, with certain sensing distance requirements. It also requires a certain contrast between the pendulum pointer and the relevant background. When using different types of water meter heads, if the pendulum pointer position moves, the sensor position of the testing device must also be changed. If the recognition conditions (such as background contrast) are not met, recognition will fail. Therefore, both of the above methods have significant drawbacks. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for multimodal recognition of water meter dials based on AI vision, so as to solve the problems mentioned in the background art.

[0005] To address the aforementioned technical problems, this invention provides the following technical solution: a multimodal recognition method for water meter gauges based on AI vision, the method comprising:

[0006] S1. Place the produced water meter on the water meter fixing device and capture the image of the water meter on the water meter fixing device using a camera;

[0007] S2. Based on the constructed AI visual analysis model, the water meter number area, digital flow area, barcode area and pointer dial area in the collected water meter image are automatically located. Based on image recognition technology, the corresponding water meter number, digital flow, pointer flow and the confidence level of each recognition result in the collected water meter image are obtained respectively, and the obtained image recognition results are sent to the production test terminal.

[0008] S3. Based on the confidence levels corresponding to each of the obtained recognition results, the production test end judges the recognition operation process. When the confidence level corresponding to the water meter image recognition result is less than the preset value, it jumps to S1 to fix the water meter and re-acquire and recognize the image. Before jumping to S1, the obtained water meter image recognition result and the corresponding water meter image acquisition result are sent to the manual error correction processing sequence to wait for the corresponding administrator to perform manual error correction processing.

[0009] S4. When the confidence scores of the water meter image recognition results are all greater than or equal to the preset values, the water meter number in the water meter image recognition results is called to perform duplicate number detection, and the image recognition results of the corresponding water meters are managed in combination with the duplicate number detection results.

[0010] Furthermore, during the automatic localization of the water meter number area, digital flow area, barcode area, and pointer dial area in the acquired water meter image based on the constructed Ai visual analysis model, S2 obtains the smallest rectangular box area corresponding to the water meter number area, digital flow area, barcode area, and pointer dial area in each water meter image in historical data. It then obtains the image texture features within each obtained smallest rectangular box area based on the corresponding recognition area type. The recognition area types include water meter number area type, digital flow area type, barcode area type, and pointer dial area type. The texture feature represents the pixel matrix after binarization of the corresponding minimum rectangular region image. Each pixel in the region image corresponds to an element in the pixel matrix. The elements in the pixel matrix are the binarized results of the corresponding grayscale values ​​of the pixels. The positional relationship between different elements in the pixel matrix is ​​the same as the positional relationship between corresponding pixels in the corresponding region image. The Ai visual analysis model is a summary of the image texture features of the water meter number region, digital flow region, barcode region, and pointer dial region in each water meter image in historical data, based on the corresponding recognition region types.

[0011] The system calls upon the bounding box specifications corresponding to each recognition region type in the constructed AI visual analysis model, and translates the bounding boxes of the corresponding specifications. It then extracts the image regions within the corresponding bounding boxes in the collected water meter images in real time, denoting them as the bounding box image regions of the corresponding specifications at the corresponding translation positions. The similarity between the image texture features corresponding to the bounding box image regions of the corresponding specifications at the corresponding translation positions and the image texture features corresponding to each element within the corresponding recognition region type in the constructed AI visual analysis model is calculated. The bounding box image regions of the corresponding specifications at the corresponding translation positions with similarity greater than a preset similarity value and the highest similarity are taken as the image regions corresponding to the corresponding recognition region types. The similarity between the image texture features corresponding to the two regions is equal to the ratio of the number of elements with the same element value at the same element position in the pixel matrix corresponding to the image texture feature to the total number of elements in the pixel matrix.

[0012] Furthermore, the specific steps in S2 for obtaining the corresponding water meter number in the acquired water meter image based on image recognition technology are as follows:

[0013] S211. Extract the water meter number area from the acquired water meter image;

[0014] S212. Based on OCR technology, perform text recognition on the water meter number area in the acquired water meter image to obtain the first water meter number value;

[0015] S213. Extract the barcode region from the collected water meter image, and binarize the grayscale values ​​of the barcode region image to obtain a binarized barcode image.

[0016] S214. Based on the preset barcode parsing rules, the binary barcode image is parsed and recognized to obtain the barcode value, which is recorded as the second water meter number value.

[0017] S215. Compare the first water meter number value with the second water meter number value. If the first water meter number value and the second water meter number value are the same, then determine that the value corresponding to the first water meter number value and the second water meter number value is the water meter number, and the confidence level of the water meter number identification result is 1; if the first water meter number value and the second water meter number value are not the same, then determine that the confidence level of the water meter number identification result is 0.

[0018] Furthermore, the specific steps in S2 for obtaining the corresponding digital flow rate and pointer flow rate from the collected water meter image based on image recognition technology are as follows:

[0019] S221. Extract the digital flow area and pointer flow area from the acquired water meter image; each digital flow area or pointer flow area corresponds to a sub-dial of the water meter, and the water meter includes multiple sub-dials;

[0020] S222. Based on OCR technology, locate and identify the scale values ​​of the digital flow area on the dial, and use the Hough line detection operator to detect the straight scale in the digital flow area. The intersection of the obtained straight scale with the arc contour of the sub-dial is recorded as the scale point of the corresponding straight scale; each scale point is bound to a scale value.

[0021] The process of detecting straight line scales within a digital flow area using the Hough line detection operator in this invention includes the following specific implementation steps: 1) Image acquisition and ROI location: Read the original image of the water meter, determine the coordinates of the scale value region bounding box through target detection / template matching / contour analysis, obtain the coordinates of the rectangular region, and then extract the ROI region (region of interest); 2) Image preprocessing: Perform grayscale conversion and Gaussian blur noise reduction, and perform Canny edge detection. By adjusting the parameters, the scale lines are made clear and visible, and the noise is minimized; 3) Perform Hough line detection: Perform probabilistic Hough transform; and filter the Hough transform results through angle filtering (adjusting the angle range according to the actual scale direction, such as 75~105 degrees for vertical scales) and length filtering; 4) Result visualization: Draw the detection results on the ROI and overlay them on the entire image.

[0022] S223. Calculate the center point of each sub-dial and convert each scale point into polar coordinates based on the corresponding center point of the disk;

[0023] S224. Extract the pointer vertex position and convert it into polar coordinates based on the center point of the corresponding disk;

[0024] S225. Compare the polar coordinates of the pointer vertex position with the polar coordinates of the scale point to identify the scale range in which the pointer vertex position is located. Based on the proportion of the arc formed by the pointer vertex position, the corresponding disk center point, and the minimum scale point corresponding to the endpoint of the scale range to the arc formed by the scale points corresponding to the two endpoints of the scale range in which the pointer vertex position is located and the corresponding disk center point, predict the scale value of the corresponding pointer vertex.

[0025] S226. Sequentially identify the pointer scale of each sub-dial, summarize the scale values ​​of the pointer apex in each sub-dial, and obtain the final dial value.

[0026] The confidence level of the digital flow rate corresponding to the collected water meter image is 1; the average value of the ratio of the absolute value of the difference between the dial value identified by each water meter and the actual dial value to the corresponding actual dial value in the N most recent water meter dial identification results in historical data is recorded as the pointer confidence deviation coefficient; the difference between 1 and the corresponding pointer confidence deviation coefficient is recorded as the pointer flow rate confidence level.

[0027] Furthermore, during the process of determining the identification operation flow in the S3 production test terminal, if the confidence level corresponding to the water meter image identification result is greater than or equal to a preset value, then it will jump to S4.

[0028] Furthermore, when the confidence level corresponding to the water meter image recognition result is less than a preset value, the process jumps to S1 to re-fix the water meter and perform image acquisition and recognition. If the water meter fixing and image acquisition and recognition operations are repeatedly performed more than M times for the same water meter, the fixing, acquisition, and recognition operations on the corresponding water meter dial will be stopped, and the recognition operation process on the corresponding water meter dial will end; M is a preset constant.

[0029] In this invention, the setting "if the water meter fixing, image acquisition, and recognition operations are repeatedly performed more than M times for the same water meter, then the fixing, acquisition, and recognition operations on the corresponding water meter dial will be stopped, and the recognition operation process on the corresponding water meter dial will end" is to avoid steps S1-S3 from getting stuck in an infinite loop. By setting the value of M, the maximum number of times the fixing, acquisition, and recognition operations on the same water meter dial in step S3 are performed is given.

[0030] In step S3, if, during the process of waiting for the corresponding administrator to perform manual error correction, the confidence levels of the obtained recognition results are all greater than or equal to the preset values ​​after the water meter is fixed and the image is acquired and recognized again, the corresponding water meter image recognition results and the corresponding water meter image acquisition results sent to the manual error correction processing sequence will be deleted.

[0031] Furthermore, in the process of managing the image recognition results of the corresponding water meters by combining the duplicate number detection results in S4, if the historical water meter image recognition result data contains the water meter number in the called water meter image recognition result, it is determined that the water meter number in the called water meter image recognition result has a duplicate number, and the recognition operation process of the corresponding water meter dial is stopped; otherwise, the dial value recognized by the corresponding water meter dial is bound to the corresponding water meter dial and saved to the database.

[0032] A multimodal water meter recognition system based on AI vision, the system comprising:

[0033] The water meter fixing and image acquisition module is used to place the manufactured water meter on the water meter fixing device and acquire the water meter image on the water meter fixing device through a camera;

[0034] The AI ​​visual analysis and recognition module automatically locates the water meter number area, digital flow area, barcode area and pointer dial area in the collected water meter image based on the constructed AI visual analysis model. Based on image recognition technology, it obtains the corresponding water meter number, digital flow, pointer flow and the confidence level of each recognition result in the collected water meter image, and sends the obtained image recognition results to the production test terminal.

[0035] The dynamic management module for the identification operation process determines the identification operation process based on the confidence level corresponding to each identification result. When the confidence level corresponding to the water meter image identification result is less than the preset value, it jumps to the water meter fixing and image acquisition module to fix the water meter and re-acquire and identify the image. Before jumping to the water meter fixing and image acquisition module, the obtained water meter image identification result and the corresponding water meter image acquisition result are sent to the manual error correction waiting sequence for the corresponding administrator to perform manual error correction.

[0036] The duplicate number detection and management module is used to call the water meter number in the water meter image recognition result to perform duplicate number detection when the confidence level of the corresponding water meter image recognition result is greater than or equal to the preset value, and to manage the image recognition result of the corresponding water meter in combination with the duplicate number detection result.

[0037] Furthermore, the AI ​​visual analysis and recognition module includes a region image localization unit, a region image parameter recognition unit, and an image recognition result transmission unit.

[0038] The regional image positioning unit automatically locates the water meter number area, digital flow area, barcode area, and pointer dial area in the acquired water meter image based on the constructed Ai visual analysis model.

[0039] The regional image parameter recognition unit obtains the water meter number, digital flow rate, pointer flow rate, and confidence level corresponding to each recognition result from the collected water meter image based on image recognition technology.

[0040] The image recognition result transmission unit sends the obtained image recognition result to the production test terminal.

[0041] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0042] (1) The present invention adopts overall image recognition and AI vision algorithm for the dial, and is not sensitive to the layout and arrangement inside the dial (such as the position of the plum blossom pointer), which greatly improves the universality of recognizing various water meter dials;

[0043] (2) Before identification, the present invention performs a zeroing test on the sub-dial (detects whether the pointer of each sub-dial is zero). If it is not zeroed, the previous process needs to reset it. The present invention indirectly checks the completion quality of the previous process, while ensuring the accuracy of subsequent identification. At the same time, it can save a detection station and improve work efficiency. Attached Figure Description

[0044] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0045] Figure 1 This is a schematic diagram of the structure of the AI ​​vision-based multimodal recognition system for water meter dials according to the present invention;

[0046] Figure 2 This is a flowchart illustrating the multimodal recognition method for water meter dials based on AI vision according to the present invention.

[0047] Figure 3 This is a first schematic diagram of an embodiment of the AI ​​vision-based multimodal recognition method for water meter dials of the present invention;

[0048] Figure 4 This is a second schematic diagram of an embodiment of the AI ​​vision-based multimodal recognition method for water meter dials of the present invention;

[0049] Figure 5 This is a schematic diagram of the third embodiment of the multimodal recognition method for water meter dials based on AI vision of the present invention. Detailed Implementation

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] Please see Figures 1-2 The present invention provides a technical solution: such as Figure 1 As shown, this embodiment provides a multimodal recognition system for water meter dials based on AI vision. The system includes:

[0052] The water meter fixing and image acquisition module is used to place the manufactured water meter on the water meter fixing device and acquire the water meter image on the water meter fixing device through a camera;

[0053] The AI ​​visual analysis and recognition module includes a region image localization unit, a region image parameter recognition unit, and an image recognition result transmission unit.

[0054] The regional image positioning unit automatically locates the water meter number area, digital flow area, barcode area, and pointer dial area in the acquired water meter image based on the constructed Ai visual analysis model.

[0055] The regional image parameter recognition unit obtains the water meter number, digital flow rate, pointer flow rate, and confidence level corresponding to each recognition result from the collected water meter image based on image recognition technology.

[0056] The image recognition result transmission unit sends the obtained image recognition result to the production test terminal;

[0057] The dynamic management module for the identification operation process determines the identification operation process based on the confidence level corresponding to each identification result. When the confidence level corresponding to the water meter image identification result is less than the preset value, it jumps to the water meter fixing and image acquisition module to fix the water meter and re-acquire and identify the image. Before jumping to the water meter fixing and image acquisition module, the obtained water meter image identification result and the corresponding water meter image acquisition result are sent to the manual error correction waiting sequence for the corresponding administrator to perform manual error correction.

[0058] The duplicate number detection and management module is used to call the water meter number in the water meter image recognition result to perform duplicate number detection when the confidence level of the corresponding water meter image recognition result is greater than or equal to the preset value, and to manage the image recognition result of the corresponding water meter in combination with the duplicate number detection result.

[0059] like Figure 2 As shown, this embodiment provides a multimodal recognition method for water meter dials based on AI vision. The method includes:

[0060] S1. Place the produced water meter on the water meter fixing device and capture the image of the water meter on the water meter fixing device using a camera;

[0061] S2. Based on the constructed AI visual analysis model, the water meter number area, digital flow area, barcode area and pointer dial area in the collected water meter image are automatically located. Based on image recognition technology, the corresponding water meter number, digital flow, pointer flow and the confidence level of each recognition result in the collected water meter image are obtained respectively, and the obtained image recognition results are sent to the production test terminal.

[0062] S2, based on the constructed Ai visual analysis model, automatically locates the water meter number area, digital flow area, barcode area, and pointer dial area in the acquired water meter images. During this process, it obtains the smallest rectangular bounding box area corresponding to each of these areas in historical water meter images. It then acquires image texture features within each of these smallest rectangular bounding box areas based on the corresponding recognition area type. These recognition area types include water meter number area type, digital flow area type, barcode area type, and pointer dial area type. The image texture features represent the pixel matrix after binarization of the corresponding smallest rectangular bounding box image. Each pixel in the area image corresponds to an element in the pixel matrix, and the elements in the pixel matrix are the binarized results of the corresponding grayscale values. Furthermore, the positional relationship between pixels corresponding to different elements in the pixel matrix is ​​the same as the positional relationship between corresponding pixels in the corresponding area image. The Ai visual analysis model is a summary of the image texture features of the water meter number area, digital flow area, barcode area, and pointer dial area in historical water meter images based on the corresponding recognition area types.

[0063] The system calls upon the bounding box specifications corresponding to each recognition region type in the constructed AI visual analysis model, and translates the bounding boxes of the corresponding specifications. It then extracts the image regions within the corresponding bounding boxes in the collected water meter images in real time, denoting them as the bounding box image regions of the corresponding specifications at the corresponding translation positions. The similarity between the image texture features corresponding to the bounding box image regions of the corresponding specifications at the corresponding translation positions and the image texture features corresponding to each element within the corresponding recognition region type in the constructed AI visual analysis model is calculated. The bounding box image regions of the corresponding specifications at the corresponding translation positions with similarity greater than a preset similarity value and the highest similarity are taken as the image regions corresponding to the corresponding recognition region types. The similarity between the image texture features corresponding to the two regions is equal to the ratio of the number of elements with the same element value at the same element position in the pixel matrix corresponding to the image texture feature to the total number of elements in the pixel matrix.

[0064] The specific steps in S2 for obtaining the corresponding water meter number from the acquired water meter image based on image recognition technology are as follows:

[0065] S211. Extract the water meter number area from the acquired water meter image;

[0066] S212. Based on OCR technology, perform text recognition on the water meter number area in the acquired water meter image to obtain the first water meter number value;

[0067] S213. Extract the barcode region from the collected water meter image, and binarize the grayscale values ​​of the barcode region image to obtain a binarized barcode image.

[0068] S214. Based on the preset barcode parsing rules, the binary barcode image is parsed and recognized to obtain the barcode value, which is recorded as the second water meter number value.

[0069] S215. Compare the first water meter number value with the second water meter number value. If the first water meter number value and the second water meter number value are the same, then determine that the value corresponding to the first water meter number value and the second water meter number value is the water meter number, and the confidence level of the water meter number identification result is 1; if the first water meter number value and the second water meter number value are not the same, then determine that the confidence level of the water meter number identification result is 0.

[0070] The specific steps in S2 for obtaining the corresponding digital flow rate and pointer flow rate from the collected water meter image based on image recognition technology are as follows:

[0071] S221. Extract the digital flow area and pointer flow area from the acquired water meter image; each digital flow area or pointer flow area corresponds to a sub-dial of the water meter, and the water meter includes multiple sub-dials;

[0072] S222. Based on OCR technology, locate and identify the scale values ​​of the digital flow area on the dial, and use the Hough line detection operator to detect the straight scale in the digital flow area. The intersection of the obtained straight scale with the arc contour of the sub-dial is recorded as the scale point of the corresponding straight scale; each scale point is bound to a scale value.

[0073] S223. Calculate the center point of each sub-dial and convert each scale point into polar coordinates based on the corresponding center point of the disk;

[0074] S224. Extract the pointer vertex position and convert it into polar coordinates based on the center point of the corresponding disk;

[0075] S225. Compare the polar coordinates of the pointer vertex position with the polar coordinates of the scale point to identify the scale range in which the pointer vertex position is located. Based on the proportion of the arc formed by the pointer vertex position, the corresponding disk center point, and the minimum scale point corresponding to the endpoint of the scale range to the arc formed by the scale points corresponding to the two endpoints of the scale range in which the pointer vertex position is located and the corresponding disk center point, predict the scale value of the corresponding pointer vertex.

[0076] S226. Sequentially identify the pointer scale of each sub-dial, summarize the scale values ​​of the pointer apex in each sub-dial, and obtain the final dial value.

[0077] The confidence level of the digital flow rate corresponding to the collected water meter image is 1; the average value of the ratio of the absolute value of the difference between the dial value identified by each water meter and the actual dial value to the corresponding actual dial value in the N most recent water meter dial identification results in historical data is recorded as the pointer confidence deviation coefficient; the difference between 1 and the corresponding pointer confidence deviation coefficient is recorded as the pointer flow rate confidence level.

[0078] S3. Based on the confidence levels corresponding to each of the obtained recognition results, the production test end judges the recognition operation process. When the confidence level corresponding to the water meter image recognition result is less than the preset value, it jumps to S1 to fix the water meter and re-acquire and recognize the image. Before jumping to S1, the obtained water meter image recognition result and the corresponding water meter image acquisition result are sent to the manual error correction processing sequence to wait for the corresponding administrator to perform manual error correction processing.

[0079] During the process of determining the identification operation flow in the S3 production test terminal, if the confidence level corresponding to the water meter image identification result is greater than or equal to the preset value, then jump to S4.

[0080] Furthermore, when the confidence level corresponding to the water meter image recognition result is less than a preset value, the process jumps to S1 to re-fix the water meter and perform image acquisition and recognition. If the water meter fixing and image acquisition and recognition operations are repeatedly performed more than M times for the same water meter, the fixing, acquisition, and recognition operations on the corresponding water meter dial will be stopped, and the recognition operation process on the corresponding water meter dial will end; M is a preset constant.

[0081] In step S3, if, during the process of waiting for the corresponding administrator to perform manual error correction, the confidence levels of the obtained recognition results are all greater than or equal to the preset values ​​after the water meter is fixed and the image is acquired and recognized again, the corresponding water meter image recognition results and the corresponding water meter image acquisition results sent to the manual error correction processing sequence will be deleted.

[0082] S4. When the confidence scores of the water meter image recognition results are all greater than or equal to the preset values, the water meter number in the water meter image recognition results is called to perform duplicate number detection, and the image recognition results of the corresponding water meters are managed in combination with the duplicate number detection results.

[0083] In the process of managing the image recognition results of the corresponding water meters by combining the duplicate number detection results in S4, if the historical water meter image recognition result data contains the water meter number in the called water meter image recognition result, it is determined that the water meter number in the called water meter image recognition result has a duplicate number, and the recognition operation process of the corresponding water meter dial is stopped; otherwise, the dial value recognized by the corresponding water meter dial is bound to the corresponding water meter dial and saved to the database.

[0084] In this embodiment, the reading of the pointer on the dial is recognized, and the recognition logic is as follows:

[0085] Step 1: There are a total of 4 sub-dials on the instrument (multiply by 0.1, multiply by 0.01, multiply by 0.001, and multiply by 0.0001 respectively). First, use circular contour detection to extract and locate the images of the 4 dial areas.

[0086] Step 2: Locate and identify the scale values ​​on the dial, and use the Huffman line detection operator to detect the straight scale values ​​within the scale value area. The intersection of the straight line and the arc contour of the sub-dial is the scale point.

[0087] Step 3: Calculate the center point of each sub-dial and convert each scale point into polar coordinates based on that center point.

[0088] Step 4: Perform key point detection on the vertex of the pointer and convert the vertex into polar coordinates based on the center point of the disk.

[0089] Step 5: Compare the polar coordinates of the pointer vertex with the polar coordinates of the scale to calculate which scale range the pointer vertex is in. Based on the relative proportion, the scale value of the pointer vertex can be estimated.

[0090] Step 6: Sequentially identify the pointer scales of the four dials, and summarize the scale values ​​of the four dials to obtain the final dial value.

[0091] In this embodiment, the accuracy of pointer reading recognition is very high; except for the last pointer digit, the accuracy of other pointer recognition is 100%. For the recognition of the last pointer digit (0.0001 increment), this example uses an estimation method. First, it determines which graduation value the pointer is closest to; if it is clearly identifiable, the nearest graduation value is used. Figure 3 , Figure 4 As shown, when the pointer falls between two adjacent scale values, Ai can identify the position and thus determine the reading by comparing the images.

[0092] like Figure 5 As shown, Ai can identify the pointer in the circled box as being between 1 and 2. Based on the judgment of the pointer at the 0.001 liter position below, it can basically determine the corresponding position, making the recognition accuracy of the pointer at the 0.01 liter position 100%.

[0093] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0094] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multimodal recognition method for water meter dials based on AI vision, characterized in that, The method includes: S1. Place the produced water meter on the water meter fixing device and capture the image of the water meter on the water meter fixing device using a camera; S2. Based on the constructed AI visual analysis model, the water meter number area, digital flow area, barcode area and pointer dial area in the collected water meter image are automatically located. Based on image recognition technology, the corresponding water meter number, digital flow, pointer flow and the confidence level of each recognition result in the collected water meter image are obtained respectively, and the obtained image recognition results are sent to the production test terminal. S2, based on the constructed Ai visual analysis model, automatically locates the water meter number area, digital flow area, barcode area, and pointer dial area in the acquired water meter images. During this process, it obtains the smallest rectangular bounding box area corresponding to each of these areas in historical water meter images. It then acquires image texture features within each of these smallest rectangular bounding box areas based on the corresponding recognition area type. These recognition area types include water meter number area type, digital flow area type, barcode area type, and pointer dial area type. The image texture features represent the pixel matrix after binarization of the corresponding smallest rectangular bounding box image. Each pixel in the area image corresponds to an element in the pixel matrix, and the elements in the pixel matrix are the binarized results of the corresponding grayscale values. Furthermore, the positional relationship between pixels corresponding to different elements in the pixel matrix is ​​the same as the positional relationship between corresponding pixels in the corresponding area image. The Ai visual analysis model is a summary of the image texture features of the water meter number area, digital flow area, barcode area, and pointer dial area in historical water meter images based on the corresponding recognition area types. The system calls upon the bounding box specifications corresponding to each recognition region type in the constructed AI visual analysis model, and translates the bounding boxes of the corresponding specifications. It then extracts the image regions within the corresponding bounding boxes in the collected water meter images in real time, denoting them as the bounding box image regions of the corresponding specifications at the corresponding translation positions. The similarity between the image texture features corresponding to the bounding box image regions of the corresponding specifications at the corresponding translation positions and the image texture features corresponding to each element within the corresponding recognition region type in the constructed AI visual analysis model is calculated. The bounding box image regions of the corresponding specifications at the corresponding translation positions with similarity greater than a preset similarity value and the highest similarity are taken as the image regions corresponding to the corresponding recognition region types. The similarity between the image texture features corresponding to the two regions is equal to the ratio of the number of elements with the same element value at the same element position in the pixel matrix corresponding to the image texture feature to the total number of elements in the pixel matrix. The specific steps in S2 for obtaining the corresponding digital flow rate and pointer flow rate from the collected water meter image based on image recognition technology are as follows: S221. Extract the digital flow area and pointer flow area from the acquired water meter image; each digital flow area or pointer flow area corresponds to a sub-dial of the water meter, and the water meter includes multiple sub-dials; S222. Based on OCR technology, locate and identify the scale values ​​of the digital flow area on the dial, and use the Hough line detection operator to detect the straight scale in the digital flow area. The intersection of the obtained straight scale with the arc contour of the sub-dial is recorded as the scale point of the corresponding straight scale; each scale point is bound to a scale value. S223. Calculate the center point of each sub-dial and convert each scale point into polar coordinates based on the corresponding center point of the disk; S224. Extract the pointer vertex position and convert it into polar coordinates based on the center point of the corresponding disk; S225. Compare the polar coordinates of the pointer vertex position with the polar coordinates of the scale point to identify the scale range in which the pointer vertex position is located. Based on the proportion of the arc formed by the pointer vertex position, the corresponding disk center point, and the minimum scale point corresponding to the endpoint of the scale range to the arc formed by the scale points corresponding to the two endpoints of the scale range in which the pointer vertex position is located and the corresponding disk center point, predict the scale value of the corresponding pointer vertex. S226. Sequentially identify the pointer scale of each sub-dial, summarize the scale values ​​of the pointer apex in each sub-dial, and obtain the final dial value. The confidence level of the digital flow rate corresponding to the collected water meter image is 1; the average value of the ratio of the absolute value of the difference between the water meter reading and the actual water meter reading in the N most recent water meter reading recognition results in the historical data to the corresponding actual water meter reading is recorded as the pointer confidence deviation coefficient; the difference between 1 and the corresponding pointer confidence deviation coefficient is recorded as the pointer flow rate confidence level. S3. Based on the confidence levels corresponding to each of the obtained recognition results, the production test end judges the recognition operation process. When the confidence level corresponding to the water meter image recognition result is less than the preset value, it jumps to S1 to fix the water meter and re-acquire and recognize the image. Before jumping to S1, the obtained water meter image recognition result and the corresponding water meter image acquisition result are sent to the manual error correction processing sequence to wait for the corresponding administrator to perform manual error correction processing. S4. When the confidence scores of the water meter image recognition results are all greater than or equal to the preset values, the water meter number in the water meter image recognition results is called to perform duplicate number detection, and the image recognition results of the corresponding water meters are managed in combination with the duplicate number detection results.

2. The multimodal recognition method for water meter dials based on AI vision according to claim 1, characterized in that: The specific steps in S2 for obtaining the corresponding water meter number from the acquired water meter image based on image recognition technology are as follows: S211. Extract the water meter number area from the acquired water meter image; S212. Based on OCR technology, perform text recognition on the water meter number area in the acquired water meter image to obtain the first water meter number value; S213. Extract the barcode region from the collected water meter image, and binarize the grayscale values ​​of the barcode region image to obtain a binarized barcode image. S214. Based on the preset barcode parsing rules, the binary barcode image is parsed and recognized to obtain the barcode value, which is recorded as the second water meter number value. S215. Compare the first water meter number value with the second water meter number value. If the first water meter number value and the second water meter number value are the same, then determine that the value corresponding to the first water meter number value and the second water meter number value is the water meter number, and the confidence level of the water meter number identification result is 1; if the first water meter number value and the second water meter number value are not the same, then determine that the confidence level of the water meter number identification result is 0.

3. The multimodal recognition method for water meter dials based on AI vision according to claim 1, characterized in that: During the process of determining the identification operation flow in the S3 production test terminal, if the confidence level corresponding to the water meter image identification result is greater than or equal to the preset value, then jump to S4. Furthermore, when the confidence level corresponding to the water meter image recognition result is less than a preset value, the process jumps to S1 to re-fix the water meter and perform image acquisition and recognition. If the water meter fixing and image acquisition and recognition operations are repeatedly performed more than M times for the same water meter, the fixing, acquisition, and recognition operations on the corresponding water meter dial will be stopped, and the recognition operation process on the corresponding water meter dial will end; M is a preset constant. In step S3, if, during the process of waiting for the corresponding administrator to perform manual error correction, the confidence levels of the obtained recognition results are all greater than or equal to the preset values ​​after the water meter is fixed and the image is acquired and recognized again, then the corresponding water meter image recognition results and the corresponding water meter image acquisition results sent to the manual error correction processing sequence will be deleted.

4. The multimodal recognition method for water meter dials based on AI vision according to claim 1, characterized in that: In the process of managing the image recognition results of the corresponding water meter by combining the duplicate number detection results in S4, when the historical water meter image recognition result data contains the water meter number in the called water meter image recognition result, it is determined that the water meter number in the called water meter image recognition result has a duplicate number, and the recognition operation process of the corresponding water meter dial is stopped. Conversely, the meter value identified by the corresponding water meter dial is bound to the corresponding water meter dial and saved to the database.

5. A multimodal recognition system for water meter dials based on AI vision, employing the multimodal recognition method for water meter dials based on AI vision as described in any one of claims 1-4, characterized in that, The system includes: The water meter fixing and image acquisition module is used to place the manufactured water meter on the water meter fixing device and acquire the water meter image on the water meter fixing device through a camera; The AI ​​visual analysis and recognition module automatically locates the water meter number area, digital flow area, barcode area and pointer dial area in the collected water meter image based on the constructed AI visual analysis model. Based on image recognition technology, it obtains the corresponding water meter number, digital flow, pointer flow and the confidence level of each recognition result in the collected water meter image, and sends the obtained image recognition results to the production test terminal. The dynamic management module for the identification operation process determines the identification operation process based on the confidence level corresponding to each identification result. When the confidence level corresponding to the water meter image identification result is less than the preset value, it jumps to the water meter fixing and image acquisition module to fix the water meter and re-acquire and identify the image. Before jumping to the water meter fixing and image acquisition module, the obtained water meter image identification result and the corresponding water meter image acquisition result are sent to the manual error correction waiting sequence for the corresponding administrator to perform manual error correction. The duplicate number detection and management module is used to call the water meter number in the water meter image recognition result to perform duplicate number detection when the confidence level of the corresponding water meter image recognition result is greater than or equal to the preset value, and to manage the image recognition result of the corresponding water meter in combination with the duplicate number detection result.

6. The AI ​​vision-based multimodal recognition system for water meter gauges according to claim 5, characterized in that: The AI ​​visual analysis and recognition module includes a region image localization unit, a region image parameter recognition unit, and an image recognition result transmission unit. The regional image positioning unit automatically locates the water meter number area, digital flow area, barcode area, and pointer dial area in the acquired water meter image based on the constructed Ai visual analysis model. The regional image parameter recognition unit obtains the water meter number, digital flow rate, pointer flow rate, and confidence level corresponding to each recognition result from the collected water meter image based on image recognition technology. The image recognition result transmission unit sends the obtained image recognition result to the production test terminal.

Images