Digital image recognition method, device, electronic device and readable storage medium

Abstract

This application provides a method, apparatus, electronic device, and readable storage medium for recognizing digital images, relating to the field of computer technology. The method includes: determining the image type of the image to be recognized; then determining first feature points of the image to be recognized and obtaining descriptors corresponding to each first feature point; determining descriptors of second feature points corresponding to the image type of the image to be recognized from a feature library; determining target feature points by matching the descriptors of the first and second feature points; determining digital character images from the image to be recognized based on the target feature points; and finally recognizing digital information in the digital character images using a digital image recognition model. This method achieves rapid and accurate localization of digital regions based on feature points of the image to be recognized, improving the accuracy of digital character recognition.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, electronic device and readable storage medium for recognizing a digital image. Background Art

[0002] Human beings are entering the information age, and computers will increasingly enter almost all fields. Computer vision is to use various imaging systems to replace visual organs as input sensitive means, and computers to replace the brain to complete processing and interpretation. The ultimate research goal of computer vision is to enable computers to observe and understand the world through vision like humans, and have the ability to adapt to the environment autonomously, so that computers can replace humans to do more work.

[0003] Among them, digital display recognition is a common task in the field of computer vision, which can be understood as the recognition of digital images in essence. In the existing technology, when faced with complex digital dial structures and various digital characters, the recognition accuracy is not high enough and it is difficult to meet the needs of actual applications. Summary of the invention

[0004] The purpose of the embodiments of the present application is to solve the problem of insufficient accuracy in recognizing numeric characters.

[0005] The present invention provides a method for recognizing a digital image, the method comprising:

[0006] Acquire an image to be identified and determine the image type of the image to be identified;

[0007] Determine a plurality of first feature points of the image to be identified, and obtain a descriptor corresponding to each first feature point; wherein the first feature point is a key point in the image to be identified;

[0008] Determine descriptors of a plurality of second feature points corresponding to the image type of the image to be identified in a preset feature library, match the descriptors of the plurality of first feature points with the descriptors of the plurality of second feature points, and use the first feature points that are successfully matched as target feature points;

[0009] A digital character image is determined from the image to be recognized based on the target feature points; and digital information in the digital character image is recognized based on the trained digital image recognition model.

[0010] In an optional embodiment of the first aspect, the method further comprises:

[0011] Obtain at least one template image corresponding to each image type;

[0012] Determine the second feature points of each template image based on a feature point detection algorithm, and obtain the descriptors corresponding to each second feature point;

[0013] A feature library is established based on the template images corresponding to the respective image types, the second feature points of the respective template images, and the descriptors corresponding to the respective second feature points.

[0014] In an optional embodiment of the first aspect, the method further comprises:

[0015] Determine the corner point coordinates of the graphics corresponding to each template image, and extract the second image features of each template image; establish a template library based on each template image, the corner point coordinates of each template image, and the second image features;

[0016] Determining the image type of the image to be identified based on the first image feature includes:

[0017] Calculating the similarity between the first image feature and each second image feature in the template library;

[0018] Based on the calculated similarity, a target template image that is most similar to the image to be identified is determined from the template library, and the image type corresponding to the target template image is used as the image type of the image to be identified.

[0019] In an optional embodiment of the first aspect, matching the descriptors of the plurality of first feature points with the descriptors of the plurality of second feature points, and taking the successfully matched first feature points as target feature points, includes:

[0020] Matching the descriptors of the plurality of first feature points with the descriptors of the plurality of second feature points to determine the first feature points that match the respective second feature points;

[0021] For each second feature point, the similarity between the descriptor of the second feature point and the descriptor of at least one matching first feature point is calculated, and the first feature point corresponding to the similarity less than a preset threshold is used as a candidate feature point, and the target feature point with the smallest distance to the second feature point is determined from the candidate feature points.

[0022] In an optional embodiment of the first aspect, determining a digital character image from an image to be recognized based on target feature points includes:

[0023] Acquire the corner point coordinates of the target template image corresponding to the image to be identified from the template library; the plurality of second feature points corresponding to the image type of the image to be identified are the feature points of the target template image corresponding to the image to be identified;

[0024] Obtaining a transformation relationship between the second feature point and the target feature point;

[0025] Obtaining the coordinates of the corner points of the image to be identified based on the transformed relationship and the coordinates of the corner points of the target template image;

[0026] A digital character image is determined from the image to be recognized based on the coordinates of the corner points of the image to be recognized.

[0027] In an optional embodiment of the first aspect, determining a digital character image from the image to be recognized based on the coordinates of the corner points of the image to be recognized includes:

[0028] According to the coordinates of the corner points of the graphics corresponding to the image to be identified, a digital region image is cut out from the image to be identified;

[0029] Correcting the digital region image through a perspective transformation operation to obtain a corrected digital region image;

[0030] The rectified digital region image is preprocessed, and the digital character image is segmented from the preprocessed digital region image by a vertical projection method.

[0031] In an optional embodiment of the first aspect, recognizing digital information in a digital character image based on a trained digital image recognition model includes:

[0032] The digital character image is input into the digital image recognition model to obtain a digital recognition result, which includes digital information in the digital character image.

[0033] In a second aspect, a digital image recognition device is provided, the device comprising:

[0034] A type determination module, used to obtain an image to be identified and determine the image type of the image to be identified;

[0035] A feature point detection module, used to determine a plurality of first feature points of the image to be identified, and obtain a descriptor corresponding to each first feature point;

[0036] A feature point matching module, used to determine the descriptors of multiple second feature points corresponding to the image type of the image to be identified in a preset feature library, match the descriptors of multiple first feature points with the descriptors of multiple second feature points, and use the first feature points that are successfully matched as target feature points; wherein the first feature points are key points in the image to be identified;

[0037] The digital recognition module is used to determine the digital character image from the image to be recognized based on the target feature points; and recognize the digital information in the digital character image based on the trained digital image recognition model.

[0038] In a third aspect, an electronic device is provided, the electronic device comprising:

[0039] A memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the digital image recognition method of any of the above embodiments is implemented.

[0040] In a fourth aspect, a readable storage medium is provided, on which a program is stored, and when the program is executed by a processor, the digital image recognition method of any of the above embodiments is implemented.

[0041] The above-mentioned digital image recognition method includes: determining the image type of the image to be recognized, then determining the first feature point of the image to be recognized, and obtaining the descriptor corresponding to each first feature point. Determining the descriptor of the second feature point corresponding to the image type of the image to be recognized in the feature library, determining the target feature point by matching the descriptor of the first feature point with the descriptor of the second feature point, and determining the digital character image from the image to be recognized based on the target feature point, and finally recognizing the digital information in the digital character image through the digital image recognition model, so as to realize the rapid and accurate positioning of the digital area based on the feature points of the image to be recognized, and improve the accuracy of digital character recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0043] Figure 1a A diagram of an application environment of a digital image recognition method provided in an embodiment of the present application;

[0044] Figure 1b A schematic diagram of a flow chart of a digital image recognition method provided in an embodiment of the present application;

[0045] Figure 2a A schematic diagram of a flow chart of a digital image recognition method provided in an embodiment of the present application;

[0046] Figure 2b A schematic diagram of marking the coordinates of four corner points in a digital image recognition method provided in an embodiment of the present application;

[0047] Figure 2c A schematic diagram of a corrected digital region image in a digital image recognition method provided in an embodiment of the present application;

[0048] Figure 2d A schematic diagram of a digital character segmentation result in a digital image recognition method provided in an embodiment of the present application;

[0049] Figure 2e A schematic diagram of a digital character recognition result in a digital image recognition method provided in an embodiment of the present application;

[0050] Figure 3 A schematic diagram of the structure of a digital image recognition device provided in an embodiment of the present application;

[0051] Figure 4 A schematic diagram of the structure of an electronic device for digital image recognition provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the implementation methods described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0053] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application refer to that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation as other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the one element may be directly connected or coupled to the other element, or it may refer to that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" may be implemented as "A", or as "B", or as "A and B".

[0054] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0055] First, several terms involved in this application are introduced and explained:

[0056] DROM is the abbreviation of digital display instrument. Due to its visual intuitiveness, convenient setting and high degree of intelligent control, and with the rapid development of current sensor technology, the functions of digital display instruments are becoming increasingly powerful and the accuracy is constantly improving. Digital display instruments are replacing traditional mechanical instruments in large quantities and have been widely used in all walks of life and daily life.

[0057] Image features can refer to a collection of attributes that characterize the characteristics or content of an image, mainly including natural image features (such as brightness, color, texture, etc.) and artificial image features (such as image spectrum, image histogram, etc.). Image features mainly include color features, texture features, shape features, and spatial relationship features.

[0058] The scale-invariant feature transform (SIFT) algorithm is a computer vision algorithm used to detect and describe local features in images. It searches for extreme points in the spatial scale and extracts their position, scale, and rotation invariants.

[0059] Morphology is one of the most widely used techniques in image processing. It is mainly used to extract image components that are meaningful for expressing and depicting the shape of the region, so that subsequent recognition work can grasp the most essential shape features of the target object, such as boundaries and connected areas. At the same time, techniques such as thinning, pixelation, and burr trimming are also often used in image preprocessing and postprocessing, becoming a powerful supplement to image enhancement technology. Basic morphological processing methods include image erosion, image dilation, opening operation, and closing operation.

[0060] Digital display recognition is a common task in the field of computer vision. It can be essentially understood as the recognition of digital images, which can include two main stages: digital area positioning and digital character recognition.

[0061] For the task of locating digital areas, existing methods include target detection-based methods and segmentation-based methods, which usually directly use a single detection or recognition algorithm to directly recognize digital images. Among them, the detection-based method is faster, but the positioning accuracy of the digital area is poor; the segmentation-based method has high accuracy in locating the digital area when the dial is relatively clean, but when there is a lot of occlusion on the dial, the positioning accuracy of the digital area is poor.

[0062] For digital character recognition tasks, existing methods include sequence recognition methods based on recurrent neural networks and single digit recognition methods based on convolutional neural networks. Among them, the sequence recognition method has a simple process, inputs the entire digital area, and outputs the reading, but the recognition effect for half characters is not good; the single digit recognition method includes single digit segmentation and recognition. Although the recognition effect of full characters and half characters is good, the process is more complicated.

[0063] In the prior art, a single detection or recognition algorithm is usually used to directly recognize digital images. When faced with complex digital dial structures and various digital characters, the positioning accuracy of the digital area is poor, and the accuracy of recognizing digital characters is not high enough, which makes it difficult to meet actual application needs.

[0064] The digital image recognition method, device, electronic device and readable storage medium provided in this application are intended to solve the above technical problems in the prior art.

[0065] The following describes several exemplary embodiments to illustrate the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application. It should be noted that the following embodiments can refer to, draw on or combine with each other, and the same terms, similar features and similar implementation steps in different embodiments will not be described repeatedly.

[0066] Figure 1a is a schematic diagram of an application environment of a digital image recognition method provided by an embodiment of the present invention, see Figure 1a , the application environment may include a terminal 101 and a server 102. The terminal 101 sends the image to be identified to the server 102, and the server 102 determines the image type of the image to be identified; determines multiple first feature points of the image to be identified, and obtains the descriptors corresponding to each first feature point. The server 102 determines the descriptors of multiple second feature points corresponding to the image type of the image to be identified in the preset feature library, matches the descriptors of the multiple first feature points with the descriptors of the multiple second feature points, and uses the first feature points that successfully match as target feature points; the server 102 determines a digital character image from the image to be identified based on the target feature points; and recognizes digital information in the digital character image based on the trained digital image recognition model.

[0067] Understandably, Figure 1a This is an application scenario in an example, and does not limit the application scenario of the image processing method of the present application. In other application scenarios, the terminal may directly process the image to be identified to obtain digital information; or the image acquisition device may acquire the image to be identified and send it to the server for processing to obtain digital image information.

[0068] It can be understood by technicians in this technical field that the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server or server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The terminal can be a smart phone (such as Android phones, iOS phones, etc.), a tablet computer, a laptop computer, a digital broadcast receiver, a MID (Mobile Internet Devices), a PDA (personal digital assistant), a desktop computer, a smart home appliance, a vehicle-mounted terminal (such as a vehicle-mounted navigation terminal, a vehicle-mounted computer, etc.), a smart speaker, a smart watch, etc. The terminal and the server can be directly or indirectly connected by wired or wireless communication, but are not limited to this. The specific can also be determined based on the actual application scenario requirements, and is not limited here.

[0069] The image processing method provided in the embodiment of the present application can be applied in a terminal or in a server.

[0070] The present application provides a method for recognizing a digital image, taking the execution subject as a server as an example. Figure 1b As shown, the method includes:

[0071] Step S101, obtaining an image to be identified and determining the image type of the image to be identified.

[0072] In an embodiment of the present application, the image to be identified may be a network image or a local image, or may be an image acquired in real time by an image acquisition device, such as a camera, and the present application does not impose any limitation thereto.

[0073] Specifically, the server may first extract a first image feature of the image to be identified, and then determine the image type according to the first image feature.

[0074] In the specific implementation process, the server can extract the first image feature of the image to be identified through a pre-trained convolutional neural network model. The first image feature can be used to describe the characteristics of the image to be identified, including but not limited to brightness, edge, texture, color, etc. The first image feature can be expressed in the form of a feature vector.

[0075] Among them, the algorithms that can be used in the convolutional neural network model include but are not limited to the VGGNet algorithm and the Resnet algorithm.

[0076] In the embodiment of the present application, the image type of the image to be identified can be determined according to the preset template library and the first image feature. The configuration method of the template library will be described below.

[0077] Specifically, before processing the image to be recognized, a template library and a corresponding feature library may be pre-established for different types of digital images.

[0078] Among them, the feature library corresponding to each image type can be established in the following ways:

[0079] (1) Obtain at least one template image corresponding to each image type. For example, in the application scenario of digital display meter recognition, the image type can be determined according to the digital display meter type, and a digital display meter image corresponding to each digital display meter type is obtained as the template image corresponding to the type.

[0080] (2) Determine the second feature points of each template image based on a feature point detection algorithm, and obtain the descriptors corresponding to each second feature point.

[0081] The preset feature point detection algorithm can be a scale-invariant feature transform (SIFT) algorithm, which is used to detect and describe local features in the image. In the present application, it can be used to determine the first feature point of the image to be identified and the second feature point of each template image, as well as the descriptor of each feature point.

[0082] Among them, the first feature point is a coordinate point in the image to be detected, which is a key point in the image to be identified, and the first feature descriptor is a high-dimensional vector corresponding to the coordinate point in the image to be detected. Similarly, the second feature point is a coordinate point of the template image, and the descriptor of the second feature point is a high-dimensional vector corresponding to the coordinate point in the template image. As an example, the method for feature point detection using the SIFT algorithm may include the following steps:

[0083] ① Extreme value detection in scale space: Search images in all scale spaces and identify potential candidate points in the image that are invariant to scale and rotation through Gaussian differential functions.

[0084] ② Feature point positioning: At the position of each candidate point, a fine fitting model is used to determine the position scale, and the feature points are selected based on their stability.

[0085] ③ Feature direction assignment: Based on the local gradient direction of the image, one or more directions are assigned to each feature point position. All subsequent operations transform the direction, scale and position of the feature points to provide invariance of these features.

[0086] ④ Feature point description: In the neighborhood around each feature point, the local gradient of the image is measured at a selected scale, and these gradients are transformed into a representation that allows relatively large local shape deformations and illumination changes. This representation can be called a descriptor. The purpose of using descriptors for feature description is to describe the feature point with a set of vectors after the feature point is calculated. This description includes not only the feature point, but also the pixels around the feature point that contribute to it. Descriptors can be used as a basis for target matching, and can also make feature points have more invariant characteristics, such as illumination changes, 3D viewpoint changes, etc.

[0087] By performing feature point detection on each template image through the SIFT algorithm, the second feature points corresponding to each template image and the descriptors of each second feature point can be obtained.

[0088] Since each template image corresponds to a different image type, when the type of image to be identified is determined, the second feature point corresponding to the image type can be further determined according to the image type.

[0089] (3) A feature library is established based on the template images corresponding to the image types, the second feature points of the template images, and the descriptors corresponding to the second feature points.

[0090] In the established feature library, each template image and the second feature point of the template image are associated with the corresponding image type, and the second feature point corresponding to the image type can be determined according to the image type.

[0091] The template libraries corresponding to each image type can be established in the following ways:

[0092] (1) The coordinates of the four corner points corresponding to each template image can be determined, and the second image features of each template image can be extracted. The coordinates of the four corner points corresponding to each template image can be the coordinates of the four corner points of the digital area in each template image, which are recorded as x1, y1, x2, y2, x3, y3, x4, y4. The method for extracting the second image features of each template image can be the same as the method for extracting the first image features of the image to be identified. As an example, a deep convolutional neural network model pre-trained on ImageNet can be used for feature extraction. Specifically, each template image can be input into a deep convolutional neural network to obtain a deep feature x=(x1, ..., x p ), where p represents the dimension of the depth feature.

[0093] (2) A template library is established based on each template image, the coordinates of the four corner points of each template image and the second image features.

[0094] In the embodiment of the present application, determining the image type of the image to be identified based on the first image feature may include the following steps:

[0095] (1) Based on a preset similarity algorithm, the similarity between the first image feature and each second image feature in the template library is calculated. Specifically, the Euclidean norm can be used as the similarity criterion to calculate the similarity between the first image feature of the image to be identified and the second image feature of each template image.

[0096] (2) Based on the calculated similarity, a target template image that is most similar to the image to be identified is determined from the template library, and the image type corresponding to the target template image is used as the image type of the image to be identified.

[0097] Among them, the similarities calculated by different similarity calculation methods may correspond to different comparison standards. For example, when the l2 norm is used as the similarity criterion, the smaller the result, the more similar it is; and when the cosine similarity is used, the larger the result, the more similar it is. This application does not impose any restrictions on this, and the template image that is most similar to the image to be identified based on the similarity is used as the target template image.

[0098] Each template image has a corresponding image type. The image type corresponding to the target template image can be used as the image type of the image to be identified, so as to determine the second feature point corresponding to the image to be identified in the feature library according to the image type.

[0099] Step S102: determine a plurality of first feature points of the image to be identified, and obtain a descriptor corresponding to each first feature point.

[0100] The first feature point is a key point in the image to be identified.

[0101] Specifically, the server may perform feature point detection on the image to be identified based on a preset feature point algorithm, for example, using a SIFT algorithm, to obtain first feature points corresponding to the image to be identified and descriptors of each first feature point.

[0102] Step S103, determining descriptors of multiple second feature points corresponding to the image type of the image to be identified in a preset feature library, matching the descriptors of the multiple first feature points with the descriptors of the multiple second feature points, and taking the successfully matched first feature points as target feature points.

[0103] The preset feature library includes each template image and the association relationship between the second feature point of the template image and the corresponding image type. The second feature point and descriptor corresponding to the image type of the image to be identified can be determined based on the association relationship.

[0104] In the embodiment of the present application, matching the descriptors of multiple first feature points with the descriptors of multiple second feature points, and taking the successfully matched first feature points as target feature points, may include the following steps:

[0105] For each second feature point, the distance between the descriptor of each second feature point and the descriptors of all the first feature points is calculated. If the ratio of the closest distance to the second closest distance is less than a preset threshold, the descriptor of the first feature point closest to the descriptor of the second feature point is the final matching descriptor, and the corresponding first feature point is the matching point of the second feature point.

[0106] As an example, the present application may use a neighbor algorithm (KNN, K-NearestNeighbor) to match the descriptors of multiple first feature points with the descriptors of multiple second feature points to obtain the first feature points corresponding to the second feature points.

[0107] The template image includes multiple descriptors (k) of the second feature points, and the image to be identified contains multiple descriptors (d) of the first feature points. Wherein, k and d are both positive integers. The first step is to calculate the distance between the descriptor of the first feature point and the descriptor of the second feature point, and a distance matrix of k rows and d columns can be obtained; the second step is to sort the rows from small to large; the third step, k can be 2, then take the first two of each row, corresponding to the two descriptors of the image to be identified; the fourth step, if the ratio of the first value and the second value taken from each row is less than the preset threshold, then take the first value as the final descriptor matched by a feature descriptor of the template image, that is, the descriptor of the first feature point closest to the descriptor of the second feature point is the final matched descriptor, and the first feature point corresponding to the matched descriptor is the matching point of the second feature point; otherwise, a feature descriptor of a template image may not match the feature descriptor of the image to be identified.

[0108] The descriptor of the match after filtering obtained according to the above steps is smaller than the descriptor of the second feature point on the initial template image.

[0109] According to the above method, the target feature points corresponding to the second feature points can be determined.

[0110] Step S104, determining a digital character image from the image to be recognized based on the target feature points; and recognizing digital information in the digital character image based on the trained digital image recognition model.

[0111] Specifically, the server may determine a digital character image from the image to be recognized based on the transformation relationship between the target feature point and the corresponding second feature point, where the digital character image is an image that includes a digital region in the image to be recognized.

[0112] The digital character image can be input into the trained digital image recognition model to obtain the digital information of the image to be recognized.

[0113] In an embodiment of the present application, recognizing digital information in a digital character image based on a trained digital image recognition model may include the following steps: inputting the digital character image into the digital image recognition model to obtain a digital recognition result, which includes the digital information in the digital character image.

[0114] The above-mentioned digital image recognition method includes: determining the image type of the image to be recognized, then determining the first feature point of the image to be recognized, and obtaining the descriptor corresponding to each first feature point. Determining the descriptor of the second feature point corresponding to the image type of the image to be recognized in the feature library, determining the target feature point by matching the descriptor of the first feature point with the descriptor of the second feature point, and determining the digital character image from the image to be recognized based on the target feature point, and finally recognizing the digital information in the digital character image through the digital image recognition model, so as to realize the rapid and accurate positioning of the digital area based on the feature points of the image to be recognized, and improve the accuracy of digital character recognition.

[0115] In an embodiment of the present application, a possible implementation method is provided, which can include the following steps to determine a digital character image from an image to be recognized based on target feature points:

[0116] (1) Obtain the corner point coordinates of the target template image corresponding to the image to be recognized from the template library.

[0117] The figure corresponding to the target template image may have multiple corner points. For example, if the figure corresponding to the target template image is a quadrilateral, it may have four corner point coordinates.

[0118] Among them, the multiple second feature points corresponding to the image type of the image to be identified are feature points of the target template image corresponding to the image to be identified.

[0119] Specifically, the image type of the image to be identified is consistent with the image type of the target template image, and there is a corresponding relationship between the second feature point of the target template image and the image type of the target template image. Therefore, the second feature point corresponding to the image type of the image to be identified is the feature point of the target template image.

[0120] The template library may include the coordinates of the corner points of each template image, and the coordinates of the corner points of the target template image may be determined from the template library.

[0121] (2) Obtaining the transformation relationship between the second feature point and the target feature point. The aforementioned steps can determine the target feature points corresponding to each second feature point, and then determine the transformation relationship between each second feature point and the corresponding target feature point. The transformation relationship can describe the correspondence between the second feature point and the corresponding target feature point.

[0122] (3) Obtain the coordinates of the corner points of the image to be identified based on the transformed relationship and the coordinates of the corner points of the target template image.

[0123] Taking the target template image as an example, the four corner point coordinates include four. Assuming that the transformation relationship is expressed as H, the four corner point coordinates of the target template image are written as homogeneous coordinates q i =(x i ,t i ,1) T , where i = 1, 2, 3, 4; according to the transformation relationship, the method for calculating the coordinates of the corner points of the image to be identified can be:

[0124] According to formula 1, the homogeneous coordinates q′ of the image to be identified are calculated i =(x′ i ,y′ i ,z i ) T , where i=1,2,3,4.

[0125] q′ i =H*q i (Formula 1)

[0126] Then the homogeneous coordinates of the image to be identified can be converted into corner coordinates. Specifically, (x′ i / z i ,y′ i / z i ) Get the corner coordinates of the image to be recognized Among them, i=1,2,3,4.

[0127] (4) Determine a digital character image from the image to be recognized based on the coordinates of the corner points of the image to be recognized.

[0128] In an embodiment of the present application, a possible implementation method is provided, which can include the following steps to determine a digital character image from an image to be recognized based on the coordinates of the corner points of the image to be recognized:

[0129] (1) According to the coordinates of the corner points of the image to be identified, a digital region image is cropped from the image to be identified.

[0130] (2) The digital region image is corrected through a perspective transformation operation to obtain a corrected digital region image.

[0131] (3) Preprocessing the rectified digital region image, and segmenting the digital character image from the preprocessed digital region image by a vertical projection method. The preprocessing process may include but is not limited to: first grayscale processing the rectified digital region image, then using Gaussian filtering to reduce the influence of noise, then using the opening and closing operations in morphological operations to remove impurities outside and inside the digital region image, and finally performing adaptive binarization processing on the digital region image to obtain the preprocessed digital region image.

[0132] The digital character image is segmented from the preprocessed digital region image using a vertical projection method.

[0133] Based on a preset digital image data set, a deep convolutional neural network model for digital image classification can be trained, the segmented digital character image is input into the convolutional neural network model, and the corresponding digital recognition result is output. The digital recognition result includes the digital information in the digital character image.

[0134] The digital image recognition method provided in the present application can be applied to recognize the readings of a digital display meter to enhance the computer's real-time detection and analysis capabilities for the digital display meter. Specifically, Figure 2a As shown, the following steps are included:

[0135] Step S201, establishing a digital display meter template library and a corresponding digital display meter feature library for different types of digital display meters.

[0136] DROM is the abbreviation of digital display instrument. Due to its visual intuitiveness, convenient setting and high degree of intelligent control, and with the rapid development of current sensor technology, the functions of digital display instruments are becoming increasingly powerful and the accuracy is constantly improving. Digital display instruments are replacing traditional mechanical instruments in large quantities and have been widely used in all walks of life and daily life.

[0137] Digital display meters can cover a variety of types, and the classification method can be based on function classification, such as water meters, ammeters, voltmeters, and steam meters, etc. This application does not limit this. Different types of digital display meters may have different digital display forms and styles. Therefore, a digital display meter template library and a corresponding digital display meter feature library can be established for different types of digital display meters.

[0138] Step S201 may specifically include the following sub-steps:

[0139] Step S2011, taking a certain type of water meter as an example, given an image of the water meter of this type, mark the four corner points of the digital area clockwise, such as Figure 2b shown.

[0140] Step S2012: Detect feature points based on the SIFT algorithm on the water meter image and calculate M sift feature descriptors (for example, M=128). The sift feature descriptor has excellent properties such as rotation invariance, scale invariance, and good invariance to projection transformation. A feature library can be constructed based on the sift feature descriptors.

[0141] Step S2013, using a pre-trained model, such as resnet34, to extract the water meter image feature x = (x1, ..., x p ), p represents the dimension of feature extraction (e.g. p = 1000). The ResNet series of models have achieved high accuracy in image classification tasks and are often used as the backbone of target detection frameworks. Therefore, the ResNet series of models can be used for feature extraction.

[0142] Step S2014, establishing a template library and a feature library, wherein the template library may contain image features and corner point coordinates, and the feature library may contain M sift feature descriptors (eg, M=128).

[0143] Step S202: acquiring an image to be identified, extracting image features of the image to be identified, and determining the image type of the image to be identified based on the image features of the image to be identified.

[0144] Specifically, step S202 may include the following sub-steps:

[0145] Step S2021, obtaining an image to be identified through camera acquisition, and the image to be identified may specifically be an image of a digital display meter to be identified.

[0146] Step S2022, using a pre-trained model, such as resnet34, to extract the test image feature y = (y1, ..., y p ), p represents the dimension of the extracted features. The pre-trained model used here can be consistent with the pre-trained model used in S2013. Different pre-trained models will extract different types of features. Therefore, using differentiated pre-trained models will bring uncertainty factors to template matching.

[0147] Step S2023, using a preset similarity algorithm, calculates the similarity between the image features of the digital display meter to be identified and the image features in the template library, and determines the type of digital display meter corresponding to the digital display meter image to be identified.

[0148] When the preset similarity algorithm uses the l2 norm as the similarity criterion, the smaller the result, the more similar it is. The l2 norm expression is: where x=(x1,...,x p ) represents the image feature of a template, y=(y1,...,y p) represents the feature of the test image. Different similarity criteria can be used in this step, such as cosine similarity, l1 norm, etc., which are not limited in this application.

[0149] Step S203: perform feature point matching to obtain the coordinates of the corner points of the image to be identified.

[0150] Specifically, step S203 may include the following sub-steps:

[0151] Step S2031, determining feature points of the digital display meter image to be identified based on a preset feature point detection algorithm, and obtaining N sift feature descriptors (for example, N=M or N=200) corresponding to each feature point.

[0152] Step S2032, matching the SIFT feature descriptors of the digital display meter image to be identified and the template image.

[0153] Specifically, the KNN method can be used for matching, K can be 2, and then the matched sift feature descriptors are filtered according to certain criteria to obtain N finally matched sift feature descriptors. The filtering criteria can be: using the KNN method, each sift feature descriptor of the template image can be matched to the K sift feature descriptors on the digital display image to be identified. If the distance ratio of these K sift feature descriptors to the sift feature descriptors of the template image is less than a preset distance threshold, then the sift feature descriptor with the smallest distance from the sift feature descriptor of the digital display image to be identified to the sift feature descriptor of the template image is retained. The present application can use an approximate KNN matching algorithm, which can improve the generalization performance compared to the best matching algorithm, making the technical solution in the present application more suitable for complex scenes.

[0154] Step S204, according to the coordinates of the corner points of the digital display image to be identified (the image to be identified), a digital region image is cut out from the image to be identified; and the digital region image is corrected by a perspective transformation operation to obtain a corrected digital region image.

[0155] Step S204 may specifically include the following sub-steps:

[0156] Step S2041, according to the corner point coordinates obtained in the previous step, a digital area image in the digital display meter image to be identified is cut out, and the digital area image may be a rectangle;

[0157] Step S2042, using perspective transformation operation to perform regular correction on the digital region image, the correction example is as follows: Figure 2c Correcting the digital area map is helpful to improve the accuracy of digital segmentation and recognition.

[0158] Step S205, identifying the numeric characters in the corrected numeric region image.

[0159] Specifically, step S205 may include the following sub-steps:

[0160] Step S2051, use the collected digital display meter data set to build a training set and a validation set of the digital image recognition model, and pre-train a 20-category resnet50 model for digital image recognition.

[0161] Step S2052, obtain the width w of the corrected digital region image, then accurately calculate the left and right boundary indexes of each number based on the number of numbers in the digital region, and crop to obtain a single number based on the boundary index. Figure 2d shown.

[0162] Step S2053, input the single number obtained in the above step into the pre-trained resnet50 model, and output the recognition result. Figure 2e shown.

[0163] The present application embodiment provides a digital image recognition device, such as Figure 3 As shown, the digital image recognition device 30 may include: a type determination module 301, a feature point detection module 302, a feature point matching module 303 and a digital recognition module 304, wherein:

[0164] The type determination module 301 is used to obtain an image to be identified and determine the image type of the image to be identified;

[0165] The feature point detection module 302 is used to determine a plurality of first feature points of the image to be identified and obtain a descriptor corresponding to each first feature point; wherein the first feature point is a key point in the image to be identified;

[0166] A feature point matching module 303 is used to determine the descriptors of multiple second feature points corresponding to the image type of the image to be identified in a preset feature library, match the descriptors of the multiple first feature points with the descriptors of the multiple second feature points, and use the first feature points that are successfully matched as target feature points;

[0167] The digital recognition module 304 is used to determine the digital character image from the image to be recognized based on the target feature points; and recognize the digital information in the digital character image based on the trained digital image recognition model.

[0168] The above-mentioned digital image recognition device includes: determining the image type of the image to be recognized, then determining the first feature point of the image to be recognized, and obtaining the descriptor corresponding to each first feature point. Determining the descriptor of the second feature point corresponding to the image type of the image to be recognized in the feature library, determining the target feature point by matching the descriptor of the first feature point with the descriptor of the second feature point, and determining the digital character image from the image to be recognized based on the target feature point, and finally recognizing the digital information in the digital character image through the digital image recognition model, so as to realize the rapid and accurate positioning of the digital area based on the feature points of the image to be recognized, and improve the accuracy of digital character recognition.

[0169] In the embodiment of the present application, a feature library establishment module is also included, which is specifically used to:

[0170] Obtain at least one template image corresponding to each image type;

[0171] Determine the second feature points of each template image based on a feature point detection algorithm, and obtain the descriptors corresponding to each second feature point;

[0172] A feature library is established based on the template images corresponding to the respective image types, the second feature points of the respective template images, and the descriptors corresponding to the respective second feature points.

[0173] In the embodiment of the present application, a template library establishment module is also included, which is specifically used to:

[0174] Determine the corner point coordinates of the graphics corresponding to each template image, and extract the second image features of each template image; establish a template library based on each template image, the corner point coordinates of each template image, and the second image features;

[0175] When the type determination module determines the image type of the image to be identified based on the first image feature, it is specifically used to:

[0176] Based on a preset similarity algorithm, calculating the similarity between the first image feature and each second image feature in the template library;

[0177] Based on the calculated similarity, a target template image that is most similar to the image to be identified is determined from the template library, and the image type corresponding to the target template image is used as the image type of the image to be identified.

[0178] In the embodiment of the present application, when the feature point matching module matches the descriptors of the plurality of first feature points with the descriptors of the plurality of second feature points and takes the first feature point that has been successfully matched as the target feature point, it is specifically used to:

[0179] Matching the descriptors of the plurality of first feature points with the descriptors of the plurality of second feature points to determine the first feature points that match the respective second feature points;

[0180] For each second feature point, the distance between the descriptor of the second feature point and the descriptor of at least one matching first feature point is calculated, and the first feature point corresponding to the distance less than a preset distance threshold is used as a candidate feature point, and the target feature point with the smallest distance to the second feature point is determined from the candidate feature points.

[0181] In the embodiment of the present application, when the digital recognition module determines the digital character image from the image to be recognized based on the target feature points, it is specifically used to:

[0182] Acquire the corner point coordinates of the target template image corresponding to the image to be identified from the template library; the plurality of second feature points corresponding to the image type of the image to be identified are the feature points of the target template image corresponding to the image to be identified;

[0183] Obtaining a transformation relationship between the second feature point and the target feature point;

[0184] Obtaining the coordinates of the corner points of the image to be identified based on the transformed relationship and the coordinates of the corner points of the target template image;

[0185] A digital character image is determined from the image to be recognized based on the coordinates of the corner points of the image to be recognized.

[0186] In the embodiment of the present application, when the digital recognition module determines the digital character image from the image to be recognized based on the corner point coordinates of the image to be recognized, it is specifically used to:

[0187] According to the coordinates of the corner points of the image to be identified, a digital region image is cut out from the image to be identified;

[0188] Correcting the digital region image through a perspective transformation operation to obtain a corrected digital region image;

[0189] The rectified digital region image is preprocessed, and the digital character image is segmented from the preprocessed digital region image by a vertical projection method.

[0190] In the embodiment of the present application, when the digital recognition module recognizes digital information in a digital character image based on a trained digital image recognition model, it is specifically used to:

[0191] The digital character image is input into the digital image recognition model to obtain a digital recognition result, which includes digital information in the digital character image.

[0192] The device of the embodiments of the present application can execute the method provided by the embodiments of the present application, and the implementation principles are similar. The actions performed by each module in the device of each embodiment of the present application correspond to the steps in the method of each embodiment of the present application. For the detailed functional description of each module of the device, please refer to the description in the corresponding method shown in the previous text, which will not be repeated here.

[0193] An electronic device (computer device / equipment / system) is provided in an embodiment of the present application, including a memory, a processor and a computer program stored in the memory. The processor executes the above-mentioned computer program to implement the steps of the digital image recognition method. Compared with the related art, it can achieve: based on the image features and feature points of the image to be recognized, the digital area is quickly and accurately located, thereby improving the accuracy of digital character recognition.

[0194] In an alternative embodiment, an electronic device is provided, such as Figure 4 As shown, Figure 4 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, such as through a bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0195] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0196] The bus 4002 may include a path to transmit information between the above components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0197] The memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation herein.

[0198] The memory 4003 is used to store the computer program for executing the embodiment of the present application, and the execution is controlled by the processor 4001. The processor 4001 is used to execute the computer program stored in the memory 4003 to implement the steps shown in the above method embodiment.

[0199] The electronic devices include, but are not limited to, mobile terminals such as mobile phones, notebook computers, PADs, and fixed terminals such as digital TVs and desktop computers.

[0200] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.

[0201] The embodiment of the present application also provides a computer program product, including a computer program, which can implement the steps and corresponding contents of the aforementioned method embodiment when executed by a processor.

[0202] The terms "first", "second", "third", "fourth", "1", "2", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that shown or described in the drawings.

[0203] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the implementation order of these steps is not limited to the order indicated by the arrows. Unless clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages may be executed at the same time, and each sub-step or stage in these sub-steps or stages may also be executed at different times respectively. In different scenarios of execution time, the execution order of these sub-steps or stages may be flexibly configured according to demand, and the embodiment of the present application does not limit this.

[0204] The above is only an optional implementation method for some implementation scenarios of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the solution of the present application, other similar implementation methods based on the technical ideas of the present application are also within the protection scope of the embodiments of the present application.

Claims

1. A method for recognizing a digital image, characterized in that: include: Acquire an image to be identified, extract a first image feature of the image to be identified, and calculate the similarity between the first image feature and each second image feature in a template library; wherein the template library is established in the following manner: determine the corner point coordinates of a graphic corresponding to each template image, and extract the second image feature of each template image; establish the template library based on each template image, the corner point coordinates of each template image, and the second image feature; Determine a target template image that is most similar to the image to be identified from the template library based on the calculated similarity, and use the image type corresponding to the target template image as the image type of the image to be identified; Determine a plurality of first feature points of the image to be identified, and obtain a descriptor corresponding to each first feature point; wherein the first feature point is a key point in the image to be identified; Determine descriptors of a plurality of second feature points corresponding to the image type of the image to be identified in a preset feature library, match the descriptors of the plurality of first feature points with the descriptors of the plurality of second feature points, and determine the first feature points to which each second feature point matches; For each second feature point, calculate the similarity between the descriptor of the second feature point and the descriptor of at least one matched first feature point, and take the first feature point with a similarity less than a preset threshold as a candidate feature point, and determine the target feature point with the smallest distance from the second feature point from the candidate feature points; wherein the preset feature library is established in the following manner: obtain at least one template image corresponding to each image type; determine the second feature point of each template image based on a feature point detection algorithm, and obtain the descriptor corresponding to each second feature point; establish the feature library based on the template images corresponding to each image type, the second feature points of each template image, and the descriptors corresponding to each second feature point; A digital character image is determined from the image to be recognized based on the target feature points; and digital information in the digital character image is recognized based on a trained digital image recognition model.

2. The digital image recognition method according to claim 1, characterized in that: The step of determining a digital character image from the image to be recognized based on the target feature points includes: Acquire the coordinates of the four corner points of the target template image corresponding to the image to be identified from the template library, and the plurality of second feature points corresponding to the image type of the image to be identified are the feature points of the target template image corresponding to the image to be identified; Acquire a transformation relationship between the second feature point and the target feature point; Acquire the coordinates of the corner points of the image to be recognized based on the transformed relationship and the coordinates of the corner points of the target template image; A digital character image is determined from the image to be recognized based on the corner point coordinates of the image to be recognized.

3. The digital image recognition method according to claim 2, characterized in that: The step of determining a digital character image from the image to be recognized based on the coordinates of the corner points of the image to be recognized comprises: According to the coordinates of the corner points of the graphic corresponding to the image to be identified, cutting out a digital region image from the image to be identified; Correcting the digital region image by a perspective transformation operation to obtain a corrected digital region image; The rectified digital region image is preprocessed, and the digital character image is segmented from the preprocessed digital region image by a vertical projection method.

4. The digital image recognition method according to any one of claims 1 to 3, characterized in that: The step of recognizing digital information in the digital character image based on the trained digital image recognition model includes: The digital character image is input into the digital image recognition model to obtain a digital recognition result, wherein the digital recognition result includes digital information in the digital character image.

5. A digital image recognition device, characterized in that: include: A type determination module, used to obtain an image to be identified, extract a first image feature of the image to be identified, and calculate the similarity between the first image feature and each second image feature in a template library; wherein the template library is established in the following manner: determining the corner point coordinates of the graphics corresponding to each template image, and extracting the second image feature of each template image; establishing the template library based on each template image, the corner point coordinates of each template image, and the second image feature; Determine a target template image that is most similar to the image to be identified from the template library based on the calculated similarity, and use the image type corresponding to the target template image as the image type of the image to be identified; A feature point detection module, used to determine a plurality of first feature points of the image to be identified, and obtain a descriptor corresponding to each first feature point; wherein the first feature point is a key point in the image to be identified; A feature point matching module, used to determine the descriptors of a plurality of second feature points corresponding to the image type of the image to be identified in a preset feature library, match the descriptors of the plurality of first feature points with the descriptors of the plurality of second feature points, and determine the first feature point to which each second feature point matches; For each second feature point, calculate the similarity between the descriptor of the second feature point and the descriptor of at least one matched first feature point, and take the first feature point with a similarity less than a preset threshold as a candidate feature point, and determine the target feature point with the smallest distance from the second feature point from the candidate feature points; wherein the preset feature library is established in the following manner: obtain at least one template image corresponding to each image type; determine the second feature point of each template image based on a feature point detection algorithm, and obtain the descriptor corresponding to each second feature point; establish the feature library based on the template images corresponding to each image type, the second feature points of each template image, and the descriptors corresponding to each second feature point; The digital recognition module is used to determine a digital character image from the image to be recognized based on the target feature points; and recognize digital information in the digital character image based on a trained digital image recognition model.

6. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the digital image recognition method according to any one of claims 1 to 4 are implemented.

Images