Nameplate information positioning and identification method based on computer vision and related device

Abstract

The invention relates to the technical field of data processing, and discloses a nameplate information positioning and identification method based on computer vision and a related device, and the method comprises the steps: obtaining nameplate image data containing nameplate information; preprocessing the nameplate image data to obtain preprocessed image data; based on deep learning, an improved YOLOX neural network framework is adopted, and a nameplate information positioning model is constructed and trained; identifying and positioning the preprocessed image data through a nameplate information positioning model to obtain nameplate area image data; and performing cutting and character recognition processing on the nameplate area image data to obtain target nameplate character information. By implementing the method provided by the invention, the accuracy of the nameplate information positioning and character recognition process can be effectively improved, and the purpose of more accurate and more efficient nameplate information positioning and character recognition can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a nameplate information positioning and recognition method based on computer vision and a related device. Background Art

[0002] In modern industrial production and daily life, nameplates, as important identifiers of equipment, products or other objects, carry a lot of key information, such as manufacturer name, model specifications, production date, serial number, etc. This key information is crucial for equipment maintenance, management, tracking and product quality control. However, the traditional way of obtaining nameplate information mainly relies on manual reading and recording, which is not only inefficient but also prone to errors. Especially in the case of a large number of nameplates or nameplate information is vague and the location is hidden, the difficulty and error rate of manual processing will increase significantly.

[0003] With the rapid development of computer vision technology, it has become possible to automatically recognize text in images using image processing and machine learning algorithms. Computer vision technology can simulate human visual perception and analyze and process images efficiently and accurately. In the field of nameplate information recognition, computer vision technology can obtain image data containing nameplate information through image acquisition devices, and use image processing algorithms to pre-process the images to improve image quality and improve the accuracy of text recognition.

[0004] However, the existing nameplate information recognition technology based on computer vision still has some shortcomings. On the one hand, due to the diversity of the position, size, shape and background environment of the nameplate, the precise positioning of the nameplate area becomes a challenging task. Traditional positioning methods often rely on manually designed feature extraction algorithms, which are not effective for nameplate positioning in complex scenarios. On the other hand, the text on the nameplate may be more difficult to recognize due to factors such as font, size, color, wear or pollution, and a more robust text recognition algorithm is needed to deal with it. Therefore, how to improve the accuracy of nameplate information positioning and text recognition process has become a problem that needs to be solved urgently. Summary of the invention

[0005] The present invention provides a nameplate information positioning and recognition method and related devices based on computer vision to solve the technical problem of how to improve the accuracy of nameplate information positioning and text recognition process.

[0006] In a first aspect, a method for positioning and identifying nameplate information based on computer vision is provided, comprising:

[0007] Acquire nameplate image data containing nameplate information;

[0008] Preprocessing the nameplate image data to obtain preprocessed image data;

[0009] Based on deep learning, the improved YOLOX neural network framework is used to build and train the nameplate information positioning model;

[0010] The pre-processed image data is identified and located by the nameplate information positioning model to obtain the nameplate area image data;

[0011] The nameplate area image data is cropped and processed for text recognition to obtain the target nameplate text information.

[0012] In a second aspect, a nameplate information positioning and recognition device based on computer vision is provided, comprising:

[0013] An acquisition module, used for acquiring nameplate image data containing nameplate information;

[0014] A first processing module, used for preprocessing the nameplate image data to obtain preprocessed image data;

[0015] The second processing module is used to construct and train a nameplate information positioning model based on deep learning and using an improved YOLOX neural network framework;

[0016] The third processing module is used to identify and locate the pre-processed image data through the nameplate information positioning model to obtain the nameplate area image data;

[0017] The fourth processing module is used to perform cropping and text recognition processing on the nameplate area image data to obtain the target nameplate text information.

[0018] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the above-mentioned computer vision-based nameplate information positioning and identification method are implemented.

[0019] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned nameplate information positioning and identification method based on computer vision are implemented.

[0020] In the scheme implemented by the above-mentioned computer vision-based nameplate information positioning and recognition method and related devices, by obtaining nameplate image data containing nameplate information, the nameplate image data can be preprocessed to obtain preprocessed image data, and based on deep learning, an improved YOLOX neural network framework is used to construct and train a nameplate information positioning model, so that the preprocessed image data can be identified and positioned through the nameplate information positioning model to obtain nameplate area image data, and then the nameplate area image data can be cropped and text recognition processed to obtain more accurate target nameplate text information, which can improve the accuracy of the nameplate information positioning and text recognition process, and is conducive to achieving more accurate and efficient nameplate information positioning and text recognition purposes. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0022] Figure 1 is a schematic diagram of an application environment of a nameplate information positioning and recognition method based on computer vision in one embodiment of the present invention;

[0023] Figure 2 It is a flow chart of a method for positioning and identifying nameplate information based on computer vision in one embodiment of the present invention;

[0024] Figure 3 It is a structural schematic diagram of a nameplate information positioning and recognition device based on computer vision in one embodiment of the present invention;

[0025] Figure 4 is a schematic diagram of a structure of a computer device in one embodiment of the present invention;

[0026] Figure 5 It is another structural schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] The nameplate information positioning and recognition method based on computer vision provided by the embodiment of the present invention can be applied in the following aspects: Figure 1 In an application environment, the client communicates with the server through a network. Exemplarily, taking the nameplate information positioning and text recognition scenario as an example, the target user (R&D personnel or management personnel) can upload the nameplate image data through the client. Accordingly, the server can obtain the nameplate image data containing the nameplate information through the client, pre-process the nameplate image data to obtain the pre-processed image data, and based on deep learning, use the improved you only look once-X (youonly look once-X, YOLOX) neural network framework to build and train a nameplate information positioning model, so that the pre-processed image data can be identified and positioned through the nameplate information positioning model to obtain the nameplate area image data, and then the nameplate area image data can be cropped and text recognized to obtain more accurate target nameplate text information, and the target nameplate text information is fed back to the client. Correspondingly, the client can receive the target nameplate text information from the server, and can display the target nameplate text information on the client for the target user to query or browse. By adopting the computer vision-based nameplate information positioning and recognition method provided in this application, the accuracy of the nameplate information positioning and text recognition process can be improved, which is conducive to achieving more accurate and efficient nameplate information positioning and text recognition.

[0029] The client may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices. The server may be implemented by an independent server or a server cluster consisting of multiple servers. The present invention is described in detail below through specific embodiments.

[0030] See also Figure 2 As shown, Figure 2 A flowchart of a method for positioning and identifying nameplate information based on computer vision provided by an embodiment of the present invention includes the following steps:

[0031] S10: Acquire nameplate image data including nameplate information.

[0032] The server can obtain images containing nameplate information through appropriate image acquisition devices (such as cameras, etc.). In actual operation, it is possible to ensure that the light and other conditions of the acquisition environment are suitable to ensure that clear, complete and accurate image data that reflects the nameplate information is obtained. For example, in a dark environment, auxiliary lighting equipment may be required to improve the image quality to avoid the nameplate information being blurred due to insufficient light, which affects subsequent processing and recognition.

[0033] S20: Preprocessing the nameplate image data to obtain preprocessed image data.

[0034] The preprocessing step is crucial to improving the accuracy and efficiency of subsequent model processing, and can mainly include image denoising, image enhancement, and image grayscale processing, which are not restricted in this application.

[0035] It should be understood that preprocessing the nameplate image data to obtain preprocessed image data refers to the process of how to perform preprocessing. In step S20, that is, preprocessing the nameplate image data to obtain preprocessed image data, includes the following steps:

[0036] S21: using a Gaussian filtering method to perform denoising on the nameplate image data to obtain denoised image data;

[0037] S22: performing image enhancement processing on the denoised image data to obtain enhanced image data;

[0038] S23: performing image grayscale processing on the enhanced image data to obtain pre-processed image data.

[0039] The server can use a denoising method such as Gaussian filtering to replace the value of each pixel in the image with the weighted average of the pixel values ​​in its neighborhood. Optionally, the server can use the following formula to implement the step of denoising the nameplate image data using Gaussian filtering to obtain denoised image data:

[0040]

[0041] Among them, G(x,y) is the denoised image data, that is, the Gaussian function value at the two-dimensional coordinate (x,y) of the image; (x,y) is the coordinate of the image in the two-dimensional space, x is the horizontal coordinate of the image in the two-dimensional space, and y is the vertical coordinate of the image in the two-dimensional space; σ is the standard deviation of the Gaussian distribution; e is the base of the natural logarithm; π is the number of pi.

[0042] The server can determine the standard deviation σ of the Gaussian function, which determines the smoothness of the Gaussian filter. The larger the σ value, the more obvious the smoothing effect, but it may also cause the loss of image details. The server can multiply the value of each pixel in the neighborhood by its corresponding Gaussian function value, and then sum them to obtain a weighted average as the new value of the current pixel. The server can traverse all pixels in the image and repeat the above capture to complete the denoising of the entire image.

[0043] It should be understood that performing image enhancement processing on the denoised image data to obtain enhanced image data refers to the process of how to perform image enhancement processing. Among them, step S22, that is, performing image enhancement processing on the denoised image data to obtain enhanced image data, includes the following steps:

[0044] S221: using a Sobel detection operator to perform a convolution operation on the denoised image data, calculate the gradient strength and direction of each pixel point, and obtain a pixel gradient strength set and a pixel direction set;

[0045] S222: traverse each pixel point, obtain the gradient intensity and the gradient intensity of the adjacent pixel points in the gradient direction from the pixel gradient intensity set and the pixel direction set, and obtain the local maximum pixel point;

[0046] S223: Determine a strong contour point according to a high threshold condition and the pixel gradient strength set;

[0047] S224: determining weak contour points according to the low threshold condition and the pixel gradient intensity set;

[0048] S225: Determine enhanced image data according to the strong contour point, the weak contour point and the local maximum pixel point.

[0049] The Sobel operator is a discrete differential operator that can be used mainly for edge detection of images. The Sobel operator can calculate the gradient strength and direction of each pixel in the image by performing a convolution operation on the image. The Sobel operator can detect the edge in the image. The gradient strength indicates the strength of the edge, and the gradient direction indicates the direction of the edge.

[0050] The server can adopt non-maximum suppression, such as traversing each pixel in the image and comparing its gradient strength with the gradient strength of the neighboring pixels in its gradient direction. For example, if the gradient strength of the current pixel is not the local maximum, its gradient strength is set to 0 to suppress non-edge pixels.

[0051] Since the gradient strength of pixels usually peaks at the edge of an image, after calculating the gradient strength of each pixel, we can find the local maximum of the edge in the image by traversing each pixel and comparing its gradient strength with that of the adjacent pixels in the gradient direction. For example, for a straight edge, the gradient strength of the pixels on the edge will be greater than the gradient strength of the adjacent pixels on both sides.

[0052] The above method can effectively highlight the edge features in the image. By retaining the pixel points of the above local maximum value, the gradient changes caused by noise or other non-edge factors can be filtered out, so that the real edge is clearer, which can provide more accurate information for subsequent contour extraction and enhancement operations.

[0053] The server can adopt dual threshold detection, such as setting high threshold conditions and low threshold conditions, which is a dual threshold edge detection strategy. Among them, the high threshold condition can be used to determine strong contour points. If the gradient strength of a pixel is higher than the high threshold, the pixel can be considered a strong contour point and is very likely to be located on the real edge of the object. If the gradient strength of a pixel is lower than the high threshold but higher than the low threshold, the pixel can be classified as a weak contour point. The pixel may be part of the edge or may be affected by some noise or other weaker edges. Pixels with gradient strength lower than the low threshold can be judged as non-contour points.

[0054] For strong contour points, they can usually be directly retained or highlighted, such as setting their color to white (in a binary image) or increasing their brightness in the original image. For weak contour points, it can be decided whether to retain or further process them based on their connection with strong contour points. If a weak contour point is adjacent to a strong contour point, the weak contour point may be part of the real edge and can be enhanced to a strong contour point or made more obvious in some way.

[0055] According to the relationship between strong contour points and weak contour points, contours are connected. If a weak contour point is connected to a strong contour point, or is indirectly connected to a strong contour point through other weak contour points, it is retained as a contour point; otherwise, it is considered a non-contour point and removed. In this way, the contour of the image can be enhanced, making the edge of the object (such as the edge of the nameplate) clearer and more prominent in the image, which is helpful for subsequent operations such as positioning and identifying the nameplate area.

[0056] It should be understood that performing image grayscale processing on the enhanced image data to obtain preprocessed image data refers to the process of how to perform image grayscale processing. Among them, step S23, that is, performing image grayscale processing on the enhanced image data to obtain preprocessed image data, includes the following steps:

[0057] S231: traverse each pixel point, obtain the RGB value corresponding to each pixel point, and obtain a pixel RGB value set;

[0058] S232: using a color value conversion method to convert each pixel RGB value in the pixel RGB value set to obtain a pixel grayscale value set;

[0059] S233: applying each pixel grayscale value in the pixel grayscale value set to each pixel to obtain preprocessed image data;

[0060] In a color image, each pixel is represented by the values ​​of the three color channels: red (Red, R), green (Green, G) and blue (Blue, B). These three values ​​together constitute the RGB value of the pixel. By traversing each pixel in the enhanced image data, extracting the corresponding R, G, B channel values, and then collecting the RGB values ​​of all pixels, a pixel RGB value set can be obtained.

[0061] Optionally, the server may implement the step of converting each pixel RGB value in the pixel RGB value set by using the following formula to obtain a pixel grayscale value set:

[0062] Gray=0.3*R+0.59*G+0.11*B

[0063] Among them, Gray is the set of pixel gray values; R is the component of the pixel in the red domain; G is the component of the pixel in the green domain; B is the component of the pixel in the blue domain.

[0064] After obtaining the pixel grayscale value set, these grayscale values ​​can be reassigned to the corresponding pixel points, thereby converting the color enhanced image data into a grayscale image to obtain the preprocessed image data. That is, for each pixel in the image, the grayscale value corresponding to it in the pixel grayscale value set replaces the original RGB value.

[0065] The above-mentioned image grayscale processing can simplify the image data, thereby reducing the amount of calculation for subsequent processing (such as subsequent nameplate information positioning and text recognition, etc.), and in many cases does not affect the processing and recognition of key information in the image (such as nameplate information).

[0066] For example, taking a color image containing nameplate information as an example, first perform image denoising, select Gaussian filtering, set the standard deviation to 1.0, smooth the image, and remove noise interference; then perform image enhancement, use the Sobel detection operator to calculate the gradient strength and direction, set the high threshold to 100 and the low threshold to 50, perform double threshold detection and contour connection, and enhance the nameplate contour in the image; finally, perform image grayscale processing, and use a weighted method to convert the color image into a grayscale image to facilitate subsequent text recognition processing.

[0067] S30: Based on deep learning, the improved YOLOX neural network framework is used to build and train the nameplate information positioning model.

[0068] It should be noted that the improved YOLOX neural network framework has good target detection performance. The nameplate information positioning model constructed in this application may include a deformable multi-branch gradient path module, which may include multiple parallel gradient flow branches, such as a main gradient flow branch and a secondary gradient flow branch.

[0069] Exemplarily, the deformable multi-branch gradient path module in this embodiment includes multiple gradient flow branches arranged in parallel, which work together to extract and fuse feature information at different levels. Specifically, these branches include a main gradient flow branch and several sub-gradient flow branches.

[0070] Among them, the main gradient flow branch is the core part of this module, which is responsible for extracting the main features in the image. The main gradient flow branch can use a deformable residual component to adaptively adjust the convolution kernel structure. Furthermore, the deformable multi-branch gradient path module can splice the output of the main gradient flow branch with the secondary gradient flow branch, and can adjust the number of channels through the convolution layer to fuse the features of multiple branches. Through the above model design, the model's feature extraction ability for small targets at a distance (such as nameplates that may occupy a small proportion of the image) can be enhanced, thereby more accurately locating the nameplate area.

[0071] The sub-gradient flow branch can be used to extract auxiliary features in the image, which are equally important for the location of the nameplate information, but may not be significant enough in the main gradient flow. The number and structure of the sub-gradient flow branches can be flexibly adjusted according to actual application requirements to adapt to feature extraction tasks in different scenarios. Each sub-gradient flow branch contains a series of convolutional layers and pooling layers for gradually extracting and abstracting image features.

[0072] After the main gradient flow branch and the sub-gradient flow branch extract their own features, this module concatenates these features. The concatenation operation can be achieved through a simple channel connection, merging the output feature maps of the main gradient flow branch and the sub-gradient flow branch in the channel dimension. In order to further fuse the features of these multiple branches, this module also sets a convolution layer to adjust the number of channels of the concatenated feature map and deeply fuse the features of different branches through convolution operations. This step ensures that the model fully utilizes the feature information from different branches and improves the richness and robustness of feature representation.

[0073] Exemplarily, taking the nameplate information positioning of a certain model of equipment as an example, the image contains a small nameplate at a distance. When constructing the nameplate information positioning model, the deformable multi-branch gradient path module is embedded in the model. The main gradient flow branch adaptively adjusts the convolution kernel structure through the deformable residual component to capture the subtle features of the nameplate. At the same time, several sub-gradient flow branches are set to extract auxiliary features such as background information and edge information in the image. Finally, the output features of the main gradient flow branch and the sub-gradient flow branch are spliced ​​and fused through the convolution layer. The fused feature map is sent to the subsequent neural network layer for further processing and positioning of the nameplate information. In this way, the deformable multi-branch gradient path module in this embodiment can effectively improve the accuracy and robustness of the nameplate information positioning model.

[0074] After the model is built, a large amount of annotated nameplate image data is needed to train the model so that the model can learn the characteristic patterns of the nameplate and its position rules in different image scenes, so that it can accurately identify and locate the nameplate area in subsequent practical applications.

[0075] S40: using the nameplate information positioning model, identifying and positioning the pre-processed image data to obtain nameplate area image data.

[0076] The server can input the preprocessed image data into the trained nameplate information positioning model. The nameplate information positioning model can identify and locate the nameplate area in the preprocessed image data based on its learned features and rules, and further output relevant information such as the coordinates of the nameplate area in the image, thereby obtaining the nameplate area image data.

[0077] S50: performing cropping and text recognition processing on the nameplate area image data to obtain target nameplate text information.

[0078] The server can crop the sub-image containing the nameplate from the original image according to the coordinates of the nameplate area in the original image to obtain the image data of the nameplate area, and then perform text area preprocessing operations such as image size adjustment, brightness and contrast adjustment, and image binarization on the cropped image in sequence. Finally, optical character recognition (OCR) technology is used to perform text recognition on the preprocessed image to obtain accurate target nameplate text information.

[0079] It should be understood that the process of cropping and text recognition processing the nameplate area image data to obtain the target nameplate text information refers to the process of cropping and text recognition processing. Among them, step S50, that is, cropping and text recognition processing the nameplate area image data to obtain the target nameplate text information, includes the following steps:

[0080] S51: according to the nameplate area image data, cropping the cropped nameplate area sub-image data including the nameplate from the nameplate image data;

[0081] S52: resizing the cropped nameplate region sub-image data to obtain sized nameplate region sub-image data;

[0082] S53: adjusting the brightness and contrast of the size nameplate area sub-image data to obtain brightness nameplate area sub-image data;

[0083] S54: performing image binarization processing on the brightness nameplate region sub-image data to obtain binary nameplate region sub-image data;

[0084] S55: using optical character recognition to perform text recognition processing on the binary nameplate region sub-image data to obtain target nameplate text information.

[0085] The server can crop a sub-image containing the nameplate from the original image (i.e., the nameplate image data) according to the nameplate area coordinates given in the nameplate area image data. The server can obtain the nameplate area coordinates by, for example, processing the original image through a positioning model (such as a target detection model based on deep learning) to output the coordinates of the nameplate area in the image, including the horizontal and vertical coordinates of the upper left corner and the width and height of the nameplate area. It is understandable that the integrity and clarity of the nameplate area must be maintained during the cropping process to ensure that subsequent text recognition is not affected.

[0086] During the text recognition process, the server may first perform further preprocessing on the cropped nameplate image, which may include but is not limited to image size adjustment, brightness contrast adjustment, and image binarization processing.

[0087] Among them, image resizing can be performed based on the input requirements of optical character recognition technology, and after determining the target size of the cropped nameplate image, a bilinear interpolation image scaling algorithm can be further used to scale the cropped nameplate image to the target size. Brightness and contrast adjustment can adjust the brightness and contrast of the image through a suitable algorithm to improve the readability of the text and the recognition accuracy of the OCR technology, and make the text part of the image clearer and more legible.

[0088] Image binarization processing can set a suitable grayscale threshold according to the grayscale distribution of the cropped nameplate image, and further compare each pixel of the cropped nameplate image with the grayscale threshold, so that pixels greater than the threshold are set to white (usually representing the text area), and pixels less than the threshold are set to black (usually representing the background area), thereby converting the image into a binary image with only black and white colors, which is convenient for subsequent OCR processing.

[0089] Furthermore, after completing the above-mentioned text area preprocessing, the OCR technology can be used to perform text recognition on the preprocessed nameplate image. Among them, OCR technology is a technology that converts the text in the image into editable text. It can analyze and recognize the features of the text in the image, match the shape, structure and other information of the text with the predefined character template or the character pattern learned by the machine learning algorithm, so as to determine the text content in the image.

[0090] Different OCR systems may have different recognition accuracy and application scope. In practical applications, it is necessary to select a suitable OCR system according to the specific situation. For example, some OCR systems are better at recognizing printed text, while others are also good at recognizing handwritten text. After the pre-processed nameplate image is input into the selected OCR system, the system will recognize the text in the image and finally output the target nameplate text information, that is, accurately convert the text on the nameplate image into editable text form.

[0091] By cutting the nameplate area image data and performing a series of text recognition processing steps, the target nameplate text information can be effectively extracted from the image containing the nameplate information, thereby achieving accurate acquisition of the nameplate text content.

[0092] This application uses an image acquisition device to automatically acquire nameplate image data, and combines image preprocessing technology to effectively remove noise in the image, enhance image contrast, and make the nameplate area clearer. At the same time, using a nameplate information positioning model based on deep learning, the nameplate area in the image can be accurately and quickly located, avoiding the inefficiency and errors of manual positioning.

[0093] The nameplate information positioning model constructed by this application adopts an improved YOLOX neural network framework and introduces a deformable multi-branch gradient path module, which enables the model to better capture the characteristics of small targets at a distance and enhances the positioning ability of the model in complex backgrounds, different lighting conditions, and nameplates with varying shapes and sizes. Therefore, this application can be widely used in various industrial production and daily life scenarios, effectively solving the limitations of traditional methods in these scenarios.

[0094] In view of the problems that may exist in the font, size, color, wear, pollution, etc. on the nameplate, this application performs cropping and further preprocessing after locating the nameplate area, providing a clearer and more standardized input image for text recognition. Combined with the text recognition algorithm, this application can accurately recognize various text information on the nameplate, and can maintain a high recognition accuracy even in the face of poor quality nameplate images.

[0095] It can be seen that in the above scheme, the server can pre-process the nameplate image data by obtaining the nameplate image data containing the nameplate information to obtain the pre-processed image data, and based on deep learning, use the improved YOLOX neural network framework to build and train a nameplate information positioning model, so that the pre-processed image data can be identified and located through the nameplate information positioning model to obtain the nameplate area image data, and then the nameplate area image data can be cropped and processed for text recognition to obtain more accurate target nameplate text information, which can improve the accuracy of the nameplate information positioning and text recognition process, and is conducive to achieving more accurate and efficient nameplate information positioning and text recognition.

[0096] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0097] In one embodiment, a nameplate information positioning and recognition device based on computer vision is provided, and the nameplate information positioning and recognition device based on computer vision corresponds to the nameplate information positioning and recognition method based on computer vision in the above embodiment. Figure 3 As shown, the nameplate information positioning and identification device based on computer vision includes an acquisition module 101, a determination module 102 and a processing module 103. The detailed description of each functional module is as follows:

[0098] An acquisition module 101 is used to acquire nameplate image data containing nameplate information;

[0099] A first processing module, used for preprocessing the nameplate image data to obtain preprocessed image data;

[0100] The second processing module is used to construct and train a nameplate information positioning model based on deep learning and using an improved YOLOX neural network framework;

[0101] A third processing module, used to identify and locate the pre-processed image data through the nameplate information positioning model to obtain nameplate area image data;

[0102] The fourth processing module is used to perform cropping and text recognition processing on the nameplate area image data to obtain target nameplate text information.

[0103] In one embodiment, the fourth processing module is used to perform cropping and text recognition processing on the nameplate area image data to obtain target nameplate text information, specifically for:

[0104] According to the nameplate area image data, cropping the cropped nameplate area sub-image data including the nameplate from the nameplate image data;

[0105] resizing the cropped nameplate region sub-image data to obtain sized nameplate region sub-image data;

[0106] Adjusting the brightness and contrast of the size nameplate area sub-image data to obtain brightness nameplate area sub-image data;

[0107] Performing image binarization processing on the brightness nameplate region sub-image data to obtain binarized nameplate region sub-image data;

[0108] Optical character recognition is used to perform text recognition processing on the binary nameplate area sub-image data to obtain target nameplate text information.

[0109] In one embodiment, the first processing module is used to preprocess the nameplate image data to obtain preprocessed image data, specifically for:

[0110] Using Gaussian filtering method to perform denoising on the nameplate image data to obtain denoised image data;

[0111] Performing image enhancement processing on the denoised image data to obtain enhanced image data;

[0112] Performing image grayscale processing on the enhanced image data to obtain preprocessed image data;

[0113] The following formula is used to implement the step of denoising the nameplate image data by using the Gaussian filtering method to obtain denoised image data:

[0114]

[0115] Among them, G(x,y) is the denoised image data, that is, the Gaussian function value at the two-dimensional coordinate (x,y) of the image; (x,y) is the coordinate of the image in the two-dimensional space, x is the horizontal coordinate of the image in the two-dimensional space, and y is the vertical coordinate of the image in the two-dimensional space; σ is the standard deviation of the Gaussian distribution; e is the base of the natural logarithm; π is the number of pi.

[0116] In one embodiment, the first processing module is used to perform image enhancement processing on the denoised image data to obtain enhanced image data, specifically for:

[0117] Using a Sobel detection operator, performing a convolution operation on the denoised image data, calculating the gradient strength and direction of each pixel point, and obtaining a pixel gradient strength set and a pixel direction set;

[0118] Traversing each pixel point, obtaining the gradient intensity and the gradient intensity of the adjacent pixel points in the gradient direction from the pixel gradient intensity set and the pixel direction set, and obtaining the local maximum pixel point;

[0119] Determining strong contour points according to a high threshold condition and the pixel gradient strength set;

[0120] Determining weak contour points according to a low threshold condition and the pixel gradient strength set;

[0121] Enhanced image data is determined according to the strong contour point, the weak contour point and the local maximum pixel point.

[0122] In one embodiment, the first processing module is used to perform image grayscale processing on the enhanced image data to obtain pre-processed image data, specifically for:

[0123] Traverse each pixel point, obtain the RGB value corresponding to each pixel point, and obtain the pixel RGB value set;

[0124] Using a color value conversion method, convert each pixel RGB value in the pixel RGB value set to obtain a pixel grayscale value set;

[0125] Apply each pixel gray value in the pixel gray value set to each pixel to obtain preprocessed image data;

[0126] The following formula is used to implement the step of converting each pixel RGB value in the pixel RGB value set by using the color value conversion method to obtain the pixel gray value set:

[0127] Gray=0.3*R+0.59*G+0.11*B

[0128] Among them, Gray is the set of pixel gray values; R is the component of the pixel in the red domain; G is the component of the pixel in the green domain; B is the component of the pixel in the blue domain.

[0129] The present invention provides a nameplate information positioning and identification device based on computer vision. By acquiring nameplate image data containing nameplate information, the nameplate image data can be preprocessed to obtain preprocessed image data, and based on deep learning, an improved YOLOX neural network framework is adopted to construct and train a nameplate information positioning model, so that the preprocessed image data can be identified and positioned by the nameplate information positioning model to obtain nameplate area image data, and then the nameplate area image data can be cropped and text recognition processed to obtain more accurate target nameplate text information, which can improve the accuracy of the nameplate information positioning and text recognition process, and is conducive to achieving more accurate and efficient nameplate information positioning and text recognition purposes.

[0130] For the specific definition of the nameplate information positioning and identification device based on computer vision, please refer to the definition of the nameplate information positioning and identification method based on computer vision above, which will not be repeated here. Each module in the above-mentioned nameplate information positioning and identification device based on computer vision can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0131] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client through a network connection. When the computer program is executed by the processor, it realizes the functions or steps of a service-side method for positioning and identifying nameplate information based on computer vision.

[0132] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 5As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, the functions or steps on the client side of a nameplate information positioning and identification method based on computer vision are realized.

[0133] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0134] Acquire nameplate image data containing nameplate information;

[0135] Preprocessing the nameplate image data to obtain preprocessed image data;

[0136] Based on deep learning, the improved YOLOX neural network framework is used to build and train the nameplate information positioning model;

[0137] The pre-processed image data is identified and located by the nameplate information positioning model to obtain nameplate area image data;

[0138] The nameplate area image data is cropped and subjected to text recognition processing to obtain target nameplate text information.

[0139] The present invention provides a computer device, which can pre-process the nameplate image data by acquiring nameplate image data containing nameplate information to obtain pre-processed image data, and based on deep learning, adopt an improved YOLOX neural network framework to construct and train a nameplate information positioning model, so that the pre-processed image data can be identified and positioned by the nameplate information positioning model to obtain nameplate area image data, and then the nameplate area image data can be cropped and text recognition processed to obtain more accurate target nameplate text information, which can improve the accuracy of nameplate information positioning and text recognition processes, and is conducive to achieving more accurate and efficient nameplate information positioning and text recognition purposes.

[0140] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0141] Acquire nameplate image data containing nameplate information;

[0142] Preprocessing the nameplate image data to obtain preprocessed image data;

[0143] Based on deep learning, the improved YOLOX neural network framework is used to build and train the nameplate information positioning model;

[0144] The pre-processed image data is identified and located by the nameplate information positioning model to obtain nameplate area image data;

[0145] The nameplate area image data is cropped and subjected to text recognition processing to obtain target nameplate text information.

[0146] The present invention provides a computer-readable storage medium. By acquiring nameplate image data containing nameplate information, the nameplate image data can be preprocessed to obtain preprocessed image data, and based on deep learning, an improved YOLOX neural network framework is used to construct and train a nameplate information positioning model, so that the preprocessed image data can be identified and positioned by the nameplate information positioning model to obtain nameplate area image data, and then the nameplate area image data can be cropped and text recognition processed to obtain more accurate target nameplate text information, which can improve the accuracy of nameplate information positioning and text recognition processes, and is conducive to achieving more accurate and efficient nameplate information positioning and text recognition purposes.

[0147] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0148] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0149] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0150] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for positioning and identifying nameplate information based on computer vision, characterized in that: The nameplate information positioning and recognition method based on computer vision includes: Acquire nameplate image data containing nameplate information; Preprocessing the nameplate image data to obtain preprocessed image data; Based on deep learning, the improved YOLOX neural network framework is used to build and train the nameplate information positioning model; The pre-processed image data is identified and located by the nameplate information positioning model to obtain nameplate area image data; The nameplate area image data is cropped and subjected to text recognition processing to obtain target nameplate text information.

2. The nameplate information positioning and recognition method based on computer vision according to claim 1 is characterized in that: The step of performing cropping and text recognition processing on the nameplate area image data to obtain target nameplate text information includes: According to the nameplate area image data, cropping the cropped nameplate area sub-image data including the nameplate from the nameplate image data; resizing the cropped nameplate region sub-image data to obtain sized nameplate region sub-image data; Adjusting the brightness and contrast of the size nameplate area sub-image data to obtain brightness nameplate area sub-image data; Performing image binarization processing on the brightness nameplate region sub-image data to obtain binarized nameplate region sub-image data; Optical character recognition is used to perform text recognition processing on the binary nameplate area sub-image data to obtain target nameplate text information.

3. The nameplate information positioning and recognition method based on computer vision according to claim 2 is characterized in that: The preprocessing of the nameplate image data to obtain preprocessed image data includes: Using Gaussian filtering method to perform denoising on the nameplate image data to obtain denoised image data; Performing image enhancement processing on the denoised image data to obtain enhanced image data; Performing image grayscale processing on the enhanced image data to obtain preprocessed image data; The following formula is used to implement the step of denoising the nameplate image data by using the Gaussian filtering method to obtain denoised image data: Among them, G(x,y) is the denoised image data, that is, the Gaussian function value at the two-dimensional coordinate (x,y) of the image; (x,y) is the coordinate of the image in the two-dimensional space, x is the horizontal coordinate of the image in the two-dimensional space, and y is the vertical coordinate of the image in the two-dimensional space; σ is the standard deviation of the Gaussian distribution; e is the base of the natural logarithm; π is the number of pi.

4. The nameplate information positioning and recognition method based on computer vision according to claim 3 is characterized in that: The performing image enhancement processing on the denoised image data to obtain enhanced image data includes: Using a Sobel detection operator, performing a convolution operation on the denoised image data, calculating the gradient strength and direction of each pixel point, and obtaining a pixel gradient strength set and a pixel direction set; Traversing each pixel point, obtaining the gradient intensity and the gradient intensity of the adjacent pixel points in the gradient direction from the pixel gradient intensity set and the pixel direction set, and obtaining the local maximum pixel point; Determining strong contour points according to a high threshold condition and the pixel gradient strength set; Determining weak contour points according to a low threshold condition and the pixel gradient strength set; Enhanced image data is determined according to the strong contour point, the weak contour point and the local maximum pixel point.

5. The nameplate information positioning and recognition method based on computer vision according to any one of claims 1 to 4, characterized in that: Performing image grayscale processing on the enhanced image data to obtain preprocessed image data includes: Traverse each pixel point, obtain the RGB value corresponding to each pixel point, and obtain the pixel RGB value set; Using a color value conversion method, convert each pixel RGB value in the pixel RGB value set to obtain a pixel grayscale value set; Apply each pixel gray value in the pixel gray value set to each pixel to obtain preprocessed image data; The following formula is used to implement the step of converting each pixel RGB value in the pixel RGB value set by using the color value conversion method to obtain the pixel gray value set: Gray=0.3*R+0.59*G+0.11*B Among them, Gray is the set of pixel gray values; R is the component of the pixel in the red domain; G is the component of the pixel in the green domain; B is the component of the pixel in the blue domain.

6. A nameplate information positioning and recognition device based on computer vision, characterized in that: The nameplate information positioning and identification device based on computer vision includes: An acquisition module, used for acquiring nameplate image data containing nameplate information; A first processing module, used for preprocessing the nameplate image data to obtain preprocessed image data; The second processing module is used to construct and train a nameplate information positioning model based on deep learning and using an improved YOLOX neural network framework; A third processing module, used to identify and locate the pre-processed image data through the nameplate information positioning model to obtain nameplate area image data; The fourth processing module is used to perform cropping and text recognition processing on the nameplate area image data to obtain target nameplate text information.

7. The nameplate information positioning and identification device based on computer vision according to claim 6, characterized in that: The fourth processing module is used to perform cropping and text recognition processing on the nameplate area image data to obtain target nameplate text information, specifically for: According to the nameplate area image data, cropping the cropped nameplate area sub-image data including the nameplate from the nameplate image data; resizing the cropped nameplate region sub-image data to obtain sized nameplate region sub-image data; Adjusting the brightness and contrast of the size nameplate area sub-image data to obtain brightness nameplate area sub-image data; Performing image binarization processing on the brightness nameplate region sub-image data to obtain binarized nameplate region sub-image data; Optical character recognition is used to perform text recognition processing on the binary nameplate area sub-image data to obtain target nameplate text information.

8. The nameplate information positioning and identification device based on computer vision according to claim 7, characterized in that: The first processing module is used to preprocess the nameplate image data to obtain preprocessed image data, specifically for: Using Gaussian filtering method to perform denoising on the nameplate image data to obtain denoised image data; Performing image enhancement processing on the denoised image data to obtain enhanced image data; Performing image grayscale processing on the enhanced image data to obtain preprocessed image data; The following formula is used to implement the step of denoising the nameplate image data by using the Gaussian filtering method to obtain denoised image data: Among them, G(x,y) is the denoised image data, that is, the Gaussian function value at the two-dimensional coordinate (x,y) of the image; (x,y) is the coordinate of the image in the two-dimensional space, x is the horizontal coordinate of the image in the two-dimensional space, and y is the vertical coordinate of the image in the two-dimensional space; σ is the standard deviation of the Gaussian distribution; e is the base of the natural logarithm; π is the number of pi.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the nameplate information positioning and recognition method based on computer vision as described in any one of claims 1 to 5 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for locating and identifying nameplate information based on computer vision as described in any one of claims 1 to 5 is implemented.

Images