Work piece surface character recognition method and device, computer equipment and storage medium

Abstract

The invention relates to a workpiece surface character recognition method and device, computer equipment, a storage medium and a computer program product. The method comprises the steps of obtaining a to-be-recognized image shot for a placement station where a workpiece is located, performing region division on the to-be-recognized image, determining a character region containing character information from the to-be-recognized image, and screening out a non-character region, thereby improving the accuracy of determining the character information. Affine transformation processing is carried out on the area boundary of the character area to obtain the target image area, the posture of the character information is corrected, and the accuracy of determining the character information is improved. And performing character extraction on the target image area to obtain the character information on the surface of the workpiece, and by adopting the method, performing affine transformation processing on the area boundary of the character area to obtain the target image area, the identification error caused by character deformation is reduced, and the accuracy of determining the character information is improved.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method, device, computer equipment, storage medium and computer program product for text recognition on a workpiece surface. Background Art

[0002] In industrial scenarios, image recognition technology is widely used in areas such as product quality inspection in automated production lines and label recognition.

[0003] Traditional image recognition technology faces many challenges when faced with images in complex industrial environments, such as poor image quality, complex backgrounds, and deformed characters. The image information extracted by traditional image recognition technology has low accuracy. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for text recognition on the surface of a workpiece, which can improve the accuracy of the extracted information, in order to solve the above technical problems.

[0005] In the first aspect, the present application provides a method for text recognition on the surface of a workpiece. The method comprises: obtaining an image to be recognized taken at a workstation where the workpiece is placed; dividing the image to be recognized into regions, and determining a text region containing text information from the image to be recognized; performing affine transformation processing on the region boundary of the text region to obtain a target image region; and performing text extraction on the target image region to obtain text information on the surface of the workpiece.

[0006] In one of the embodiments, dividing the image to be identified into regions and determining the text region containing text information from the image to be identified includes: extracting a plurality of basic image features of the image to be identified at a plurality of different scales; fusing the basic image features to obtain image features of the image to be identified; dividing the image to be identified into regions based on the image features and determining the text region containing text information from the image to be identified.

[0007] In one of the embodiments, the method of dividing the image to be identified into regions based on the image features and determining the text region containing text information from the image to be identified includes: inputting the image features into a size prediction sub-model to determine the region center position and region size corresponding to the text information; inputting the image features into an angle prediction sub-model to determine the region angle corresponding to the text information; and determining the text region containing the text information from the image to be identified by combining the region center position, the region size and the region angle.

[0008] In one of the embodiments, performing affine transformation processing on the area boundary of the text area to obtain the target image area includes: determining an affine transformation matrix that matches the text area based on the undulation changes of the workpiece surface where the text area is located; and performing affine transformation processing on the area boundary of the text area according to the affine transformation matrix to obtain the target image area.

[0009] In one of the embodiments, the text extraction of the target image area to obtain text information on the workpiece surface includes: acquiring environmental information of the environment in which the workpiece is located; performing image enhancement processing on the target image area based on image enhancement parameters matched with the environmental information to obtain an enhanced image; and extracting text from the enhanced image to obtain text information on the workpiece surface.

[0010] In one embodiment, the environmental information includes light intensity; the image enhancement processing is performed on the target image area based on the image enhancement parameters matched with the environmental information to obtain an enhanced image, including: when the light intensity does not meet the light intensity condition, determining the histogram equalization processing parameters that match the light intensity; according to the histogram equalization processing parameters, performing histogram equalization processing on the pixel information of the target image area to obtain an enhanced image.

[0011] In a second aspect, the present application also provides a device for identifying text on the surface of a workpiece. The device comprises:

[0012] The acquisition module is used to acquire the image to be identified taken at the placement station where the workpiece is located; the division module is used to divide the image to be identified into regions and determine the text region containing text information from the image to be identified; the transformation module is used to perform affine transformation processing on the region boundary of the text region to obtain the target image region; the extraction module is used to extract text from the target image region to obtain the text information on the surface of the workpiece.

[0013] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0014] Acquire an image to be identified that is taken at the workstation where the workpiece is placed; divide the image to be identified into regions, and determine a text region containing text information from the image to be identified; perform affine transformation processing on the region boundary of the text region to obtain a target image region; perform text extraction on the target image region to obtain text information on the workpiece surface.

[0015] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0016] Acquire an image to be identified that is taken at the workstation where the workpiece is placed; divide the image to be identified into regions, and determine a text region containing text information from the image to be identified; perform affine transformation processing on the region boundary of the text region to obtain a target image region; perform text extraction on the target image region to obtain text information on the workpiece surface.

[0017] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0018] Acquire an image to be identified that is taken at the workstation where the workpiece is placed; divide the image to be identified into regions, and determine a text region containing text information from the image to be identified; perform affine transformation processing on the region boundary of the text region to obtain a target image region; perform text extraction on the target image region to obtain text information on the workpiece surface.

[0019] The above-mentioned method, device, computer equipment, storage medium and computer program product for text recognition on the surface of a workpiece obtain an image to be recognized taken at the workstation where the workpiece is placed, divide the image to be recognized into regions, determine the text area containing text information from the image to be recognized, and filter out non-text areas, thereby improving the accuracy of determining the text information. Affine transformation is performed on the regional boundary of the text area to obtain the target image area, and the posture of the text information is corrected to improve the accuracy of determining the text information. Text is extracted from the target image area to obtain the text information on the surface of the workpiece. The above-mentioned method is used to obtain the target image area by performing affine transformation on the regional boundary of the text area, thereby reducing the recognition error caused by character deformation and improving the accuracy of determining the text information. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A diagram showing an application environment of a method for recognizing text on a workpiece surface in one embodiment;

[0021] Figure 2 A schematic diagram of a flow chart of a method for text recognition on a workpiece surface in one embodiment;

[0022] Figure 3 A schematic diagram of a general industrial OCR detection method in one embodiment;

[0023] Figure 4 is a structural block diagram of a text recognition device for a workpiece surface in one embodiment;

[0024] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0026] The text recognition method on the workpiece surface provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the image acquisition terminal 102 communicates with the server 104 through a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers.

[0027] The server 104 obtains the image to be identified taken at the placement station where the workpiece is located from the image acquisition terminal 102, the server 104 divides the image to be identified into regions, determines the text region containing text information from the image to be identified, the server 104 performs affine transformation processing on the region boundary of the text region to obtain the target image region, and the server 104 extracts text from the target image region to obtain text information on the surface of the workpiece.

[0028] The image acquisition terminal 102 may be a device with image acquisition capability, such as a visual camera or other devices. The server 104 may be implemented as an independent server or a server cluster composed of multiple servers.

[0029] In one embodiment, Figure 2 As shown, a method for text recognition on the surface of a workpiece is provided, and the method is applied to Figure 1 The following is an example of a server in the example, including:

[0030] S202, obtaining an image to be identified that is taken at the placement station where the workpiece is located.

[0031] The workpiece can be a finished product or semi-finished product produced by different product lines in the factory. The workpiece can also be various types of equipment in the factory. In actual situations, in order to identify specific text on the workpiece, for example, the workpiece number of the workpiece and the equipment number on the surface of the equipment. The workpiece (equipment) where the workpiece number and equipment number are located may appear in the camera field of view at different angles, causing the text in the image to be distorted and deformed.

[0032] The placement station refers to a station for placing workpieces, and the placement station can be a station for long-term placement of workpieces, such as a workpiece storage station, a station for placing various equipment, etc. The placement station can also be a station for short-term placement of workpieces, such as a conveyor belt for workpieces, a temporary storage station for workpieces, etc.

[0033] Specifically, the camera is controlled to shoot toward the placement station where the workpiece is located, so as to obtain an image to be identified including the workpiece.

[0034] The image to be identified is an image taken at the workstation where the workpiece is placed. The image to be identified includes the image area to be selected of the workpiece and the background image area that does not include the background.

[0035] Specifically, the server may screen out the background image area in the image to be identified to obtain a candidate image area containing only the workpiece, wherein the candidate image area includes a text-related image area and a non-text-related image area.

[0036] Furthermore, the image to be selected is divided into image regions related to text and image regions not related to text are obtained. Subsequently, further text recognition is performed on the image regions related to text, which can improve the accuracy of determining text information.

[0037] S204, dividing the image to be recognized into regions, and determining a text region containing text information from the image to be recognized.

[0038] Partitioning the image to be recognized into regions means dividing the image to be recognized into a text region containing text information and a background region not containing text information.

[0039] The text information indicates that there is text information in the image area. For example, for an image of a certain device, the image is divided into regions to obtain a text region including a device code and a device model, and a background region not including a device code and a device model.

[0040] Specifically, the methods of dividing the region of the image to be recognized include: clustering division and model-based division.

[0041] Among them, clustering division can be to pre-divide the image to be identified to obtain multiple regions, and then cluster the image features contained in each region to obtain a clustering result of a text region containing text information and a background region not containing text information. Since there is a large difference between the region containing text information and the region not containing text information, that is, there is a large difference between the target and the background, each region can be divided by clustering division to obtain a text region and a background region. Specifically, the server can extract the pixel information of each pixel in the image to be identified, and further calculate the pixel information difference between each pixel and the surrounding pixels, and then, based on the pixel information difference, perform clustering processing on the image to be identified, determine the region boundary with relatively large pixel information difference from the image to be identified, and then obtain the text region represented by the region boundary.

[0042] Among them, the model-based division can be to input the image to be identified into the region division model, divide the image to be identified through the region division model and based on the image features of the image to be identified, and determine the text area containing text information from the image to be identified.

[0043] Specifically, the server inputs the image features of the image to be identified into a pre-trained rotation target prediction module, and outputs the text area containing text information. Before outputting the text area containing text information, it is necessary to determine the area parameters such as the area center position, area size, and area angle of the text area. The area center position represents the center point of the block of text information, the area size represents the area, perimeter, and other geometric parameters of the area with the area center as the center point, and the area angle represents the angle of the area relative to the image coordinate axis.

[0044] S206, performing affine transformation processing on the region boundary of the text region to obtain a target image region.

[0045] The region boundary represents the vertex information or side length information of the text region. Specifically, the region boundary can be represented by the region boundary point. For example, if the text region is a rectangle or other quadrilateral region, the region boundary can be the four vertex information of the quadrilateral, for example, vertex coordinate values.

[0046] The affine transformation process refers to a correction process for image defects such as tilt and rotation. Specifically, the image to be corrected is corrected by controlling the edge points of the image to be corrected, changing the coordinates of the edge points, and obtaining a corrected image.

[0047] Affine transformation processing includes operations such as translation, rotation, and scaling of an image while keeping the basic properties of straight and parallel lines in the image unchanged. The affine transformation matrix is ​​the core tool for implementing these operations. For example, the boundary of the text area is affine transformed by the affine transformation matrix involved in the affine transformation processing to obtain the target image area.

[0048] The affine transformation processing method can be determined according to the concave-convex change of the workpiece surface where the text area is located. For example, for different concave-convex change conditions, the determined affine transformation processing method (affine transformation matrix) is different.

[0049] S208, extracting text from the target image area to obtain text information on the workpiece surface.

[0050] The target image region refers to the region processed by affine transformation. The posture of the text information in the target image region is more standard than the posture of the text information in the text region.

[0051] Specifically, the server can extract text from the target image area based on OCR technology to obtain text information on the surface of the workpiece.

[0052] Specifically, industrial robots are equipped with OCR text recognition systems that can automatically scan and recognize these text information on parts. Through the recognition results, the robot can accurately identify and classify parts to ensure that they are correctly assembled into products, thereby improving production efficiency and product quality.

[0053] Specifically, an OCR model based on deep learning is adopted, such as CRNN (product recursive neural network) or Transformer-based model; a joint text detection and recognition network is constructed, including multiple convolutional layers for feature extraction, text detection branch and text recognition branch; an attention mechanism is introduced to enable the model to automatically focus on the text area in the image and suppress background interference.

[0054] In the above-mentioned method for text recognition on the surface of a workpiece, an image to be recognized taken at the workstation where the workpiece is placed is obtained, the image to be recognized is divided into regions, the text region containing text information is determined from the image to be recognized, and the non-text region is screened out, thereby improving the accuracy of determining the text information. Affine transformation is performed on the region boundary of the text region to obtain the target image region, and the posture of the text information is corrected to improve the accuracy of determining the text information. Text is extracted from the target image region to obtain the text information on the surface of the workpiece. The above-mentioned method is used to obtain the target image region by performing affine transformation on the region boundary of the text region, thereby reducing the recognition error caused by character deformation and improving the accuracy of determining the text information.

[0055] In one embodiment, the image to be identified is divided into regions, and a text region containing text information is determined from the image to be identified, including: extracting multiple basic image features of the image to be identified at multiple different scales, fusing the basic image features to obtain image features of the image to be identified, dividing the image to be identified into regions based on the image features, and determining a text region containing text information from the image to be identified.

[0056] Among them, different scales represent the scales extracted from the image to be identified. Specifically, a feature extraction network with multiple layers of convolution and pooling operations can be constructed to extract image features of different scales. These features capture the detailed information and global information of the image at different levels.

[0057] By extracting image features of different scales, multi-scale image features reflect the detailed and global features of the image to be identified, which facilitates the subsequent fusion of detailed and global features and improves the model's ability to detect objects of different sizes and shapes.

[0058] Among them, the method of fusing the basic image features can be a method of fusing based on a feature fusion layer. Specifically, the basic image features of different scales are fused to obtain the image features of the image to be identified. The feature fusion layer can adopt upsampling and downsampling methods to combine features of different scales in the same dimension.

[0059] There are many ways to divide the image to be identified based on image features. For example, a clustering method is used to divide the image to be identified based on image features, cluster the text-type areas, and determine the text areas containing text information from the image to be identified; cluster the background-type areas, and determine the background areas that do not contain text information from the image to be identified.

[0060] For another example, the image to be identified is input into the region division model by using the model prediction method, and the model outputs the text region containing text information and the background region not containing text information according to the image features. Specifically, the image to be identified is input into the region division model, and the model determines the target detection frame containing text information according to the image features, and further adjusts the size of the target detection frame so that the target detection frame fits the region corresponding to the text information, and then determines the region demarcated by the target detection frame in the image to be identified as the text region containing text information.

[0061] Since the fused features contain multi-level information, dividing the image to be identified into regions based on image features and determining the text region containing text information from the image to be identified is conducive to improving the accuracy of determining the text information.

[0062] In this embodiment, at multiple different scales, multiple basic image features of the image to be identified are extracted respectively, and the basic image features are fused to obtain image features of the image to be identified, and the image to be identified is divided into regions based on the image features, and the text region containing text information is determined from the image to be identified. Since the fused features contain multi-level information, the image to be identified is divided into regions based on the image features, and the text region containing text information is determined from the image to be identified, which is conducive to improving the accuracy of determining the text information.

[0063] In one embodiment, an image to be identified is divided into regions based on image features, and a text region containing text information is determined from the image to be identified, including: inputting image features into a size prediction sub-model to determine a region center position and region size corresponding to the text information, inputting image features into an angle prediction sub-model to determine a region angle corresponding to the text information, and comprehensively considering the region center position, region size and region angle to determine a text region containing text information from the image to be identified.

[0064] Among them, the size prediction sub-model is used to predict the center position of the region corresponding to the text information and the size of the region using the bounding box regression method. The size prediction sub-model can be used to preliminarily predict the center position of the region corresponding to the text information. The center position of the region represents the image coordinates of the center of the region. The size prediction sub-model can also be used to preliminarily predict the size of the region corresponding to the text information. The region size represents the length of the region box corresponding to the text information and the width of the region box. Exemplarily, the region size represents the rectangular length and rectangular width of the region rectangular box containing the text information. The shape of the region is not limited to a rectangle, and can be a triangle or other polygon.

[0065] The role of the angle prediction sub-model is to accurately describe the regional angle corresponding to the text information. The angle prediction sub-model is used to predict the regional angle corresponding to the text information through a regression algorithm. The regional angle represents the angle between the region corresponding to the text information and the coordinate axis of the image to be recognized.

[0066] Among them, the size prediction sub-model and the angle prediction sub-model can both belong to the area division model.

[0067] After obtaining the region center position, region size and region angle, the text region containing text information can actually be determined from the image to be recognized.

[0068] In this embodiment, the image features are input into the size prediction sub-model to determine the area center position and area size corresponding to the text information. The image features are input into the angle prediction sub-model to determine the area angle corresponding to the text information. The area center position, area size and area angle are comprehensively considered to determine the text area containing the text information from the image to be identified. The accuracy of determining the text information is improved through rotation detection.

[0069] In one embodiment, an affine transformation is performed on the region boundary of the text region to obtain a target image region, including: determining an affine transformation matrix that matches the text region based on the undulations of the workpiece surface where the text region is located, and performing an affine transformation on the region boundary of the text region according to the affine transformation matrix to obtain the target image region.

[0070] Among them, since the surface of the workpiece has ups and downs, for example, the workpiece number of a workpiece is printed at the corner of the workpiece surface, the text information in the image to be recognized has certain bending and deformation, etc., and the bending and deformation need to be corrected.

[0071] The undulating changes in the workpiece surface where the text area is located can represent non-flat areas such as rounded corners, curved corners, and defect points on the workpiece surface.

[0072] Specifically, the server can perform affine transformation on these uneven areas. The greater the degree of fluctuation of the workpiece surface where the text area is located, the greater the degree of correction required for the text area. If the degree of fluctuation of the workpiece surface where the text area is located is small or negligible, there is no need to correct the text area, and there is no need to determine the affine transformation matrix.

[0073] Among them, the affine transformation can be represented by a matrix, which defines the linear mapping relationship from the original image coordinates to the target image coordinates. The transformed image coordinates can be calculated from the original image coordinates, including rotation, scaling, translation and shearing operations.

[0074] Specifically, when the fluctuation of the surface of the workpiece where the text area is located reaches a certain level, the server determines the affine transformation matrix for matching the text area according to the fluctuation of the surface of the text area. It should be noted that the affine transformation matrix corresponding to the fluctuation of the surface of the workpiece where the text area is located can be queried in the image processing library by using an image processing library, and then the affine transformation matrix can be used as the affine transformation matrix for matching the text area.

[0075] Among them, the affine transformation matrix is ​​used to perform operations such as translation, rotation, and scaling on the image while keeping the basic properties of straight and parallel lines in the image unchanged. The affine transformation matrix is ​​the core tool for implementing these operations. The affine transformation matrix can be used to perform affine transformation on the region boundary of the text area to obtain the target image area.

[0076] The region boundary of the text region may refer to the frame boundary of the text region, such as the side length of the rectangular frame, and the region boundary may also refer to the frame vertex of the text region, such as the vertex of the rectangular frame. Exemplarily, affine transformation is performed on the four vertices of the text region to obtain the target image region. It can be understood that the posture of the text information in the target image region is a standard posture or a posture close to the standard posture.

[0077] In this embodiment, according to the fluctuation of the workpiece surface where the text area is located, the affine transformation matrix matching the text area is determined, and the regional boundary of the text area is affinely transformed according to the affine transformation matrix to obtain the target image area. The affine transformation can maintain the straight lines and proportional relationships in the image without causing obvious geometric distortion. The computational complexity of the affine transformation is low, and the image can be efficiently geometrically corrected. The affine transformation can be used for geometric correction of various two-dimensional images, including rotation, scaling, translation, and shearing operations. The corrected image has high geometric accuracy, and the character area becomes regular, which is convenient for further text extraction in the target image area, and improves the accuracy of determining text information.

[0078] In one embodiment, text is extracted from a target image area to obtain text information on the workpiece surface, including: obtaining environmental information of the environment in which the workpiece is located, performing image enhancement processing on the target image area based on image enhancement parameters matched with the environmental information to obtain an enhanced image, and extracting text from the enhanced image to obtain text information on the workpiece surface.

[0079] The environment where the workpiece is located refers to the environmental information around the workpiece, and may also be the environmental information where the workpiece is located. The environmental information refers to information such as the light intensity of the environment, the noise level of the environment, the visibility of the environment, and the temperature of the environment.

[0080] The image enhancement parameter represents the direction parameter of image enhancement under different environmental information conditions. For example, when the environmental information is lack of light, the image enhancement parameter can represent the processing parameter in the histogram equalization process. For another example, when the environmental information is low visibility (severe gas pollution affects visibility in the factory), the image enhancement parameter can be the processing parameter in the noise removal process. Specifically, based on different processing parameters, the target image area is subjected to image enhancement processing that matches the processing parameters to obtain an enhanced image.

[0081] Among them, the text information in the enhanced image is better recognized, and text extraction from the enhanced image can obtain more accurate text information on the workpiece surface.

[0082] In this embodiment, environmental information of the environment in which the workpiece is located is obtained, and image enhancement processing is performed on the target image area based on image enhancement parameters that match the environmental information to obtain an enhanced image. The text information in the enhanced image is better recognized, and text extraction is performed on the enhanced image to obtain more accurate text information on the workpiece surface.

[0083] In one embodiment, the environmental information includes light intensity, and image enhancement processing is performed on the target image area based on image enhancement parameters matching the environmental information to obtain an enhanced image, including: when the light intensity does not meet the light intensity condition, determining histogram equalization processing parameters matching the light intensity, and performing histogram equalization processing on pixel information of the target image area according to the histogram equalization processing parameters to obtain an enhanced image.

[0084] The light intensity represents the luminous flux of visible light received per unit area. When the light intensity is lower than the light intensity threshold, it is considered that the light intensity is not sufficient, and the environment is dim and the light is insufficient. At this time, the histogram equalization processing parameters matching the light intensity can be determined.

[0085] It should be noted that histogram equalization is also called histogram equalization, adaptive histogram equalization, etc. It stretches the histogram of the image to make it evenly distributed, thereby improving the contrast and brightness of the image. This method is particularly effective when the lighting conditions are poor.

[0086] The histogram equalization processing parameters can represent the target distribution of each pixel in the histogram of the stretched image.

[0087] Specifically, based on the target distribution, the server performs histogram equalization processing on the pixel information of the target image area to obtain an enhanced image. The histogram equalization processing on the pixel information of the target image area means that the pixel values ​​of each pixel in the target image area are stretched, compressed, etc. to obtain an enhanced image.

[0088] In this embodiment, the environmental information includes light intensity. When the light intensity does not meet the light intensity condition, a histogram equalization processing parameter matching the light intensity is determined, and histogram equalization processing is performed on the pixel information of the target image area according to the histogram equalization processing parameters to obtain an enhanced image. In low light conditions, an enhanced image can still be obtained, thereby improving the accuracy of determining text information under different environmental conditions.

[0089] Traditional OCR detection methods are prone to the following limitations when dealing with complex environments in industrial sites: 1. Unfixed shooting angles: Objects on industrial production lines may appear in the camera's field of view at different angles, causing distortion and deformation of text in the image. 2. Changing lighting conditions: The lighting conditions in industrial environments are complex and changeable, and shadows, reflections, etc. may occur, affecting image quality. 3. Complex background interference: Industrial images usually contain complex background information, which interferes with the detection and recognition process of the OCR model. 4. Complex image preprocessing: A large number of preprocessing steps are required, which increases the complexity of the system. In view of this, in one embodiment, Figure 3 As shown, a general industrial OCR detection method is provided, including:

[0090] The first part, acquiring the image, includes:

[0091] S302, obtaining an image to be identified that is taken at the placement station where the workpiece is located.

[0092] The second part, the application of the rotation target detection model, includes:

[0093] S304: extracting a plurality of basic image features of the image to be identified at a plurality of different scales respectively.

[0094] S306, fusing the basic image features to obtain image features of the image to be identified.

[0095] S308, inputting the image features into the size prediction sub-model to determine the center position and size of the region corresponding to the text information.

[0096] S310, inputting the image features into the angle prediction sub-model to determine the area angle corresponding to the text information.

[0097] S312, comprehensively considering the center position of the region, the size of the region and the angle of the region, and determining a text region containing text information from the image to be recognized.

[0098] Among them, the training part of the rotation target detection model includes:

[0099] The data set collection includes: (1) collecting image data containing various industrial scenes. These images should cover various possible complex backgrounds, different lighting conditions, and multiple character types as much as possible. (2) annotating the images and determining the rotation target detection box in each image, that is, the four vertex coordinates of the character area to be detected. (3) using data enhancement technology to simulate various environmental changes and improve the robustness of the model. Data enhancement methods include rotation, scaling, cropping, noise addition, etc.

[0100] The model architecture includes: (1) Input layer. The size of the input image is usually fixed (such as 512x512 or other suitable sizes) to facilitate model processing and feature extraction. (2) Feature extraction network includes: Convolution layer: Use multiple convolution layers to extract image features. Each convolution layer contains several convolution kernels to detect different features in the image. The size and number of convolution kernels are designed according to the specific application, usually starting with a smaller convolution kernel, such as 3x3, and then gradually increasing. Pooling layer: After the convolution layer, a pooling layer (such as maximum pooling or average pooling) is added to reduce the dimension of the feature map while retaining important features. The pooling layer can reduce the amount of calculation and improve the computational efficiency of the model. (3) Multi-scale feature fusion, including: Multi-scale feature extraction: Construct a feature extraction network with multiple layers of convolution and pooling operations to extract image features of different scales. These features capture the details and global information of the image at different levels. Feature fusion layer: Fusion features of different scales. The feature fusion layer can use upsampling and downsampling methods to combine features of different scales on the same dimension. The fused features contain multi-level information, which improves the model's ability to detect objects of different sizes and shapes. (4) Attention mechanism, including: Attention module: The attention mechanism is introduced to enable the model to automatically focus on the target detection box area and suppress background interference. The attention mechanism dynamically adjusts the feature value of each position by calculating the importance weight of each position in the feature map. Spatial attention mechanism: Focuses on the spatial features of the image, emphasizes the target area and suppresses background noise by calculating the weight of each position in the feature map. Channel attention mechanism: Focuses on the channel features of the feature map, and enhances important feature channels and suppresses unimportant channels by calculating the weight of each channel. (5) Rotated target detection box prediction, including: Bounding box regression: The bounding box regression method is used to predict the coordinates of the four vertices of the target detection box. Bounding box regression directly predicts the coordinate values ​​of the target detection box through the regression model. Rotation angle prediction: In order to accurately describe the rotation angle of the rotated target detection box, a rotation angle prediction module is added. This module predicts the rotation angle of the target detection box through the regression model. (6) Loss function, including: Position loss: Calculates the error between the predicted vertex coordinates of the target detection box and the true coordinates. Position loss usually uses L2 loss or smooth L1 loss. Angle loss: Calculate the error between the predicted rotation angle and the true angle. Angle loss usually uses L2 loss. Total loss: The weighted sum of position loss and angle loss. The model parameters are continuously optimized through the back propagation algorithm until the loss converges.

[0101] The training process includes: (1) using supervised learning methods to train the model. (2) using the coordinates of the vertex of the labeled detection box and the rotation angle as supervision signals to calculate the loss between the predicted value and the true value. (3) continuously optimizing the model parameters through the back propagation algorithm until the loss converges.

[0102] Model optimization includes: (1) Hyperparameter adjustment: According to the performance of the model on the training set and the validation set, adjust the model's hyperparameters, such as learning rate, number of convolution kernels, pooling layer size, etc. (2) Regularization technology: Use L2 regularization or Dropout and other technologies to prevent model overfitting and improve the generalization ability of the model. (3) Data enhancement: During the training process, continuously apply data enhancement technology to simulate various environmental changes and improve the robustness of the model.

[0103] The third part, the image correction part based on affine transformation, includes:

[0104] S314, determining an affine transformation matrix matching the text area according to the fluctuations of the workpiece surface where the text area is located.

[0105] S316, performing affine transformation processing on the region boundary of the text region according to the affine transformation matrix to obtain a target image region.

[0106] The fourth part, the text extraction based on OCR, includes:

[0107] S318, obtaining environmental information of the environment in which the workpiece is located.

[0108] S320: When the light intensity does not satisfy the light intensity condition, determine a histogram equalization processing parameter that matches the light intensity.

[0109] The environmental information includes light intensity.

[0110] S322, performing histogram equalization processing on pixel information of the target image area according to the histogram equalization processing parameters to obtain an enhanced image.

[0111] S324, extracting text from the enhanced image to obtain text information on the workpiece surface.

[0112] The OCR text extraction of the target image area may be processed based on an OCR text extraction model. Specifically, the target image area is input into the OCR text extraction model, and text information on the workpiece surface is output.

[0113] Among them, the training process of the OCR text extraction model includes: preparation of the rectified image dataset, OCR model architecture and training process.

[0114] The preparation of the rectified image dataset includes: (1) using the rectified images after affine transformation to construct a new OCR training dataset. (2) annotating the rectified images, including the location and content information of the characters. (3) using data enhancement technology to simulate various environmental changes, such as rotation, scaling, cropping, noise addition, etc.

[0115] The OCR model architecture includes: (1) using an OCR model based on deep learning, such as CRNN (convolutional recurrent neural network) or Transformer-based model. (2) building a joint text detection and recognition network, including multiple convolutional layers for feature extraction, text detection branch and text recognition branch. (3) introducing an attention mechanism so that the model can automatically focus on the text area in the image and suppress background interference.

[0116] The training process includes: (1) using the rectified image dataset to train the OCR model. (2) jointly training the text detection and text recognition tasks, and using a multi-task loss function to optimize the model parameters.

[0117] In this embodiment, the method obtains the image to be identified taken at the workstation where the workpiece is placed, divides the image to be identified into regions, determines the text area containing text information from the image to be identified, and filters out the non-text area, thereby improving the accuracy of determining the text information. Affine transformation is performed on the regional boundary of the text area to obtain the target image area, and the posture of the text information is corrected to improve the accuracy of determining the text information. Text is extracted from the target image area to obtain the text information on the surface of the workpiece. The above method is used to perform affine transformation on the regional boundary of the text area to obtain the target image area, thereby reducing the recognition error caused by character deformation and improving the accuracy of determining the text information. The method, 1. Improves detection accuracy and adaptability. Through multi-scale feature fusion, the model can more accurately detect the target area in the image, especially in the case of character deformation and complex background interference. The adaptive learning strategy enables the model to maintain a high detection accuracy in different industrial environments and enhances the robustness of the system. 2. Improves the accuracy of image correction. Through affine transformation, the character area in the image is corrected to a standard form, reducing the recognition error caused by character deformation and improving the accuracy and robustness of OCR recognition. The manual feature extraction steps are reduced, the image preprocessing process is simplified, and the processing efficiency of the system is improved. 3. Improve OCR recognition accuracy. The deep learning model can automatically extract image features. Combined with the attention mechanism, the model can still maintain high recognition accuracy under complex backgrounds and changing lighting conditions. Reduced dependence on image preprocessing, improved the overall processing speed and efficiency of the system. 4. Enhanced system robustness and stability. Phased training enables each model to achieve optimal performance on its respective task, and the overall system performs more stably in complex environments. Model training at each stage is independent of each other, easy to adjust and optimize, and improves the maintainability and scalability of the system.

[0118] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0119] Based on the same inventive concept, the embodiment of the present application also provides a device for recognizing text on the surface of a workpiece for implementing the method for recognizing text on the surface of a workpiece involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the method above, so the specific limitations in the embodiments of the device for recognizing text on the surface of one or more workpieces provided below can refer to the limitations of the method for recognizing text on the surface of a workpiece above, and will not be repeated here.

[0120] In one embodiment, Figure 4 As shown, a device for identifying text on the surface of a workpiece is provided, comprising: an acquisition module 402, a division module 404, a transformation module 406 and an extraction module 408, wherein:

[0121] An acquisition module 402 is used to acquire an image to be identified taken at a placement station where the workpiece is located;

[0122] A division module 404 is used to divide the image to be recognized into regions, and determine a text region containing text information from the image to be recognized;

[0123] The transformation module 406 is used to perform affine transformation processing on the region boundary of the text region to obtain a target image region;

[0124] The extraction module 408 is used to extract text from the target image area to obtain text information on the workpiece surface.

[0125] In one embodiment, the segmentation module 404 is further used to extract multiple basic image features of the image to be identified at multiple different scales; fuse the basic image features to obtain the image features of the image to be identified; divide the image to be identified into regions based on the image features, and determine the text region containing text information from the image to be identified.

[0126] In one embodiment, the division module 404 is also used to input the image features into the size prediction sub-model to determine the area center position and area size corresponding to the text information; input the image features into the angle prediction sub-model to determine the area angle corresponding to the text information; and comprehensively consider the area center position, area size and area angle to determine the text area containing the text information from the image to be identified.

[0127] In one embodiment, the transformation module 406 is further used to determine an affine transformation matrix matching the text area according to the undulations of the workpiece surface where the text area is located; and perform affine transformation processing on the area boundary of the text area according to the affine transformation matrix to obtain a target image area.

[0128] In one embodiment, the extraction module 408 is also used to obtain environmental information of the environment in which the workpiece is located; based on the image enhancement parameters matched with the environmental information, perform image enhancement processing on the target image area to obtain an enhanced image; and perform text extraction on the enhanced image to obtain text information on the surface of the workpiece.

[0129] In one of the embodiments, the environmental information includes light intensity; the extraction module 408 is further used to determine a histogram equalization processing parameter that matches the light intensity when the light intensity does not meet the light intensity condition; according to the histogram equalization processing parameters, the pixel information of the target image area is subjected to histogram equalization processing to obtain an enhanced image.

[0130] Each module in the above-mentioned workpiece surface text recognition device can be fully or partially implemented by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each of the above modules.

[0131] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store image data to be identified. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for text recognition on the surface of a workpiece is implemented.

[0132] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0133] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the above method steps are implemented when the processor executes the computer program.

[0134] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above method steps are implemented.

[0135] In one embodiment, a computer program product is provided, comprising a computer program, which implements the above method steps when executed by a processor.

[0136] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. In addition, the acquisition, storage, processing, and transmission of data are in compliance with relevant laws and regulations.

[0137] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0138] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0139] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for text recognition on a workpiece surface, characterized in that: The method comprises: Acquire an image to be identified taken at the workstation where the workpiece is placed; Dividing the image to be identified into regions, and determining a text region containing text information from the image to be identified; Performing affine transformation processing on the region boundary of the text region to obtain a target image region; Text extraction is performed on the target image area to obtain text information on the workpiece surface.

2. The method according to claim 1, characterized in that The step of dividing the image to be identified into regions and determining a text region containing text information from the image to be identified includes: Extracting a plurality of basic image features of the image to be identified at a plurality of different scales respectively; Fusion of the basic image features to obtain image features of the image to be identified; The image to be identified is divided into regions based on the image features, and a text region containing text information is determined from the image to be identified.

3. The method according to claim 2, characterized in that The step of dividing the image to be identified into regions based on the image features and determining a text region containing text information from the image to be identified includes: Inputting the image features into the size prediction sub-model to determine the center position and size of the region corresponding to the text information; Inputting the image features into an angle prediction sub-model to determine the area angle corresponding to the text information; The text area containing the text information is determined from the image to be recognized by comprehensively considering the center position of the area, the size of the area and the angle of the area.

4. The method according to claim 1, characterized in that: The step of performing affine transformation on the region boundary of the text region to obtain a target image region includes: Determining an affine transformation matrix matching the text area according to the fluctuations of the workpiece surface where the text area is located; According to the affine transformation matrix, an affine transformation process is performed on the region boundary of the text region to obtain a target image region.

5. The method according to claim 1, characterized in that The step of extracting text from the target image area to obtain text information on the workpiece surface includes: Acquiring environmental information of the environment in which the workpiece is located; Based on the image enhancement parameters matched with the environmental information, image enhancement processing is performed on the target image area to obtain an enhanced image; Text extraction is performed on the enhanced image to obtain text information on the workpiece surface.

6. The method according to claim 5, characterized in that The environmental information includes light intensity; The image enhancement parameters matched based on the environmental information are used to perform image enhancement processing on the target image area to obtain an enhanced image, including: When the light intensity does not meet the light intensity condition, determining a histogram equalization processing parameter that matches the light intensity; According to the histogram equalization processing parameters, histogram equalization processing is performed on the pixel information of the target image area to obtain an enhanced image.

7. A device for identifying text on the surface of a workpiece, characterized in that: The device comprises: An acquisition module, used to acquire an image to be identified taken at the placement station where the workpiece is located; A division module, used for dividing the image to be identified into regions, and determining a text region containing text information from the image to be identified; A transformation module, used for performing affine transformation processing on the region boundary of the text region to obtain a target image region; The extraction module is used to extract text from the target image area to obtain text information on the workpiece surface.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images