A method, device and storage medium for barcode label recognition applied to AOI

Abstract

The present application discloses a method, device and storage medium for barcode label recognition applied to AOI, which is used to detect and decode barcode labels. The present application method includes: obtaining a screen image of a screen with a barcode attached; processing the screen image using a Laplace operator; performing grayscale threshold segmentation on the processed image, and removing the noise on the binary image after grayscale threshold segmentation to obtain a backbone binary image; segmenting each connected area in the backbone binary image into independent sub-connected areas; performing feature recognition on each sub-connected area based on Blob analysis to obtain a result image; performing Hough line detection on the result image to obtain the line direction angle of the barcode; creating a rotation matrix and setting the rotation center point and rotation angle; performing adaptive affine transformation on the result image based on the rotation center point and rotation angle; decoding and identifying the barcode.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method, device and storage medium for barcode label recognition applied to AOI. Background Art

[0002] With the rapid growth in demand for consumer electronic products (such as monitors, laptops and mobile phones) and professional display products (such as commercial displays, car displays, industrial control displays and medical equipment displays), my country's display panel industry has shown an overall growth trend in recent years. Correspondingly, display panel defect detection technology has also received more and more attention. Traditional manual visual inspection methods have been unable to meet the needs of large-scale, high-precision inspection. In order to improve inspection efficiency and accuracy, the application of automatic optical inspection (AOI) and image processing technology in display panel defect detection has become more and more widespread.

[0003] AOI and image processing technology are based on visual image acquisition and analysis, software visualization processing and automated control, and use automated means to achieve efficient and accurate defect detection. Compared with manual inspection, the AOI system can process a large amount of image data in a short time, identify tiny defects, and ensure the quality of display panels.

[0004] During the panel production process, barcode labels are often attached to the panel surface for identification purposes, which interferes with subsequent image analysis. Therefore, accurate detection and shielding of barcode labels has become an important part of image processing. Traditional barcode detection methods have problems such as long detection time, low decoding rate and inaccurate positioning, which limit the overall detection efficiency and accuracy. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides a method, device and storage medium for barcode label recognition applied to AOI.

[0006] The technical solution provided in this application is described below:

[0007] In a first aspect, the present application provides a barcode label recognition method applied to AOI, comprising:

[0008] Acquire a screen image of the screen with the barcode attached;

[0009] Processing the screen image using a Laplacian operator to obtain a prominent barcode edge;

[0010] Perform grayscale threshold segmentation on the processed image, remove the noise on the binary image after grayscale threshold segmentation, retain the connected area of ​​the barcode, and obtain the backbone binary image;

[0011] Segmenting each connected region in the backbone binary image into independent sub-connected regions;

[0012] Based on Blob analysis, feature recognition is performed on each sub-connected region, and the target sub-connected regions that meet the preset feature conditions are retained to obtain a result graph;

[0013] Performing Hough line detection on the result image to obtain the line direction angle of the barcode;

[0014] Create a rotation matrix and set the rotation center point and rotation angle, where the rotation angle is the direction angle of the line;

[0015] performing an adaptive affine transformation on the result image based on a rotation center and the rotation angle;

[0016] Based on the result image after adaptive affine transformation, the barcode is decoded and recognized.

[0017] Optionally, before using the Laplace operator to process the screen image to obtain a prominent barcode edge, the method further includes:

[0018] Determining an image defect type of the screen image, the image defect type comprising: low contrast, blurred image, and dark background with light-colored barcode;

[0019] Based on the determined image defect type, a corresponding pre-processing operation is performed, wherein the pre-processing operation includes shading enhancement, image sharpening, and image inversion.

[0020] Optionally, before using the Laplace operator to process the screen image to obtain a prominent barcode edge, the method further includes:

[0021] Gaussian smoothing is performed on the screen image.

[0022] Optionally, performing an adaptive affine transformation on the result image based on the rotation center and the rotation angle includes:

[0023] Obtaining vertex coordinates of four vertex pixels of the result image, and respectively calculating relative coordinates of the four vertex pixels relative to the rotation center;

[0024] Using the rotation matrix, coordinate transformation is performed on the relative coordinates of each vertex pixel to obtain corresponding intermediate transformation coordinates;

[0025] Based on the coordinates of the rotation center, transform the intermediate conversion coordinates into coordinates of the original coordinate system to obtain new coordinate sets of four vertices;

[0026] A transformation is performed on the result map based on the new set of coordinates.

[0027] Optionally, after obtaining the new coordinate sets of the four vertices, the method further includes:

[0028] Get the minimum value of the transformed rows and columns;

[0029] Determine whether the minimum value is a negative value;

[0030] If it is a negative value, based on the minimum value, the result graph is shifted in the opposite direction along the x-axis and / or the y-axis.

[0031] Optionally, the decoding and identifying of the barcode based on the result image after the adaptive affine transformation includes:

[0032] Inputting the transformed result graph into a pre-built decoding model, wherein the decoding model is pre-configured with decoding parameters;

[0033] Find and read the barcode symbol in the result map using predefined operators and return the decoded data;

[0034] When the returned decoded data indicates that there are multiple decoded characters corresponding to the barcodes, the length of each decoded character is determined, and a decoded character that meets the preset character length is output.

[0035] Optionally, also include:

[0036] Get the center coordinate position, major axis length and minor axis length of the minimum circumscribed rectangular area of ​​the barcode;

[0037] Expand the minimum circumscribed rectangular area based on a predefined major axis expansion ratio and a minor axis expansion ratio to obtain a ROI area;

[0038] Creating a measurement model and adding a rectangular measurement object to the measurement model;

[0039] Setting parameters of the rectangular measurement object;

[0040] Based on the rectangular measurement object, performing measurement on the ROI area and outputting a measurement result;

[0041] Extracting edge point data from the measurement results, and generating an XLD profile based on the edge point data;

[0042] Fitting the XLD profile into a straight line or a plurality of line segments using a fitting algorithm;

[0043] Generate a rectangular area according to the fitted straight line or multiple line segments, wherein the rectangular area represents the position of the positioning barcode label;

[0044] Create a mask based on the rectangular area;

[0045] Apply the mask to the screen image to shield the barcode area.

[0046] Optionally, the light and dark enhancement includes:

[0047] The grayscale of the screen image is adjusted using the following formula:

[0048] g′=g*Mult+Add;

[0049] Among them, g and g′ are the grayscale values ​​of the image before and after the transformation, Mult and Add are weights and biases, which are used to adjust the brightness and contrast of the image.

[0050] The values ​​of Mult and Add are determined by the following formula:

[0051] Mult=255 / (G_max-G_min);

[0052] Add=-Mult*G_min.

[0053] Optionally, the image sharpening includes:

[0054] Processing the screen image using a low-pass filter;

[0055] The sharpened value res of each pixel is calculated by the following formula:

[0056] res=round((orig-mean)*Factor)+orig;

[0057] Among them, orig represents the grayscale value of a pixel in the original image, mean represents the image mean calculated within the filter kernel size range, which represents the average grayscale value of the area around the pixel; Factor represents the contrast intensity, which is used to adjust the degree of sharpening; round represents the rounding function, which is used to ensure that the adjusted pixel value is an integer.

[0058] Optionally, the image inversion includes:

[0059] The grayscale of the screen image is adjusted using the following formula:

[0060] g'=255-g;

[0061] Where g and g′ are the grayscale values ​​of the image before and after the transformation, respectively.

[0062] Optionally, the rotation matrix is ​​expressed as:

[0063] ;

[0064] in, represents the rotation angle, u represents the horizontal coordinate of the new coordinate after rotation; v represents the vertical coordinate of the new coordinate after rotation; 0 and 1 represent constant terms in the homogeneous coordinates; x represents the horizontal coordinate of the original coordinate; y represents the vertical coordinate of the original coordinate.

[0065] Optionally, the step of obtaining the center coordinate position, major axis length, and minor axis length of the minimum circumscribed rectangular area of ​​the barcode includes:

[0066] When decoding and identifying a barcode, the "find_bar_code" operator is used to find and read the barcode symbol in the image. After decoding, the decoded data returned contains the "SymbolRegions" parameter.

[0067] The center coordinate position, major axis length and minor axis length of the minimum circumscribed rectangular area of ​​the barcode are obtained by reading the SymbolRegions parameter.

[0068] A second aspect of the present application provides a device for barcode label recognition applied to AOI, the device comprising:

[0069] Image acquisition unit: acquires the screen image of the screen with the barcode attached;

[0070] A Laplace processing unit, used for processing the screen image using a Laplace operator to obtain a prominent barcode edge;

[0071] A grayscale threshold segmentation unit is used to perform grayscale threshold segmentation on the processed image, remove noise on the binary image after grayscale threshold segmentation, retain the connected area of ​​the barcode, and obtain a backbone binary image;

[0072] A connected region segmentation unit, used for segmenting each connected region in the backbone binary image into independent sub-connected regions;

[0073] A feature recognition unit is used to perform feature recognition on each sub-connected region based on Blob analysis, and retain the target sub-connected region that meets the preset feature conditions to obtain a result map;

[0074] A Hough line detection unit, used for performing Hough line detection on the result image to obtain the line direction angle of the barcode;

[0075] A matrix creation unit, used to create a rotation matrix and set a rotation center point and a rotation angle, wherein the rotation angle is the line direction angle;

[0076] An affine transformation unit, configured to perform an adaptive affine transformation on the result image based on a rotation center and the rotation angle;

[0077] The decoding and recognition unit is used to decode and recognize the barcode based on the result image after the adaptive affine transformation.

[0078] A third aspect of the present application provides a device for barcode label recognition applied to AOI, the device comprising:

[0079] Processor, memory, input-output unit, and bus;

[0080] The processor is connected to the memory, the input and output unit, and the bus;

[0081] The memory stores a program, and the processor calls the program to execute the first aspect and any optional method in the first aspect.

[0082] A fourth aspect of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed on a computer, the program executes the first aspect and any optional method in the first aspect.

[0083] It can be seen from the above technical solutions that this application has the following advantages:

[0084] 1. By using the Laplacian operator to process the screen image, this method can effectively highlight the edge of the barcode, which is helpful for subsequent image processing and recognition. The Laplacian operator can enhance the fast-changing areas in the image, making the edge of the barcode clearer.

[0085] 2. By performing grayscale threshold segmentation on the processed image and removing noise on the binary image, this method can retain the connected areas of the barcode and obtain a backbone binary image. This step helps to improve the accuracy of barcode recognition because the interference of noise and other non-barcode elements is effectively reduced.

[0086] 3. Based on Blob analysis, feature recognition is performed on each sub-connected region, and the target sub-connected regions that meet the preset feature conditions are retained. This method can further purify the barcode area and exclude areas that do not meet the barcode characteristics, thereby improving the accuracy of recognition.

[0087] 4. By performing Hough line detection on the result image, the barcode line direction angle is obtained, and adaptive affine transformation is performed accordingly. This step can correct the direction of the barcode to make it conform to the standard direction for decoding and recognition, which helps to improve the accuracy and efficiency of decoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] In order to more clearly illustrate the technical solution in the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0089] Figure 1 A schematic flow chart of an embodiment of a method for barcode label recognition applied to AOI provided in this application;

[0090] Figure 2 This is a schematic flow chart of another embodiment of a method for barcode label recognition applied to AOI provided in this application;

[0091] Figure 3 A schematic flow chart of an implementation method of performing adaptive affine transformation in the barcode label recognition method applied to AOI provided in this application;

[0092] Figure 4 A flowchart of an implementation method for decoding and identifying a barcode in a barcode label identification method applied to AOI provided in this application;

[0093] Figure 5 This is a schematic flow chart of another embodiment of a method for barcode label recognition applied to AOI provided in the present application;

[0094] Figure 6 This is a schematic diagram of an embodiment of a device for barcode label recognition applied to AOI provided in this application;

[0095] Figure 7 This is a schematic diagram of the structure of another embodiment of a device for barcode label recognition applied to AOI provided in this application. DETAILED DESCRIPTION

[0096] It should be noted that the barcode label recognition method for AOI provided in this application can be applied to a terminal or a system, and can also be applied to a server. For example, the terminal can be a smart phone or a computer, a tablet computer, a smart TV, a smart watch, a portable computer terminal, or a fixed terminal such as a desktop computer. For the convenience of explanation, this application uses the system as an example for the execution subject.

[0097] See also Figure 1 The present application first provides an embodiment of a method for barcode label recognition applied to AOI, the embodiment comprising:

[0098] S101, obtaining a screen image of a screen with a barcode attached thereto;

[0099] In this step, an image acquisition device (such as an industrial camera or scanner) is used to capture the image of the screen. A barcode label has been attached to the screen, and the image acquisition can be performed under uniform lighting conditions to ensure the quality of the image and the clarity of the barcode.

[0100] S102, using a Laplace operator to process the screen image to obtain a prominent barcode edge;

[0101] The acquired screen image is processed using the Laplace operator. The Laplace operator is a second-order differential operator that enhances the edge features of the image by calculating the second-order derivatives of the pixels in the image. The processed image can highlight the edge of the barcode, facilitating subsequent image processing steps.

[0102] The image acquisition device (such as an industrial camera or scanner) reads the screen image.

[0103] Make sure the image is loaded into memory and you can use an image processing library such as OpenCV to read the image.

[0104] The Laplacian operator is a second-order differential operator that can be implemented through convolution operations. Use the cv2.Laplacian function in OpenCV for processing. The Laplacian operator uses a kernel (such as a 3x3 or 5x5 convolution kernel) to calculate the second-order derivative of each pixel in the image and enhance edge features. Before using the Laplacian operator for processing, the image can be smoothed with a Gaussian smoothing filter to reduce the sensitivity of the Laplacian operator operation to noise. The resulting image processed by the Laplacian operator usually contains positive and negative values ​​and needs to be converted into a visual form. Take the absolute value of the image and then convert it to an 8-bit image format for display or subsequent processing. The following is an example to illustrate:

[0105] First, use the cv2.imread function to read the image file. The cv2.IMREAD_GRAYSCALE parameter reads the image as a grayscale image because edge detection works better on grayscale images.

[0106] The image is processed using the cv2.Laplacian function. The parameter cv2.CV_64F specifies that the data type of the output image is a 64-bit floating point number to retain negative values ​​in the calculation.

[0107] The Laplacian operator enhances edge features in an image by calculating the second-order derivative of each pixel.

[0108] The resulting image of the Laplacian operator may contain negative values ​​and values ​​greater than 255, so it needs to be taken in absolute value and then converted to an 8-bit unsigned integer (0 to 255) for display and saving.

[0109] You can try different kernel sizes (such as 3x3 or 5x5) to adjust the effect of edge enhancement. Before applying the Laplacian operator, you can use Gaussian blur (cv2.GaussianBlur) to remove noise in the image and improve the effect of edge detection.

[0110] S103, performing gray threshold segmentation on the processed image, removing noise on the binary image after gray threshold segmentation, retaining the connected area of ​​the barcode, and obtaining a backbone binary image;

[0111] The image processed by Laplacian operator is segmented by grayscale threshold. Grayscale threshold segmentation converts the image into a binary image, in which the barcode part and the background part correspond to different grayscale values. Then, the noise on the binary image is removed by morphological operations (such as opening and closing operations), and only the connected areas of the barcode are retained to generate a backbone binary image. Use a suitable threshold to separate the barcode and the background in the image. Use morphological operations (such as opening and closing operations) to remove noise on the binary image. Opening operation (erosion followed by dilation) is used to remove small noise points, and closing operation (dilation followed by erosion) is used to fill small holes. The connected areas in the image after morphological processing are retained, and these connected areas correspond to the backbone of the barcode.

[0112] The following example illustrates the process of this step:

[0113] First, use the cv2.threshold function to binarize the image. The parameter 127 is the threshold. The pixel values ​​above the threshold are set to 255, and the pixel values ​​below the threshold are set to 0. This method divides the image into foreground (barcode) and background.

[0114] Create a 3x3 rectangular kernel for morphological operations.

[0115] Use cv2.morphologyEx function to perform opening operation to remove small noise in the image. The opening operation first performs erosion operation and then dilation operation.

[0116] Use cv2.morphologyEx function again to perform closing operation to fill the small holes in the image. The closing operation first performs dilation operation and then erosion operation.

[0117] Use the cv2.connectedComponentsWithStats function to find all connected regions in the image and get the statistical information (such as area, bounding rectangle, etc.) of each connected region.

[0118] Filter connected regions based on features such as area and keep only regions that meet the criteria. Keep regions that meet the criteria in the new binary image.

[0119] S104, dividing each connected region in the backbone binary image into independent sub-connected regions;

[0120] In this step, the connected regions in the backbone binary image can be segmented into independent sub-connected regions using a connected region labeling algorithm. Each sub-connected region corresponds to an independent barcode or noise block.

[0121] In this step, the connected region labeling algorithm can be used to label each connected region in the backbone binary image. The attribute information of each connected region, such as area, bounding rectangle, etc., is obtained. According to the labeling results, the backbone binary image is divided into independent sub-connected regions. Each sub-connected region is further processed and analyzed.

[0122] First, you need to choose a suitable connected region marking algorithm. The optional connected region marking algorithms include One-Pass algorithm, Two-Pass algorithm (two-pass scanning method) and Seed Filling algorithm. These algorithms can effectively mark the connected regions in binary images.

[0123] Taking the Two-Pass algorithm as an example, the specific steps are as follows:

[0124] First scan:

[0125] Visit each pixel in the image and mark those points with value 1 (foreground pixels).

[0126] If the neighborhood of the current pixel is all background pixels (value 0), a new label is assigned to the current pixel.

[0127] If there are marked foreground pixels in the neighborhood of the current pixel, the minimum value of the labels in the neighborhood is assigned to the current pixel, and the equality relationship between these labels is recorded.

[0128] Second scan:

[0129] According to the label equality relationship recorded in the first scan, the labels with equality relationship are unified into a minimum label.

[0130] Through this scan, pixels with the same label form a connected region.

[0131] After the connected region labeling algorithm is used, each connected region in the image is assigned a unique label. These connected regions may correspond to independent barcodes or noise blocks.

[0132] After completing the marking of the connected regions, the marked connected regions can be further screened and identified according to the characteristics of the barcode (such as aspect ratio, area, etc.) to distinguish the real barcode region from the noise block.

[0133] Here is a practical example:

[0134] You can use the cv2.connectedComponentsWithStats function to label the connected components of a binary image. This function returns the following:

[0135] num_labels: The number of connected regions, including background regions.

[0136] Labels: Label the image, the value of each pixel represents the label of the connected region to which it belongs.

[0137] stats: Statistics of each connected region, such as upper left corner coordinates, width, height, and area.

[0138] centroids: The centroid coordinates of each connected region.

[0139] The connectivity of the connected area marker is set to 8, indicating that 8 connectivity is used.

[0140] Create a blank image of the same size as the original image to store the resulting image.

[0141] Traverse all connected regions and skip the background region (labeled 0).

[0142] For each connected region, create a mask image, set the pixel values ​​belonging to the connected region to 255, and the remaining pixel values ​​to 0. Use bit operations to copy each connected region to the result image, and the final result image contains all independent sub-connected regions.

[0143] S105, performing feature recognition on each sub-connected region based on Blob analysis, and retaining target sub-connected regions that meet preset feature conditions to obtain a result graph;

[0144] In this step, each independent sub-connected region will be subjected to Blob analysis to extract features such as the shape, size, and position of the region. Blob analysis is an image processing technique used to detect and analyze connected regions (or "blobs") in an image. Blob analysis can be used to extract specific features in an image, such as the shape, size, and position of the target object. According to preset feature conditions (such as area, aspect ratio, etc.), the target sub-connected regions that meet the conditions are screened out. These target regions are likely to be barcodes, and these regions are retained to obtain the result map.

[0145] The specific implementation can be carried out as follows:

[0146] First, determine the features used to distinguish barcodes from other non-barcode areas. Features include area, aspect ratio, boundary complexity (such as the ratio of perimeter to area), etc.

[0147] Based on experience or experimental data, set a reasonable threshold range for the selected features. For example, barcodes usually have a specific aspect ratio and area range.

[0148] For each sub-connected region segmented in step S104, a Blob analysis method is used to extract its features.

[0149] Calculate the area: Count the number of pixels in each sub-connected region as the area of ​​the region.

[0150] Calculate aspect ratio: Calculate the aspect ratio of each connected subregion by fitting the minimum enclosing rectangle or ellipse. Barcodes usually have a large aspect ratio.

[0151] Other features can be extracted as needed, such as the centroid position of the region and the complexity of the boundary (such as using Fourier descriptors to describe the boundary shape).

[0152] The characteristics of each sub-connected region are compared with the preset characteristic conditions. For example, check whether the area is within the preset range, whether the aspect ratio conforms to the typical value of the barcode, etc.

[0153] The sub-connected regions that meet all the preset conditions are retained and are considered as potential barcode regions.

[0154] The selected target sub-connected regions are merged into a new image to form a result map, which will only contain areas that may contain barcodes, making it easier to process later.

[0155] The following example illustrates this:

[0156] Use the cv2.connectedComponentsWithStats function to analyze the segmented connected region image and extract the statistical information of each connected region. The statistical information includes:

[0157] x, y: coordinates of the upper left corner of the connected region.

[0158] w, h: width and height of the connected region.

[0159] area: The area of ​​the connected region.

[0160] Calculate the aspect ratio of each connected region, i.e. w / h.

[0161] Feature condition screening:

[0162] According to the preset characteristic conditions (area and aspect ratio), the connected areas that meet the conditions are selected. The conditions here are:

[0163] Condition 1: The area is between min_area and max_area.

[0164] Condition 2: The aspect ratio is between min_aspect_ratio and max_aspect_ratio.

[0165] These conditions can be adjusted according to actual needs to accommodate different types of barcodes and backgrounds.

[0166] For connected regions that meet the conditions, copy them to the result image. Use bitwise operation (cv2.bitwise_or) to add each connected region mask that meets the conditions to the result image to generate the final result map.

[0167] S106, performing Hough line detection on the result image to obtain the line direction angle of the barcode;

[0168] Perform Hough line detection on the result image to identify the lines in the barcode and calculate their orientation angles. Hough transform is an algorithm that maps a set of points in an image into a parameter space and determines the lines by voting. Based on the orientation angles of the barcode lines, the rotation angle of the barcode can be determined.

[0169] Hough line detection is a technique for detecting lines from binary images. Here are the detailed steps, including code examples and explanations of key technical points:

[0170] Before performing Hough line detection, you can do some preprocessing on the image to ensure that the barcode lines in the image are clearly visible. For example:

[0171] Grayscale: Convert an image to grayscale.

[0172] Edge Detection: Use the Canny edge detector to extract edges in the image.

[0173] In this step, you can use the HoughLines or HoughLinesP function in OpenCV to perform Hough line detection. HoughLines detects infinitely extended lines, while HoughLinesP detects line segments of finite length. Then calculate the line direction angle and extract it from the result of Hough line detection. The line direction angle is the angle theta in the polar coordinates of the line. These line direction angles can be used to determine the rotation angle of the barcode, providing a basis for the subsequent adaptive affine transformation.

[0174] S107, creating a rotation matrix and setting a rotation center point and a rotation angle, wherein the rotation angle is the line direction angle;

[0175] Create a rotation matrix based on the line direction angles obtained by Hough line detection. Set the center point of rotation (usually the center of the image) and the rotation angle (i.e. the line direction angle of the barcode) to adjust the barcode to a horizontal position.

[0176] The specific implementation of this step can include the following steps:

[0177] First, you need to calculate the center point of the rotation, which is usually set to the center point of the image. Then use OpenCV's getRotationMatrix2D function to create a rotation matrix. This matrix is ​​used to rotate the barcode to a horizontal position. Then use OpenCV's warpAffine function to apply the rotation matrix to the image and perform an affine transformation on the image.

[0178] The following is a detailed code example showing how to create a rotation matrix and set the rotation center point and rotation angle:

[0179] importcv2

[0180] importnumpyasnp

[0181] #Assume the result image is stored in result_image

[0182] result_image=cv2.imread('result_image.png')

[0183] #Get the size of the image

[0184] (h,w)=result_image.shape[:2]

[0185] # Calculate the center point of the image

[0186] center=(w / / 2,h / / 2)

[0187] #Assume that the barcode line direction angle calculated in the previous step is angle

[0188] #Convert it to rotation angle here (the negative sign is because the image coordinate system is different from the mathematical coordinate system)

[0189] rotation_angle=-angle

[0190] #Create a rotation matrix

[0191] M=cv2.getRotationMatrix2D(center,rotation_angle,1.0)

[0192] # Calculate the size of the rotated image

[0193] cos = np.abs(M[0,0])

[0194] sin = np.abs(M[0,1])

[0195] new_w=int((h*sin)+(w*cos))

[0196] new_h=int((h*cos)+(w*sin))

[0197] # Adjust the translation part of the rotation matrix

[0198] M[0,2]+=(new_w / 2)-center[0]

[0199] M[1,2]+=(new_h / 2)-center[1]

[0200] # Apply the rotation matrix to perform an affine transformation

[0201] rotated_image=cv2.warpAffine(result_image,M,(new_w,new_h))

[0202] # Display the rotated image

[0203] cv2.imshow('RotatedImage',rotated_image)

[0204] cv2.waitKey(0)

[0205] cv2.destroyAllWindows()

[0206] Through the above steps, the rotation matrix can be successfully created and the image can be adaptively affine transformed to adjust the barcode to a horizontal position, which facilitates subsequent decoding and recognition.

[0207] S108, performing an adaptive affine transformation on the result image based on the rotation center and the rotation angle;

[0208] The rotation matrix is ​​used to perform an adaptive affine transformation on the result image. Affine transformation includes operations such as translation, scaling, and rotation, which can adjust the barcode in the image to a horizontal position and eliminate the tilt effect.

[0209] The specific implementation steps of this step include:

[0210] Get the image size and calculate the center point: You need to get the image size and calculate the center point of the image.

[0211] Create a rotation matrix: Create a rotation matrix using the center point and the rotation angle obtained by Hough line detection.

[0212] Adjust the translation part of the rotation matrix: To ensure that the rotated image is not cropped, adjust the translation part of the rotation matrix.

[0213] Perform an affine transformation: Perform an affine transformation on the image using a rotation matrix to generate a new transformed image.

[0214] S109, decoding and identifying the barcode based on the result image after the adaptive affine transformation;

[0215] Finally, the barcode is decoded and recognized on the result image after adaptive affine transformation. The barcode in the image is parsed using the barcode recognition algorithm to extract the information encoded in it and complete the barcode recognition.

[0216] Through the above steps, the method can accurately and efficiently identify the barcode label attached to the screen, and is suitable for an automatic optical inspection (AOI) system.

[0217] First, determine the appropriate barcode decoding library (such as ZBar, OpenCV, etc.) and load the modules and functions required by the library. The image after affine transformation can be preprocessed, such as grayscale, binarization, etc., to improve the accuracy of barcode decoding. Use the barcode decoding algorithm to detect the barcode in the image, decode it, and extract the encoded information in the barcode. Process the decoding results, such as storing, displaying, or transmitting to subsequent systems for further processing.

[0218] See also Figure 2The present application also provides another embodiment of a method for barcode label recognition applied to AOI. Before using the Laplace operator to process the screen image to obtain a prominent barcode edge, the embodiment also needs to perform corresponding preprocessing operations according to the image defect type to improve the accuracy of image decoding. The embodiment is as follows:

[0219] S201, obtaining a screen image of a screen with a barcode attached thereto;

[0220] S202, determining the image defect type of the screen image, the image defect type including: low contrast, blurred image, dark background and light-colored barcode;

[0221] In the initial step of image processing, it is necessary to determine whether there are image defects in the screen image that affect barcode recognition. Common types of image defects include:

[0222] Low contrast: The contrast between the barcode and the background in the image is low, resulting in unclear edges of the barcode, making it difficult to detect and decode.

[0223] Image blur: The barcode in the image is blurred due to movement, inaccurate focus, etc., making the lines of the barcode unclear.

[0224] Dark background and light barcode: There is a large contrast between the barcode and the background, but the background is dark and the barcode is light, which may cause errors in barcode recognition.

[0225] Analyze the acquired screen image to determine whether the image contains the above three image defects. You can use the following methods:

[0226] Low contrast detection: Calculate the contrast of the image (e.g., using a global contrast measurement method) and determine whether the contrast is below a preset threshold.

[0227] Image blur detection: Use image blur detection algorithms (such as Laplacian variance) to calculate the clarity of the image and determine whether the image is blurry.

[0228] Color inversion detection: Analyze the color distribution of the image to determine whether the color contrast between the barcode and the background is large and the background color is dark.

[0229] S203 . Based on the determined image defect type, perform a corresponding preprocessing operation, wherein the preprocessing operation includes light and dark enhancement, image sharpening, and image inversion.

[0230] According to the type of image defect detected in step S202, a corresponding preprocessing operation is performed to improve the accuracy of barcode recognition. The preprocessing operation includes light and dark enhancement, image sharpening and image inversion.

[0231] Preprocessing operations include:

[0232] Brightness and Darkness Enhancement: For low-contrast images, by adjusting the brightness and contrast of the image, the contrast between the barcode and the background is enhanced, and the edge of the barcode is highlighted. Brightness and Darkness Enhancement includes:

[0233] The grayscale of the screen image is adjusted using the following formula:

[0234] g′=g*Mult+Add;

[0235] Among them, g and g′ are the grayscale values ​​of the image before and after the transformation, Mult and Add are weights and biases, which are used to adjust the brightness and contrast of the image.

[0236] The values ​​of Mult and Add are determined by the following formula:

[0237] Mult=255 / (G_max-G_min);

[0238] Add=-Mult*G_min.

[0239] Image sharpening: For blurred images, image sharpening processing (such as using the Laplacian operator) can be used to enhance the details and edges of the image and make the lines of the barcode clearer.

[0240] Image sharpening includes:

[0241] Processing the screen image using a low-pass filter;

[0242] The sharpened value res of each pixel is calculated by the following formula:

[0243] res = round((orig - mean) * Factor) + orig;

[0244] Among them, orig represents the grayscale value of a pixel in the original image, mean represents the image mean calculated within the filter kernel size range, which represents the average grayscale value of the area around the pixel; Factor represents the contrast intensity, which is used to adjust the degree of sharpening; round represents the rounding function, which is used to ensure that the adjusted pixel value is an integer.

[0245] Image inversion: For images with dark background and light-colored barcode, the image pixel values ​​are reversed through the image inversion operation, so that the barcode and background colors are reversed, which is convenient for barcode recognition.

[0246] Image inversion includes:

[0247] The grayscale of the screen image is adjusted using the following formula:

[0248] g'=255-g;

[0249] Where g and g′ are the grayscale values ​​of the image before and after the transformation, respectively.

[0250] Through the above steps, the accuracy of image decoding can be effectively improved, providing better image quality assurance for subsequent barcode recognition and decoding.

[0251] S204, using a Laplace operator to process the screen image to obtain a prominent barcode edge;

[0252] S205, performing grayscale threshold segmentation on the processed image, and removing noise on the binary image after grayscale threshold segmentation, retaining the connected area of ​​the barcode, and obtaining a backbone binary image;

[0253] S206, dividing each connected region in the backbone binary image into independent sub-connected regions;

[0254] S207, performing feature recognition on each sub-connected region based on Blob analysis, and retaining target sub-connected regions that meet preset feature conditions to obtain a result graph;

[0255] S208, performing Hough line detection on the result image to obtain the line direction angle of the barcode;

[0256] S209, creating a rotation matrix and setting a rotation center point and a rotation angle, wherein the rotation angle is the line direction angle;

[0257] S210, performing an adaptive affine transformation on the result image based on the rotation center and the rotation angle;

[0258] S211. Decode and identify the barcode based on the result image after the adaptive affine transformation.

[0259] Steps S204 to S211 in this embodiment are similar to the description of the relevant steps in the aforementioned embodiment, and are not repeated here.

[0260] See also Figure 3 The present application provides an embodiment, which includes a specific implementation method of performing adaptive affine transformation, namely step S108, and the embodiment includes:

[0261] S301, obtaining a screen image of a screen with a barcode attached thereto;

[0262] S302, using a Laplace operator to process the screen image to obtain a prominent barcode edge;

[0263] S303, performing gray threshold segmentation on the processed image, and removing noise on the binary image after gray threshold segmentation, retaining the connected area of ​​the barcode, and obtaining a backbone binary image;

[0264] S304, dividing each connected region in the backbone binary image into independent sub-connected regions;

[0265] S305, performing feature recognition on each sub-connected region based on Blob analysis, and retaining target sub-connected regions that meet preset feature conditions to obtain a result graph;

[0266] S306, performing Hough line detection on the result image to obtain the line direction angle of the barcode;

[0267] Steps S301 to S306 in this embodiment are similar to the description of steps S101 to S106 in the aforementioned embodiment, and are not described again here.

[0268] S307, creating a rotation matrix and setting a rotation center point and a rotation angle, wherein the rotation angle is the line direction angle;

[0269] First, define a rotation matrix based on the previously calculated rotation angle. The rotation matrix is ​​used to rotate each point in the image according to the specified angle. The rotation matrix can be expressed as:

[0270]

[0271] This rotation matrix represents a two-dimensional rotation matrix used to rotate the original coordinates (x, y) by a certain angle. The new coordinates (u, v) are obtained. The following is an explanation of the symbols in the rotation matrix:

[0272] :

[0273] u: The horizontal coordinate of the new coordinate after rotation.

[0274] v: The vertical coordinate of the new coordinate after rotation.

[0275] 1: Constant term in homogeneous coordinates.

[0276] :

[0277] x: The horizontal coordinate of the original coordinate.

[0278] y: The ordinate of the original coordinate.

[0279] 1: Constant term in homogeneous coordinates.

[0280] :

[0281] This matrix is ​​used for two-dimensional plane rotation transformation. The specific elements have the following meanings:

[0282] First line:

[0283] : The first element of the rotation matrix, used to calculate the new horizontal coordinate.

[0284] : The second element of the rotation matrix, used to calculate the new horizontal coordinate.

[0285] 0: The constant term in the third column does not affect the rotation calculation.

[0286] Second line:

[0287] : The fourth element of the rotation matrix, used to calculate the new vertical coordinate.

[0288] : The fifth element of the rotation matrix, used to calculate the new vertical coordinate.

[0289] 0: The constant term in the third column does not affect the rotation calculation.

[0290] Third line:

[0291] 0: Constant term in homogeneous coordinates, ensuring that the homogeneous coordinate system is maintained after matrix transformation.

[0292] 1: Constant term in homogeneous coordinates, ensuring that the homogeneous coordinate system is maintained after matrix transformation.

[0293] Suppose a point (x, y) needs to be rotated around the origin Angle, the new coordinates (u, v) can be obtained by expanding the above matrix:

[0294]

[0295]

[0296] In this way, the new coordinates after transformation can be obtained.

[0297] S308, obtaining vertex coordinates of four vertex pixels of the result image, and respectively calculating relative coordinates of the four vertex pixels relative to the rotation center;

[0298] In this step, the pixel coordinates of the four vertices of the result image are obtained, including the upper left, upper right, lower left, and lower right. Then, the relative coordinates of these vertices relative to the rotation center are calculated.

[0299] S309, using the rotation matrix to perform coordinate transformation on the relative coordinates of each vertex pixel to obtain corresponding intermediate transformation coordinates;

[0300] In this step, the rotation matrix is ​​applied to the relative coordinates of each vertex, and the relative coordinates of each vertex are transformed using the rotation matrix to obtain the intermediate transformed coordinates.

[0301] S310, based on the coordinates of the rotation center, transform the intermediate conversion coordinates into coordinates of the original coordinate system to obtain new coordinate sets of four vertices;

[0302] In this step, the intermediate transformation coordinates are transformed from the relative coordinate system back to the original coordinate system to obtain the new vertex coordinates.

[0303] S311. Transform the result graph based on the new coordinate set.

[0304] In this step, the new vertex coordinate set calculated previously is used to determine the correspondence before and after the transformation and construct the affine transformation matrix.

[0305] You can use OpenCV's cv2.getPerspectiveTransform function to get the affine transformation matrix.

[0306] After performing the affine transformation, the minimum row coordinates and the minimum column coordinates of all pixels in the result image can also be obtained. And it is determined whether the minimum row coordinate or the minimum column coordinate is a negative value. If the minimum value is negative, the image needs to be translated so that its coordinates are non-negative. For example, the result image is translated in the opposite direction along the x-axis and / or y-axis respectively.

[0307] S312, decoding and identifying the barcode based on the result image after the adaptive affine transformation;

[0308] Finally, the barcode is decoded and recognized on the result image after adaptive affine transformation. The barcode in the image is parsed using the barcode recognition algorithm to extract the information encoded in it and complete the barcode recognition.

[0309] See also Figure 4 The present application also provides an embodiment of a method for barcode label recognition applied to AOI, which includes a specific embodiment of decoding and identifying the barcode, that is, step S109, which includes:

[0310] S401, obtaining a screen image of a screen with a barcode attached thereto;

[0311] S402, using a Laplace operator to process the screen image to obtain a prominent barcode edge;

[0312] S403, performing gray threshold segmentation on the processed image, removing noise on the binary image after gray threshold segmentation, retaining the connected area of ​​the barcode, and obtaining a backbone binary image;

[0313] S404, dividing each connected region in the backbone binary image into independent sub-connected regions;

[0314] S405, performing feature recognition on each sub-connected region based on Blob analysis, and retaining target sub-connected regions that meet preset feature conditions to obtain a result graph;

[0315] S406, performing Hough line detection on the result image to obtain the line direction angle of the barcode;

[0316] S407, creating a rotation matrix and setting a rotation center point and a rotation angle, wherein the rotation angle is the line direction angle;

[0317] S408, performing an adaptive affine transformation on the result image based on the rotation center and the rotation angle;

[0318] Steps S401 to S408 in this embodiment are similar to the step descriptions of steps S101 to S108 in the aforementioned embodiment, and are not repeated here.

[0319] S409, inputting the transformed result graph into a pre-built decoding model, wherein the decoding model is pre-configured with decoding parameters;

[0320] Pre-train a barcode decoding model using a machine learning or deep learning framework (such as TensorFlow, PyTorch). The model accepts image input and outputs decoded barcode characters. Set relevant decoding parameters in the model, such as barcode type (EAN-13, QR code, etc.), scanning direction, barcode area, etc. The transformed result image obtained in step S108 is used as the input image and passed to the decoding model.

[0321] S410, using a predefined operator to search and read the barcode symbol in the result graph, and returning decoded data;

[0322] Use predefined image processing operators (such as morphological operations, feature detection) to locate the barcode region in the image. Extract the bounding box or region of the barcode symbol. In the located barcode region, use the barcode decoding algorithm (such as ZBar, Zxing) to read the barcode symbol. The decoding algorithm converts the barcode symbol into the corresponding decoded character data and returns the decoded result.

[0323] S411. When the returned decoded data indicates that there are decoded characters corresponding to multiple barcodes, the length of each decoded character is determined, and a decoded character that meets a preset character length is output.

[0324] Traverse the returned decoded data and determine the length of each barcode decoded character. According to the preset character length conditions, filter out the decoded characters that meet the conditions and output them.

[0325] The following is a specific example to further illustrate:

[0326] First, create a barcode detection model create_bar_code_model. This model can use techniques from the field of computer vision, such as feature-based methods or deep learning methods. The purpose of this model is to locate the position and bounding box of the barcode from the image.

[0327] Set the decoding parameters set_bar_code_param. These parameters include element_size_min, barcode_height_min and barcode_width_min.

[0328] The following are the introductions to each parameter:

[0329] element_size_min: It defines the minimum size of the barcode, which includes the width and spacing of the barcode. For large codes, this parameter should be set larger to reduce processing time.

[0330] barcode_height_min and barcode_width_min: These two parameters define the minimum height and width of a barcode, respectively. This ensures that only sufficiently large barcodes are detected and decoded.

[0331] Extract ROI (Region of Interest), you can further extract the region of interest (ROI), which is to reduce decoding time and improve efficiency. By extracting ROI, the algorithm can only detect and decode the barcode in the region of interest without having to process the entire image.

[0332] The operator find_bar_code finds and reads the barcode symbol in the given image is the key barcode detection and decoding process. In this process, the previously created model and the set decoding parameters are used to find and read the barcode symbol in the given image. Once the barcode is found, the decoding operation can be performed and the decoded data is returned.

[0333] Finally, the decoded string is returned in DecodedDataStrings. This string contains the barcode data decoded from the image.

[0334] If multiple barcodes are detected, the algorithm will determine the length of the decoded characters as needed. Only the decoded results that meet the preset length requirements will be output. This step ensures the accuracy and controllability of the final output decoded results. For each successfully read barcode, the index from 0 to (n-1) can be used as a candidate handle to retrieve the expected data for a specific barcode result. This mechanism allows the algorithm to obtain the decoded results of a specific barcode as needed.

[0335] The implementation provided in this embodiment is an efficient barcode detection and decoding method, which can effectively improve the speed and accuracy of decoding by setting appropriate decoding parameters and using ROI technology.

[0336] The method provided in this application can be applied in the field of AOI technology. In order to reduce unnecessary false detection in the automatic optical inspection process, it is necessary to shield the barcode label area in the image. Based on this, this application also provides a more specific embodiment of the method for barcode label recognition applied to AOI, see Figure 5 , the embodiment comprises:

[0337] S501, obtaining a screen image of a screen with a barcode attached thereto;

[0338] S502, using a Laplace operator to process the screen image to obtain a prominent barcode edge;

[0339] S503, performing gray threshold segmentation on the processed image, and removing noise on the binary image after gray threshold segmentation, retaining the connected area of ​​the barcode, and obtaining a backbone binary image;

[0340] S504, dividing each connected region in the backbone binary image into independent sub-connected regions;

[0341] S505, performing feature recognition on each sub-connected region based on Blob analysis, and retaining target sub-connected regions that meet preset feature conditions to obtain a result graph;

[0342] S506, performing Hough line detection on the result image to obtain the line direction angle of the barcode;

[0343] S507, creating a rotation matrix and setting a rotation center point and a rotation angle, wherein the rotation angle is the line direction angle;

[0344] S507, performing an adaptive affine transformation on the result image based on the rotation center and the rotation angle;

[0345] S508: Decode and identify the barcode based on the result image after the adaptive affine transformation.

[0346] Steps S501 to S508 in this embodiment are similar to the description of steps S101 to S108 in the aforementioned embodiment, and are not described again here.

[0347] S509, obtaining the center coordinate position, major axis length and minor axis length of the minimum circumscribed rectangular area of ​​the barcode;

[0348] This step is to obtain the position and size information of the minimum bounding rectangle of the barcode. Usually, the minimum bounding rectangle is the smallest rectangular area that can completely surround the barcode. The information obtained includes the center coordinate position, major axis length and minor axis length of the rectangle.

[0349] When decoding and identifying a barcode, the "find_bar_code" operator can be used to find and read the barcode symbol in the image. The decoded data returned after decoding includes a "SymbolRegions" parameter; by reading the SymbolRegions parameter, the center coordinate position, major axis length and minor axis length of the minimum circumscribed rectangular area of ​​the barcode are obtained.

[0350] S510, expanding the minimum circumscribed rectangular area based on a predefined major axis expansion ratio and a minor axis expansion ratio to obtain a ROI area;

[0351] In this step, the minimum bounding rectangle is expanded using the predefined major and minor axis expansion ratios. This expands the rectangular area to ensure that it contains the entire barcode and a certain area around it, resulting in a new area called ROI (region of interest).

[0352] S511, creating a measurement model, and adding a rectangular measurement object to the measurement model;

[0353] In this step, a measurement model is created and a rectangular measurement object is added to the model. This measurement object will be used in subsequent measurement operations to obtain the boundary information of the ROI area.

[0354] S512, setting parameters of the rectangular measurement object;

[0355] Set the parameters of the rectangular measurement object, including the center coordinates, main axis direction, border length, etc. These parameters will affect the behavior of the measurement object in the ROI area.

[0356] S513, based on the rectangular measurement object, performing measurement on the ROI area and outputting a measurement result;

[0357] Use the set rectangle measurement object to measure the ROI area and output the measurement results, which can include the boundary information and corner point information of the rectangle.

[0358] S514, extracting edge point data from the measurement results, and generating an XLD profile based on the edge point data;

[0359] The edge point data is extracted from the measurement results and used to generate an XLD (eXtended LineData) profile. An XLD profile is a data structure that represents the edge of a barcode.

[0360] S515, using a fitting algorithm to fit the XLD profile into a straight line or multiple line segments;

[0361] The XLD contour is fitted using a fitting algorithm to fit it into a straight line or multiple line segments. These straight lines or line segments will be used to determine the boundaries of the barcode area.

[0362] S516, generating a rectangular area according to the fitted straight line or multiple line segments, wherein the rectangular area indicates the position of the positioning barcode label;

[0363] Generate a rectangular area based on the fitted straight line or line segment information. This rectangular area will be used to locate the position of the barcode label.

[0364] S517, creating a mask based on the rectangular area;

[0365] Using the generated rectangular area information, create a mask. The mask is a binary image that indicates which areas need to be blocked.

[0366] S518: Apply the mask to the screen image to shield the barcode area.

[0367] Finally, the generated mask is applied to the original image to mask out the barcode area. In this way, the barcode area in the image will be blocked, thereby reducing the possibility of false detection.

[0368] Take a specific example to describe the steps in the above automatic optical inspection process in detail:

[0369] Suppose there is an image containing a barcode, and you want to mask out the barcode label area in the image and locate it accurately. Here is the implementation process:

[0370] Using the barcode recognition algorithm, a barcode is detected in the image. The center coordinate position of the minimum bounding rectangle of the barcode area is (Row, Col), the major axis length is Len1, and the minor axis length is Len2.

[0371] Set the expansion ratios of the long and short sides ExpandRatio_Len1 and ExpandRatio_Len2, assuming that ExpandRatio_Len1=1.2 and ExpandRatio_Len2=1.1.

[0372] A rectangular region Pre_ROI is generated based on the center coordinate position (Row, Col), the major axis length Len1 and the minor axis length Len2, and the set expansion ratio. Assume that the calculated rectangle has the following boundaries:

[0373] The coordinates of the upper left vertex are:

[0374] (Row-Len1*ExpandRatio_Len1 / 2,Col-Len2*ExpandRatio_Len2 / 2).

[0375] The coordinates of the lower right vertex are:

[0376] (Row+Len1*ExpandRatio_Len1 / 2,Col+Len2*ExpandRatio_Len2 / 2).

[0377] Create a measurement model and set its image size to the same as the original image. Then, use the add_metrology_object_rectangle2_measure operator to add a rectangular measurement object to the measurement model.

[0378] Set the parameters of the rectangular measurement object, where:

[0379] Row and Col are set to the row and column coordinates of the center point of the rectangle, respectively.

[0380] Phi is the direction of the principal axis of the rectangle, usually an angle.

[0381] ColumnEnd is set to the end point of the measurement area column coordinates.

[0382] Length1 and Length2 are half the length of the long side and short side of the rectangle respectively.

[0383] MeasureLength1 and MeasureLength2 are half the length of the measurement area perpendicular to the boundary and tangent to the boundary, respectively.

[0384] MeasureSigma is the sigma value of the Gaussian function, which is used for smoothing.

[0385] MeasureThreshold is the measurement threshold, which determines the boundary of the measurement.

[0386] Perform the measurement operation and obtain the measurement result. The measurement result will include the location information of the rectangle boundary and the values ​​of other measurement parameters.

[0387] Through this process, the location of the barcode label can be accurately determined and a mask for occlusion can be generated. In this way, in subsequent processing, this mask can be applied to the original image to block out the barcode area, thereby reducing the possibility of false detection.

[0388] The above embodiment describes in detail the method for barcode label recognition applied to AOI provided in the present application. The following describes in detail the device for barcode label recognition applied to AOI provided in the present application. Figure 6 , the embodiment comprises:

[0389] A device for barcode label recognition applied to AOI, the device comprising:

[0390] Image acquisition unit 601: acquiring a screen image of a screen with a barcode attached thereto;

[0391] A Laplace processing unit 602, configured to process the screen image using a Laplace operator to obtain a prominent barcode edge;

[0392] A gray threshold segmentation unit 603 is used to perform gray threshold segmentation on the processed image, remove noise on the binary image after gray threshold segmentation, retain the connected area of ​​the barcode, and obtain a backbone binary image;

[0393] A connected region segmentation unit 604 is used to segment each connected region in the backbone binary image into independent sub-connected regions;

[0394] A feature recognition unit 605 is used to perform feature recognition on each sub-connected region based on Blob analysis, and retain the target sub-connected region that meets the preset feature conditions to obtain a result map;

[0395] A Hough line detection unit 606 is used to perform Hough line detection on the result image to obtain the line direction angle of the barcode;

[0396] A matrix creation unit 607, used to create a rotation matrix and set a rotation center point and a rotation angle, wherein the rotation angle is the line direction angle;

[0397] An affine transformation unit 608, configured to perform an adaptive affine transformation on the result image based on a rotation center and the rotation angle;

[0398] The decoding and recognition unit 609 is used to decode and recognize the barcode based on the result image after the adaptive affine transformation.

[0399] Optionally, a pre-processing unit 610 is further included, configured to:

[0400] Determining an image defect type of the screen image, the image defect type comprising: low contrast, blurred image, and dark background with light-colored barcode;

[0401] Based on the determined image defect type, a corresponding pre-processing operation is performed, wherein the pre-processing operation includes shading enhancement, image sharpening, and image inversion.

[0402] Optionally, a Gaussian smoothing unit 611 is further included, which is used to:

[0403] Before processing the screen image using the Laplacian operator to obtain a prominent barcode edge, Gaussian smoothing is performed on the screen image.

[0404] Optionally, the affine transformation unit is specifically used for:

[0405] defining a rotation matrix according to the rotation angle;

[0406] Obtaining vertex coordinates of four vertex pixels of the result image, and respectively calculating relative coordinates of the four vertex pixels relative to the rotation center;

[0407] Using the rotation matrix, coordinate transformation is performed on the relative coordinates of each vertex pixel to obtain corresponding intermediate transformation coordinates;

[0408] Based on the coordinates of the rotation center, transform the intermediate conversion coordinates into coordinates of the original coordinate system to obtain new coordinate sets of four vertices;

[0409] A transformation is performed on the result map based on the new set of coordinates.

[0410] Optionally, the affine transformation unit is specifically used for:

[0411] Get the minimum value of the transformed rows and columns;

[0412] Determine whether the minimum value is a negative value;

[0413] If it is a negative value, based on the minimum value, the result graph is shifted in the opposite direction along the x-axis and / or y-axis.

[0414] Optionally, the decoding and identification unit is specifically used for:

[0415] Inputting the transformed result graph into a pre-built decoding model, wherein the decoding model is pre-configured with decoding parameters;

[0416] Find and read the barcode symbol in the result map using predefined operators and return the decoded data;

[0417] When the returned decoded data indicates that there are multiple decoded characters corresponding to the barcodes, the length of each decoded character is determined, and a decoded character that meets the preset character length is output.

[0418] Optionally, a shielding unit 612 is further included, which is specifically used for:

[0419] Get the center coordinate position, major axis length and minor axis length of the minimum circumscribed rectangular area of ​​the barcode;

[0420] Expand the minimum circumscribed rectangular area based on a predefined major axis expansion ratio and a minor axis expansion ratio to obtain a ROI area;

[0421] Creating a measurement model and adding a rectangular measurement object to the measurement model;

[0422] Setting parameters of the rectangular measurement object;

[0423] Based on the rectangular measurement object, performing measurement on the ROI area and outputting a measurement result;

[0424] Extracting edge point data from the measurement results, and generating an XLD profile based on the edge point data;

[0425] Fitting the XLD profile into a straight line or a plurality of line segments using a fitting algorithm;

[0426] Generate a rectangular area according to the fitted straight line or multiple line segments, wherein the rectangular area represents the position of the positioning barcode label;

[0427] Create a mask based on the rectangular area;

[0428] Apply the mask to the screen image to shield the barcode area.

[0429] Optionally, the preprocessing unit is specifically used for:

[0430] The grayscale of the screen image is adjusted using the following formula:

[0431] g′=g*Mult+Add;

[0432] Among them, g and g′ are the grayscale values ​​of the image before and after the transformation, Mult and Add are weights and biases, which are used to adjust the brightness and contrast of the image.

[0433] The values ​​of Mult and Add are determined by the following formula:

[0434] Mult=255 / (G_max-G_min);

[0435] Add=-Mult*G_min.

[0436] Optionally, the preprocessing unit is specifically used for:

[0437] Processing the screen image using a low-pass filter;

[0438] The sharpened value res of each pixel is calculated by the following formula:

[0439] res = round((orig - mean) * Factor) + orig;

[0440] Among them, orig represents the grayscale value of a pixel in the original image, mean represents the image mean calculated within the filter kernel size range, which represents the average grayscale value of the area around the pixel; Factor represents the contrast intensity, which is used to adjust the degree of sharpening; round represents the rounding function, which is used to ensure that the adjusted pixel value is an integer.

[0441] Optionally, the preprocessing unit is specifically used for:

[0442] The grayscale of the screen image is adjusted using the following formula:

[0443] g'=255-g;

[0444] Where g and g′ are the grayscale values ​​of the image before and after the transformation, respectively.

[0445] Optionally, the shielding unit is specifically used for:

[0446] When decoding and identifying a barcode, the "find_bar_code" operator is used to find and read the barcode symbol in the image. After decoding, the decoded data returned contains the "SymbolRegions" parameter;

[0447] The center coordinate position, major axis length and minor axis length of the minimum circumscribed rectangular area of ​​the barcode are obtained by reading the SymbolRegions parameter.

[0448] See also Figure 7 The present application also provides a device for barcode label recognition applied to AOI, comprising:

[0449] Processor 701, memory 702, input and output unit 703, bus 704;

[0450] The processor 701 is connected to the memory 702, the input and output unit 703 and the bus 704;

[0451] The memory 702 stores a program, and the processor 701 calls the program to execute any of the above methods.

[0452] The present application also relates to a computer-readable storage medium, on which a program is stored. When the program is run on a computer, the computer executes any of the above methods.

[0453] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0454] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0455] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0456] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0457] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A method for barcode label recognition applied to AOI, characterized in that: The method comprises: Acquire a screen image of the screen with the barcode attached; Processing the screen image using a Laplacian operator to obtain a prominent barcode edge; Perform grayscale threshold segmentation on the processed image, remove the noise on the binary image after grayscale threshold segmentation, retain the connected area of ​​the barcode, and obtain the backbone binary image; Segmenting each connected region in the backbone binary image into independent sub-connected regions; Based on Blob analysis, feature recognition is performed on each sub-connected region, and the target sub-connected regions that meet the preset feature conditions are retained to obtain a result graph; Performing Hough line detection on the result image to obtain the line direction angle of the barcode; Create a rotation matrix and set the rotation center point and rotation angle, where the rotation angle is the direction angle of the line; performing an adaptive affine transformation on the result image based on a rotation center and the rotation angle; Based on the result image after adaptive affine transformation, the barcode is decoded and recognized; Also includes: Get the center coordinate position, major axis length and minor axis length of the minimum circumscribed rectangular area of ​​the barcode; Expand the minimum circumscribed rectangular area based on a predefined major axis expansion ratio and a minor axis expansion ratio to obtain a ROI area; Creating a measurement model and adding a rectangular measurement object to the measurement model; Setting parameters of the rectangular measurement object; Based on the rectangular measurement object, performing measurement on the ROI area and outputting a measurement result; Extracting edge point data from the measurement results, and generating an XLD profile based on the edge point data; Fitting the XLD profile into a straight line or a plurality of line segments using a fitting algorithm; Generate a rectangular area according to the fitted straight line or multiple line segments, wherein the rectangular area represents the position of the positioning barcode label; Create a mask based on the rectangular area; Applying the mask to the screen image to shield the barcode area; The step of dividing each connected region in the backbone binary image into independent sub-connected regions comprises: Access each pixel in the binary image and mark the points with a value of 1; If the current pixel's neighborhood is full of pixels with a value of 0, a new label is assigned to the current pixel; If there is a pixel in the neighborhood of the current pixel that has been marked with a value of 1, the minimum value of the label in the neighborhood is assigned to the current pixel, and the equality relationship between these labels is recorded; According to the label equality relationship recorded in the first scan, the labels with equality relationship are unified into a minimum label, so that the pixels with the same label form a connected area.

2. The method for barcode label recognition applied to AOI according to claim 1, characterized in that: Before using the Laplace operator to process the screen image, the method further includes: Determining an image defect type of the screen image, the image defect type comprising: low contrast, blurred image, and dark background with light-colored barcode; Based on the determined image defect type, a corresponding pre-processing operation is performed, wherein the pre-processing operation includes shading enhancement, image sharpening, and image inversion.

3. The method for barcode label recognition applied to AOI according to claim 1, characterized in that: Before using the Laplacian operator to process the screen image to obtain a prominent barcode edge, the method further includes: Gaussian smoothing is performed on the screen image.

4. The method for barcode label recognition applied to AOI according to claim 1, characterized in that: The performing an adaptive affine transformation on the result image based on the rotation center and the rotation angle comprises: Obtaining vertex coordinates of four vertex pixels of the result image, and respectively calculating relative coordinates of the four vertex pixels relative to the rotation center; Using the rotation matrix, coordinate transformation is performed on the relative coordinates of each vertex pixel to obtain corresponding intermediate transformation coordinates; Based on the coordinates of the rotation center, transform the intermediate conversion coordinates into coordinates of the original coordinate system to obtain new coordinate sets of four vertices; A transformation is performed on the result map based on the new set of coordinates.

5. The method for barcode label recognition applied to AOI according to claim 4, characterized in that: After obtaining the new coordinate sets of the four vertices, the method further includes: Get the minimum value of the transformed rows and columns; Determine whether the minimum value is a negative value; If it is a negative value, based on the minimum value, the result graph is shifted in the opposite direction along the x-axis and / or the y-axis.

6. The method for barcode label recognition applied to AOI according to claim 1, characterized in that: The decoding and recognition of the barcode based on the result image after the adaptive affine transformation includes: Inputting the transformed result graph into a pre-built decoding model, wherein the decoding model is pre-configured with decoding parameters; Find and read the barcode symbol in the result map using predefined operators and return the decoded data; When the returned decoded data indicates that there are multiple decoded characters corresponding to the barcodes, the length of each decoded character is determined, and a decoded character that meets the preset character length is output.

7. The method for barcode label recognition applied to AOI according to claim 2, characterized in that: The light and dark enhancement includes: The grayscale of the screen image is adjusted using the following formula: g′=g*Mult+Add; Among them, g and g′ are the grayscale values ​​of the image before and after the transformation, Mult and Add are weights and biases, which are used to adjust the brightness and contrast of the image. The values ​​of Mult and Add are determined by the following formula: Mult=255 / (G_max-G_min); Add=-Mult*G_min.

8. The method for barcode label recognition applied to AOI according to claim 2, characterized in that: The image sharpening includes: Processing the screen image using a low-pass filter; The sharpened value res of each pixel is calculated by the following formula: res=round((orig-mean)*Factor)+orig; Among them, orig represents the grayscale value of a pixel in the original image, mean represents the image mean calculated within the filter kernel size range, which represents the average grayscale value of the area around the pixel; Factor represents the contrast intensity, which is used to adjust the degree of sharpening; round represents the rounding function, which is used to ensure that the adjusted pixel value is an integer.

9. The method for barcode label recognition applied to AOI according to claim 2, characterized in that: The image inversion comprises: The grayscale of the screen image is adjusted using the following formula: g'=255-g; Where g and g′ are the grayscale values ​​of the image before and after the transformation.

10. The method for barcode label recognition applied to AOI according to claim 4, characterized in that: The rotation matrix is ​​expressed as: ； in, represents the rotation angle, u represents the horizontal coordinate of the new coordinate after rotation; v represents the vertical coordinate of the new coordinate after rotation; 0 and 1 represent constant terms in the homogeneous coordinates; x represents the horizontal coordinate of the original coordinate; y represents the vertical coordinate of the original coordinate.

11. The method for barcode label recognition applied to AOI according to claim 1, characterized in that: The method of obtaining the center coordinate position, major axis length and minor axis length of the minimum circumscribed rectangular area of ​​the barcode includes: When decoding and identifying a barcode, the "find_bar_code" operator is used to find and read the barcode symbol in the image. After decoding, the decoded data returned contains the "SymbolRegions" parameter. The center coordinate position, major axis length and minor axis length of the minimum circumscribed rectangular area of ​​the barcode are obtained by reading the SymbolRegions parameter.

12. A device for barcode label recognition applied to AOI, characterized in that: The device comprises: Image acquisition unit: acquires the screen image of the screen with the barcode attached; A Laplace processing unit, used for processing the screen image using a Laplace operator to obtain a prominent barcode edge; A grayscale threshold segmentation unit is used to perform grayscale threshold segmentation on the processed image, remove noise on the binary image after grayscale threshold segmentation, retain the connected area of ​​the barcode, and obtain a backbone binary image; A connected region segmentation unit, used for segmenting each connected region in the backbone binary image into independent sub-connected regions; A feature recognition unit is used to perform feature recognition on each sub-connected region based on Blob analysis, and retain the target sub-connected region that meets the preset feature conditions to obtain a result map; A Hough line detection unit, used for performing Hough line detection on the result image to obtain the line direction angle of the barcode; A matrix creation unit, used to create a rotation matrix and set a rotation center point and a rotation angle, wherein the rotation angle is the line direction angle; An affine transformation unit, configured to perform an adaptive affine transformation on the result image based on a rotation center and the rotation angle; A decoding and recognition unit, used for decoding and recognizing the barcode based on the result image after the adaptive affine transformation; Also includes: also includes a shielding unit, specifically used for: Get the center coordinate position, major axis length and minor axis length of the minimum circumscribed rectangular area of ​​the barcode; Expand the minimum circumscribed rectangular area based on a predefined major axis expansion ratio and a minor axis expansion ratio to obtain a ROI area; Creating a measurement model and adding a rectangular measurement object to the measurement model; Setting parameters of the rectangular measurement object; Based on the rectangular measurement object, performing measurement on the ROI area and outputting a measurement result; Extracting edge point data from the measurement results, and generating an XLD profile based on the edge point data; Fitting the XLD profile into a straight line or a plurality of line segments using a fitting algorithm; Generate a rectangular area according to the fitted straight line or multiple line segments, wherein the rectangular area represents the position of the positioning barcode label; Create a mask based on the rectangular area; Applying the mask to the screen image to shield the barcode area; The connected region segmentation unit is specifically used for: Access each pixel in the binary image and mark the points with a value of 1; If the current pixel's neighborhood is full of pixels with a value of 0, a new label is assigned to the current pixel; If there is a pixel in the neighborhood of the current pixel that has been marked with a value of 1, the minimum value of the label in the neighborhood is assigned to the current pixel, and the equality relationship between these labels is recorded; According to the label equality relationship recorded in the first scan, the labels with equality relationship are unified into a minimum label, so that the pixels with the same label form a connected area.

13. A device for barcode label recognition applied to AOI, characterized in that: The device comprises: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a program stored thereon, wherein the program, when executed on a computer, performs the method according to any one of claims 1 to 11.

Images