Method, device and computer-readable storage medium for processing graphic symbols

Abstract

The present application discloses a method and apparatus for processing graphic symbols, wherein the method comprises: obtaining a luminance component image from an image to be processed, performing binarization processing on the luminance component image to obtain a binarized image, and performing a grayscale morphological operation on the binarized image to obtain a target image. This method can improve the efficiency of image restoration.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and more specifically, to a method and device for processing graphic symbols. Background Art

[0002] Images of Quick Codes (QS) and barcodes (BS) captured in real-world environments can be difficult to decode due to uneven brightness, stains, and noise due to environmental factors like contamination and lighting. Therefore, image restoration is often necessary to accurately identify the information contained in the captured images.

[0003] In existing technologies, trained deep learning models are often used to repair images. However, deep learning models need to be trained with a large amount of data before they can be applied. The training process is cumbersome and highly dependent on data, which affects the efficiency of image restoration. Summary of the Invention

[0004] The method for processing graphic symbols provided in the embodiment of the present application can improve the efficiency of image restoration.

[0005] In a first aspect, a method for processing a graphic symbol is provided, comprising:

[0006] Obtaining the brightness component image of the image to be processed;

[0007] Performing binarization processing on the brightness component image to obtain a binarized image;

[0008] Performing grayscale morphological operation on the binary image to obtain a target image;

[0009] The image to be processed contains graphic symbols.

[0010] In the above technical solution, by extracting the brightness component image of the image to be processed captured by the image acquisition device and performing binarization processing and morphological operations based on the brightness component image, a high-quality target image can be obtained. The graphics contained in the target image have clear contours and less noise, which is conducive to improving the subsequent graphic recognition accuracy and efficiency.

[0011] In some possible implementations of the first aspect, obtaining a luminance component image of a to-be-processed image includes: converting the color space of the to-be-processed image into a target color space based on a mapping relationship between the color space of the to-be-processed image and a target color space; and extracting the luminance component image of the target color space. The color space of the to-be-processed image may be an RGB color space or a BGR color space; and the target color space may be a YCbCr color space, a YCrCb color space, or a YUV color space.

[0012] Through the above implementation, an image containing only color components is mapped to a color space containing brightness components, making the image mapping method simpler and more robust, and facilitating the recognition of regions of interest in the image.

[0013] In some possible implementations of the first aspect, the mapping relationship between the color space of the image to be processed and the target color space is shown in the following formula:

[0014] Y=k r R+k g G+k b B

[0015] Where Y is the brightness value of the pixel in the brightness component image, R, G, and B are the red, green, and blue chromaticity values ​​of the pixel in the original image respectively; k r 、k g 、k b is the weighting factor, and satisfies the following relationship:

[0016] k r +k g +k b =1.

[0017] Through the above implementation, the image component containing the most sufficient information on texture and structural features in the image to be processed can be obtained, which is beneficial to subsequent processing and recognition.

[0018] In some possible implementations of the first aspect, before performing binarization processing based on the luminance component image, the method for processing graphic symbols further includes: performing image enhancement on the luminance component image.

[0019] Through the above implementation, the visual effect of the image can be improved, the image can be made clearer, and the interpretation and recognition effect of the graphic symbols in the image can be enhanced.

[0020] In some possible implementations, performing image enhancement on the luminance component image includes: performing image enhancement on the luminance component image using a point operation algorithm.

[0021] Through the above implementation, the grayscale range occupied by the image data can be changed, and the contrast of the feature of interest can be expanded.

[0022] In some possible implementations of the first aspect, using a point operation algorithm to enhance the luminance component image includes: performing contrast stretching on the luminance component image.

[0023] Through the above implementation, the grayscale can be adjusted according to the characteristics of the image, so that the contrast of the image is transformed into an appropriate range, and the difference in grayscale between different areas in the image is expanded, which facilitates subsequent binarization processing.

[0024] In some possible implementations of the first aspect, contrast stretching of a luminance component image includes: traversing the pixel points in the luminance component image to determine the grayscale value of each pixel in the luminance component image; determining a contrast stretching function based on the grayscale range in which the grayscale value of each pixel in the luminance component lies; and performing grayscale transformation on the pixel points in the luminance component image according to the contrast stretching function.

[0025] The above implementation method can expand the difference between the foreground and the background, making the feature of interest more prominent.

[0026] In some possible implementations of the first aspect, performing binarization processing based on the luminance component image includes: performing binarization processing on the luminance component image using a local adaptive binarization algorithm.

[0027] Through the above implementation, the luminance component image is binarized, which can remove the light and dark information contained in the grayscale image and convert the grayscale image into a black and white image, which is beneficial to the processing and recognition of the image in subsequent processes.

[0028] In some possible implementations of the first aspect, binarizing the luminance component using a local adaptive binarization algorithm includes:

[0029] Determine the size of the binarization processing window; traverse each pixel of the luminance component image with the binarization processing window; calculate the sum of the pixel values ​​of all pixels covered by the binarization processing window; determine the grayscale threshold covered by the binarization processing window; when the sum of the pixel values ​​is greater than or equal to the grayscale threshold, set the pixel value of the pixel corresponding to the center of the binarization processing window to 1, otherwise set the pixel value of the pixel corresponding to the center of the binarization processing window to 0.

[0030] The grayscale threshold can be determined according to the following formula:

[0031]

[0032] Among them, T is the grayscale threshold, n is the side length of the binary processing window, and v ij represents the grayscale value of the pixel in the i-th row and j-th column of the binarization processing window. C is a constant term, and the value of the constant term can be determined according to the actual image processing requirements.

[0033] The gray threshold T can be optimized by using single variable control method, Bayesian optimization or other parameter optimization methods.

[0034] The above implementation allows the binarization threshold at each pixel to be determined by the distribution of its surrounding pixels rather than being fixed. This increases the binarization threshold for areas with higher brightness, while correspondingly decreases it for areas with lower brightness. This effectively enables the proper binarization of local image regions with varying brightness, contrast, and texture. This reduces the impact of significant differences in pixel grayscale in local areas of the image due to uneven illumination in actual captured images.

[0035] In some possible implementations of the first aspect, before performing grayscale morphological operations based on the binary image, the image processing method further includes: filtering the binary image.

[0036] In some possible implementations of the first aspect, filtering the binarized image includes: performing edge-preserving filtering on the binarized image.

[0037] Through the above implementation, noise in the image can be filtered out, which is beneficial to image recognition, and the edge-preserving filtering can filter out as much noise as possible while retaining more edge details.

[0038] In some possible implementations of the first aspect, performing edge-preserving filtering on a binary image includes: converting the binary image into an RGB image; performing color mean shift on all pixels on the RGB image; and converting the RGB image after color mean shift into a binary image.

[0039] Through the above implementation, the binary image can be made smoother and the computational complexity of morphological operations can be reduced.

[0040] In some possible implementations of the first aspect, performing grayscale morphological operations based on the binary image includes: performing morphological closing and opening operations on the binary image.

[0041] Through the above implementation, it is possible to filter out noise in an image while keeping the area of ​​the image unchanged.

[0042] In some possible implementations of the first aspect, the closing operation includes: selecting a first structural element based on the binary image; and performing dilation processing and erosion processing on the closing operation area in sequence according to the first structural element and a preset closing operation rule.

[0043] Through the above implementation, small gaps in bright areas of an image can be filled.

[0044] In some possible implementations of the first aspect, the opening operation includes: selecting a second structural element based on the image after the closing operation; and performing corrosion processing and expansion processing on the opening operation area in sequence according to the second structural element and preset opening operation rules.

[0045] Through the above implementation, the boundaries can be smoothed, small spikes can be eliminated, narrow connections can be disconnected, and small holes in the image can be eliminated; the noise that cannot be filtered out in the edge-preserving filtering step can be further filtered out, thereby achieving the filtering of most noise in the image.

[0046] In some possible implementations of the first aspect, the graphic symbol in the image to be processed is a QR code or a barcode.

[0047] Through the above implementation, an image containing a QR code or a barcode can be repaired, and the outline of the QR code or barcode in the repaired image is clear, so that it is easy to be recognized by a code scanning device.

[0048] In a second aspect, a device for processing graphic symbols is provided, comprising: an acquisition module for acquiring a luminance component image of an image to be processed; a binarization processing module for performing binarization processing based on the luminance component image and outputting a binarized image; and an operation module for performing grayscale morphological operations based on the binarized image and outputting a target image. The image to be processed is an image containing graphic symbols. In a summary of some possible implementations of the second aspect, the graphic symbols contained in the image to be processed are QR codes or barcodes.

[0049] In the above technical solution, by extracting the brightness component image of the image to be processed captured by the image acquisition device and performing binarization processing and morphological operations based on the brightness component image, a high-quality target image can be obtained. The graphics contained in the target image have clear contours and less noise, which is conducive to improving the subsequent graphic recognition accuracy and efficiency.

[0050] In some possible implementations of the second aspect, the acquisition module is used to: convert the color space of the image to be processed into the target color space according to a mapping relationship between the color space of the image to be processed and the target color space; and extract a brightness component image of the target color space.

[0051] In some possible implementations of the second aspect, the color space of the image to be processed is an RGB color space or a BGR color space, and the target color space is a YCbCr color space, a YCrCb color space, or a YUV color space.

[0052] In some possible implementations of the second aspect, the acquisition module is configured to convert the color space of the image to be processed into the target color space according to the mapping relationship shown in the following formula:

[0053] Y=k r R+k g G+k b B

[0054] Where Y is the brightness value of the pixel in the brightness component image, R, G, and B are the red, green, and blue chromaticity values ​​of the pixel in the image to be processed respectively; k r 、k g 、k b is the weighting factor, and satisfies the following relationship:

[0055] k r +k g +k b =1.

[0056] In some possible implementations of the second aspect, the image processing apparatus further includes an image enhancement module configured to perform image enhancement on the luminance component image before binarization.

[0057] In some possible implementations of the second aspect, the image enhancement module is configured to enhance the brightness component image using a point operation algorithm.

[0058] In some possible implementations of the second aspect, the image enhancement module is used to perform contrast stretching on the luminance component image.

[0059] In some possible implementations of the second aspect, the image enhancement module is used to traverse the pixel points in the brightness component image to determine the grayscale value of each pixel in the brightness component image; determine a contrast stretching function based on the grayscale range of the grayscale value of each pixel in the brightness component; and perform grayscale transformation on the pixel points in the brightness component image according to the contrast stretching function.

[0060] In some possible implementations of the second aspect, the binarization processing module is used to perform binarization processing on the enhanced image using a local adaptive binarization algorithm.

[0061] In some possible implementations of the second aspect, the binarization processing module is used to: determine the size of the binarization processing window; traverse each pixel of the brightness component image or the enhanced image with the binarization processing window; calculate the sum of the pixel values ​​of all pixels covered by the binarization processing window; when the sum of the pixel values ​​is greater than or equal to a preset threshold, set the pixel value of the pixel corresponding to the center of the window to 1, otherwise set the pixel value of the pixel corresponding to the center of the window to 0.

[0062] In some possible implementations of the second aspect, the image processing apparatus further includes a filtering module configured to filter the binarized image before the morphological operation.

[0063] In some possible implementations of the second aspect, the filtering module is used to perform edge-preserving filtering on the binarized image.

[0064] In some possible implementations of the second aspect, the filtering module is used to: convert the binary image into an RGB image; perform color mean shift on all pixels on the RGB image; and convert the RGB image after color mean shift into a binary image.

[0065] In some possible implementations of the second aspect, the operation module is used to perform morphological closing operations and opening operations on the binary image in sequence.

[0066] In some possible implementations of the second aspect, the operation module includes a closing operation unit for selecting a first structural element based on the binary image; and performing dilation processing and erosion processing on the closing operation area in sequence according to the first structural element and the preset closing operation rule.

[0067] In some possible implementations of the second aspect, the operation module also includes an opening operation unit, which is used to select a second structural element based on the image after the closing operation; and perform corrosion processing and expansion processing on the opening operation area in sequence according to the second structural element and the preset opening operation rules.

[0068] In a third aspect, a device for processing an image is provided, comprising a processor and a memory, wherein the memory is used to store a program, and the processor is used to call and run the program from the memory to execute the method for processing graphic symbols in the first aspect or any possible implementation of the first aspect.

[0069] In a fourth aspect, a computer-readable storage medium is provided for storing a computer program. When the computer program is run on a computer, the computer executes the method for processing graphic symbols in the above-mentioned first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the drawings without creative work.

[0071] Figure 1 It is a structural diagram of the system architecture provided;

[0072] Figure 2 is a schematic flow chart of a method for processing graphic symbols according to an embodiment of the present application;

[0073] Figure 3 is a schematic flow chart of a method for processing graphic symbols according to another embodiment of the present application;

[0074] Figure 4 is an image of each component of the target color space in the method for processing graphic symbols in an embodiment of the present application;

[0075] Figure 5 is a grayscale histogram before and after contrast stretching in the method for processing graphic symbols in an embodiment of the present application;

[0076] Figure 6 This is a process image of the method for processing graphic symbols according to an embodiment of the present application;

[0077] Figure 7 is a schematic structural block diagram of a device for processing graphic symbols according to an embodiment of the present application;

[0078] Figure 8 It is a schematic structural block diagram of a device for processing graphic symbols in an embodiment of the present application. DETAILED DESCRIPTION

[0079] The embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and examples. The detailed descriptions and drawings of the following examples are used to illustrate the principles of the present application, but are not intended to limit the scope of the present application, that is, the present application is not limited to the described embodiments. It should be understood that the specific examples herein are only intended to help those skilled in the art better understand the embodiments of the present application, and are not intended to limit the scope of the embodiments of the present application.

[0080] It should also be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0081] Unless otherwise indicated, all technical and scientific terms used in the embodiments of this application have the same meaning as those commonly understood by those skilled in the art in the art of this application. The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit the scope of this application. The term "and / or" used in this application includes any and all combinations of one or more related listed items.

[0082] The embodiments of the present application are applicable to processing images containing features such as shape and texture, thereby facilitating the subsequent interpretation of information contained in the image. The embodiments of the present application include, but are not limited to, processing QR code images. Such images can be captured by a charge coupled device (CCD) camera, other cameras, or captured via screenshots. The embodiments disclosed in this application do not limit the image acquisition method.

[0083] Due to their advantages of large amounts of information, easy identification, and low cost, QR codes have experienced rapid development and widespread application in recent years. Scanning a QR code enables a variety of functions, including obtaining information, mobile payment, anti-counterfeiting and traceability, and account login. Common QR codes include QR (Quick Response Code), PDF (Portable Data File) 417 QR codes, and Data Matrix (Data Matrix) QR codes. Data Matrix QR codes are designed for industrial products. They have the smallest minimum size of any barcode currently available, making them particularly suitable for labeling small parts and for direct printing on physical objects. Each Data Matrix QR code consists of a data area, a finder pattern, alignment patterns, and a blank area. The data area is composed of regularly arranged square modules, surrounded by finder patterns, which in turn are surrounded by blank areas. The data area is further separated by alignment patterns. The positioning pattern forms the boundary of the data area. Two adjacent edges are dark solid lines, primarily used to define physical size and positioning. The other two adjacent edges consist of alternating dark and light modules, primarily used to define the symbol's unit structure and also assist in determining physical size and distortion. Because Data Matrix codes only require reading 20% ​​of the data for accurate reading, they are well-suited for use in applications where barcodes are easily damaged, such as those printed on parts exposed to extreme environments like high heat, chemical cleaning agents, and mechanical abrasion. When a QR code is damaged, image restoration is required before the information contained within can be read. Existing techniques typically grayscale and enhance the resulting color QR code. A deep learning model is then trained based on large amounts of data to repair the damaged QR code within the QR code image area, producing a restored QR code image that can then be read. However, training a deep learning model first requires collecting sample sets and training sets. The training set consists of various types of damaged QR code images, and the sample set consists of clear QR code images corresponding to the damaged QR code images in the training set. The sample set and training set also need to undergo manual classification and other processes before they can be used for model training. Therefore, using deep learning methods to repair QR code images consumes a lot of time in collecting data, analyzing data, organizing data, and training models, which is inefficient. In addition, it is highly dependent on data, and the trained model is difficult to use in a general sense.

[0084] In view of this, an embodiment of the present application provides a method for processing graphic symbols, which uses a digital image processing method to extract brightness components, enhance the image, binarize the image, filter and perform morphological operations on the collected image containing graphic symbols. The clear shape features of the area of ​​interest can be displayed on the final target image. Using this method, the image can be quickly repaired, and the Data Matrix two-dimensional code in the repaired image can be correctly recognized.

[0085] like Figure 1 As shown, the embodiment of the present application provides a system architecture 100. Figure 1 In the embodiment, the image acquisition device 110 is used to input the image to be processed 1101 into the image processing device 120. The image acquisition device 110 can be any device with an image capture or acquisition function, such as a camera, a camcorder, a scanner, a mobile phone, a tablet computer or a barcode scanner, and the image to be processed 1101 is an image captured or acquired by the above device; it can also be a device with a data storage function, and the image to be processed 1101 is stored in the device. The present application does not limit the type of image acquisition device 110. With respect to the image processing method of the embodiment of the present application, the image to be processed is an image containing graphic symbols, and optionally, it can be an image containing a QR code or a barcode. The image acquisition device 120 is used to process the image to be processed 1101 and output a target image 1201. The target image 1201 is an image that can clearly reflect certain specific features such as shape and texture after being processed by the image processing device 120, that is, an image that can clearly reflect certain features that the user is interested in. When the image to be processed 1101 is an image containing a QR code or barcode, the target image 1201 can be an image that can be correctly recognized by a QR code or barcode recognition device after processing. The image processing device 120 can be any device with image processing capabilities, such as a computer, a smartphone, a workstation, or other device with a central processing unit. This application does not limit the type of image processing device 120.

[0086] In some embodiments, the image acquisition device 110 and the image processing device 120 may be the same device. For example, the image acquisition device 110 and the image processing device 120 are both smartphones, or both are barcode scanners.

[0087] In other embodiments, the image acquisition device 110 and the image processing device 120 may be different devices. For example, the image acquisition device 110 may be a terminal device, while the image processing device 120 may be a computer, workstation, or other device. The image acquisition device 110 may interact with the image processing device 120 via a communication network using any communication mechanism or standard. The communication network may be a wide area network, a local area network, a point-to-point connection, or any combination thereof.

[0088] The image to be processed in this embodiment is a QR code image captured by an image acquisition device, which can be a QR code scanning device, such as a barcode scanner, or a camera. This method is not limited to processing QR code images and can also be applied to processing other images. This application does not limit the application of this method.

[0089] Figure 2 and Figure 3 A flow chart of a method 200 for processing graphic symbols according to an embodiment of the present application is shown. The method can be used to Figure 1 The image processing device 120 shown in FIG.

[0090] S210: Obtain a brightness component image of the image to be processed.

[0091] A luminance component image represents the brightness information of an image. Different image formats correspond to different color spaces, and different color spaces contain different channels, such as luma and chroma. The component image corresponding to the luma channel is a grayscale image, or an image represented by grayscale. The sampling rate of the luma channel is higher than that of the chroma channels. Therefore, luma component images are generally referred to as grayscale images. Grayscale images can be obtained using floating-point methods, component methods, maximum methods, average methods, weighted average methods, or gamma correction algorithms. They can also be generated using the target color adjustment tools included in multimedia editing software. These target color adjustment tools often generate grayscale images by performing the aforementioned calculations on the image. Alternatively, they can be obtained through color space conversion.

[0092] Optionally, in some embodiments of the present application, step S210 of obtaining a luminance component image containing luminance information in the image to be processed includes:

[0093] Convert the image to be processed from the original color space to the target color space and extract the brightness component image of the target color space.

[0094] The original color space is the color space of the image to be processed. This original color space depends on the format of the image to be processed. It can be a color space that uses colors to describe colors, such as RGB or BGR. It can also be a color space that originally includes a luminance channel, such as YUV or YCbCr. Or other color spaces. RGB and BGR are two commonly used color spaces for color images. RGB color space includes three channels: R (Red), G (Green), and B (Blue). The target color space is a color space that includes a luminance channel, such as YUV or YCbCr. The YCbCr color space includes a Y (luminance) channel, a Cb (blue chrominance) channel, and a Cr (red chrominance) channel.

[0095] When the color space of the image to be processed does not contain brightness information, it needs to be converted to the target color space first to extract the brightness component. Through the conversion of the color space, the brightness and chromaticity can be separated, and the required information can be seen more intuitively.

[0096] Optionally, color space conversion may be performed according to a mapping relationship between the color space of the image to be processed and the target color space.

[0097] When the original color space of the image to be processed is RGB color space or BGR color space, and the target color space is YCbCr color space, the mapping relationship is as shown in the following formula:

[0098] Y=k r R+k g G+k b B

[0099] Where Y is the brightness value of the pixel in the brightness component image, R, G, and B are the red, green, and blue chromaticity values ​​of the pixel in the original image respectively; k r 、k g 、k b is the weighting factor, and satisfies the following relationship:

[0100] k r +k g +k b =1

[0101] In Data Matrix code recognition algorithms, the YCbCr color space mapping method is more intuitive and robust than the RGB or BGR color space mapping methods. After color space conversion, the brightness and chromaticity of the image can be separated, making the information contained in the image more intuitive. This not only makes the outline features of the QR code more clearly displayed, but also facilitates Data Matrix code recognition.

[0102] S230 , performing image binarization processing based on the brightness component image to obtain a binarized image.

[0103] Image binarization involves identifying all pixels in an image with a grayscale greater than or equal to a certain threshold as belonging to a specific object, with their grayscale value set to 255. Otherwise, these pixels are excluded from the object region and have a grayscale value of 0, representing the background or an exceptional object region. This process renders the entire image in a distinctly black and white effect. Image binarization can significantly reduce the amount of data in an image, thereby highlighting the outline of the target.

[0104] Optionally, in some embodiments, the point operation algorithm may be a global fixed threshold method, a local adaptive threshold method, or an OTSU binarization algorithm. The global fixed threshold method uses the same threshold for binarization across the entire image, while the local adaptive threshold method determines the binarization threshold at a pixel location based on the pixel value distribution of its neighborhood.

[0105] Due to factors such as uneven ambient lighting, the pixel grayscale of different local areas of the original image may vary significantly. The adaptive binarization algorithm can binarize the image into foreground and background parts according to the image's grayscale characteristics, thereby obtaining a binary image. This allows for reasonable binarization of local image areas with different brightness, contrast, and texture.

[0106] Optionally, in some embodiments, the point operation algorithm is a local adaptive binarization algorithm.

[0107] Optionally, the adaptive binarization algorithm may be a wolf local adaptive binarization algorithm, a Niblack binarization algorithm, or a sauvola binarization algorithm.

[0108] In some embodiments, the local adaptive binarization algorithm includes the following steps:

[0109] Determine the size of the binarization processing window; traverse each pixel of the luminance component image with the binarization processing window; calculate the sum of the pixel values ​​of all pixels covered by the binarization processing window; determine the grayscale threshold covered by the binarization processing window; when the sum of the pixel values ​​is greater than or equal to the grayscale threshold, set the pixel value of the pixel corresponding to the center of the binarization processing window to 1, otherwise set the pixel value of the pixel corresponding to the center of the binarization processing window to 0.

[0110] The grayscale threshold can be determined according to the following formula:

[0111]

[0112] Among them, T is the grayscale threshold, n is the side length of the binary processing window, and v ij Represents the grayscale value of the pixel in the i-th row and j-th column in the binarization processing window. C is a constant selected according to the actual image processing requirements.

[0113] The gray threshold T can be optimized by using single variable control method, Bayesian optimization or other parameter optimization methods.

[0114] In this embodiment, an adaptive binarization algorithm is used to binarize the enhanced image. The binarization threshold at each pixel position is not fixed, but is determined by the distribution of its surrounding pixels. This allows the image area with higher brightness to have a higher binarization threshold, while the image area with lower brightness has a relatively lower binarization threshold. Figure 6 As can be seen from (c) and (d), the contours in the binarized image after local adaptive binarization are clearer than those in the enhanced image.

[0115] S250 , performing grayscale morphological operations based on the binary image to obtain a target image.

[0116] Grayscale morphological operations include erosion, dilation, opening, and closing operations. Through the above operations, parameters such as the grayscale value and spatial size of the original image can be adjusted.

[0117] Optionally, the grayscale morphological operation in step S250 may be a morphological closing operation and an opening operation performed sequentially based on the binary image;

[0118] The closing operation includes: selecting a first structural element based on the binary image; and sequentially performing dilation processing and erosion processing on the closing operation area according to the first structural element and a preset closing operation rule.

[0119] The opening operation includes: selecting a second structural element based on the image after the opening operation; and sequentially performing corrosion processing and expansion processing on the opening operation area according to the second structural element and a preset opening operation rule.

[0120] The first structural element and the second structural element may be the same structural element or different structural elements, and the sizes of the structural elements may be adjusted according to actual image processing effects.

[0121] Alternatively, as Figure 3 As shown, in some embodiments of the present application, the above image processing method may further include, before step 230:

[0122] S220, performing image enhancement on the luminance component image to obtain an enhanced luminance component image;

[0123] Image enhancement is the process of improving the visual quality of an image to meet specific analytical needs. Alternatively, spatial algorithms, such as point operations or neighborhood denoising, can be used for image enhancement. Frequency-domain algorithms can also be used.

[0124] Optionally, in step S220 , a point operation algorithm may be used to enhance the brightness component image.

[0125] Optionally, a point operation algorithm is used to enhance the brightness component image, which can be:

[0126] Perform contrast stretching on the luminance component image, or perform grayscale histogram shift on the luminance component image.

[0127] Optionally, the contrast stretching method may be a linear stretching method, i.e., linearly scaling the pixel values ​​of the luminance component image. According to some embodiments of the present application, the luminance component image may optionally be linearly stretched using methods such as global linear stretching, 2% linear stretching, piecewise linear stretching, and grayscale window slicing.

[0128] Optionally, the method used for contrast stretching can be a nonlinear stretching method, that is, the image is stretched using a nonlinear function. According to some embodiments of the present application, optionally, the brightness component image can be nonlinearly stretched using functions such as exponential functions, logarithmic functions, and Gaussian functions.

[0129]

[0130] In some embodiments of the present application, the contrast stretching process can be expressed as follows:

[0131] Among them, I(x,y) is the grayscale value of the pixel in the brightness component image, (x,y) is the coordinate value of the pixel, I min is the minimum grayscale value of the luminance component image, I maxis the maximum grayscale value of the luminance component image, MIN and MAX are the minimum and maximum grayscale values ​​of the grayscale space to be stretched. Optionally, the minimum value can be 0 and the maximum value can be 255.

[0132] Optionally, in some embodiments, when the brightness component image acquired in step S210 is relatively clear or the information provided is relatively sufficient, step S220 may not be performed.

[0133] Optionally, in other embodiments, when the luminance component image quality is low or blurred, the luminance component image may be enhanced in step S220 before step S230 to magnify the differences between features of different objects in the image and suppress features that are not of interest, thereby improving image quality and enhancing subsequent image interpretation and recognition. In this case, step S230 may be:

[0134] The enhanced luminance component image is binarized to obtain a binarized image.

[0135] Optionally, in some embodiments of the present application, before step S250, the image processing method may further include:

[0136] S240 , filtering the binarized image to obtain a filtered binarized image.

[0137] Specifically, in step S240 , edge-preserving filtering is performed on the binary image, which can filter out noise as much as possible while retaining more edge details.

[0138] In some possible implementations, performing edge-preserving filtering on a binary image includes: converting the binary image into an RGB image; performing color mean shift on all pixels on the RGB image; and converting the RGB image after color mean shift into a binary image.

[0139] Through the above implementation, the binary image can be made smoother and the computational complexity of morphological operations can be reduced.

[0140] Among them, performing color mean shift on all pixel points on the RGB image includes: determining the physical space radius and the color space radius used to establish an iteration space; taking any pixel point on the RGB image as the initial center point, and establishing a space sphere based on the physical space radius and the color space radius; taking the space sphere as the iteration space, and calculating the vector sum of the color vectors of all pixel points in the iteration space relative to the center point; moving the center point to the end point of the sum vector, and recalculating the vector sum until the end point of the vector sum coincides with the center point, and taking the end point of the vector sum as the final center point; updating the color value of the initial center point to the color value of the final center point.

[0141] In some embodiments of the present application, when the binarized image obtained in step S230 is of high quality and has less noise, step S240 may not be performed.

[0142] In some embodiments of the present application, when there are many noise points in the binarized image obtained in step S230, a filtering operation can be performed on the binarized image in step S240 before step S250. The filtering operation can eliminate the noise in the image that is easier to eliminate, thereby reducing the computational complexity of step S250 and enhancing the denoising effect. In this case, step S250 can be:

[0143] The morphological opening and closing operations are performed on the filtered binary image in sequence.

[0144] like Figures 4 to 6 In the embodiment shown, the above-mentioned method for processing an image is applied to the processing of a DataMatrix two-dimensional code image. The image to be processed is a color two-dimensional code image input by an image acquisition device, and the color space is an RGB color space. The processing steps are as follows:

[0145] Step 1: Convert the color QR code image from RGB color space to YCbCr color space, separate the Y component, Cb component, and Cr component, and extract the Y component of the QR code image, that is, the brightness component image; Figure 4 The image of the three components after the color space of the Data Matrix two-dimensional code image is converted into the YCbCr color space by the method of processing graphic symbols of the present application, wherein (a) is the image of the Y component, (b) is the image of the Cb component, and (c) is the image of the Cr component. Figure 4 It can be seen that the Y component image contains more information than the other two component images; Figure 5 The process diagram of processing Data Matrix two-dimensional code images by the method of the embodiment of the present application is shown, wherein (a) is the image to be processed, (b) is the image of the Y channel extracted after color space conversion, and Figure 6 As can be seen from (a) and (b), the Y component can provide more sufficient information. Therefore, in this embodiment, the three components of the YCbCr color space are separated, and the Y component image is extracted as the basic image for subsequent operations, which is more conducive to subsequent image processing and QR code recognition.

[0146] Step 2: Perform image enhancement on the Y component of the QR code image using a contrast stretching method to enlarge the grayscale difference between the foreground and background, thereby obtaining a stretched QR code image, i.e., a grayscale image. Figure 5The grayscale histograms of the Data Matrix two-dimensional code image before and after image enhancement are shown, where (a) is the grayscale histogram of the Y channel component image before contrast stretching, and (b) is the grayscale histogram of the enhanced image obtained after contrast stretching. Figure 5 It can be seen that the grayscale of the image before contrast stretching is relatively concentrated, so it is difficult to reflect the texture and shape features in the image. After contrast stretching, the grayscale values ​​are dispersed within the range of 0-255, and the contrast is enhanced. Figure 6 (c) is the enhanced image obtained after contrast stretching; Figure 6 As can be seen from (b) and (c), the image before contrast stretching is relatively blurry, while the enhanced image after contrast stretching is clearer and can better reflect the texture and shape features contained in the image, which is conducive to the recognition of QR codes.

[0147] Step 3: Perform local adaptive binarization on the stretched QR code image, and divide the image into foreground and background according to the grayscale characteristics of the image to obtain a binarized QR code image; Figure 6 Figure (d) is the binarized image obtained after binarization. It can be seen that the image contains a lot of noise, which interferes with the subsequent recognition process. Therefore, based on the characteristics of DataMatrix two-dimensional code images, edge-preserving filtering is performed on them to initially filter out noise while ensuring the clarity of the image edges.

[0148] Step 4: Perform edge-preserving filtering on the binarized QR code image to remove noise as much as possible while retaining more edge details, thereby obtaining a filtered QR code image. Figure 6 (e) is the filtered image obtained after filtering, through Figure 6 As can be seen in (d) and (e), the edge-preserving filtered image shows reduced noise compared to the binary image. However, despite some noise removal, small particles of noise, such as spikes and holes, still remain in the image. Therefore, in the next step, morphological operations are performed on the QR code to remove these small particles.

[0149] Step 5: Perform morphological closing and opening operations on the filtered two-dimensional code image in sequence to obtain the target two-dimensional code image; in this embodiment, when the sizes of the first structure element and the second structure element are both 3*3, the two-dimensional code after morphological operation is clearest.

[0150] Figure 6 (f) is the target image obtained after image morphological operation; Figure 6As can be seen in (e) and (f), the target image obtained after the morphological opening and closing operations has a clear outline and almost no noise that affects QR code recognition. After a series of processing methods described above, the blurred QR code image becomes clear and can be recognized by the code scanning device.

[0151] The above describes in detail the method embodiment of the embodiment of the present application. The following describes the device embodiment of the embodiment of the present application. The device embodiment and the method embodiment correspond to each other. Therefore, for the parts not described in detail, please refer to the previous method embodiment. The device can implement any possible implementation method of the above method.

[0152] Figure 7 A schematic block diagram of an apparatus 400 for processing graphic symbols according to an embodiment of the present application is shown, comprising: an acquisition module 410 for obtaining a luminance component image containing luminance information in an image to be processed; a binarization processing module 430 for performing binarization processing based on the luminance component image and outputting a binarized image; and an operation module 450 for performing grayscale morphological operations based on the binarized image and outputting a target image.

[0153] In the above technical solution, by extracting the brightness component image of the image to be processed captured by the image acquisition device and performing binarization processing and morphological operations based on the brightness component image, a high-quality target image can be obtained. The graphics contained in the target image have clear contours and less noise, which is conducive to improving the subsequent graphic recognition accuracy and efficiency.

[0154] Optionally, in some embodiments, the acquisition module 410 is used to: convert the color space of the image to be processed into the target color space according to a mapping relationship between the color space of the image to be processed and the target color space; and extract a brightness component image of the target color space.

[0155] Optionally, in some embodiments, the color space of the image to be processed is an RGB color space or a BGR color space, and the target color space is a YCbCr color space, a YCrCb color space or a YUV color space.

[0156] Optionally, in some embodiments, the acquisition module 410 is configured to convert the color space of the image to be processed into the target color space according to the mapping relationship shown in the following formula:

[0157] Y=k r R+k g G+k b B

[0158] Where Y is the brightness value of the pixel in the brightness component image, R, G, and B are the red, green, and blue chromaticity values ​​of the pixel in the image to be processed respectively; kr 、k g 、k b is the weighting factor, and satisfies the following relationship:

[0159] k r +k g +k b =1.

[0160] Optionally, in some embodiments, the image processing apparatus 400 further includes an image enhancement module 420 configured to perform image enhancement on the luminance component image and output an enhanced image.

[0161] Optionally, in some embodiments, the image enhancement module 420 is configured to enhance the brightness component image using a point operation algorithm.

[0162] Optionally, in some embodiments, the image enhancement module 420 is configured to perform contrast stretching on the luminance component image.

[0163] Optionally, in some embodiments, the image enhancement module 420 is used to traverse the pixel points in the luminance component image to determine the grayscale value of each pixel in the luminance component image; determine the contrast stretching function according to the grayscale range of the grayscale value of each pixel in the luminance component; and perform grayscale transformation on the pixel points in the luminance component image according to the contrast stretching function.

[0164] Optionally, in some embodiments, the binarization processing module 430 is configured to perform binarization processing on the enhanced image using a local adaptive binarization algorithm.

[0165] Optionally, in some embodiments, the binarization processing module is used to:

[0166] Determine the size of the binarization processing window; traverse each pixel of the luminance component image with the binarization processing window; calculate the sum of the pixel values ​​of all pixels covered by the binarization processing window; determine the grayscale threshold covered by the binarization processing window; when the sum of the pixel values ​​is greater than or equal to the grayscale threshold, set the pixel value of the pixel corresponding to the center of the binarization processing window to 1, otherwise set the pixel value of the pixel corresponding to the center of the binarization processing window to 0.

[0167] The grayscale threshold can be determined according to the following formula:

[0168]

[0169] Among them, T is the grayscale threshold, n is the side length of the binary processing window, and v ij represents the grayscale value of the pixel in the i-th row and j-th column in the binarization processing window, and C is a constant selected according to actual image processing requirements.

[0170] Optionally, in some embodiments, the image processing apparatus 400 further includes a filtering module 440 for filtering the binary image.

[0171] Optionally, in some embodiments, the filtering module 440 is configured to perform edge-preserving filtering on the binarized image.

[0172] Optionally, in some embodiments, the operation module 450 is configured to perform morphological closing and opening operations on the binary image in sequence.

[0173] Optionally, in some embodiments, the operation module 450 includes a closing operation unit for selecting a first structural element based on the binary image; and performing dilation processing and erosion processing on the closing operation area in sequence according to the first structural element and a preset closing operation rule.

[0174] Optionally, in some embodiments, the operation module 450 further includes an opening operation unit for selecting a second structural element based on the image after the closing operation; and performing corrosion processing and expansion processing on the opening operation area in sequence according to the second structural element and a preset opening operation rule.

[0175] Figure 8 Schematic diagram of the hardware structure of the image processing device according to an embodiment of the present application. Figure 5 The image processing apparatus 500 shown includes a memory 501 , a processor 502 , a communication interface 503 , and a bus 504 . The memory 501 , the processor 502 , and the communication interface 503 are connected to each other via the bus 504 .

[0176] Memory 501 can be a read-only memory (ROM), a static storage device, or a random access memory (RAM). Memory 501 can store programs. When the program stored in memory 501 is executed by processor 502, processor 502 and communication interface 503 are used to perform the various steps of the method for processing a graphic symbol according to an embodiment of the present application.

[0177] The processor 502 can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), a graphics processing unit (GPU) or one or more integrated circuits to execute relevant programs to implement the functions required to be performed by the units in the device for processing graphic symbols in the embodiment of the present application, or to execute the method for processing graphic symbols in the embodiment of the present application.

[0178] The processor 502 may also be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the method for processing graphic symbols in the embodiment of the present application may be completed by hardware integrated logic circuits in the processor 502 or software instructions.

[0179] The processor 502 may also be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application may be directly implemented and executed by a hardware processor, or by a combination of hardware and software modules within the processor. The software modules may be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The storage medium is located in the memory 501. The processor 502 reads the information in the memory 501 and, in conjunction with its hardware, performs the functions required by the units included in the apparatus for processing graphic symbols in the embodiments of this application, or executes the method for processing graphic symbols in the embodiments of this application.

[0180] The communication interface 503 uses a transceiver device such as, but not limited to, a transceiver to implement communication between the device 500 and other devices or communication networks. For example, traffic data of an unknown device can be obtained through the communication interface 503.

[0181] The bus 504 may include a path for transmitting information between various components of the device 500 (eg, the memory 501 , the processor 502 , and the communication interface 503 ).

[0182] It should be noted that although the above-mentioned device 500 only shows a memory, a processor, and a communication interface, in the specific implementation process, those skilled in the art should understand that the device 500 may also include other devices necessary for normal operation. At the same time, according to specific needs, those skilled in the art should understand that the device 500 may also include hardware devices that implement other additional functions. In addition, those skilled in the art should understand that the device 500 may also include only the devices necessary to implement the embodiments of the present application, and does not necessarily include Figure 5 All devices shown in .

[0183] An embodiment of the present application further provides a computer-readable storage medium, on which is stored a program code for execution by a device, the program code including instructions for executing the steps in the above-mentioned method for processing graphic symbols.

[0184] An embodiment of the present application also provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the above-mentioned method for processing graphic symbols.

[0185] The aforementioned computer-readable storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0186] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0187] In the several embodiments provided herein, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units described is merely a logical functional division. In actual implementation, other divisions may be used, such as combining or integrating multiple units or components into another system, or omitting or disabling certain features. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interface, or the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0188] The terms used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and claims, unless the context clearly indicates otherwise, the singular forms "a" and "the" are intended to include the plural forms as well. Similarly, the terms "and / or" as used in this application refer to any and all possible combinations of one or more associated listings. In addition, when used in this application, the term "comprising" refers to the presence of stated features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or these groups.

[0189] The various aspects, implementations, implementations, or features of the described embodiments can be used individually or in any combination. The various aspects of the described embodiments can be implemented by software, hardware, or a combination of software and hardware. The described embodiments can also be embodied by a computer-readable medium storing computer-readable code, the computer-readable code comprising instructions executable by at least one computing device. The computer-readable medium can be associated with any data storage device capable of storing data that can be read by a computer system. Example computer-readable media can include read-only memory, random access memory, compact disc read-only memory (CD-ROM), hard disk drive (HDD), digital video disc (DVD), magnetic tape, and optical data storage devices. The computer-readable medium can also be distributed among computer systems connected via a network so that the computer-readable code can be stored and executed in a distributed manner.

[0190] The above technical description may refer to the accompanying drawings, which form a part of this application and illustrate implementation methods in accordance with the described embodiments in the drawings. Although these embodiments are described in sufficient detail to enable those skilled in the art to implement these embodiments, these embodiments are non-limiting; other embodiments can be used and changes can be made without departing from the scope of the described embodiments. For example, the order of operations described in the flowchart is non-limiting, so the order of two or more operations illustrated in the flowchart and described according to the flowchart can be changed according to several embodiments. As another example, in several embodiments, one or more operations illustrated in the flowchart and described according to the flowchart are optional or deletable. In addition, certain steps or functions can be added to the disclosed embodiments, or the order of two or more steps can be replaced. All these changes are considered to be included in the disclosed embodiments and the claims.

[0191] In addition, the above technical description uses terms to provide a thorough understanding of the described embodiments. However, excessively detailed details are not required to implement the described embodiments. Therefore, the above description of the embodiments is presented for the purpose of illustration and description. The embodiments presented in the above description and the examples disclosed based on these embodiments are provided separately to add context and facilitate understanding of the described embodiments. The above description is not intended to be exhaustive or to limit the described embodiments to the precise form of the present application. Based on the above teachings, several modifications, selective applications, and variations are possible. In some cases, well-known processing steps are not described in detail to avoid unnecessarily affecting the described embodiments. Although the present application has been described with reference to preferred embodiments, various improvements may be made thereto and components therein may be replaced with equivalents without departing from the scope of the present application. In particular, the various technical features mentioned in the various embodiments may be combined in any manner as long as there are no structural conflicts. The present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for processing graphic symbols, characterized in that: The method comprises: Obtaining the brightness component image of the image to be processed; Performing binarization processing on the brightness component image to obtain a binarized image; Performing binarization processing based on the brightness component image, including: Determining the size of a binarization processing window; traversing each pixel of the luminance component image with the binarization processing window; determining a grayscale threshold covered by the binarization processing window; when the sum of pixel values ​​is greater than or equal to the grayscale threshold, setting the pixel value of the pixel corresponding to the center of the binarization processing window to 1; otherwise, setting the pixel value of the pixel corresponding to the center of the binarization processing window to 0; Perform grayscale morphological operation based on the binary image to obtain the target image. Wherein, the image to be processed contains graphic symbols; Before the binarization process is performed, the method further includes: Traversing the pixel points in the luminance component image, determining the grayscale value of each pixel in the luminance component image; determining a contrast stretching function according to the grayscale range of the grayscale value of each pixel in the luminance component; performing grayscale transformation on the pixel points in the luminance component image according to the contrast stretching function; before performing the grayscale morphological operation, the method further includes: performing edge-preserving filtering on the binary image, including: The binary image is converted into an RGB image; color mean shift is performed on all pixels on the RGB image; the RGB image after color mean shift is converted into a binary image after edge filtering; the grayscale threshold is determined according to the following formula: Wherein, T is the grayscale threshold, n is the side length of the binarization processing window, and v ij is the grayscale value of the pixel in the i-th row and j-th column in the binarization processing window, and C is a constant term.

2. The method according to claim 1, characterized in that The step of obtaining a brightness component image of an image to be processed includes: Converting the color space of the image to be processed into the target color space according to a mapping relationship between the color space of the image to be processed and the target color space; Extract the luminance component image of the target color space.

3. The method according to claim 2, characterized in that The color space of the image to be processed is an RGB color space or a BGR color space, and the target color space is a YCbCr color space, a YCrCb color space or a YUV color space.

4. The method according to claim 2 or 3, characterized in that The mapping relationship between the color space of the image to be processed and the target color space is shown in the following formula: Y=k r R+k g G+k b B Where Y is the brightness value of the pixel in the brightness component image, R, G, and B are the red, green, and blue chromaticity values ​​of the pixel in the image to be processed respectively; k r 、k g 、k b is the weighting factor and satisfies the following relationship: k r +k g +k b =1。 5. The method according to claim 1, wherein The binarization processing based on the brightness component image includes: A local adaptive binarization algorithm is used to perform binarization processing on the brightness component image.

6. The method according to claim 1, characterized in that The grayscale morphological operation based on the binary image comprises: Morphological closing and opening operations are sequentially performed on the binary image.

7. The method according to claim 6, characterized in that The opening operation includes: selecting a first structuring element based on the binarized image; performing dilation processing and erosion processing on the closed operation area in sequence according to the first structural element and a preset closing operation rule; The closing operation includes: selecting a second structuring element based on the image after the closing operation; The opening operation area is sequentially subjected to erosion processing and dilation processing according to the second structural element and a preset opening operation rule.

8. The method according to claim 1, characterized in that The graphic symbol in the image to be processed is a QR code or a bar code.

9. A device for processing graphic symbols implemented using the method according to any one of claims 1 to 8, characterized in that: The device comprises: An acquisition module, used for acquiring a brightness component image of an image to be processed; A binarization processing module, configured to perform binarization processing based on the brightness component image and output a binarized image; An operation module, configured to perform grayscale morphological operation based on the binary image and output a target image; Wherein, the image to be processed has graphic symbols.

10. A device for processing graphic symbols, characterized in that: The system comprises a processor and a memory, wherein the memory is used to store a program, and the processor is used to call and run the program from the memory to execute the method for processing graphic symbols according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that The medium stores a computer program, and when the computer program is run on a computer, the computer is caused to execute the method for processing graphic symbols according to any one of claims 1 to 8.

Images