Image processing method and device, electronic equipment and storage medium

Abstract

The invention provides an image processing method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining an original image, and carrying out the processing of the original image, and obtaining a mask image; the mask image comprises a target area and a background area; in the target area, determining an effective target area comprising a character area; identifying a character area and a non-character area in the effective target area; determining gray features of a character region and gray features of a non-character region in the effective target region; determining relative gray features of the character region and the non-character region based on the gray features of the character region and the gray features of the non-character region; performing feature enhancement processing on the effective target area based on the relative gray features; and performing feature optimization on the effective target area after feature enhancement processing to obtain a target image. By applying the scheme of the invention, the accuracy of character feature extraction can be greatly improved.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image processing method, apparatus, electronic device and storage medium. Background Technology

[0002] In the process of character detection accuracy, it is first necessary to acquire a design document and an actual image of the target object for comparison. The design document contains the standard character information that the target object should have, while the actual image presents the real character state on the target object. The final detection result is then obtained through comparative analysis of the two. During the comparison process, it is necessary to establish a mapping relationship between the characters in the actual image and the characters in the design document; only by clearly defining the correspondence between the two can this mapping relationship be established. However, current technologies may exhibit differences or deviations in the extraction of character features from the actual image, thus affecting the accuracy of establishing the mapping relationship and ultimately interfering with the overall accuracy of the character detection results. Summary of the Invention

[0003] This disclosure provides an image processing method, apparatus, electronic device, and storage medium to at least solve the above-mentioned technical problems existing in the prior art.

[0004] A first aspect of this disclosure provides an image processing method, the method comprising: The original image is acquired, and the original image is processed to obtain a mask image; the mask image includes a target region and a background region. Within the target area, a valid target area including the character area is determined; Identify character regions and non-character regions within the valid target area; Determine the grayscale features of character regions and non-character regions within the effective target area; Based on the grayscale features of the character region and the grayscale features of the non-character region, the relative grayscale features of the character region and the non-character region are determined. Based on the relative grayscale features, feature enhancement processing is performed on the effective target region; Feature optimization is performed on the effective target region after feature enhancement processing to obtain the target image.

[0005] In one possible implementation, processing the original image to obtain a mask image includes: Feature extraction is performed on the original image to obtain multi-scale features, including global and local features; Based on the multi-scale features, the original image is segmented to obtain the target region and the background region; A mask image is obtained based on the target region and the background region.

[0006] In one possible implementation, determining the grayscale features of character regions and non-character regions within the effective target area includes: Calculate the first average gray value of all pixels within the character region, and use the first average gray value as the gray value feature of the character region within the effective target region; Calculate the second average gray value of all pixels within the non-character region, and use the second average gray value as the gray value feature of the non-character region within the effective target region.

[0007] In one possible implementation, the mask image includes one or more target regions; The determination of a valid target region, including a character region, within the target region includes: Determine the total gradient change for each target region; If the total gradient change in a target region exceeds a preset gradient threshold, the target region is determined to be a valid target region.

[0008] In one possible implementation, the feature enhancement processing of the effective target region based on the relative grayscale features includes: In response to the relative grayscale feature indicating that the first average grayscale value is lower than the second average grayscale value, the dark detail features of the effective target region are extracted, and the dark detail features are subjected to feature enhancement processing. In response to the relative grayscale feature indicating that the first average grayscale value is not lower than the second average grayscale value, the bright detail features of the effective target region are extracted, and the bright detail features are subjected to feature enhancement processing.

[0009] In one possible implementation, the step of performing feature optimization on the effective target region after feature enhancement processing to obtain the target image includes: The effective target region after feature enhancement is subjected to convolutional filtering to obtain the filtered image; The filtered image is binarized to obtain a binarized image; The binarized image is then subjected to noise filtering to obtain the target image.

[0010] In one possible implementation, the step of filtering noise from the binarized image to obtain the target image includes: Connectivity analysis is performed on the binarized image to obtain multiple connected components; Determine the pixel dimensions of the bounding rectangle for each connected component; Filter out connected components whose pixel size information does not meet the preset character size; The target image is obtained by connecting regions that satisfy the preset character size based on pixel size information.

[0011] A second aspect of this disclosure provides an image processing apparatus, the apparatus comprising: The preprocessing module is used to acquire the original image, process the original image to obtain a mask image; the mask image includes a target region and a background region; The recognition module is used to determine a valid target region, including a character region, within the target region. The recognition module is also used to identify character regions and non-character regions within the valid target area; The recognition module is also used to determine the grayscale features of character regions and non-character regions within the effective target area; The recognition module is also used to determine the relative grayscale features of the character region and the non-character region based on the grayscale features of the character region and the grayscale features of the non-character region; An image processing module is used to perform feature enhancement processing on the effective target region based on the relative grayscale features; The image processing module is also used to perform feature optimization on the effective target region after feature enhancement processing to obtain the target image.

[0012] According to a third aspect of this disclosure, an electronic device is provided, comprising: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods described in this disclosure.

[0013] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described in this disclosure.

[0014] The image processing method, apparatus, electronic device, and storage medium disclosed herein perform targeted feature enhancement processing on effective target regions based on the grayscale feature differences between character regions and non-character regions. This avoids interference from the object's own color differences, significantly improves the separation between character regions and non-character regions, strengthens the salience of character features, and thus improves the accuracy of subsequent character feature matching.

[0015] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0016] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which: In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0017] Figure 1 A schematic diagram illustrating the implementation flow of an image processing method according to an embodiment of the present disclosure is shown; Figure 2 A schematic diagram illustrating the implementation flow of another image processing method according to an embodiment of the present disclosure is shown; Figure 3 A schematic diagram illustrating the implementation flow of another image processing method according to an embodiment of the present disclosure is shown; Figure 4 This illustration shows an application scenario diagram of an image processing method according to an embodiment of the present disclosure; Figure 5 This illustration shows an application scenario diagram of another image processing method according to an embodiment of the present disclosure; Figure 6 A schematic diagram of the composition structure of an image processing apparatus according to an embodiment of the present disclosure is shown; Figure 7 A schematic diagram of the composition structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0018] To make the objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0019] A first aspect of this disclosure provides an image processing method, such as... Figure 1 As shown, the method includes: Step 101: Obtain the original image, process the original image to obtain a mask image; the mask image includes the target area and the background area.

[0020] First, photographs are taken of objects containing features for which characters to be extracted, such as the C-side of a laptop or the operating area of ​​an instrument, to obtain their raw images. If the acquired raw images are color images, they are first converted to grayscale images. Then, a segmentation model is used to analyze the grayscale images, identifying and segmenting regions that may contain the characters to be extracted—the target regions—based on the object's pre-defined physical shape (such as geometric shape, size regularity, and other structural features). Examples include the keycap area on the C-side of a laptop or the button area in the operating area of ​​an instrument. Finally, the target regions are masked, and the remaining areas are classified as background regions, resulting in a masked image that distinguishes the target regions from the background regions.

[0021] Step 102: In the target area, determine the valid target area that includes the character area.

[0022] The target regions obtained in the previous step are regions that may contain the characters to be extracted, based on the segmentation of the object's physical shape. Therefore, there may be target regions that do not contain the characters to be extracted. For example, the space bar in the keycap area of ​​a laptop's C-side, or function keys without character labels in the operation area of ​​an instrument. This step uses pixel feature analysis to continue to detect each target region one by one, filtering out valid target regions that contain the characters to be extracted. Specifically, pixel feature analysis methods such as gradient change analysis, edge contour detection, and character texture recognition can be used to detect the target regions one by one: if a region has gradient changes, edge contours, or texture features that match the character shape, it is determined to be a valid target region containing the characters to be extracted; otherwise, it is determined to be an invalid target region.

[0023] Step 103: Identify character regions and non-character regions within the valid target area.

[0024] Based on the differences in pixel features within the effective target area, character regions and non-character regions are distinguished. Specifically, identification can be based on the inherent differences in grayscale value distribution and area between character and non-character regions. For example, if the area occupied by non-character regions in the effective target area is significantly larger than that of character regions, then the set of pixels whose grayscale values ​​deviate from the main grayscale range is identified as character regions. The set of pixels whose grayscale values ​​are close to the main grayscale range is identified as non-character regions. If the area difference between character and non-character regions is not significant, they can be distinguished by detecting character-specific structural features. For example, gradient detection algorithms can be used to capture the grayscale transition boundaries between character and non-character regions, and morphological operations can be used to filter out regions that conform to the size and shape rules of characters, etc. This step does not limit the specific identification methods.

[0025] Step 104: Determine the grayscale features of the character regions and the grayscale features of the non-character regions within the effective target area.

[0026] For character regions, parameters such as average grayscale value, maximum grayscale value, minimum grayscale value, and grayscale variance can be calculated as their grayscale features. For non-character regions, the same statistical dimensions are used to calculate their grayscale features, ensuring that the grayscale features of the two types of regions are comparable. These grayscale features are used to characterize the overall distribution pattern and quantization attributes of pixel grayscale values ​​within their respective regions.

[0027] Step 105: Determine the relative grayscale features of the character region and the non-character region based on the grayscale features of the character region and the non-character region.

[0028] Relative grayscale features characterize the contrast between character and non-character regions. Specifically, the relative grayscale feature can be determined by calculating the difference between the corresponding grayscale features of the two types of regions. For example, subtracting the grayscale feature of the non-character region from the grayscale feature of the character region will result in a negative value, indicating that the character region is darker than the non-character region, and a positive value, indicating that the character region is brighter than the non-character region. Furthermore, the relative grayscale feature of two regions can also be determined through the ratio of their grayscale features or the degree of grayscale deviation.

[0029] Step 106: Based on the relative grayscale features, perform feature enhancement processing on the effective target region.

[0030] Specifically, if the relative grayscale feature indicates that the character area is darker than the non-character area, that is, the non-character area is mainly characterized by brightness and the character area is mainly characterized by darkness, then the original image is enhanced with dark detail features by increasing the grayscale contrast of dark pixels and strengthening the character edge contours to highlight the character features in the image. If the relative grayscale feature indicates that the character area is brighter than the non-character area, that is, the non-character area is mainly characterized by darkness and the character area is mainly characterized by lightness, then the image is enhanced with bright detail features by reducing the brightness of the dark background and optimizing the bright pixels to highlight the character features in the image.

[0031] Step 107: Perform feature optimization on the effective target region after feature enhancement processing to obtain the target image.

[0032] Finally, convolutional filtering is applied to the enhanced target region. The convolution kernel strengthens the contour edges and details of the character region, further widening the grayscale difference between the character and non-character regions and improving their separation. Then, noise filtering and other operations are performed to remove redundant interference. The final output is a target image with clear character features and no redundant interference.

[0033] The solution in this embodiment is based on the difference in grayscale features between character regions and non-character regions. It performs targeted feature enhancement processing on the effective target region, avoids the interference of the object's own color differences, significantly improves the separation between character regions and non-character regions, strengthens the salience of character features, and thus improves the accuracy of subsequent character feature matching.

[0034] In another embodiment of this disclosure, processing the original image to obtain a mask image includes: extracting features from the original image to obtain multi-scale features including global and local features; segmenting the original image based on the multi-scale features to obtain a target region and a background region; and obtaining a mask image based on the target region and the background region.

[0035] First, feature extraction is performed on the original image: a basic network (such as a convolutional neural network) captures the low-level visual information of the image, and then global and local features are generated. Global features correspond to macroscopic information such as the overall layout and shape of the target region in the image, while local features correspond to detailed information such as the edges and textures of the target region. These two types of features are then fused to obtain multi-scale features encompassing information at different scales. Next, based on these multi-scale features, the original image is segmented into target regions requiring focused processing and background regions that do not need attention. Finally, target regions and background regions are distinguished and labeled using different identifiers (such as different colors or values), resulting in a masked image that clearly defines the two types of regions.

[0036] This embodiment first extracts global and local features from the image and fuses them to generate multi-scale features. Then, based on these multi-scale features, it performs accurate segmentation of the target and background regions, ultimately generating a mask image that clearly marks the two types of regions. This approach balances macroscopic layout and detailed features, effectively improving the accuracy of region segmentation and providing a clear and reliable processing range for subsequent character region recognition.

[0037] In another embodiment of this disclosure, determining the grayscale features of the character region and the grayscale features of the non-character region within the effective target area includes: calculating a first average grayscale value of all pixels within the character region, and using the first average grayscale value as the grayscale feature of the character region within the effective target area; calculating a second average grayscale value of all pixels within the non-character region, and using the second average grayscale value as the grayscale feature of the non-character region within the effective target area.

[0038] Specifically, for character regions, the grayscale values ​​of all pixels within the region are first obtained and their sum is calculated, while the total number of pixels in the region is also counted. The first average grayscale value of all pixels within the character region is calculated by using the ratio of the sum of grayscale values ​​to the total number of pixels. Similarly, for non-character regions, the sum of grayscale values ​​of all pixels within the region and the total number of pixels are first obtained, and then the second average grayscale value of all pixels within the non-character region is calculated by using the ratio of the sum of grayscale values ​​to the total number of pixels.

[0039] The scheme in this embodiment uses the average gray value of the character region and the non-character region as the gray value feature of their respective regions, which can accurately characterize the overall brightness and darkness attributes of the two types of regions, and the calculation logic is simple and efficient.

[0040] In another embodiment of this disclosure, the mask image includes one or more target regions; determining a valid target region including a character region within the target region includes: determining the total gradient change of each target region; if the total gradient change of a target region is greater than a preset gradient threshold in each target region, the target region is determined to be a valid target region.

[0041] Each mask image may contain one or more target regions. For example, the mask image corresponding to the C-side of a laptop may contain multiple keycaps, i.e., multiple target regions. The mask image corresponding to the operation panel of some instruments may contain only one key, i.e., a single target region. In this embodiment, each target region needs to be detected one by one to filter out the valid target regions that actually contain character regions.

[0042] Specifically, firstly, gradient detection is performed on each target region, such as Sobel gradient detection or Prewitt gradient detection, to calculate the gradient values ​​of all pixels within that region. Then, the gradient values ​​of all pixels are summed, or the sum of squares is calculated, to obtain the total gradient change of the target region. The total gradient change is a quantitative representation of the degree of abrupt change in pixel grayscale values ​​within the region. Since a significant difference in pixel grayscale values ​​forms at the boundary between character and non-character regions, regions containing characters will exhibit a higher total gradient change. Next, a preset gradient threshold is obtained. This preset gradient threshold can be pre-determined based on statistical analysis of sample images. Then, the total gradient change is compared with the preset gradient threshold. If the total gradient change of the target region exceeds the preset gradient threshold, it indicates a significant grayscale abrupt change within the region, and the target region can be determined to include character regions, making it a valid target region. Conversely, if the total gradient change does not exceed the preset gradient threshold, it indicates a relatively uniform grayscale distribution within the region, and the target region can be determined to not include character regions, making it an invalid target region.

[0043] In one possible implementation, if all character regions in the same mask image have the same grayscale distribution characteristics (such as all keycap characters on the same keyboard having the same grayscale contrast mode with the background), then based on any determined valid target region, the grayscale features and relative grayscale features of its character regions and non-character regions can be calculated, without needing to repeatedly calculate the relative grayscale features of other valid target regions. At this point, target regions can be sequentially determined to be valid target regions: if the current target region is determined to be a valid target region, the determination process for subsequent target regions can be stopped; if the current target region is determined to be an invalid target region, the detection of the next target region continues until at least one valid target region is selected.

[0044] The solution in this embodiment determines whether a target area contains a character region by measuring the total gradient change within that region. This approach is logically simple, less susceptible to interference from the object's own color and lighting, and accurately filters out valid target areas.

[0045] In another embodiment of this disclosure, the step of performing feature enhancement processing on the effective target region based on the relative grayscale features includes: in response to the relative grayscale features indicating that the first average grayscale value is lower than the second average grayscale value, extracting dark detail features of the effective target region and performing feature enhancement processing on the dark detail features; in response to the relative grayscale features indicating that the first average grayscale value is not lower than the second average grayscale value, extracting bright detail features of the effective target region and performing feature enhancement processing on the bright detail features.

[0046] In this context, when the relative grayscale feature, representing the first average grayscale value, is lower than the second average grayscale value, it means that the character region exists as dark details in the non-character region, such as a black character on a white keycap. In this case, a morphological opening operation (erosion followed by dilation) can be used to extract the bright background component from the effective target region. Subtracting this bright background component from the original effective target region image separates the dark detail features, which are the character features. Simultaneously, this difference operation enhances the dark detail features, amplifying the grayscale contrast between the character and non-character regions. Conversely, when the first average grayscale value is not lower than the second average grayscale value, it indicates that the character exists as bright details in the non-character region, such as a white character on a black keycap. In this case, a morphological closing operation (dilation followed by erosion) can be used to extract the dark background component from the effective target region. Subtracting this dark background component from the original effective target region image separates the bright detail features, which are the character features. Simultaneously, this difference operation enhances the bright detail features, amplifying the grayscale contrast between the character and non-character regions.

[0047] This embodiment determines the contrast between the brightness of characters and non-character areas by using relative grayscale features, and performs targeted feature enhancement processing to ensure that character features can be effectively highlighted in different brightness scenarios such as bright background with dark characters and dark background with bright characters.

[0048] In another embodiment of this disclosure, the step of performing feature optimization on the effective target region after feature enhancement processing to obtain a target image includes: performing convolution filtering on the effective target region after feature enhancement processing to obtain a filtered image; performing binarization processing on the filtered image to obtain a binarized image; and performing noise filtering on the binarized image to obtain the target image.

[0049] Since the position of the machine in the camera's field of view is not fixed, the enhanced image may exhibit inconsistent brightness. To further highlight the contrast between character features and the background, convolutional filtering is required on the enhanced image. By weakening non-target noise and refining character edges, the gray-level contrast between character features and the background is indirectly improved, resulting in a filtered image. Specifically, a preset convolutional kernel (such as a mean filter kernel or a Gaussian filter kernel) is used to traverse the image pixels, performing a weighted average or smoothing operation based on the gray levels of neighboring pixels. While preserving the core outline and details of the characters, scattered non-target noise is weakened, ultimately resulting in a filtered image with more regular edges and preliminary noise suppression. Then, to completely separate the gray-level gradient between the characters and the background and make the character outlines clearer, the filtered image is binarized. This involves setting an appropriate gray-level threshold to uniformly map image pixels to two gray-level values: 0 and 255, resulting in a binary image containing only black and white values. Finally, to address the isolated noise that may be generated after binarization, a noise filtering algorithm can be used to remove isolated pixels or interference regions with an area smaller than a set threshold, while retaining complete character connected regions. This results in a target image with clean character features and no background interference, providing a high-quality feature carrier for subsequent character recognition.

[0050] In another embodiment of this disclosure, the step of filtering noise from the binarized image to obtain a target image includes: performing connected component analysis on the binarized image to obtain multiple connected components; determining the pixel size information of the bounding rectangle of each connected component; filtering out connected components whose pixel size information does not meet the preset character size; and obtaining the target image based on connected components whose pixel size information meets the preset character size.

[0051] Specifically, connected component analysis is performed on the binarized image to obtain multiple connected components. A connected component refers to an interconnected region in the image composed of pixels with the same grayscale value. Next, the pixel size information of the bounding rectangle of each connected component is extracted. Specifically, the minimum and maximum row and column indices of each connected component are determined: in the row direction, the row coordinates of all pixels within the connected component are traversed, and the minimum and maximum row indices are recorded; similarly, in the column direction, the column coordinates of all pixels are traversed, and the minimum and maximum column indices are obtained. Then, the height pixel value of the bounding rectangle is obtained by calculating the difference between the maximum and minimum row indices. The width pixel value of the bounding rectangle is obtained by calculating the difference between the maximum and minimum column indices. The height and width pixel values ​​constitute the pixel size information of the bounding rectangle.

[0052] After obtaining the pixel size information of all connected components, a filtering process is performed using a preset character size. The preset character size is a pre-defined range based on the actual size of the target character, including width and height threshold ranges. The pixel size information of each connected component is compared with the preset character size. If the size information of a connected component is outside the range specified by the preset character size, it is filtered out. If it is within the range specified by the preset character size, it is retained. Finally, the target image is obtained based on all retained connected components.

[0053] The solution in this embodiment can effectively distinguish noise from characters through size features, which ensures both filtering accuracy and high processing efficiency.

[0054] To better understand the embodiments of this disclosure, the following uses the C-side of a laptop computer as an example to describe the above embodiments in detail. Figure 2 As shown, this example includes the following steps: 1. Obtain the original image of the C-side of the laptop. For the input raw image Perform color space conversion to convert a color image to a grayscale image. ; 2. Convert the grayscale image The image is input into the Fast SAM pre-trained model to obtain the mask image. Its expression can be: ,in, The visualized segmentation rendering image output synchronously by the model can be used to verify the segmentation effect.

[0055] Specific steps are as follows Figure 3 As shown: grayscale image After entering the Image Encoder module, the CNN Backbone first extracts basic keyboard features from the grayscale image, such as the shape and edges of the keycaps, and outputs a corresponding keyboard feature map. Then, the Feature Pyramid Network (FPN) generates multi-scale features based on this feature map, thereby enhancing the model's ability to detect small targets like keycaps. Finally, the Mask Decoder combines the multi-scale features output by the FPN to generate a segmentation mask for the keyboard keycaps. In the final output, each bounding box corresponds to an independent keycap region, i.e., the target region. This yields a mask image that distinguishes the keycap region (target region) from the background region. .

[0056] 3. For masked images Sort all keycap areas to obtain a set. ,in n This indicates the number of keycap areas. Then, according to preset position coordinate rules, such as the physical arrangement of the keyboard areas from left to right and top to bottom, each keycap area in set R is sorted, resulting in a sorted set. ; 4. In the set In the middle, randomly select a keycap area. Through character detection function This determines whether the area contains printed characters. Specifically, it determines whether the selected keycap area contains printed characters. The Sobel operator is applied to calculate the horizontal and vertical gradients separately. The pixel grayscale gradients from both directions are then combined to generate a gradient magnitude image that highlights edge features. The total gradient change T of all pixels in this image is calculated and compared with a preset gradient threshold. Comparison. If T exceeds This indicates that there are edge features within the region that conform to the shape of printed characters, and a determination is made. ,Right now It is the keycap area containing printed characters; if T does not exceed Then determine ,Right now It is the keycap area that does not contain printed characters; 5. For keycap areas containing printed characters The region consisting of its non-zero pixels is defined as First calculate Average gray value of all pixels And introduce a grayscale deviation parameter σ. This will adjust the grayscale value in... arrive Pixels within the range are divided into the background pixel set. ,Right now ,in Representing coordinates The grayscale value of that pixel. Then, statistics are calculated. Number of pixels and the sum of the grayscale values ​​of all pixels in the set. Furthermore, through The average grayscale value of the non-character region is calculated, i.e., the second average grayscale value, which is used to characterize the grayscale features of the non-character region. A set of pixels for the character region is defined. for Except The remaining pixels outside are mathematically defined as .statistics Number of pixels and the sum of gray values ,pass The average gray value of the character region is calculated, which is the first average gray value and is used to characterize the gray-scale features of the character region.

[0057] 6. Grayscale features based on character regions and grayscale features of non-character areas Determine the brightness relationship between character regions and non-character regions. Specifically, if... This indicates that the overall brightness of non-character areas is higher than that of character areas, meaning the keyboard surface is predominantly bright, and characters appear as dark details, such as black characters on white keycaps. In this case, a dark detail enhancement method is used. Generate enhanced feature image ;like Less than This indicates that the brightness of the character area is higher than that of the non-character area, meaning the keyboard surface is predominantly dark with bright details on the characters, such as white characters on black keycaps. In this case, a bright detail enhancement method is used. Generate enhanced feature image .in, This represents the processed image. B represents the input image, and B represents the structuring element used to define the shape and size of the domain. This indicates an expansion operation. This indicates a corrosion operation.

[0058] 7. Because the camera's position within the imaging field of view is not fixed during shooting, inconsistencies in brightness may occur after processing steps 1-6. Therefore, this step first processes the image... A convolutional filtering enhancement operation is performed. Using a specific convolutional kernel, each pixel in the image and its neighboring pixels are weighted and calculated to suppress uneven background interference and enhance the grayscale differences between character edges and details. In other words, the filtering operation improves the grayscale contrast between character features and the background, making the character outlines clearer. Then, a thresholding operation is performed on the convolutionally filtered image to finally obtain a binary image L containing only black and white pixels.

[0059] 8. The pixel grayscale values ​​in the binarized image L contain only 0 and 255. Label each non-zero independent connected region in the binarized image L as k. The specific steps are as follows: First, perform connected component labeling processing on the binarized image L(x,y) to obtain the labeled image. Assign a unique number k to each connected component, where Let N represent the total number of connected components in the image. For each connected component, calculate its minimum bounding rectangle. The specific steps are as follows: First, find the extreme values ​​of the row and column indices of all pixels (x, y) in the connected component k: the minimum row index. Maximum row index Minimum column index Maximum column index Then, based on the aforementioned index extrema, calculate the circumscribed rectangle. The width and height are used as the pixel size information of the bounding rectangle, where the width is... The height is .

[0060] 9. For each connected component, the external rectangle... Based on its pixel width and pixel height Perform size verification and filtering. Pre-set the standard pixel height for each character. and standard pixel width This serves as the preset character size. The target image is obtained through size matching and filtering. : ,in, This indicates the allowable error range for the area width. This indicates the permissible error range for the area height.

[0061] refer to Figure 4 It displays a scene on the C-side of a laptop where the keycaps are dark and the characters are light, and the corresponding processed target image. (Reference) Figure 5 It displays a scene on the C-side of a laptop where the keycaps are light-colored and the characters are dark-colored, and the corresponding processed target image.

[0062] In this example, the FastSAM model is used to achieve high-quality segmentation of the keyboard image, laying a precise regional foundation for subsequent character feature extraction. The differences between the characters within the keycaps and the background are analyzed, and different methods are used to highlight the detailed features of the characters, avoiding the influence of the keyboard's own color features. Then, the set features of the character regions are used to filter and retain valid character regions. This provides high-purity, high-discrimination target data for subsequent character feature matching, thereby significantly improving the accuracy and reliability of feature matching.

[0063] According to a second aspect of the embodiments of this disclosure, an image processing apparatus is provided, such as... Figure 6 As shown, the device includes: A preprocessing module 201 is used to acquire an original image and process the original image to obtain a mask image; the mask image includes a target region and a background region; a recognition module 202 is used to determine a valid target region, including a character region, within the target region; the recognition module 202 is also used to recognize character regions and non-character regions within the valid target region; the recognition module 202 is also used to determine the grayscale features of the character regions and the grayscale features of the non-character regions within the valid target region; the recognition module 202 is also used to determine the relative grayscale features between the character regions and the non-character regions based on the grayscale features of the character regions and the grayscale features of the non-character regions; an image processing module 203 is used to perform feature enhancement processing on the valid target region based on the relative grayscale features; the image processing module 203 is also used to perform feature optimization on the feature-enhanced valid target region to obtain a target image.

[0064] In another embodiment of this disclosure, the preprocessing module 201 is further configured to extract features from the original image to obtain multi-scale features including global features and local features; based on the multi-scale features, the original image is segmented to obtain a target region and a background region; and based on the target region and the background region, a mask image is obtained.

[0065] In another embodiment of this disclosure, the recognition module 202 is further configured to calculate a first average gray value of all pixels within the character region, and use the first average gray value as the gray value of the character region within the effective target region; calculate a second average gray value of all pixels within the non-character region, and use the second average gray value as the gray value of the non-character region within the effective target region.

[0066] In another embodiment of this disclosure, the identification module 202 is further configured to determine the total gradient change of each target region; if the total gradient change of a target region exceeds a preset gradient threshold, the target region is determined to be a valid target region.

[0067] In another embodiment of this disclosure, the image processing module 203 is further configured to, in response to the relative grayscale feature indicating that the first average grayscale value is lower than the second average grayscale value, extract dark detail features of the effective target region and perform feature enhancement processing on the dark detail features; and in response to the relative grayscale feature indicating that the first average grayscale value is not lower than the second average grayscale value, extract bright detail features of the effective target region and perform feature enhancement processing on the bright detail features.

[0068] In another embodiment of this disclosure, the image processing module 203 is further configured to perform convolution filtering on the effective target region after feature enhancement processing to obtain a filtered image; perform binarization processing on the filtered image to obtain a binarized image; and perform noise filtering on the binarized image to obtain a target image.

[0069] In another embodiment of this disclosure, the image processing module 203 is further configured to perform connected component analysis on the binarized image to obtain multiple connected components; determine the pixel size information of the bounding rectangle of each connected component; filter out connected components whose pixel size information does not meet the preset character size; and obtain the target image based on the connected components whose pixel size information meets the preset character size.

[0070] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium.

[0071] Figure 7 A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0072] like Figure 7 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0073] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0074] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as image processing methods. For example, in some embodiments, the image processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform image processing methods by any other suitable means (e.g., by means of firmware).

[0075] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0076] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0077] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0078] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0079] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0080] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0081] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this disclosure can be achieved, and this is not limited herein.

[0082] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0083] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, The method includes: The original image is acquired, and the original image is processed to obtain a mask image; the mask image includes a target region and a background region. Within the target area, a valid target area including the character area is determined; Identify character regions and non-character regions within the valid target area; Determine the grayscale features of character regions and non-character regions within the effective target area; Based on the grayscale features of the character region and the grayscale features of the non-character region, the relative grayscale features of the character region and the non-character region are determined. Based on the relative grayscale features, feature enhancement processing is performed on the effective target region; Feature optimization is performed on the effective target region after feature enhancement processing to obtain the target image.

2. The image processing method according to claim 1, characterized in that, The process of processing the original image to obtain the mask image includes: Feature extraction is performed on the original image to obtain multi-scale features, including global and local features; Based on the multi-scale features, the original image is segmented to obtain the target region and the background region; A mask image is obtained based on the target region and the background region.

3. The image processing method according to claim 1, characterized in that, The determination of the grayscale features of character regions and non-character regions within the effective target area includes: Calculate the first average gray value of all pixels within the character region, and use the first average gray value as the gray value feature of the character region within the effective target region; Calculate the second average gray value of all pixels within the non-character region, and use the second average gray value as the gray value feature of the non-character region within the effective target region.

4. The image processing method according to claim 1, characterized in that, The mask image includes one or more target regions; The determination of a valid target region, including a character region, within the target region includes: Determine the total gradient change for each target region; If the total gradient change in a target region exceeds a preset gradient threshold, the target region is determined to be a valid target region.

5. The image processing method according to claim 3, characterized in that, The feature enhancement processing of the effective target region based on the relative grayscale features includes: In response to the relative grayscale feature indicating that the first average grayscale value is lower than the second average grayscale value, the dark detail features of the effective target region are extracted, and the dark detail features are subjected to feature enhancement processing. In response to the relative grayscale feature indicating that the first average grayscale value is not lower than the second average grayscale value, the bright detail features of the effective target region are extracted, and the bright detail features are subjected to feature enhancement processing.

6. The image processing method according to claim 1, characterized in that, The step of performing feature optimization on the effective target region after feature enhancement processing to obtain the target image includes: The effective target region after feature enhancement is subjected to convolutional filtering to obtain the filtered image; The filtered image is binarized to obtain a binarized image; The binarized image is then subjected to noise filtering to obtain the target image.

7. The image processing method according to claim 6, characterized in that, The step of filtering noise from the binarized image to obtain the target image includes: Connectivity analysis is performed on the binarized image to obtain multiple connected components; Determine the pixel dimensions of the bounding rectangle for each connected component; Filter out connected components whose pixel size information does not meet the preset character size; The target image is obtained by connecting regions that satisfy the preset character size based on pixel size information.

8. An image processing apparatus, characterized in that, The device includes: The preprocessing module is used to acquire the original image, process the original image to obtain a mask image; the mask image includes a target region and a background region; The recognition module is used to determine a valid target region, including a character region, within the target region. The recognition module is also used to identify character regions and non-character regions within the valid target area; The recognition module is also used to determine the grayscale features of character regions and non-character regions within the effective target area; The recognition module is also used to determine the relative grayscale features of the character region and the non-character region based on the grayscale features of the character region and the grayscale features of the non-character region; An image processing module is used to perform feature enhancement processing on the effective target region based on the relative grayscale features; The image processing module is also used to perform feature optimization on the effective target region after feature enhancement processing to obtain the target image.

9. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

Images