Character defect detection method and device and electronic equipment

Abstract

The embodiment of the invention provides a character defect detection method and device and electronic equipment. The electronic equipment can obtain a to-be-detected image and extract image features; dividing the character region into a plurality of to-be-detected regions according to the region division parameters; dividing the image features into area image features of the areas to be detected according to feature division parameters; for each to-be-detected region, determining the similarity of the to-be-detected region according to the similarity between the region image feature of the to-be-detected region and each sample feature; and determining a character defect detection result based on the similarity of each to-be-detected area. As the sample image region belongs to a defect-free sample image, the sample feature is the feature of the sample image region. If the similarity between the image features of the to-be-detected image and the sample features is small, the to-be-detected image has defects. Therefore, the detection result can be determined based on the similarity of each to-be-detected area, multiple models do not need to be trained, and the complexity of character defect detection can be reduced.

Description

Technical Field

[0001] This application relates to the field of defect detection technology, and in particular to a character defect detection method, apparatus and electronic device. Background Technology

[0002] Character defect detection is a technique for detecting character defects in character images. Character defects include dirt, wear, spots, deformation, and ghosting. Character defect detection can be used to detect defects in printed text information on product surfaces, labels, etc. Current character defect detection methods require training multiple models in stages to achieve detection. Specifically, first, a matching and alignment network is trained to correct sample positions; then, an unsupervised model is trained based on the output of the matching and alignment network; finally, the output of the unsupervised model is used as labels to train a weakly supervised defect detection model. The trained model then enables the detection of character defects in character images.

[0003] The above-mentioned character defect detection methods require training multiple models in stages, which is a complex and cumbersome process, resulting in high complexity in character defect detection. Summary of the Invention

[0004] The purpose of this application is to provide a character defect detection method, apparatus, and electronic device to reduce the complexity of character defect detection. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of this application provide a character defect detection method, the method comprising:

[0006] Acquire an image to be detected, wherein the image to be detected includes characters;

[0007] Extract image features from the character regions in the image to be detected;

[0008] According to the pre-stored region division parameters, the character region is divided into multiple regions to be detected, wherein the region division parameters are parameters corresponding to the region division method for dividing the sample character region in the defect-free sample image that are pre-recorded.

[0009] According to the pre-stored feature segmentation parameters, the image features are divided into region image features corresponding to each region to be detected. The feature segmentation parameters are obtained by transforming the region segmentation parameters based on the size relationship between the image features and the character regions in the image to be detected.

[0010] For each region to be detected, the similarity of the region to be detected is determined based on the similarity between the region image features of the region to be detected and the pre-stored sample features. The sample features are the features of the sample image regions obtained by dividing the sample character regions in the defect-free sample image according to the region division method.

[0011] Based on the similarity corresponding to each region to be detected, the character defect detection result of the image to be detected is determined.

[0012] Optionally, before the step of acquiring the image to be detected, the method further includes:

[0013] Acquire multiple defect-free sample images;

[0014] Extract image features from the character regions of the sample in the defect-free sample image;

[0015] Based on the size of the sample character region, determine the number of regions corresponding to the sample character region, and based on the grayscale distribution of the sample character region, determine the size of the sample image region to be divided.

[0016] The sample character region is divided according to the region division method to obtain multiple sample image regions corresponding to each defect-free sample image.

[0017] Based on the grayscale distribution of the sample image region, the image features corresponding to the sample image region are sampled to obtain sample features;

[0018] The sample features and the region division parameters that characterize the region division method are stored as a sample feature library.

[0019] Optionally, before the step of sampling the image features corresponding to the sample image region based on the grayscale distribution of the sample image region to obtain the sample features, the method further includes:

[0020] The feature partitioning method is determined based on the size relationship between the sample character region and the image features of the sample character region, as well as the region partitioning method.

[0021] According to the feature division method, the image features of the sample character region are divided to obtain the image features corresponding to each sample image region;

[0022] The feature partitioning parameters that characterize the feature partitioning method are stored in the sample feature library.

[0023] Optionally, the step of extracting image features of the sample character region in the defect-free sample image includes:

[0024] For each defect-free sample image, the sample character region in the defect-free sample image is input into a pre-trained feature extractor to obtain the intermediate layer features of the sample character region in the defect-free sample image extracted by the intermediate layer of the feature extractor.

[0025] The intermediate layer features are input from the attention layer to introduce global information into the intermediate layer features, thereby obtaining global features;

[0026] The intermediate layer features are input into the local feature aggregation module to obtain the local features output by the local feature aggregation module;

[0027] The global features and the local features are stacked according to channels to form the image features of the sample character region in the defect-free sample image.

[0028] Optionally, the step of sampling image features corresponding to the sample image region based on the grayscale distribution of the sample image region to obtain sample features includes:

[0029] Based on the grayscale distribution of the sample image region, the image features corresponding to the sample image region are sampled according to the preset sampling constraints to obtain the sample features;

[0030] The preset sampling constraint ensures that the proportion of image features corresponding to sample image regions with fewer than a preset number of grayscale value types is less than a preset proportion, the proportion of image features corresponding to sample image regions with more than or equal to the preset number of grayscale value types is not less than the preset proportion, and the maximum distance between the sampled image features is within a preset range.

[0031] Optionally, the step of determining the similarity of the region to be detected based on the similarity between the region image features of the region to be detected and the features of each pre-stored sample includes:

[0032] Calculate the feature distance map between the regional image features of the region to be detected and the features of each sample in the sample feature library;

[0033] The feature distance map with the smallest distance between the region image features of the region to be detected and the region to be detected is used as the anomaly localization segmentation map corresponding to the region to be detected. The anomaly localization segmentation map is used to identify the maximum similarity between the region to be detected and each sample feature in the sample feature library.

[0034] Optionally, the step of determining the character defect detection result of the image to be detected based on the similarity corresponding to each region to be detected includes:

[0035] According to the position of each region to be detected in the image to be detected represented by the region division parameters, the anomaly localization segmentation maps corresponding to each region to be detected are merged to obtain the anomaly localization map corresponding to the image to be detected.

[0036] Based on a predetermined maximum distance, the anomaly localization map is normalized to obtain a normalized anomaly localization map, wherein the maximum distance is the maximum distance between the sample features;

[0037] If the maximum value of the pixel value in the normalized anomaly localization map is greater than the preset anomaly score, the detection area corresponding to the pixel value greater than the preset anomaly score is determined, and the determined detection area is determined as the character defect area in the image to be detected.

[0038] If the maximum pixel value of the normalized anomaly localization map is not greater than the preset anomaly score, it is determined that the image to be detected does not have character defects.

[0039] Secondly, embodiments of this application provide a character defect detection device, the device comprising:

[0040] An image acquisition module is used to acquire an image to be detected, wherein the image to be detected includes characters;

[0041] The feature extraction module is used to extract image features of character regions in the image to be detected;

[0042] The region segmentation module is used to divide the character region into multiple regions to be detected according to the pre-stored region segmentation parameters, wherein the region segmentation parameters are a pre-recorded region segmentation method for dividing the sample character region in a defect-free sample image.

[0043] The feature segmentation module is used to segment the image features into region image features corresponding to each region to be detected according to the pre-stored feature segmentation parameters, wherein the feature segmentation parameters are obtained by transforming the region segmentation parameters based on the size relationship between the image features and the character regions in the image to be detected;

[0044] The similarity determination module is used to determine the similarity of each region to be detected based on the similarity between the region image features of the region to be detected and the pre-stored sample features. The sample features are the features corresponding to the sample image regions obtained by dividing the sample character regions in the defect-free sample image according to the region division method.

[0045] The result determination module is used to determine the character defect detection result of the image to be detected based on the similarity corresponding to each region to be detected.

[0046] Optionally, the device further includes:

[0047] The sample image acquisition module is used to acquire multiple defect-free sample images;

[0048] The sample feature extraction module is used to extract image features of the sample character region in the defect-free sample image;

[0049] The region division method determination module is used to determine the number of regions corresponding to the sample character region based on the size of the sample character region, and to determine the size of the sample image region to be divided based on the grayscale distribution of the sample character region.

[0050] The sample region division module is used to divide the sample character region according to the region division method to obtain multiple sample image regions corresponding to each defect-free sample image.

[0051] The feature sampling module is used to sample the image features corresponding to the sample image region based on the grayscale distribution of the sample image region, so as to obtain the sample features;

[0052] The feature library construction module is used to store the sample features and the region division parameters that characterize the region division method as a sample feature library.

[0053] Optionally, the device further includes:

[0054] The feature segmentation method determination module is used to determine the feature segmentation method based on the size relationship between the sample character region and the image features of the sample character region and the region segmentation method;

[0055] The image feature segmentation module is used to segment the image features of the sample character region according to the feature segmentation method, so as to obtain the image features corresponding to each sample image region.

[0056] The parameter storage module is used to store the feature partitioning parameters that characterize the feature partitioning method into the sample feature library.

[0057] Optionally, the sample feature extraction module includes:

[0058] The feature extraction submodule is used to input the defect-free sample image into a pre-trained feature extractor for each defect-free sample image in order to obtain the intermediate layer features of the sample character region in the defect-free sample image extracted by the intermediate layer of the feature extractor.

[0059] The global feature acquisition submodule is used to input the intermediate layer features from the attention layer to introduce global information into the intermediate layer features and obtain global features;

[0060] The local feature acquisition submodule is used to input the intermediate layer features into the local feature aggregation module to obtain the local features output by the local feature aggregation module;

[0061] The feature stacking submodule is used to stack the global features and the local features according to channels, as image features of the sample character region in the defect-free sample image.

[0062] Optionally, the feature sampling module includes:

[0063] The feature sampling submodule is used to sample the image features corresponding to the sample image region based on the grayscale distribution of the sample image region and according to the preset sampling constraints to obtain the sample features;

[0064] The preset sampling constraint ensures that the proportion of image features corresponding to sample image regions with fewer than a preset number of grayscale value types is less than a preset proportion, the proportion of image features corresponding to sample image regions with more than or equal to the preset number of grayscale value types is not less than the preset proportion, and the maximum distance between the sampled image features is within a preset range.

[0065] Optionally, the similarity determination module includes:

[0066] The distance map calculation submodule is used to calculate the feature distance map between the regional image features of the region to be detected and the features of each sample in the sample feature library.

[0067] The segmentation map determination submodule is used to take the feature distance map with the smallest distance between the region image features represented by the multiple calculated feature distance maps and the region to be detected as the anomaly localization segmentation map corresponding to the region to be detected. The anomaly localization segmentation map is used to identify the maximum similarity between the region to be detected and each sample feature in the sample feature library.

[0068] Optionally, the result determination module includes:

[0069] The segmentation map merging submodule is used to merge the anomaly localization segmentation maps corresponding to each region to be detected according to the position of each region to be detected in the image to be detected represented by the region division parameters, so as to obtain the anomaly localization map corresponding to the image to be detected.

[0070] The normalization processing submodule is used to normalize the anomaly localization map based on a predetermined maximum distance to obtain a normalized anomaly localization map, wherein the maximum distance is the maximum distance between the sample features;

[0071] The first result determination submodule is used to determine the detection area corresponding to the pixel value greater than the preset anomaly score if the maximum value of the pixel value of the normalized anomaly localization map is greater than the preset anomaly score, and to determine the determined detection area as the character defect area in the image to be detected.

[0072] The second result determination submodule is used to determine that the image to be detected does not have character defects if the maximum value of the pixel value of the normalized anomaly localization map is not greater than the preset anomaly score.

[0073] Thirdly, embodiments of this application provide an electronic device, including:

[0074] Memory, used to store computer programs;

[0075] When a processor executes a program stored in memory, it implements any of the methods described in the first aspect above.

[0076] Fourthly, a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the methods described in the first aspect.

[0077] Beneficial effects of the embodiments in this application:

[0078] In the technical solution provided in this application embodiment, an electronic device can acquire an image to be detected, wherein the image to be detected includes characters; extract image features of the character regions in the image to be detected; divide the character regions into multiple regions to be detected according to pre-stored region division parameters, wherein the region division parameters are parameters corresponding to the region division method for dividing the sample character regions in a defect-free sample image that are pre-recorded; divide the image features into region image features corresponding to each region to be detected according to pre-stored feature division parameters, wherein the feature division parameters are obtained by transforming the region division parameters based on the size relationship between the image features and the character regions in the image to be detected; for each region to be detected, determine the similarity corresponding to the region to be detected based on the similarity between the region image features of the region to be detected and each pre-stored sample feature, wherein the sample feature is the feature corresponding to the sample image region obtained by dividing the sample character regions in a defect-free sample image according to the region division method; and determine the character defect detection result of the image to be detected based on the similarity corresponding to each region to be detected. Since the sample image region belongs to a defect-free sample image, the sample feature is the feature corresponding to the sample image region. Therefore, if the similarity between the image features corresponding to the character region in the image to be detected and the sample features is small, it indicates that the image to be detected has a defect; conversely, it may not have a defect. Since the sample image regions are obtained by dividing the defect-free sample image into regions according to region division parameters, the image to be detected can be divided into regions according to the region division parameters, and the image features can be divided according to the feature division parameters to obtain the image region features corresponding to each region to be detected. In this way, the electronic device can calculate the similarity between the region image features of each region to be detected and the features of each sample for each region to be detected. Then, based on the similarity corresponding to each region to be detected, the character defect detection result is determined, thereby achieving character defect detection. This eliminates the need to train multiple models for character defect detection, reducing the complexity of character defect detection. Of course, implementing any product or method of this application does not necessarily require achieving all the advantages described above simultaneously. Attached Figure Description

[0079] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0080] Figure 1 A flowchart illustrating a character defect detection method provided in an embodiment of this application;

[0081] Figure 2 Based on Figure 1A flowchart illustrating a method for constructing a sample feature library in the illustrated embodiment;

[0082] Figure 3 Based on Figure 1 A flowchart illustrating a method for determining image features corresponding to a sample image region in the illustrated embodiment;

[0083] Figure 4 for Figure 2 A specific flowchart of step S202 in the illustrated embodiment;

[0084] Figure 5 for Figure 1 A specific flowchart of step S104 in the illustrated embodiment;

[0085] Figure 6 for Figure 1 A specific flowchart of step S105 in the illustrated embodiment;

[0086] Figure 7(a) shows the results based on Figure 1 The flowchart shown is a process for constructing a normal character feature library and classifying and locating the character image to be detected.

[0087] Figure 7(b) is a schematic diagram of the structure of each unit in the offline stage based on the embodiment shown in Figure 7(a);

[0088] Figure 7(c) is a schematic diagram of the structure of each unit in the online stage based on the embodiment shown in Figure 7(a);

[0089] Figure 8(a) shows the results based on Figure 1 A schematic diagram of a character defect detection process in the embodiment shown;

[0090] Figure 8(b) shows the results based on... Figure 1 A schematic diagram of result fusion of the image to be detected in the embodiment shown;

[0091] Figure 9 This is a schematic diagram of the structure of a character defect detection device provided in an embodiment of this application;

[0092] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0093] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0094] To reduce the complexity of character defect detection, this application provides a character defect detection method, apparatus, electronic device, computer-readable storage medium, and computer program product. The character defect detection method provided in this application is described below.

[0095] The character defect detection method provided in this application can be applied to any electronic device that needs to detect character defects, such as a defect detector, image detector, or detection device, etc., without specific limitations. For clarity, it will be referred to as an electronic device herein.

[0096] like Figure 1 As shown, a cue learning method for a visual language model includes:

[0097] S101, acquire the image to be detected;

[0098] The image to be detected includes characters.

[0099] S102, Extract image features of the character region in the image to be detected;

[0100] S103, the character region is divided into multiple regions to be detected according to the pre-stored region division parameters;

[0101] The region division parameters are parameters corresponding to the region division method for dividing the sample character region in the defect-free sample image, which are recorded in advance.

[0102] S104, according to the pre-stored feature division parameters, divide the image features into region image features corresponding to each region to be detected;

[0103] The feature segmentation parameters are obtained by transforming the region segmentation parameters based on the size relationship between the image features and the character regions in the image to be detected.

[0104] S105, For each region to be detected, determine the similarity of the region to be detected based on the similarity between the region image features of the region to be detected and the features of each sample stored in advance;

[0105] The sample features are the features corresponding to the sample image regions obtained by dividing the sample character regions in the defect-free sample image according to the region division method.

[0106] S106, Based on the similarity corresponding to each region to be detected, determine the character defect detection result of the image to be detected.

[0107] As can be seen, in this embodiment of the application, the electronic device can acquire an image to be detected, wherein the image to be detected includes characters; extract image features of the character regions in the image to be detected; divide the character regions into multiple regions to be detected according to pre-stored region division parameters, wherein the region division parameters are parameters corresponding to the region division method for dividing the sample character regions in a defect-free sample image that are pre-recorded; divide the image features into region image features corresponding to each region to be detected according to pre-stored feature division parameters, wherein the feature division parameters are obtained by transforming the region division parameters based on the size relationship between the image features and the character regions in the image to be detected; for each region to be detected, determine the similarity corresponding to the region to be detected based on the similarity between the region image features of the region to be detected and each pre-stored sample feature, wherein the sample feature is the feature corresponding to the sample image region obtained by dividing the sample character regions in a defect-free sample image according to the region division method; and determine the character defect detection result of the image to be detected based on the similarity corresponding to each region to be detected. Since the sample image region belongs to a defect-free sample image, the sample feature is the feature corresponding to the sample image region. Therefore, if the similarity between the image features corresponding to the character region in the image to be detected and the sample features is small, it indicates that the image to be detected has a defect; conversely, it may not have a defect. Since the sample image regions are obtained by dividing the defect-free sample image into regions according to region division parameters, the image to be detected can be divided into regions according to the region division parameters, and the image features can be divided according to the feature division parameters to obtain the image region features corresponding to each region to be detected. In this way, the electronic device can calculate the similarity between the region image features of each region to be detected and the features of each sample for each region to be detected. Then, based on the similarity corresponding to each region to be detected, the character defect detection result is determined, thus achieving character defect detection. This eliminates the need to train multiple models for character defect detection, reducing the complexity of character defect detection.

[0108] In step S101, the electronic device can acquire the image to be detected. The characters in the image to be detected can be numbers or text from different languages. For example, they can be Chinese, English, Japanese, etc., without specific limitations. In one embodiment, the region of interest, i.e., the character region, of the image to be detected can be extracted in advance. Then, the character region in the image to be detected is used as the image to be detected. In this way, the electronic device can acquire the pre-processed image to be detected. In another embodiment, the electronic device can extract the character region from the image to be detected after acquiring it. The above step of extracting the character region can be implemented in at least any of the following three ways:

[0109] In the first approach, when image alignment is achieved—that is, when the image to be detected is similar in size and the regions of interest (ROIs) are located close to each defect-free sample image—the positions of the ROIs can be pre-set. Based on these pre-set ROI positions, the image to be detected is cropped to obtain the character regions. In the second approach, character regions can be identified using OCR (Optical Character Recognition). In the third approach, deep learning models can be used to locate character regions in the image to be detected; for example, Grounding DINO (Open Set Object Detection Algorithm) can be employed.

[0110] To determine the features of the characters included in the image to be detected, the electronic device can extract image features of the character regions in the image to be detected, i.e., perform step S102. The image features can be feature vectors or feature matrices. As one implementation, the electronic device can input the character regions in the image to be detected into a pre-trained feature extractor to obtain fine-grained features of the character regions in the image to be detected extracted by the feature extractor.

[0111] To detect character defects in different regions of the image to be inspected, thereby locating the defects, a sample feature library is constructed. Each defect-free sample image is divided into multiple sample image regions according to region partitioning parameters. The sample features and region partitioning parameters corresponding to these regions are stored in the sample feature library. Since the image to be inspected and the defect-free sample image have the same image size, during character defect detection, the electronic device can divide the character region into multiple regions to be inspected according to the pre-stored region partitioning parameters, i.e., execute step S103. This ensures that the region partitioning method for the character region is the same as that for the sample character region.

[0112] The region division parameters are pre-recorded parameters corresponding to the region division method used to divide the sample character regions in the defect-free sample image. These parameters can be the coordinates of the intersection points of each dividing line with the boundary of the defect-free sample image in the image coordinate system, the boundary coordinates or vertex coordinates of each divided sample image region, etc., and are not specifically limited here.

[0113] For example, the length and width of a defect-free sample image are 4000 pixels and 800 pixels, respectively. The region is divided into four equally sized sample image regions by dividing the defect-free sample image horizontally and vertically. An image coordinate system is established with the top left corner of the defect-free sample image as the origin. The vertex coordinates of each sample image region are (0,0), (2000,0), (0,400), (2000,400); (2000,0), (4000,0), (2000,400), (4000,400); (0,400), (2000,400), (0,800), (2000,800); (2000,400), (4000,400), (2000,800), (4000,800). The coordinates of the intersection points of the horizontal and vertical cutting lines with the boundary of the defect-free sample image are (0,400), (4000,400) and (2000,0), (2000,800), respectively.

[0114] Therefore, the vertex coordinates of each sample image region or the coordinates of the intersection points of each dividing line and the boundary of the defect-free sample image can be stored as region partitioning parameters. Then, according to these region partitioning parameters, the image to be detected can be divided horizontally and vertically to obtain four equally sized detection regions, with the vertex coordinates of each detection region representing the coordinates indicated by the region partitioning parameters.

[0115] Since the region partitioning parameters correspond to the region partitioning method of the sample character regions, and there is a certain size relationship between the image features of the sample character regions, the feature partitioning parameters can be obtained by transforming the region partitioning parameters according to the aforementioned size relationship when constructing the sample feature library. For each defect-free sample image, the image features of the sample character regions in the defect-free sample image are partitioned according to the feature partitioning parameters to obtain the sample features corresponding to each sample image region of the defect-free sample image, and the feature partitioning parameters are stored in the sample feature library. In this way, it can be ensured that each sample feature corresponds one-to-one with each sample character region. The sample features corresponding to each sample image region of the defect-free sample image are the set of features corresponding to pixel regions in the feature space, i.e., the feature pool.

[0116] For example, suppose the length and width of the sample image region are 2000 and 400 respectively, and the cutting points in the length and width directions of the image coordinate system are 1000 and 200 respectively, which are the region partitioning parameters. Suppose that in the extracted fine-grained feature space, the length and width of the image features are 250 and 50 respectively. Then, the size relationship between the sample image region and the sample features can be obtained as 8:1, and the cutting points in the length and width directions of the feature coordinate system can be determined as 125 and 25 respectively, which are the feature partitioning parameters.

[0117] In this way, the electronic device can divide the image features into region image features corresponding to each region to be detected according to the pre-stored feature division parameters, i.e., execute step S104. The feature extraction method of the image to be detected is the same as that of the defect-free sample image. Therefore, the size relationship between the image features of the sample character region and the image features of the character region in the image to be detected are the same as the size relationship between the image features of the character region and the character region. Therefore, the feature division parameters can be obtained by transforming the region division parameters based on the size relationship between the image features of the character region and the character region in the image to be detected. Among them, the feature division parameters can be the coordinates of the intersection point of each cutting line and the boundary of the image feature in the feature coordinate system, the boundary coordinates or vertex coordinates of each region image feature, etc., without specific limitations here.

[0118] Since the sample features are the features of a sample image region, and the sample image region is a defect-free sample image, if the similarity between the region image features of the area to be detected and the sample features is low, it indicates that the area to be detected has a character defect. If the similarity between the region image features of the area to be detected and the sample features is high, it indicates that the area to be detected may not have a character defect. Therefore, in order to detect character defects in the area to be detected, the electronic device can calculate the similarity between the region image features of the area to be detected and each pre-stored sample feature for each area to be detected. Specifically, the electronic device can calculate the feature distance between the region image features of the area to be detected and each pre-stored sample feature. The feature distance can be cosine distance, Euclidean distance, or Manhattan distance, etc., and is not specifically limited here.

[0119] If at least one of the multiple regions to be detected in the image to be detected is a character defect region, then it can be determined that the image to be detected contains a character defect. Therefore, after obtaining the similarity between the region image features of each region to be detected and the features of each sample, in order to determine whether the region to be detected is a character defect region, the electronic device can determine the similarity corresponding to the region to be detected based on the similarity between the region image features of the region to be detected and the features of each sample, that is, execute step S105.

[0120] In one implementation, a similarity threshold can be preset to determine whether a region to be detected contains character defects. If the maximum similarity among the similarities between the region image features of the region to be detected and the features of each sample is less than the similarity threshold, it indicates that the similarity between the region to be detected and the features of each sample is less than the similarity threshold, and thus the region to be detected can be determined as a region with character defects. Therefore, the electronic device can use the maximum similarity among the similarities between the region image features of the region to be detected and the features of each sample as the similarity score for each region to be detected.

[0121] For example, the image to be detected is divided into detection region 1 and detection region 2. The similarity between the region image features of detection region 1 and the features of each sample is 21%, 87%, 35%, 69%, and 34%, respectively, with the highest similarity being 87%. The similarity between the region image features of detection region 2 and the features of each sample is 64%, 33%, 41%, 58%, and 61%, with the highest similarity being 64%. The electronic device can use 87% and 64% as the similarity scores for detection region 1 and detection region 2, respectively.

[0122] If the similarity of each region to be detected in the image to be detected is high, then it can be determined that the image to be detected does not have character defects. If at least one region to be detected in the image to be detected has low similarity, then it can be determined that the image to be detected has character defects. Therefore, the electronic device can determine the character defect detection result of the image to be detected based on the similarity of each region to be detected, i.e., execute step S106.

[0123] In one implementation, if the similarity of each region to be detected within the image to be detected is not less than a similarity threshold, then it can be determined that the image to be detected does not have character defects; if among the similarities of each region to be detected within the image to be detected, there is a similarity less than the similarity threshold, then it can be determined that the image to be detected has character defects. Continuing with the previous example, assuming the similarity threshold is 80%, since the similarity of region 2 to be detected is 64%, which is less than the similarity threshold, it is determined that the image to be detected has character defects.

[0124] As can be seen, in this embodiment, since the sample image region is a defect-free sample image, the sample features are the features corresponding to the sample image region. Therefore, if the similarity between the image features corresponding to the character region in the image to be detected and the sample features is small, it indicates that the image to be detected has a character defect; otherwise, there may be no character defect. Since the sample image region is obtained by dividing the defect-free sample image into regions according to the region division parameters, the image to be detected can be divided into regions according to the region division parameters, and the image features can be divided according to the feature division parameters to obtain the image region features corresponding to each region to be detected. In this way, the electronic device can calculate the similarity between the region image features of each region to be detected and the features of each sample for each region to be detected. Then, the character defect detection result is determined based on the similarity corresponding to each region to be detected. Character defect detection of the image to be detected can be performed automatically without training multiple models or manually calibrating samples, avoiding a complex training process and reducing the complexity of character defect detection. Because it possesses the ability to detect both large and small targets, and has high segmentation accuracy, it can process each detection region of the image in parallel, exhibiting high real-time performance. It requires no additional modules to provide functionality, thus the unsupervised character defect detection method provided in this application is highly capable and can accurately detect character defects. Furthermore, since feature segmentation parameters are pre-recorded, when determining the region image features corresponding to each detection region, it is only necessary to extract image features from the image to be detected once, and then divide the image features into region image features corresponding to each detection region based on the feature segmentation parameters. This eliminates the need to extract region image features for each detection region separately, thereby improving the efficiency of image feature extraction.

[0125] As one implementation method of this application, such as Figure 2 As shown, prior to the step of acquiring the image to be detected, the method may further include:

[0126] S201, acquire multiple defect-free sample images;

[0127] Electronic devices can acquire multiple defect-free sample images. The types of characters included in these defect-free sample images can be determined based on the character defect detection scenario. As one implementation, the electronic device can acquire defect-free sample images of various characters, with the same number of images for each character. This ensures a relatively balanced number of defect-free sample images for each character, resulting in a more balanced number of sample features for each character stored in the sample feature library. Therefore, the accuracy of character defect detection for each character can be made more average.

[0128] In one implementation, if character defect detection is to be performed on characters of a certain language, the electronic device can acquire defect-free sample images of characters in different fonts of that language. For example, if the characters to be detected are Chinese characters, the electronic device can acquire defect-free sample images of Song typeface characters, Kai typeface characters, and Hei typeface characters, etc.

[0129] In another implementation, if character defect detection is to be performed on characters from multiple languages, the electronic device can acquire defect-free sample images including characters from the multiple languages ​​to be detected. For example, if the characters to be detected are English and Japanese, the electronic device can acquire multiple defect-free sample images including both English and Japanese characters.

[0130] In step S201, the defect-free sample images acquired by the electronic device can be pre-processed. Specifically, regions of interest (ROIs) can be extracted from the defect-free sample images to be acquired, allowing the electronic device to acquire ROIs from multiple defect-free sample images. For example, pre-processing techniques such as image filtering, affine transformation, and image cropping can be applied to the defect-free sample images. This filters out unnecessary background interference and retains the ROIs, including characters, in each defect-free sample image.

[0131] S202, Extract image features of the sample character region in the defect-free sample image;

[0132] If the image features of the character regions in the image to be detected are not very similar to the image features of the character regions in the defect-free sample image, it indicates that the image to be detected has character defects. Therefore, in order to perform character defect detection, electronic devices can extract image features from defect-free sample images. These image features can be feature vectors or feature matrices. For example, assuming defect-free sample images 1 through 100 are obtained, the electronic device can extract the image features from each of these images.

[0133] In one implementation, the electronic device can input a defect-free sample image into a pre-trained feature extractor to obtain fine-grained features of the defect-free sample image extracted by the feature extractor. The feature extractor can be pre-trained on the ImageNet dataset.

[0134] S203, determine the number of regions corresponding to the sample character regions based on the size of the sample character regions, and determine the size of the sample image regions to be divided based on the grayscale distribution of the sample character regions;

[0135] Electronic devices can divide sample character regions into areas and store the sample features of the divided sample image regions as a sample feature library. In this way, during character defect detection, the electronic device can divide the character regions in the image to be detected according to the region division method of the sample character regions. Then, the features of multiple divided regions to be detected are simultaneously compared with the sample features to determine the regions to be detected, and parallel processing of each region can be achieved.

[0136] To divide the sample character regions in a defect-free sample image, the electronic device can determine how to divide the sample character regions. Specifically, if the sample character region is large, it can be divided into more sample image regions; if the sample character region is small, it can be divided into fewer sample image regions. For example, for a sample character region with a resolution of 640×480, the number of regions can be determined to be 2; for a sample character region with a resolution of 2048×1536, the number of regions can be determined to be 4.

[0137] Since the grayscale values ​​of pixels in regions with similar grayscale distributions within a sample character area change continuously without abrupt changes, they can effectively reflect the characteristics of defect-free characters. Therefore, electronic devices can determine the size of the sample image region to be divided based on the grayscale distribution of the defect-free sample image. The determined size of the sample image region can be its length and width, or it can be the vertex coordinates of the sample image region in the image coordinate system. The steps for determining the size of the sample image region to be divided can be implemented using at least any of the following three methods:

[0138] In the first approach, an edge detection algorithm is used to locate the contours of characters in the sample character region based on the grayscale distribution, and the coordinates of the bounding moments of individual characters are extracted. The size of the sample image region is the size of the bounding moment. In the second approach, an adaptive binarization algorithm is used to process the sample character region to obtain a binarized image. Using the `findCountours` function in OpenCV (Open Source Computer Vision Library), the contours in the binarized image are located based on the grayscale distribution, and the coordinates of the bounding moments of the contours are obtained. The size of the sample image region is the size of the bounding moment.

[0139] In the third method, the average grayscale value of pixels in each row and column of the sample character region is calculated to obtain the average grayscale value for each row and column. Pixels in each row and column whose grayscale value is the average grayscale value are determined as mean points. For each mean point, the distance between that mean point and its neighboring mean points is calculated. A preset number of mean points with the largest distances are determined as the division points of the sample character region. The size of the sample image region is determined based on the size of each division point. For cases where the grayscale changes of characters in the sample character region are complex, the user can adjust the positions of the above division points.

[0140] For example, a defect-free sample image has a length and width of 2000 pixels and 500 pixels, respectively, and its grayscale distribution is uniform. Based on the length, width, and grayscale distribution of the defect-free sample image, an electronic device can determine the region division method as follows: divide the image into four equally sized sample image regions, each with a length and width of 1000 pixels and 250 pixels, respectively.

[0141] S204, the sample character region is divided according to the region division method to obtain multiple sample image regions corresponding to each defect-free sample image;

[0142] After determining the region division method for the defect-free sample image, the electronic device can divide the sample character region according to the region division method, obtaining multiple sample image regions corresponding to each defect-free sample image. Continuing from the previous example, after determining the region division method, the electronic device can divide the defect-free sample image into four equally sized sample image regions, each with a length of 1000 pixels and a width of 250 pixels.

[0143] S205, Based on the grayscale distribution of the sample image region, sample the image features corresponding to the sample image region to obtain sample features;

[0144] Since the image features of a defect-free sample image are extracted from different regions of the defect-free sample image, after obtaining multiple sample image regions corresponding to each defect-free sample image, the electronic device can determine the image features corresponding to the regions of the sample image regions in their respective defect-free sample images as the image features of the sample image regions, and store the image features as an image feature library.

[0145] Specifically, electronic devices can determine the feature partitioning parameters corresponding to the feature partitioning method of the defect-free sample image based on the size relationship between the image features of the sample character regions. Then, the image features of the defect-free sample image are partitioned according to the feature partitioning parameters to obtain image features corresponding to multiple sample image regions. In this way, features can be extracted only once from the defect-free sample image, instead of extracting features separately for each sample image region.

[0146] For example, if a defect-free sample image has a length and width of 2000 pixels and 400 pixels respectively, it can be divided according to the midpoints in the length and width directions, which are located at positions of 1000 pixels and 200 pixels respectively. This results in four sample image regions with lengths and widths of 1000 pixels and 200 pixels. Assuming the feature space of the image features has lengths and widths of 250 and 50 respectively, the ratio between the image space and the feature space can be determined to be 8:1. Therefore, the feature partitioning parameters are determined to be the midpoints of the length and width, i.e., 125 and 25. Based on these two midpoints, the image features can be segmented to obtain four image features with lengths and widths of 125 and 25 respectively.

[0147] For example, suppose the defect-free sample image is divided into four equally sized sample image regions, and the dimensions of the image feature of the defect-free sample image are [N, C, H, W]. Then, in the feature space, the cut point of the third dimension H can be determined as 0.5*H, and the cut point of the fourth dimension W can be determined as 0.5*W. Here, N is the number of defect-free sample images, C is the number of channels, H is the length of the image feature, and W is the width of the image feature.

[0148] Because image acquisition angles are fixed in industrial settings, the variation in defect-free sample images is minimal, resulting in the collection of many identical image features. Storing these image features as a sample feature library would place a significant burden on storage and computing resources, and increase redundancy. Therefore, to reduce the storage pressure and computational resource burden on the sample feature library, and to minimize redundancy, electronic devices can remove redundant information by downsampling the image features corresponding to the sample image regions. This allows for the selection of the most representative feature subset, known as the sample features, which is then stored as the core feature library, or sample feature library.

[0149] Specifically, because the grayscale distribution of character regions is more complex than that of background regions, a sampling method is used to ensure that the sampled features belong to the character regions and thus reflect the image features of defect-free characters. Electronic devices can sample the image features corresponding to the sample image regions based on their grayscale distribution to obtain the sample features.

[0150] For example, if the grayscale distribution of sample image regions 1 to 40 is relatively complex, while the grayscale distribution of sample image regions 41 to 100 is relatively simple, the image features corresponding to the sample image regions with complex grayscale distributions can better reflect the image features possessed by defect-free characters. Therefore, electronic devices can use a greedy algorithm to sample the image features corresponding to sample image regions 1 to 40 at a rate of 3%, and the image features corresponding to sample image regions 41 to 100 at a rate of 1%, using the sampled image features as the sample features.

[0151] S206, store the sample features and the region division parameters that characterize the region division method as a sample feature library.

[0152] To segment the image to be inspected into regions, the electronic device can store region segmentation parameters, representing the region segmentation method, as a sample feature library. This allows the electronic device to segment the image according to these parameters during character defect detection, ensuring that the region segmentation of the image to be inspected is identical to that of a defect-free sample image. Since sample features reflect the image characteristics of defect-free characters, the electronic device can store these sample features in a sample feature library.

[0153] As can be seen, in this embodiment, the electronic device can acquire multiple defect-free sample images; extract image features of sample character regions in the defect-free sample images; determine the number of regions corresponding to the sample character regions based on their size, and determine the size of the sample image regions to be divided based on the grayscale distribution of the sample character regions; divide the sample character regions according to the region division method to obtain multiple sample image regions corresponding to each defect-free sample image; sample the image features corresponding to the sample image regions based on the grayscale distribution of the sample image regions to obtain sample features; and store the sample features and the region division parameters characterizing the region division method as a sample feature library. Since the image size and grayscale distribution of the defect-free sample images are considered simultaneously when determining the region division method, the determined region division method can make the size of each sample image region more appropriate, and the grayscale distribution within a sample image region more uniform. To improve robustness, reduce computational resource consumption and storage pressure on the sample feature library, and reduce redundant information in the sample feature library, the electronic device can sample the image features corresponding to the sample image regions. This allows the sample features stored in the sample feature library to reflect the image features of defect-free characters. Therefore, in character defect detection, the image to be detected can be divided based on the region segmentation parameters stored in the sample feature library, and the detection result can be determined based on the similarity between the image features of the region to be detected and the sample features. This eliminates the need to train multiple models for character defect detection, thus reducing the complexity of character defect detection.

[0154] As one implementation method of this application, such as Figure 3 As shown, before the step of sampling the image features corresponding to the sample image region based on the grayscale distribution of the sample image region to obtain the sample features, the above method may further include:

[0155] S301, determine the feature division method based on the size relationship between the sample character region and the image features of the sample character region and the region division method;

[0156] Since the size relationship between the image features of the sample character regions is the proportional relationship between the region partitioning parameters and the feature partitioning parameters, and the region partitioning parameters can be determined by the region partitioning method, the electronic device can determine the feature partitioning method based on the above size relationship and region partitioning method.

[0157] For example, suppose the length and width of the sample image region are 2500 and 500 respectively, and the cutting points in the length and width directions of the image coordinate system are 1250 and 250 respectively. Assume that in the extracted fine-grained feature space, the length and width of the image features are 500 and 100 respectively. Then, the ratio between the image space and the feature space can be obtained as 5:1, thus determining the cutting points in the length and width directions of the feature coordinate system to be 250 and 50 respectively.

[0158] S302, according to the feature division method, the image features of the sample character region are divided to obtain the image features corresponding to each sample image region;

[0159] After obtaining the image features of the defect-free sample image, the electronic device can perform feature partitioning on the image features of the sample character regions in the feature space according to the feature partitioning method, obtaining the image features corresponding to each sample image region. Continuing the example in step S301, the length and width of the image features of the defect-free sample image are 500 and 100 respectively, and the feature partitioning method is to cut at two cutting points at 250 and 50 in the length and width directions. Therefore, the electronic device can perform feature partitioning on the image features of the defect-free sample image according to this feature partitioning method, obtaining the image features corresponding to each sample image region. The length and width of the image features corresponding to each sample image region are 250 and 50 respectively.

[0160] S303, store the feature partitioning parameters that characterize the feature partitioning method into the sample feature library.

[0161] To segment the image features of the image to be detected, the electronic device can store feature segmentation parameters, which represent the feature segmentation method, in a sample feature library. This allows for feature segmentation of the image features of the image to be detected according to the feature segmentation parameters during character defect detection, ensuring that the feature segmentation method of the image features of the image to be detected is the same as that of the image features of the defect-free sample image.

[0162] As can be seen, in this embodiment, the electronic device can determine the feature partitioning method based on the size relationship between the image features of the sample character regions and the region partitioning method; according to the feature partitioning method, the image features of the sample character regions are partitioned to obtain the image features corresponding to each sample image region; and the feature partitioning parameters representing the feature partitioning method are stored in the sample feature library. Since the electronic device partitions the image features of each sample image region according to the feature partitioning method after extracting the image features of the defect-free sample image, without extracting image features separately for each sample image region, only one feature extraction is required, which can improve the efficiency of determining the image features corresponding to each sample image region.

[0163] As one implementation method of this application, such as Figure 4 As shown, the steps for extracting image features of the sample character region in the defect-free sample image described above may include:

[0164] S401, For each defect-free sample image, the sample character region in the defect-free sample image is input into a pre-trained feature extractor to obtain the intermediate layer features of the sample character region in the defect-free sample image extracted by the intermediate layer of the feature extractor.

[0165] Because the shallow layers of the feature extractor contain a lot of redundant information, while the deep layers lack information about small targets, the electronic device can, in order to meet the high-precision detection requirements of small target defects and reduce redundant information in image features, input the sample character region from each defect-free sample image into the feature extractor. The electronic device can then acquire only the intermediate layer features of the defect-free sample image. These intermediate layer features can be Layer 2 features of a Wide ResNet50 network.

[0166] S402, the intermediate layer features are input from the attention layer to introduce global information into the intermediate layer features to obtain global features;

[0167] The ability to detect logical defects relies on the establishment of global information. Since the receptive field of intermediate layer features is limited and cannot acquire long-range receptive fields, in order to compensate for the lack of global information, the feature extractor can be connected to the self-attention layer. The intermediate layer features obtained in the previous step are input into the self-attention layer, introducing global information into the intermediate layer features to obtain global features.

[0168] S403, input the intermediate layer features into the local feature aggregation module to obtain the local features output by the local feature aggregation module;

[0169] The ability to detect texture structural defects relies on the establishment of local neighborhood information. Feature aggregation on the local neighborhood can preserve sufficient spatial information. Therefore, to retain the ability to detect texture structural defects, the self-attention layer can be connected to the local feature aggregation module. In this way, intermediate layer features can be input into the local feature aggregation module, which can downsample the intermediate layer features and then output local neighborhood features, i.e., local features.

[0170] S404, the global features and the local features are stacked according to channels to form the image features of the sample character region in the defect-free sample image.

[0171] Since global features can reflect logical defects and local features can reflect texture defects, stacking global and local features by channel yields image features that retain the ability to detect both texture defects and logical defects. Therefore, electronic devices can stack global and local features by channel to obtain image features, which can then be used as image features for sample character regions in defect-free sample images.

[0172] For example, assuming that the number of channels for both the global and local features of a certain sample character region is 512, the electronic device can stack the global and local features, that is, stitch them together by channel, to obtain an image feature with 1024 channels.

[0173] As can be seen, in this embodiment, the electronic device can, for each defect-free sample image, input the sample character region in the defect-free sample image into a pre-trained feature extractor to obtain the intermediate layer features of the sample character region in the defect-free sample image extracted by the intermediate layer of the feature extractor; input the intermediate layer features into the attention layer to introduce global information into the intermediate layer features to obtain global features; input the intermediate layer features into the local feature aggregation module to obtain the local features output by the local feature aggregation module; and stack the global features and local features according to channels to obtain the image features of the sample character region in the defect-free sample image. Since the intermediate layer features extracted by the feature extractor for the defect-free sample image are obtained, redundant information can be reduced and the loss of small target information can be avoided. Since the image features are obtained by stacking global features and local features according to channels, the ability to detect texture structure defects and logical defects can be preserved, thereby improving the accuracy of character defect detection.

[0174] As one embodiment of this application, the step of sampling the image features corresponding to the sample image region based on the grayscale distribution of the sample image region to obtain the sample features may include:

[0175] Based on the grayscale distribution of the sample image region, the image features corresponding to the sample image region are sampled according to the preset sampling constraints to obtain the sample features.

[0176] Since the sampling rate of image features is inversely correlated with the number of channels in the image features, the number of channels in the image features can be reduced to improve the sampling speed. Therefore, the electronic device can iterate through all image features and perform dimensionality reduction on each feature. For example, dimensionality reduction can be randomized. Then, an adaptive greedy algorithm is used to select sample features that meet preset sampling constraints from the dimensionality-reduced image features.

[0177] Since simple-distribution sample image regions are typically the background regions of defect-free sample images, with a relatively limited range of grayscale values, while complex-distribution sample image regions are typically the character regions of defect-free sample images, with a greater variety of grayscale values, a predetermined number of regions can be used to determine whether a sample image region is simple-distributed or complex.

[0178] Thus, if the number of grayscale value types of pixels in a sample image region is less than the preset number, then the sample image region can be determined to have a simple distribution; if the number of grayscale value types of pixels in a sample image region is not less than the preset number, then the sample image region can be determined to have a complex distribution. The preset number can be set according to actual needs, and can be 10, 15, 20, etc., all of which are reasonable.

[0179] To ensure that image features corresponding to simply distributed sample image regions are sampled at a lower proportion, while those corresponding to complexly distributed sample image regions are sampled at a higher proportion, a preset proportion can be determined before sampling the image features corresponding to the sample image regions. Thus, for simply distributed sample image regions, a sampling proportion lower than the preset proportion can be used; for complexly distributed sample image regions, a sampling proportion no less than the preset proportion can be used. The preset proportion can be set according to actual needs, and can be 1%, 2%, 3%, etc., all of which are reasonable.

[0180] Because the feature distance between image regions containing character areas is relatively small, a preset range can be determined to ensure that all sampled features are from image regions containing character areas. This allows for cyclic sampling of image features until the maximum distance between sampled features falls within the preset range. The maximum distance is the sum of the feature distances between multiple sampled features, and the preset range can be set according to actual detection requirements. For example, the ratio of the maximum value of the preset range to the average distance between image features can be 1.5. The electronic device can then cyclically sample image features until the ratio between the maximum distance and the average distance between sampled features is less than 1.5.

[0181] Based on the above discussion, preset sampling constraints can be determined in advance. These constraints ensure that the proportion of image features sampled from sample image regions with fewer than a preset number of grayscale value types is less than a preset proportion, the proportion of image features sampled from sample image regions with at least a preset number of grayscale value types is at least a preset proportion, and the maximum distance between sampled image features is within a preset range. In this way, the electronic device can sample image features corresponding to sample image regions according to the preset sampling constraints based on the grayscale distribution of the sample image regions, thereby obtaining sample features.

[0182] To reduce the number of channels in image features and thus improve the speed of image feature sampling, the electronic device can traverse all feature blocks. First, it performs dimensionality reduction on the features of each block, and then uses an adaptive greedy algorithm to select the core feature library that meets the sampling constraints. In other words, the electronic device can traverse the image features corresponding to each sample image region, first perform dimensionality reduction on the image features corresponding to each sample image region, and then use an adaptive greedy algorithm to select image features that meet the aforementioned preset sampling constraints. The sampled image features are then used as sample features and stored in a sample feature library.

[0183] In one embodiment, the electronic device can sample image features corresponding to simply distributed sample image regions and image features corresponding to complexly distributed sample image regions according to a first preset ratio and a second preset ratio, respectively. This continues until the ratio between the number of sampled image features belonging to simply distributed sample image regions and the total number of image features belonging to simply distributed sample image regions reaches the first preset ratio; and the ratio between the number of sampled image features belonging to complexly distributed sample image regions and the total number of image features belonging to complexly distributed sample image regions reaches the second preset ratio, and the maximum distance between the sampled image features is within a preset range. Specifically, for simply distributed sample image regions, sampling can be performed at the first preset ratio while satisfying preset sampling constraints; for complexly distributed sample image regions, cyclic sampling can be performed at the second preset ratio while satisfying preset sampling constraints.

[0184] Among them, the sample image regions with simple distribution are those with fewer than a preset number of grayscale value types, while the sample image regions with complex distribution are those with no fewer than a preset number of grayscale value types. The first preset ratio is less than a preset ratio, and the second preset ratio is no less than a preset ratio.

[0185] For example, assuming a preset ratio of 2%, the first and second preset ratios can be 1% and 3%, respectively. A simple sample image region corresponds to 1000 image features, while a complex sample image region corresponds to 5000 image features. Taking a complex sample image region as an example, the electronic device can sample from the 5000 image features according to the constraint that the maximum distance between the sampled image features is within a preset range. After each sampling, it is determined whether the number of currently sampled image features has reached 150. If it has, sampling stops; if not, sampling continues.

[0186] As can be seen, in this embodiment, the electronic device can sample image features corresponding to sample image regions according to preset sampling constraints to obtain sample features. The preset sampling constraints ensure that the proportion of image features sampled from sample image regions with fewer than a preset number of grayscale value types is less than a preset proportion, the proportion of image features sampled from sample image regions with a number of grayscale value types not less than a preset proportion is not less than a preset proportion, and the maximum distance between sampled image features is within a preset range. Because the image features corresponding to sample image regions are sampled according to preset sampling constraints, the sampling proportions corresponding to image features from sample image regions with different grayscale distributions are considered, and the maximum distance between sampled image features is within a preset range. Therefore, the storage size of the sample feature library can be reduced. Since the sample features in the sample feature library contain less redundant information and can accurately reflect the image features corresponding to defect-free sample images, the accuracy and speed of character defect detection can be improved. Furthermore, by introducing location information and typical feature selection—that is, sampling image features according to the grayscale distribution of sample image regions according to preset sampling constraints to obtain representative sample features—the high-precision detection requirements for small target defects can be met.

[0187] As one implementation method of this application, such as Figure 5 As shown, the step of determining the similarity of the region to be detected based on the similarity between the image features of the region to be detected and the features of each pre-stored sample can include:

[0188] S501, Calculate the feature distance map between the regional image features of the region to be detected and the features of each sample in the sample feature library;

[0189] Since the feature distance between the regional image features of the region to be detected and the sample features can reflect the similarity between the regional image features of the region to be detected and the sample features, in order to determine the similarity between the regional image features of the region to be detected and the sample features, the electronic device can calculate the feature distance map between the regional image features of the region to be detected and each sample feature in the sample feature library.

[0190] Assuming the dimension of the region image features of the defect-free sample image is [N, C, H, W], then the dimension of the region image features of the region to be detected is [N, C], and the dimension of the sample feature library is [M, C]. Here, M is the number of sample image regions corresponding to each defect-free sample image, which is related to the number of defect-free sample images and the sampling ratio. N and M have different sizes; specifically, M is less than N. Thus, the feature distance map between the region image features of the region to be detected with dimension [N, C] and the sample features with dimension [M, C] can be calculated.

[0191] S502, the feature distance map with the smallest distance between the calculated feature distance maps and the regional image features of the region to be detected is used as the anomaly localization segmentation map corresponding to the region to be detected.

[0192] The feature distance map between the regional image features of the region to be detected and the features of each sample can be used to characterize the distance between the region to be detected and each sample feature. The smaller the distance, the greater the similarity between the regional image features of the region to be detected and the sample features; conversely, the greater the distance, the smaller the similarity. If, among multiple sample features, there are sample features with high similarity to the regional image features of the region to be detected, it indicates that the region to be detected may not have defects; if there are no sample features with high similarity to the regional image features of the region to be detected, it indicates that the region to be detected has defects.

[0193] Therefore, electronic devices can determine whether a region to be detected has a defect simply by using the maximum similarity score. Since feature distance maps can represent similarity, electronic devices can use the feature distance map with the smallest distance to the region image features of the region to be detected among the calculated feature distance maps as the anomaly localization segmentation map corresponding to the region to be detected. The anomaly localization segmentation map is used to identify the maximum similarity between the region to be detected and each sample feature in the sample feature library.

[0194] Specifically, electronic devices can employ the KNN (K-Nearest Neighbor) algorithm to determine the feature distance map among multiple feature distance maps that represents the region image features of the region to be detected with the smallest distance. Assuming the dimension of the region image features of the region to be detected is [N,C], and the dimension of the sample features is [M,C], then calculating the Euclidean distance yields a distance matrix of dimension [N,M], which is the feature distance map. To determine the feature distance map that minimizes the distance to the region image features of the region to be detected from multiple feature distance maps, the nearest neighbors of the multiple feature distance maps can be taken, resulting in a column vector [N,1], which serves as the feature distance map with the smallest distance.

[0195] As can be seen, in this embodiment, the electronic device can calculate the feature distance map between the regional image features of the area to be detected and each sample feature in the sample feature library; the feature distance map with the smallest distance to the regional image features of the area to be detected among the calculated feature distance maps is used as the anomaly localization segmentation map corresponding to the area to be detected, wherein the anomaly localization segmentation map is used to identify the maximum similarity between the area to be detected and each sample feature in the sample feature library. If there are sample features with a large similarity to the regional image features of the area to be detected, it indicates that the area to be detected may not have defects. Since the feature distance map can represent the similarity between the regional image features of the area to be detected and the sample features, the electronic device can calculate the feature distance map with the smallest distance to the regional image features of the area to be detected among the calculated feature distance maps. In this way, it is possible to simultaneously determine whether there are defects in multiple areas to be detected in the image to be detected, realize parallel processing of different regions, and meet the real-time requirements of large-size images in actual industrial production lines.

[0196] As one implementation method of this application, such as Figure 6 As shown, the steps described above for determining the character defect detection result of the image to be detected based on the similarity corresponding to each region to be detected may include:

[0197] S601, according to the position of each region to be detected in the image to be detected represented by the region division parameters, merge the anomaly localization segmentation maps corresponding to each region to be detected to obtain the anomaly localization map corresponding to the image to be detected.

[0198] Since the region to be detected is obtained by dividing the image to be detected into regions according to the region division parameters, after determining the anomaly localization segmentation map corresponding to each region to be detected, the electronic device can merge the anomaly localization segmentation maps corresponding to each region to be detected according to the position of each region to be detected in the image to be detected represented by the region division parameters, and obtain the anomaly localization map corresponding to the image to be detected.

[0199] For example, the image to be detected is divided into detection regions 1 to 4, located in the upper left, upper right, lower left, and lower right corners of the image, respectively. The corresponding region image features 1 to 4 are all 100×100 matrices. Therefore, the anomaly localization and segmentation corresponding to detection regions 1 to 4... Figure 1 - Anomaly localization and segmentation Figure 4 All are 100×100 matrices. The electronic device can segment anomalies according to the positions of detection regions 1-4 within the image to be detected. Figure 1 - Anomaly localization and segmentation Figure 4 The images are merged to obtain the anomaly localization map corresponding to the image to be detected. Anomaly localization segmentation. Figure 1 - Anomaly localization and segmentation Figure 4 The anomalies are located in the upper left, upper right, lower left, and lower right regions, respectively, forming a 200×200 matrix.

[0200] S602, Based on the predetermined maximum distance, the anomaly location map is normalized to obtain a normalized anomaly location map; if the maximum value of the pixel value of the normalized anomaly location map is greater than the preset anomaly score, step S603 is executed; if the maximum value of the pixel value of the normalized anomaly location map is not greater than the preset anomaly score, step S604 is executed.

[0201] Since each anomaly localization segmentation map is calculated based on different regional image features and sample features of the area to be detected, each anomaly localization segmentation map has different criteria for judging whether a defect exists in the area to be detected. In order to unify the judgment criteria, the electronic device can normalize the anomaly localization map based on a predetermined maximum distance to obtain a normalized anomaly localization map.

[0202] Since the anomaly localization segmentation map represents the minimum distance between the region image features corresponding to the region to be detected and the sample features, a large minimum distance indicates that the feature distance between the region to be detected and each sample feature in the sample feature library is large, indicating that the region to be detected has a defect. Conversely, a small minimum distance indicates that there are sample features in the sample feature library that have a small feature distance to the region image features of the region to be detected, indicating that the region to be detected has a defect.

[0203] Thus, in the normalized anomaly localization map, the pixel value at each anomaly localization segmentation point can represent the probability that the corresponding area to be detected has a defect. The larger the pixel value, the greater the probability that the area to be detected has a defect; the smaller the pixel value, the smaller the probability that the area to be detected has a defect.

[0204] S603, determine the region to be detected corresponding to the pixel value that is greater than the preset abnormality score, and determine the determined region to be detected as the character defect region in the image to be detected;

[0205] To determine whether a defect exists in the area to be detected based on the pixel values ​​of the anomaly localization map, a preset anomaly score can be determined before determining the detection result of the image to be detected. The preset anomaly score can be set according to actual needs; if the detection standard is high, the preset anomaly score can be set smaller; if the detection standard is low, the preset anomaly score can be set larger. For example, the preset anomaly score can be 100, 120, 150, etc., all of which are reasonable.

[0206] In this way, the electronic device can determine the detection result of the image to be detected based on the relationship between the maximum pixel value of the normalized anomaly localization map and the preset anomaly score. In one embodiment, if there is a pixel value in the normalized anomaly localization map that is greater than the preset anomaly score, then it can be determined that the image to be detected has a defect; if none of the pixel values ​​in the normalized anomaly localization map are greater than the preset anomaly score, then it can be determined that the image to be detected does not have a defect.

[0207] For example, anomaly localization maps include anomaly localization segments. Figure 1 - Anomaly localization and segmentation Figure 4 The anomaly localization map after normalization is in the anomaly localization segment. Figure 1 - Anomaly localization and segmentation Figure 4 The pixel values ​​at the locations are 20, 168, 139, and 62, respectively. Assuming a preset anomaly score of 150, since there are pixel values ​​greater than 150, it is determined that the image to be detected has a defect.

[0208] If the region in the image to be detected that has the highest probability of containing a defect does not contain a defect, then the image to be detected is defect-free; conversely, if the region in the image to be detected that has the highest probability of containing a defect contains a defect, then the image to be detected is defective. Since the maximum pixel value in the anomaly localization map belongs to the region in the image to be detected that has the highest probability of containing a defect, the electronic device can determine the relationship between the maximum pixel value in the normalized anomaly localization map and a preset anomaly score, thereby determining whether the image to be detected contains a defect.

[0209] For example, anomaly localization maps include anomaly localization segments. Figure 1 - Anomaly localization and segmentation Figure 6 The anomaly localization map after normalization is in the anomaly localization segment. Figure 1 - Anomaly localization and segmentation Figure 6The pixel values ​​at the locations are 42, 185, 56, 83, 168, and 103, with a maximum pixel value of 185. Assuming a preset anomaly score of 150, since the maximum pixel value of 185 is greater than the preset anomaly score of 150, it can be determined that the image to be detected has a defect.

[0210] If the pixel value of the normalized anomaly localization map is greater than a preset anomaly score, it indicates that the detection area corresponding to that pixel value has a defect. Given that the image to be detected has been determined to have a defect, in order to further determine the detection area with the defect, the electronic device can identify the detection area corresponding to pixel values ​​greater than the preset anomaly score.

[0211] Continuing from the previous example, if it is determined that there is a defect in the image to be detected, the electronic device can further determine that the pixel values ​​185 and 168 are greater than the preset anomaly score of 150, and thus determine that the areas to be detected corresponding to the pixel values ​​185 and 168 are areas to be detected 2 and areas to be detected 5.

[0212] Since the pixel value corresponding to the determined detection area is greater than the preset anomaly score, the electronic device can identify the determined detection area as a character defect area in the image to be detected. Continuing from the previous example, the electronic device can identify detection areas 2 and 5 as character defect areas.

[0213] S604, determine that the image to be detected does not have character defects.

[0214] If the area in the image to be detected that is most likely to contain a defect does not contain a defect, then the image to be detected is defect-free.

[0215] As can be seen, in this embodiment, the electronic device can merge the anomaly localization segmentation maps corresponding to each region to be detected according to the position of each region to be detected in the image to be detected, as represented by the region division parameters, to obtain an anomaly localization map corresponding to the image to be detected. Based on a predetermined maximum distance, the anomaly localization map is normalized to obtain a normalized anomaly localization map, where the maximum distance is the maximum distance between sample features. If the maximum value of the pixel value in the normalized anomaly localization map is greater than a preset anomaly score, the region to be detected corresponding to the pixel value greater than the preset anomaly score is determined, and the determined region to be detected is identified as a character defect region in the image to be detected. If the maximum value of the pixel value in the normalized anomaly localization map is not greater than the preset anomaly score, it is determined that there is no character defect in the image to be detected. Since the anomaly localization segmentation map represents the minimum distance between the region image features and sample features corresponding to the region to be detected, the anomaly localization segmentation maps can be merged into an anomaly localization map, and the detection result can be determined based on the relationship between the pixel values ​​in the anomaly localization map and the preset anomaly score. In this way, the detection result of the image to be detected can be determined quickly and accurately. Furthermore, since the maximum pixel value in the anomaly localization map corresponds to the detection area in the image most likely to contain defects, the relationship between the maximum pixel value and the preset anomaly score can be determined, thereby confirming whether the image contains defects. This allows for quick and accurate determination of whether the image contains defects.

[0216] As one embodiment of this application, a flowchart for constructing a normal character feature library and classifying and locating the character image to be detected can be shown in Figure 7(a). The construction of the normal character feature library and the classification and location of the character image to be detected can be divided into an offline stage and an online stage. The offline stage includes steps S701-S702, and the online stage includes steps S703-S705, specifically including the following steps:

[0217] S701, normal character image acquisition and processing;

[0218] To determine the image features of defect-free character images, electronic devices can acquire multiple normal standard character images. Specifically, for each character, the same number of defect-free normal images that conform to the actual distribution can be selected. The electronic device can perform image preprocessing on the standard character images to eliminate background interference, ensuring that the character image acquisition meets standardization requirements. These normal standard character images are the defect-free sample images.

[0219] S702, establish a normal character feature library by region;

[0220] The electronic device can input collected standard character images into a feature extractor to obtain the image features of the standard character images extracted by the feature extractor. Next, the electronic device can select typical image features, that is, sample the image features of the standard character images. The standard character images are then processed into regions to establish a normal character feature library for each region. This normal character feature library is the sample feature library.

[0221] S703, Image acquisition of the character to be detected;

[0222] When detecting character images online, a standard character image can be preprocessed offline to obtain a standard character image to be detected. This standard character image is the one to be detected.

[0223] S704, minimum distance calculation;

[0224] For each character image to be detected, the electronic device can perform region processing on the character image to be detected, similar to the method of partitioning the standard character image in the offline stage, to obtain multiple regions to be detected for that character image. For each region to be detected, the electronic device can extract the image features of that region, calculate the distance between the image features and the normal character features of multiple different regions established in the offline stage, and determine the minimum distance.

[0225] S705, Classification and localization results.

[0226] Electronic devices can determine the presence and location of character defects based on the minimum distance between multiple regions to be detected in an image.

[0227] In one implementation, the offline phase can be implemented based on multiple units. A schematic diagram of the structure of each unit on which the offline phase is based is shown in Figure 7(b), specifically including:

[0228] The normal character image acquisition and processing unit 711 is used to acquire normal images. Specifically, it acquires the same number of normal images for each type of character, and the acquired samples should be as representative as possible of all actual normal samples. Unnecessary background interference is filtered out through preprocessing techniques such as image filtering, affine transformation, and image cropping, retaining only the region of interest for the character. The normal samples are defect-free sample images.

[0229] The normal character feature extraction unit 712 is used to extract fine-grained features from all normal samples using a feature extractor pre-trained on the ImageNet dataset. Specifically, since shallow features have a lot of redundant information and deep features lack small target information, only one layer of intermediate features is extracted. Then, a self-attention layer is added to introduce global information, and finally, a local feature aggregation module is connected to obtain the features of normal samples.

[0230] The region feature selection unit 713 is used to select typical features to represent all features. Since different regions have different grayscale distributions, the features are adaptively divided into regions according to their positions. Basic image width and height information and grayscale distribution at all positions are statistically analyzed. Based on the statistical results, adaptive position division is performed, and the division parameters are recorded. Next, all feature blocks are traversed. First, the features of each block are dimensionality reduced. Then, an adaptive greedy algorithm is used to select the core feature library that meets the sampling constraints. Specifically, a smaller sampling ratio is used for regions with simple distributions; for regions with complex distributions, multiple sampling cycles are performed to ensure that the maximum distance between normal features in each region is within a reasonable range.

[0231] Character feature library unit 714 is established by region to save the above results in order to form a normal feature library. The normal feature library mainly includes the selected typical features, the maximum distance between normal samples, and adaptive region division parameters, which are the region division parameters.

[0232] In one implementation, the online phase described above can be implemented based on multiple units. A schematic diagram of the structure of each unit on which the online phase is based can be shown in Figure 7(c), specifically including:

[0233] The character image acquisition unit 721 is used to sample the same image preprocessing method as the offline stage and extract the region of interest (ROI) of the character image to be detected.

[0234] The feature segmentation extraction unit 722 is used to divide the character image to be detected into multiple regions using the same region segmentation method as in the offline stage. A pre-trained feature extractor is then used to extract fine-grained features from all regions to be detected.

[0235] The minimum distance calculation unit 723 is used to obtain the anomaly localization segmentation map of each block by calculating the minimum distance between the fine-grained features of the region to be detected and the sample features in the offline typical feature library.

[0236] The segmentation result fusion unit 724 is used to merge the multiple results obtained from the above units according to the adaptive segmentation rules determined in the offline stage, and normalize them using the maximum distance between normal samples, thereby obtaining a complete anomaly localization map.

[0237] The classification result calculation unit 725 is used to perform Gaussian filtering on the anomaly location map of the entire image, and takes the maximum pixel value from the anomaly location map as the overall defect anomaly score of the image. This anomaly score is compared with a preset anomaly score to determine the classification result.

[0238] As one embodiment of this application, a flowchart of character defect detection can be shown in Figure 8(a):

[0239] The electronic device can acquire a real-world image containing defect-free characters, perform region of interest (ROI) recognition on the image, and obtain three ROIs, each containing the characters "TEST", "START", and "STOP". Each ROI is treated as a defect-free character image. The electronic device can input multiple defect-free character images 1001 into a feature extractor 802 to obtain the global and local features of the intermediate layer features of each defect-free character image 801 output by the feature extractor 802. These global and local features are then input into a feature dimensionality reduction module 803 to reduce the number of channels in the global and local features, thereby reducing the computational load in subsequent processes. This results in dimensionality-reduced global features 804 and local features 805. Both global features 804 and local features 805 have dimensions of M*N*C*H*W. Where M is the number of sample image regions corresponding to each defect-free character image, N is the number of defect-free character images 801, which is 3, C is the number of channels, H is the length of global feature 804 and local feature 805, and W is the width of global feature 804 and local feature 805.

[0240] Next, the electronic device can convert the region partitioning method at the image level to the feature level. In the feature space, global feature 804 and local feature 805 are partitioned into regions, such that both global feature 804 and local feature 805 are equally divided into 2*2 feature blocks. To reduce storage pressure and computational resource burden, global feature 804 and local feature 805 can be downsampled. The downsampled global feature 806 and downsampled local feature 807 are stacked to form the image feature 808 corresponding to the feature block to which the downsampled global feature 806 belongs and the feature block to which the downsampled local feature 807 belongs. The dimension of image feature 808 is N*H*W. Patches are sample image regions. In this example, each defect-free character image 801 is divided into 4 sample image regions, and each sample image region corresponds to one feature block. Thus, the image features corresponding to the 4 feature blocks can be obtained.

[0241] Taking image feature 808 as an example, an adaptive greedy algorithm is used to sample the hollow circles, as shown in image feature 810, contained in image feature library 809 according to preset sampling constraints, in order to perform adaptive core feature library selection. That is, a small proportion of sampling is used for simple distribution areas, and a large proportion of sampling is used for complex distribution areas, and the maximum distance d between the sampled image features is within a preset range. In this way, sample feature library 811 can be obtained, which contains solid circles, as shown in sample feature 812. Finally, four sample feature libraries corresponding to four feature blocks can be obtained.

[0242] An electronic device can input the image to be detected 813 into a feature extractor 802 to obtain the global and local features of the intermediate layer features of the image to be detected 813 output by the feature extractor 802. The global and local features are then stacked to obtain the image features 814 to be detected. For each feature block of the image features 814, a KNN distance is calculated with the sample features 812 stored in the sample feature library 811, resulting in a KNN distance calculation result 815. Based on the KNN distance calculation result 815, an anomaly score map 816 and an anomaly score of 0.97 are determined. The anomaly score map 816 marks the regions where defective characters are located.

[0243] As one embodiment of this application, a schematic diagram of the result fusion of the image to be detected can be shown in Figure 8(b):

[0244] Electronic devices can pre-extract regions of interest (ROIs) from the physical image to be inspected for character defects, resulting in three ROIs containing the characters "TEST", "START", and "STOP". Each ROI is treated as a test image, thus yielding the first test image, the third test image, and so on. Character defect detection is then performed on each of the three test images, resulting in an anomaly score map and an anomaly score for each image. The first anomaly score map 821 corresponding to the first test image marks the regions containing defective characters, while the second anomaly score map 822 corresponding to the second test image and the third anomaly score map 823 corresponding to the third test image do not mark the regions containing defective characters.

[0245] The anomaly scores for the first, second, and third images to be detected are 0.97, 0.15, and 0.12, respectively. The electronic device can merge the anomaly score images for each image into the original image 824, which is the physical image for character defect detection. Based on a pre-set scoring threshold, it determines whether each image has a defect (NG) or does not (OK), i.e., threshold judgment NG / OK. Ultimately, the first image to be detected is determined to have a defect (NG), and the defective area is the region containing the letters "TES". The second and third images to be detected do not have defects (OK).

[0246] In the technical solution of this application, the operations of obtaining, storing, using, processing, transmitting, providing and disclosing user personal information are all carried out with the user's authorization.

[0247] Corresponding to the character defect detection method described above, this application also provides a character defect detection device. The following describes a vehicle brand recognition device provided by this application.

[0248] like Figure 9 As shown, a character defect detection device includes:

[0249] Image acquisition module 901 is used to acquire an image to be detected, wherein the image to be detected includes characters;

[0250] Feature extraction module 902 is used to extract image features of the image to be detected;

[0251] The region segmentation module 903 is used to divide the image to be detected into multiple regions to be detected according to the pre-stored region segmentation parameters, wherein the region segmentation parameters are used to characterize the region segmentation method of pre-dividing the defect-free sample image.

[0252] The feature segmentation module 904 is used to segment the image features into region image features corresponding to each region to be detected according to the pre-stored feature segmentation parameters, wherein the feature segmentation parameters are determined based on the relationship between the image space and the feature space and the region segmentation parameters;

[0253] The similarity determination module 905 is used to determine the similarity of each region to be detected based on the similarity between the region image features of the region to be detected and the pre-stored sample features. The sample features are the features corresponding to the sample image regions obtained by dividing the defect-free sample image according to the region division method.

[0254] The result determination module 906 is used to determine the detection result of the image to be detected based on the similarity corresponding to each region to be detected.

[0255] As can be seen, in this embodiment of the application, the electronic device can acquire an image to be detected, wherein the image to be detected includes characters; extract image features of the character regions in the image to be detected; divide the character regions into multiple regions to be detected according to pre-stored region division parameters, wherein the region division parameters are parameters corresponding to the region division method for dividing the sample character regions in a defect-free sample image that are pre-recorded; divide the image features into region image features corresponding to each region to be detected according to pre-stored feature division parameters, wherein the feature division parameters are obtained by transforming the region division parameters based on the size relationship between the image features and the character regions in the image to be detected; for each region to be detected, determine the similarity corresponding to the region to be detected based on the similarity between the region image features of the region to be detected and each pre-stored sample feature, wherein the sample feature is the feature corresponding to the sample image region obtained by dividing the sample character regions in a defect-free sample image according to the region division method; and determine the character defect detection result of the image to be detected based on the similarity corresponding to each region to be detected. Since the sample image region belongs to a defect-free sample image, the sample feature is the feature corresponding to the sample image region. Therefore, if the similarity between the image features corresponding to the character region in the image to be detected and the sample features is small, it indicates that the image to be detected has a defect; conversely, it may not have a defect. Since the sample image regions are obtained by dividing the defect-free sample image into regions according to region division parameters, the image to be detected can be divided into regions according to the region division parameters, and the image features can be divided according to the feature division parameters to obtain the image region features corresponding to each region to be detected. In this way, the electronic device can calculate the similarity between the region image features of each region to be detected and the features of each sample for each region to be detected. Then, based on the similarity corresponding to each region to be detected, the character defect detection result is determined, thus achieving character defect detection. This eliminates the need to train multiple models for character defect detection, reducing the complexity of character defect detection.

[0256] As one embodiment of this application, the above-described apparatus may further include:

[0257] The sample image acquisition module is used to acquire multiple defect-free sample images;

[0258] The sample feature extraction module is used to extract image features of the sample character region in the defect-free sample image;

[0259] The region division method determination module is used to determine the number of regions corresponding to the sample character region based on the size of the sample character region, and to determine the size of the sample image region to be divided based on the grayscale distribution of the sample character region.

[0260] The sample region division module is used to divide the sample character region according to the region division method to obtain multiple sample image regions corresponding to each defect-free sample image.

[0261] The feature sampling module is used to sample the image features corresponding to the sample image region based on the grayscale distribution of the sample image region, so as to obtain the sample features;

[0262] The feature library construction module is used to store the sample features and the region division parameters that characterize the region division method as a sample feature library.

[0263] As one embodiment of this application, the above-described apparatus may further include:

[0264] The feature segmentation method determination module is used to determine the feature segmentation method based on the size relationship between the sample character region and the image features of the sample character region and the region segmentation method;

[0265] The image feature segmentation module is used to segment the image features of the sample character region according to the feature segmentation method, so as to obtain the image features corresponding to each sample image region.

[0266] The parameter storage module is used to store the feature partitioning parameters that characterize the feature partitioning method into the sample feature library.

[0267] As one embodiment of this application, the above-mentioned sample feature extraction module may include:

[0268] The feature extraction submodule is used to input the defect-free sample image into a pre-trained feature extractor for each defect-free sample image in order to obtain the intermediate layer features of the sample character region in the defect-free sample image extracted by the intermediate layer of the feature extractor.

[0269] The global feature acquisition submodule is used to input the intermediate layer features from the attention layer to introduce global information into the intermediate layer features and obtain global features;

[0270] The local feature acquisition submodule is used to input the intermediate layer features into the local feature aggregation module to obtain the local features output by the local feature aggregation module;

[0271] The feature stacking submodule is used to stack the global features and the local features according to channels, as image features of the sample character region in the defect-free sample image.

[0272] As one embodiment of this application, the feature sampling module described above may include:

[0273] The feature sampling submodule is used to sample the image features corresponding to the sample image region based on the grayscale distribution of the sample image region and according to the preset sampling constraints to obtain the sample features;

[0274] The preset sampling constraint ensures that the proportion of image features corresponding to sample image regions with fewer than a preset number of grayscale value types is less than a preset proportion, the proportion of image features corresponding to sample image regions with more than or equal to the preset number of grayscale value types is not less than the preset proportion, and the maximum distance between the sampled image features is within a preset range.

[0275] As one embodiment of this application, the similarity determination module 905 may include:

[0276] The distance map calculation submodule is used to calculate the feature distance map between the regional image features of the region to be detected and the features of each sample in the sample feature library.

[0277] The segmentation map determination submodule is used to take the feature distance map with the smallest distance between the region image features represented by the multiple calculated feature distance maps and the region to be detected as the anomaly localization segmentation map corresponding to the region to be detected. The anomaly localization segmentation map is used to identify the maximum similarity between the region to be detected and each sample feature in the sample feature library.

[0278] As one embodiment of this application, the result determination module 906 may include:

[0279] The segmentation map merging submodule is used to merge the anomaly localization segmentation maps corresponding to each region to be detected according to the position of each region to be detected in the image to be detected represented by the region division parameters, so as to obtain the anomaly localization map corresponding to the image to be detected.

[0280] The normalization processing submodule is used to normalize the anomaly localization map based on a predetermined maximum distance to obtain a normalized anomaly localization map, wherein the maximum distance is the maximum distance between the sample features;

[0281] The first result determination submodule is used to determine the detection area corresponding to the pixel value greater than the preset anomaly score if the maximum value of the pixel value of the normalized anomaly localization map is greater than the preset anomaly score, and to determine the determined detection area as the character defect area in the image to be detected.

[0282] The second result determination submodule is used to determine that the image to be detected does not have character defects if the maximum value of the pixel value of the normalized anomaly localization map is not greater than the preset anomaly score.

[0283] This application also provides an electronic device, such as... Figure 10 As shown, it includes:

[0284] Memory 1001 is used to store computer programs;

[0285] The processor 1002, when executing the program stored in the memory 1001, implements the character defect detection method described in any of the above embodiments.

[0286] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 1002, the communication interface, and the memory 1001 communicating with each other via the communication bus.

[0287] As can be seen, in this embodiment of the application, the electronic device can acquire an image to be detected, wherein the image to be detected includes characters; extract image features of the character regions in the image to be detected; divide the character regions into multiple regions to be detected according to pre-stored region division parameters, wherein the region division parameters are parameters corresponding to the region division method for dividing the sample character regions in a defect-free sample image that are pre-recorded; divide the image features into region image features corresponding to each region to be detected according to pre-stored feature division parameters, wherein the feature division parameters are obtained by transforming the region division parameters based on the size relationship between the image features and the character regions in the image to be detected; for each region to be detected, determine the similarity corresponding to the region to be detected based on the similarity between the region image features of the region to be detected and each pre-stored sample feature, wherein the sample feature is the feature corresponding to the sample image region obtained by dividing the sample character regions in a defect-free sample image according to the region division method; and determine the character defect detection result of the image to be detected based on the similarity corresponding to each region to be detected. Since the sample image region belongs to a defect-free sample image, the sample feature is the feature corresponding to the sample image region. Therefore, if the similarity between the image features corresponding to the character region in the image to be detected and the sample features is small, it indicates that the image to be detected has a defect; conversely, it may not have a defect. Since the sample image regions are obtained by dividing the defect-free sample image into regions according to region division parameters, the image to be detected can be divided into regions according to the region division parameters, and the image features can be divided according to the feature division parameters to obtain the image region features corresponding to each region to be detected. In this way, the electronic device can calculate the similarity between the region image features of each region to be detected and the features of each sample for each region to be detected. Then, based on the similarity corresponding to each region to be detected, the character defect detection result is determined, thus achieving character defect detection. This eliminates the need to train multiple models for character defect detection, reducing the complexity of character defect detection.

[0288] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0289] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0290] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0291] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0292] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described character defect detection methods.

[0293] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the character defect detection methods described in the above embodiments.

[0294] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state drive (SSD), etc.

[0295] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0296] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0297] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method for detecting character defects, characterized in that, The method includes: Acquire an image to be detected, wherein the image to be detected includes characters; Extract image features from the character regions in the image to be detected; According to the pre-stored region division parameters, the character region is divided into multiple regions to be detected, wherein the region division parameters are parameters corresponding to the region division method for dividing the sample character region in the defect-free sample image that are pre-recorded. According to the pre-stored feature segmentation parameters, the image features are divided into region image features corresponding to each region to be detected. The feature segmentation parameters are obtained by transforming the region segmentation parameters based on the size relationship between the image features and the character regions in the image to be detected. For each region to be detected, the similarity of the region to be detected is determined based on the similarity between the region image features of the region to be detected and the pre-stored sample features. The sample features are the features of the sample image regions obtained by dividing the sample character regions in the defect-free sample image according to the region division method. Based on the similarity corresponding to each region to be detected, the character defect detection result of the image to be detected is determined.

2. The method according to claim 1, characterized in that, Prior to the step of acquiring the image to be detected, the method further includes: Acquire multiple defect-free sample images; Extract image features from the character regions of the sample in the defect-free sample image; Based on the size of the sample character region, determine the number of regions corresponding to the sample character region, and based on the grayscale distribution of the sample character region, determine the size of the sample image region to be divided. The sample character region is divided according to the region division method to obtain multiple sample image regions corresponding to each defect-free sample image. Based on the grayscale distribution of the sample image region, the image features corresponding to the sample image region are sampled to obtain sample features; The sample features and the region division parameters that characterize the region division method are stored as a sample feature library.

3. The method according to claim 2, characterized in that, Before the step of sampling the image features corresponding to the sample image region based on the grayscale distribution of the sample image region to obtain the sample features, the method further includes: The feature partitioning method is determined based on the size relationship between the sample character region and the image features of the sample character region, as well as the region partitioning method. According to the feature division method, the image features of the sample character region are divided to obtain the image features corresponding to each sample image region; The feature partitioning parameters that characterize the feature partitioning method are stored in the sample feature library.

4. The method according to claim 2, characterized in that, The step of extracting image features of the sample character region in the defect-free sample image includes: For each defect-free sample image, the sample character region in the defect-free sample image is input into a pre-trained feature extractor to obtain the intermediate layer features of the sample character region in the defect-free sample image extracted by the intermediate layer of the feature extractor. The intermediate layer features are input from the attention layer to introduce global information into the intermediate layer features, thereby obtaining global features; The intermediate layer features are input into the local feature aggregation module to obtain the local features output by the local feature aggregation module; The global features and the local features are stacked according to channels to form the image features of the sample character region in the defect-free sample image.

5. The method according to claim 2, characterized in that, The step of sampling image features corresponding to the sample image region based on the grayscale distribution of the sample image region to obtain sample features includes: Based on the grayscale distribution of the sample image region, the image features corresponding to the sample image region are sampled according to the preset sampling constraints to obtain the sample features; The preset sampling constraint ensures that the proportion of image features corresponding to sample image regions with fewer than a preset number of grayscale value types is less than a preset proportion, the proportion of image features corresponding to sample image regions with more than or equal to the preset number of grayscale value types is not less than the preset proportion, and the maximum distance between the sampled image features is within a preset range.

6. The method according to any one of claims 1-5, characterized in that, The step of determining the similarity of the region to be detected based on the similarity between the region image features of the region to be detected and the features of each pre-stored sample includes: Calculate the feature distance map between the regional image features of the region to be detected and the features of each sample in the sample feature library; The feature distance map with the smallest distance between the region image features of the region to be detected and the region to be detected is used as the anomaly localization segmentation map corresponding to the region to be detected. The anomaly localization segmentation map is used to identify the maximum similarity between the region to be detected and each sample feature in the sample feature library.

7. The method according to claim 6, characterized in that, The step of determining the character defect detection result of the image to be detected based on the similarity corresponding to each region to be detected includes: According to the position of each region to be detected in the image to be detected represented by the region division parameters, the anomaly localization segmentation maps corresponding to each region to be detected are merged to obtain the anomaly localization map corresponding to the image to be detected. Based on a predetermined maximum distance, the anomaly localization map is normalized to obtain a normalized anomaly localization map, wherein the maximum distance is the maximum distance between the sample features; If the maximum value of the pixel value in the normalized anomaly localization map is greater than the preset anomaly score, the detection area corresponding to the pixel value greater than the preset anomaly score is determined, and the determined detection area is determined as the character defect area in the image to be detected. If the maximum pixel value of the normalized anomaly localization map is not greater than the preset anomaly score, it is determined that the image to be detected does not have character defects.

8. A character defect detection device, characterized in that, The device includes: An image acquisition module is used to acquire an image to be detected, wherein the image to be detected includes characters; The feature extraction module is used to extract image features of character regions in the image to be detected; The region segmentation module is used to divide the character region into multiple regions to be detected according to the pre-stored region segmentation parameters, wherein the region segmentation parameters are a pre-recorded region segmentation method for dividing the sample character region in a defect-free sample image. The feature segmentation module is used to segment the image features into region image features corresponding to each region to be detected according to the pre-stored feature segmentation parameters, wherein the feature segmentation parameters are obtained by transforming the region segmentation parameters based on the size relationship between the image features and the character regions in the image to be detected; The similarity determination module is used to determine the similarity of each region to be detected based on the similarity between the region image features of the region to be detected and the pre-stored sample features. The sample features are the features corresponding to the sample image regions obtained by dividing the sample character regions in the defect-free sample image according to the region division method. The result determination module is used to determine the character defect detection result of the image to be detected based on the similarity corresponding to each region to be detected.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains a computer program that, when executed by a processor, implements the method of any one of claims 1-7.

Images