Image processing method, electronic device and storage medium

Abstract

The present application provides an image processing method, the method comprising: obtaining an image to be recognized and a standard image; obtaining a first text area in the image to be recognized; obtaining a second text area in the standard image, extracting a text window according to the second text area, wherein the text window includes a plurality of sub-windows; obtaining a target text area in the image to be recognized based on the first text area and the text window; segmenting the target text area according to the plurality of sub-windows to obtain a first text sub-area set, and segmenting the second text area according to the plurality of sub-windows to obtain a second text sub-area set; when all first text sub-areas in the first text sub-area set are the same as the corresponding second text sub-areas in the second text sub-area set, marking the image to be recognized as a qualified image. The present application also provides an electronic device and a storage medium, which can improve production efficiency.

Description

Technical Field

[0001] The present application relates to the field of monitoring technology, and in particular to an image processing method, an electronic device and a storage medium. Background Art

[0002] Automated Optical Inspection (AOI) is an automatic visual inspection of printed circuit board manufacturing. It is a high-speed and high-precision optical image inspection system that uses machine vision to compare the object to be tested with the standard image to determine whether it meets the standard. AOI machines are commonly used on SMT (Surface Mount Technology) assembly lines to detect the quality of parts on circuit boards after solder assembly (PCB Assembly), or to check whether the solder paste meets the standard after printing. Generally speaking, AOI machine engineers will set the inspection standards for each object to be tested. If the inspection standards are set too strictly, the false alarm rate will be too high; if the inspection standards are set too loosely, product defects may be missed. Products that are currently detected as defective by AOI need to be further re-judged through manual visual inspection (Visual Inspection) to determine whether the object to be tested (such as a circuit board) really has a problem, and the results of manual visual inspection shall prevail.

[0003] For example, when the current AOI machine detects IC components, it will use the text on the component as a basis for judgment to confirm whether the component is qualified. Usually, the inspection standards for the text on the component are stricter. The same IC component has the same engraved text, but because the components are supplied by different manufacturers, the font type of the same text may be different, resulting in it being judged as a defect after inspection by the AOI machine. At this time, engineers need to reconfirm the inspection standards, or add standard images to adjust the misjudgment of the AOI machine caused by different fonts, resulting in a decrease in the overall production line efficiency. Summary of the invention

[0004] In view of this, it is necessary to provide an image processing method, an electronic device and a storage medium that can improve the production efficiency of a production line.

[0005] An embodiment of the present application provides an image processing method, the method comprising:

[0006] Obtaining an image to be identified and a standard image;

[0007] Acquire a first text area in the image to be recognized;

[0008] Acquire a second text region in the standard image, and extract a text window according to the second text region, wherein the text window includes a plurality of sub-windows;

[0009] Based on the first text area and the text window, obtaining a target text area in the image to be recognized;

[0010] Segmenting the target text region according to the multiple sub-windows to obtain a first text sub-region set, and segmenting the second text region according to the multiple sub-windows to obtain a second text sub-region set;

[0011] determining whether all first text sub-regions in the first text sub-region set are the same as corresponding second text sub-regions in the second text sub-region set; and

[0012] When all first text sub-regions in the first text sub-region set are identical to corresponding second text sub-regions in the second text sub-region set, the image to be recognized is marked as a qualified image.

[0013] In some implementations of the present application, obtaining a target text area in the image to be recognized based on the first text area and the text window includes:

[0014] intercepting a second text area in the standard image;

[0015] Matching the second text region with the first text region to find the region in the second text region that is most similar to the first text region; and

[0016] The region with the highest similarity is selected in the first text region using the text window to obtain the target text region.

[0017] In some implementations of the present application, determining whether all first text sub-regions in the first text sub-region set are the same as corresponding second text sub-regions in the second text sub-region set includes:

[0018] Calculate the similarity between a first text sub-region in the first text sub-region set and a corresponding second text sub-region in the second text sub-region set to obtain a similarity set;

[0019] Determining whether each similarity in the similarity set is greater than or equal to a preset value;

[0020] When each similarity in the similarity set is greater than or equal to a preset value, confirming that all first text sub-regions in the first text sub-region set are identical to corresponding second text sub-regions in the second text sub-region set;

[0021] When there is a text subregion in the similarity set with a similarity less than the preset value, it is confirmed that a first text subregion in the first text subregion set is different from a corresponding second text subregion in the second text subregion set.

[0022] In some implementation modes of the present application, the method for obtaining the similarity set includes:

[0023] Extracting a first feature value of each first text sub-region in the first text sub-region set by a preset classifier, and extracting a second feature value of each second text sub-region in the second text sub-region set by the preset classifier; and

[0024] The similarity between the first eigenvalue and the second eigenvalue is calculated to obtain a similarity set.

[0025] In some implementation modes of the present application, the method for obtaining the similarity set further includes:

[0026] Each first text sub-region in the first text sub-region set is input into a preset classifier to identify each first text sub-region.

[0027] In some embodiments of the present application, the method further comprises:

[0028] When there is a first text sub-region in the first text sub-region set that is different from a corresponding second text sub-region in the second text sub-region set, the image to be identified is marked as a defective image.

[0029] In some implementations of the present application, the method further includes preprocessing the image to be identified, and the preprocessing the image to be identified includes:

[0030] Filter out noise in the image to be identified by using a filter;

[0031] enhancing the first text area by using image enhancement technology; and

[0032] The image to be identified is binarized.

[0033] In some embodiments of the present application, the method further comprises:

[0034] After marking the image to be identified as a qualified image, outputting prompt information to prompt that the image to be identified is a qualified image; or

[0035] After marking the image to be identified as a defective image, output a prompt message to prompt that the image to be identified is a defective image

[0036] An embodiment of the present application provides an electronic device, the electronic device comprising:

[0037] Processor; and

[0038] A memory, wherein a plurality of program modules are stored in the memory, and the plurality of program modules are loaded by the processor to execute the image processing method as described above.

[0039] An embodiment of the present application provides a storage medium on which at least one computer instruction is stored. The instruction is loaded by a processor and executes the above-mentioned image processing method.

[0040] The embodiments of the present application provide an image processing method, an electronic device, and a storage medium. By using an image processing method and a classifier to extract features of an image to be identified, similar features can be extracted from the same text even if they have different font types, and then compared with the text features of a standard image to determine whether the image to be identified is a qualified image. The present application can significantly reduce the machine's misjudgment rate and significantly improve the overall production line efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a schematic diagram of an electronic device according to an embodiment of the present application.

[0042] Figure 2 is a flowchart of an image processing method according to one embodiment of the present application.

[0043] Figure 3 This is a standard image diagram according to this application.

[0044] Figure 4 It is a schematic diagram of obtaining a second text sub-region by matching the image to be identified with the standard image according to the present application.

[0045] FIG. 5A to FIG. 5F This is a schematic diagram of an image to be identified in this application.

[0046] Figure 6 It is a functional module diagram of an image processing system according to one embodiment of the present application.

[0047] Figure 7 is a schematic diagram of a confusion matrix according to an embodiment of the present application.

[0048] Main component symbols

[0049] Electronic device 1

[0050] Communication unit 10

[0051] Memory 11

[0052] Processor 12

[0053] Text window 30

[0054] Subwindow 301

[0055] The first text region R A , R B , R C

[0056] Region R

[0057] Image processing system 20

[0058] Get module 201

[0059] Processing module 202

[0060] Judgment module 203

[0061] Marking module 204

[0062] The following specific implementation methods will further illustrate the present application in detail in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0064] See also Figure 1 The image processing system 20 runs in an electronic device 1. The electronic device 1 includes, but is not limited to, a communication unit 10, a memory 11, and at least one processor 12. The communication unit 10, the memory 11, and the at least one processor 12 are electrically connected.

[0065] In this embodiment, the communication unit 10 is used to provide network communication for the electronic device 1. The network can be a wired network or a wireless network, such as radio, wireless fidelity (WIFI), cellular, satellite, broadcast, etc.

[0066] The electronic device 1 is connected to the AOI machine (not shown) for communication via the communication unit 10 .

[0067] In one embodiment, the electronic device 1 may be an electronic device installed with an image processing program, such as a computer, a smart phone, a personal computer, a server, etc.

[0068] Those skilled in the art should understand that Figure 1 The structure of the electronic device 1 shown does not constitute a limitation of the embodiment of the present invention. The electronic device 1 may also include Figure 1 More or less other hardware or software, or a different arrangement of components.

[0069] It should be noted that the electronic device 1 is only an example, and other existing or future electronic devices that are suitable for the present invention should also be included in the protection scope of the present invention and included here by reference.

[0070] See also Figure 2 , Figure 2 1 is a flow chart of an image processing method according to an embodiment of the present application. The image processing method is applied in an electronic device 1. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted.

[0071] Step S11: Acquire the image to be recognized and the standard image.

[0072] In this embodiment, when an AOI machine is used to detect IC-type components, the text on the IC-type components will be used as the basis for determining whether the IC products are of the same type of products. It is understandable that the text in each component of the same type of IC-type components will be affixed to the same preset position. The AOI machine can obtain the preset position information when detecting IC-type components, so that an image to be identified and a standard image including a text area can be obtained. The image to be identified and the standard image are images obtained based on the same type of product. The image to be identified is an image obtained from the AOI machine. It should be noted that each character on the standard image is clear, complete and undamaged, the text is not skewed or greatly offset, the light source is normal and the picture is clear.

[0073] It is understandable that both the image to be identified and the standard image include a plurality of pixels. The pixel refers to dividing an image into a plurality of small squares, each of which is called a pixel. The electronic device can represent the entire image by representing information such as the position, color, and brightness of these pixels.

[0074] Step S12: preprocessing the image to be recognized and obtaining a first text region in the image to be recognized. In this embodiment, in order to highlight the text region in the image to be recognized, the image to be recognized is first preprocessed.

[0075] Specifically, the preprocessing of the image to be recognized includes:

[0076] (1) Filtering out noise in the image to be identified by a filter In this embodiment, while retaining the main features of the image to be identified as much as possible, useless noise information that affects subsequent processing is removed.

[0077] (2) Enhancing the first text area by using image enhancement technology.

[0078] In this embodiment, the text in the image to be identified may become relatively blurred after the noise is removed by the filter, and it is necessary to use image enhancement technology to enhance the first text area to make the text in the first text area more obvious. For example, by enhancing the high-frequency components in the image to be identified, the outline of the first text area in the image can be clear and the details can be obvious.

[0079] (3) Binarization processing of the image to be identified.

[0080] In this embodiment, the image to be identified may be converted into a black and white image by binarizing the image to be identified, so as to facilitate distinguishing the first text area from the background area.

[0081] (4) Obtain a first text area in the image to be recognized.

[0082] In this embodiment, the image to be identified is identified by an 8-connected component method to obtain multiple connected objects, the areas of the multiple connected objects are calculated, and the connected objects with an area smaller than a preset area are deleted, the text on the image to be identified is segmented, and all the text is framed with a minimum circumscribed rectangle, which is the first text area in the image to be identified. It can be understood that the method for obtaining the first text area in the image to be identified is not limited to the above method.

[0083] In this embodiment, in order to ensure that noise can be removed and the text area can be enhanced in each image to be recognized, the above steps (1) and (2) can be used alternately.

[0084] Step S13: Acquire a second text region in the standard image, and extract a text window according to the second text region, wherein the text window includes a plurality of sub-windows.

[0085] In this embodiment, the method for obtaining the second text region in the standard image is the same as the method for obtaining the first text region in the image to be recognized, and will not be described in detail herein.

[0086] It should be noted that the text window is a bounding rectangle with the smallest area that contains all the pixels of the first text area extracted from the image. The text window includes a plurality of sub-windows, each of which is a bounding rectangle with the smallest area of ​​each pixel of a character area in the first text area. Figure 3 As shown, the second text area in the standard image includes the text "SW3", that is, the standard image includes the smallest circumscribed rectangle of the pixel points of the area corresponding to all the second text "SW3". The text window 30 includes three sub-windows 301, each of which corresponds to the letters "S", "W" and the number "3".

[0087] Step S14: Based on the first text area and the text window, a target text area in the image to be recognized is obtained.

[0088] In this embodiment, the first text area may include more text information than the second text area in the standard image. In order to find the same text information as that in the standard image in the first text area, it is necessary to use the text window 30 to slide in the first text area in the image to be identified to find the same text information as that in the standard image.

[0089] Specifically, based on the first text area and the text window 30, obtaining the target text area in the image to be recognized includes:

[0090] (1) Cut out a second text area in the standard image.

[0091] (2) Matching the second text region with the first text region to find the region in the second text region that is most similar to the first text region. In this embodiment, each pixel in the second text region is matched from left to right and from top to bottom with each pixel in the first text region to find the region in the second text region that is most similar to the first text region.

[0092] The similarity indexes defined in this application include similarity metrics commonly used to calculate different samples, such as the inverse distance (including Euclidean distance, Manhattan distance, Hamming distance, etc.), correlation coefficient, structural similarity (SSIM, Structural Similarity), complex wavelet structural similarity (CW-SSIM, Complex WaveletSSIM) and cosine similarity, etc. Different similarity indexes will be used according to different scenarios to facilitate subsequent text detection and comparison.

[0093] (3) Using the text window 30, the region with the highest similarity is selected in the first text region to obtain the target text region.

[0094] For example, Figure 4 As shown, the first text area R in the image A to be recognized A The first text region R in the image B to be recognized includes the characters "GSW30" B The first text region R in the image C to be recognized includes the characters "LBJ23". C , the second text region R in the standard image includes the characters "SW3". The second text region in the standard image is intercepted to obtain a region R including the text "SW3", and the region R is combined with the first text region R A Match and find the first text area R AIn the character area "SW3", the text window is used to open the first text area R A The character area "SW3" is selected in the box, and the target character area includes the character "SW3". B Match and find the first text area R B In the character area "BJ23", the text window 30 is used to open the first text area R B The character area "BJ23" is selected in the middle, and the target character area includes the character "BJ23". 0 With the first text area R C Match and find the first text area R C The character area in the text window 30 includes a part of the letter "E" and the character "Oti3". C A character region including part of the letter "E" and the character "Oti3" is selected in the middle box, and a target character region including part of the letter "E" and the character "Oti3" is obtained.

[0095] It should be noted that the target text area does not necessarily include complete characters, but the first text area is framed according to the size of the text window 30 to obtain the target text area.

[0096] Step S15: segmenting the target text region according to the multiple sub-windows to obtain a first text sub-region set, and segmenting the second text region according to the multiple sub-windows to obtain a second text sub-region set.

[0097] In this embodiment, in order to more accurately compare whether the target text area is consistent with the second text area in the standard image, it is necessary to segment the target text area and then compare them one by one. Specifically, the target text area is segmented according to multiple sub-windows in the text window to obtain a first text sub-region set. For example, Figure 4As shown, the text window of the standard image includes three sub-windows. The target text area in the image A to be identified is segmented by the three sub-windows to obtain a first text sub-region set, which includes the letter "S" selected by a sub-window, the letter "W" selected by a sub-window, and the number "3" selected by a sub-window; the target text area in the image B to be identified is segmented by the three sub-windows to obtain a first text sub-region set, which includes the letter "B" selected by a sub-window, the letter and number "J2" selected by a sub-window, and the number "3" selected by a sub-window; the target text area in the image C to be identified is segmented by the three sub-windows to obtain a first text sub-region set, which includes part of the letter "E" and part of the letter "O" selected by a sub-window, part of the letter "O" and the letter "ti" selected by a sub-window, and the number "3" selected by a sub-window.

[0098] In this embodiment, the second text region in the standard image is segmented using the three sub-windows to obtain a second text sub-region set, wherein the second text sub-region set includes a letter "S" selected by a sub-window, a letter "W" selected by a sub-window, and a number "3" selected by a sub-window.

[0099] It should be noted that the first text sub-region set and the second text sub-region set do not necessarily include complete characters. The size of the sub-regions in the first text sub-region set and the second text sub-region set is determined by the size of the sub-window in the text window 30 .

[0100] Step S16: determining whether all first text sub-regions in the first text sub-region set are the same as corresponding second text sub-regions in the second text sub-region set.

[0101] In this embodiment, by comparing whether the first text sub-region in the first text sub-region set is identical to the second text sub-region in the corresponding second text sub-region set, it is confirmed whether the image to be recognized is a qualified image. When all the first text sub-regions in the first text sub-region set are identical to the corresponding second text sub-regions in the second text sub-region set, the process proceeds to step S17; when there is a first text sub-region in the first text sub-region set that is different from the corresponding second text sub-region in the second text sub-region set, the process proceeds to step S18.

[0102] In this embodiment, determining whether all first text sub-regions in the first text sub-region set are the same as corresponding second text sub-regions in the second text sub-region set includes:

[0103] (a) Calculating the similarity between a first text sub-region in the first text sub-region set and a corresponding second text sub-region in the second text sub-region set to obtain a similarity set.

[0104] Specifically, the method for obtaining the similarity set includes:

[0105] (1) Inputting each first text sub-region in the first text sub-region set into a preset classifier to identify each first text sub-region. In this embodiment, the preset classifier is a classifier obtained by training based on a single character data set obtained after preprocessing the image to be processed, and additionally collecting various English letter and number data sets.

[0106] (2) extracting the first feature value of each first text sub-region in the first text sub-region set by the preset classifier, and extracting the second feature value of each second text sub-region in the second text sub-region set by the preset classifier. For example, extracting the first feature value T of the first text sub-region including the character “S” in the image A to be recognized by the preset classifier 10 , extracting the first feature value T of the first text sub-region including the character “W” in the image A to be recognized by the preset classifier 11 , and extracting the first feature value T of the first text sub-region including the character “3” in the image A to be recognized by the preset classifier 12 ; Extracting the first feature value T of the first text sub-region including the character "B" in the image B to be recognized by the preset classifier 20 , extracting the first feature value T of the first text sub-region including the character “J2” in the image B to be recognized by the preset classifier 21 , and extracting the first feature value T of the first text sub-region including the character “3” in the image B to be recognized by the preset classifier 22 ; and extracting the first feature value T of the first text sub-region including part of the letter "E" and part of the letter "O" in the image to be recognized C through the preset classifier 30 , extracting the first feature value T of the first text sub-region including part of the letter "O" and the letter "ti" in the image to be recognized C through the preset classifier 31 , and extracting the first feature value T of the first text sub-region including the character “3” in the image C to be recognized by the preset classifier 32 .

[0107] The second feature value T of the second text sub-region including the character “S” in the standard image is extracted by the preset classifier. 00, extracting the second feature value T of the second text sub-region including the character “W” in the standard image through the preset classifier 01 , and extracting the second feature value T of the second text sub-region including the character "3" in the standard image through the preset classifier 02 .

[0108] (3) Calculate the similarity between the first eigenvalue and the second eigenvalue to obtain a similarity set. For example, calculate the similarity between the first text sub-region in the first text sub-region set in the image to be identified A and the second text sub-region in the second text sub-region set of the corresponding standard image to obtain a similarity set. Specifically, calculate the first eigenvalue T 10 With the second eigenvalue T 00 The similarity between them is obtained by 00 , calculate the first eigenvalue T 11 With the second eigenvalue T 01 The similarity between them is obtained by 01 , and the first eigenvalue T 12 With the second eigenvalue T 02 The similarity between them is obtained by 02 The similarity set is {S 00 , S 01 , S 02}. Or, the similarity between the first text sub-region in the first text sub-region set in the image to be identified B and the second text sub-region in the second text sub-region set of the corresponding standard image is calculated to obtain a similarity set. Specifically, the similarity between the image to be identified B and the standard image is calculated, such as calculating the first eigenvalue T 20 With the second eigenvalue T 00 The similarity between them is obtained by 10 , calculate the first eigenvalue T 21 With the second eigenvalue T 01 The similarity between them is obtained by 11 , and the first eigenvalue T 22 With the second eigenvalue T 02 The similarity between them is obtained by 12 The similarity set is {S 10 , S 11 , S 12 Similarly, the similarity between the first text sub-region in the first text sub-region set in the to-be-recognized image C and the second text sub-region in the second text sub-region set corresponding to the standard image can be calculated to obtain a similarity set.

[0109] (b) Determine whether each similarity in the similarity set is greater than or equal to a preset value. When each similarity in the similarity set is greater than or equal to the preset value, it is confirmed that all first text sub-regions in the first text sub-region set are the same as the corresponding second text sub-regions in the second text sub-region set, that is, it is confirmed that the first text region in the image to be identified is the same as the second text region in the standard image, and the process enters step S17; when there is a similarity in the similarity set that is less than the preset value, it is confirmed that the first text sub-region in the first text sub-region set is different from the corresponding second text sub-region in the second text sub-region set, that is, it is confirmed that the first text region in the image to be identified is different from the second text region in the standard image, and the process enters step S18.

[0110] For example, the similarity S in the similarity set 00 , similarity S 01 and similarity S 02 are greater than or equal to the preset value, the process enters step S17; if the similarity S in the similarity set 00 Or similarity S 01 , or similarity S 02 If the value is less than the preset value, the process proceeds to step S18.

[0111] Step S17: Mark the image to be identified as a qualified image.

[0112] In this embodiment, when each similarity in the similarity set is greater than or equal to a preset value, it is determined that all characters in the target text area in the image to be recognized are determined to be the same as all characters in the second text area in the standard image, and the image to be recognized is marked as a qualified image. For example, if all characters in the target text area in the image A to be recognized are determined to be the same as all characters in the second text area in the standard image, the image A to be recognized can be marked as a qualified image.

[0113] In one embodiment, the image processing method may further output a prompt message to indicate that the image to be identified is a qualified image. For example, a "Pass" message may be output to indicate that the image to be identified is a qualified image. In other words, the object to be detected corresponding to the image to be identified A is a qualified object to be detected.

[0114] Step S18: marking the image to be identified as a defective image.

[0115] In this embodiment, when there is a similarity less than the preset value in the similarity set, that is, there is text in the target text area of ​​the image to be identified that is determined to be different from the text in the second text area in the standard image, indicating that the text features of this image to be identified are different from the text features of the standard image, then the image to be identified is marked as a defective image. For example, if there is text in the target text area of ​​the image to be identified B that is different from the text in the second text area in the standard image, the image to be identified B can be marked as a defective image. In other words, the object to be identified corresponding to the image to be identified B is an unqualified object to be identified.

[0116] In one embodiment, the image to be identified is marked as a defective image in a variety of possible situations. For example, the text on the image to be identified is different from the text on the standard image, such as Figure 5A , it can be confirmed that the object to be tested corresponding to the image to be identified is not the same type as the object to be tested corresponding to the standard image. For example, the object to be tested corresponding to the image to be identified and the object to be tested corresponding to the standard image are IC components produced by different manufacturers; the text on the image to be identified is greatly offset, such as Figure 5B , it can be confirmed that the object to be detected corresponding to the image to be identified has shifted, which may cause errors in subsequent use. For example, when the object to be detected corresponding to the image to be identified is a square component, the four vertices of the square component need to be soldered on the circuit board. Figure 5B If the image to be identified is missing, the circuit board cannot be soldered on it. Figure 5C At this time, it is impossible to confirm whether the object to be detected corresponding to the image to be identified is the same as the object to be detected corresponding to the standard image; the text on the image to be identified is blurred, such as Figure 5D At this time, it is also impossible to confirm whether the object to be detected corresponding to the image to be identified is the same as the object to be detected corresponding to the standard image; the text on the image to be identified is covered by foreign objects or the light source is abnormal, such as Figure 5E , at this time, it is confirmed that there may be foreign matter on the object to be tested corresponding to the image to be identified, which may affect the performance of the object to be tested; and the image to be identified is skewed, such as Fig. 5F At this time, it is confirmed that the object to be detected corresponding to the image to be identified may be skewed.

[0117] In one embodiment, the image processing method may further output a prompt message to indicate that the image to be identified is a defective image, for example, outputting a "Fail" message to indicate that the image to be identified is a defective image.

[0118] In summary, the image processing method provided by the present application uses an image processing method and a classifier to extract features of the image to be identified, so that similar features can be extracted from the same text even if they have different font types, and then compared with the text features of the standard image to determine whether the image to be identified is a qualified image. The present application can greatly reduce the machine's misjudgment rate and greatly improve the overall production line efficiency.

[0119] See also Figure 6 In this embodiment, the image processing system 20 can be divided into one or more modules, and the one or more modules can be stored in the processor 12, and the processor 12 executes the image processing method of the embodiment of the present application. The one or more modules can be a series of computer program instruction segments that can perform specific functions, and the instruction segments are used to describe the execution process of the image processing system 20 in the electronic device 1. For example, the image processing system 20 can be divided into Figure 6 The acquisition module 201, the processing module 202, the judgment module 203 and the marking module 204 in the embodiment.

[0120] The acquisition module 201 is used to acquire the image to be recognized and the standard image; the acquisition module 201 is also used to acquire the first text area in the image to be recognized; the acquisition module 201 is also used to acquire the second text area in the standard image, and extract a text window according to the second text area, wherein the text window includes multiple sub-windows; the processing module 202 is used to obtain the target text area in the image to be recognized based on the first text area and the text window; the processing module 202 is also used to segment the target text area according to the multiple sub-windows to obtain a first text sub-region set, and segment the second text area according to the multiple sub-windows to obtain a second text sub-region set; the judgment module 203 is used to judge whether all the first text sub-regions in the first text sub-region set are the same as the corresponding second text sub-regions in the second text sub-region set; and the marking module 204 is used to mark the image to be recognized as a qualified image when all the first text sub-regions in the first text sub-region set are the same as the corresponding second text sub-regions in the second text sub-region set.

[0121] Due to the variability of production line data, the present application also develops a system update mechanism, which enables the image processing system 20 to be automatically updated through continuous accumulation of data to ensure the accuracy of the model so as to achieve the effect of adapting to various products.

[0122] In one embodiment, the result of the image to be identified after being judged by this system can be compared with the result of manual processing and marking, and detection indicators such as accuracy, missed detection rate and over-kill rate can be calculated. When the detection indicators reach the standard set by the production line, it means that the overall system is stable, and the new data of the production line (i.e., the image to be identified) will continue to be judged by the image processing system 20 of this application to see if there are defects.

[0123] If the detection index does not meet the standard set by the production line, the system update mechanism is activated to retrain the preset classifier for the production line data (i.e., the image to be identified) to enhance the adaptability of the preset classifier to the production line data. The retrained result is then compared with the result of manual data labeling, and the detection index is calculated. This process is repeated until the detection index meets the requirements set by the production line, and the system update is completed.

[0124] For example, this application collects 699 images of IC-type text components that are judged as defective by the SMT production line AOI machine, of which 386 are divided into training data for developing the image processing system and 313 are verification data for verification and testing after the system development is completed. According to the confusion matrix results of the image processing system, Figure 7 shown. Figure 7 The confusion matrix on the left records the results of the training data when developing the image processing system. The actual results of manual inspection are that 252 images are qualified, marked as "PASS", and 134 images are unqualified, marked as "FAIL". The predicted results of the image processing system are that 246 images are qualified, marked as "PASS", and 140 images are unqualified, marked as "FAIL", of which 6 images are misjudged as unqualified images by the image processing system. From this, it can be calculated that the missed detection rate in the training data is 0 / (0+134)=0%, and the overkill rate is 6 / (246+6)=2.3%.

[0125] Figure 7The confusion matrix on the right records the results of the verification data when developing the image processing system. The actual results of manual inspection are that 183 images are qualified, marked as "PASS", and 130 images are unqualified, marked as "FAIL". The results predicted by the image processing system are that 179 images are qualified, of which 1 image is unqualified and misjudged as qualified, and 134 images are unqualified, of which 5 images are unqualified and misjudged as qualified. From this, it can be calculated that the missed detection rate in the verification data is 1 / (1+129)=0.7%, and the overkill rate is 5 / (178+5)=2.7%. It can be seen that in the verification data, the accuracy rate of the image processing system of this application is as high as 98%. Applying this system to the SMT production line can greatly reduce the time required for manual visual inspection, and can reduce the frequency of production line engineers adjusting AOI machine parameters, greatly improving efficiency and overall production line stability.

[0126] In one embodiment, the processor 12 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor 12 may be any other conventional processor, etc.

[0127] If the modules in the image processing system 20 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0128] It is understandable that the module division described above is a logical function division, and there may be other division methods in actual implementation. In addition, the functional modules in each embodiment of the present application may be integrated in the same processing unit, or each module may exist physically separately, or two or more modules may be integrated in the same unit. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0129] In another embodiment, the electronic device 1 may further include a memory (not shown), and the one or more modules may also be stored in the memory and executed by the processor 12. The memory may be an internal memory of the electronic device 1, that is, a memory built into the electronic device 1. In other embodiments, the memory may also be an external memory of the electronic device 1, that is, a memory externally connected to the electronic device 1.

[0130] In some embodiments, the memory is used to store program codes and various data, for example, to store program codes of the image processing system 20 installed in the electronic device 1, and to achieve high-speed and automatic access to programs or data during the operation of the electronic device 1.

[0131] The memory may include random access memory and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0132] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or basic features of the present application. Therefore, no matter from which point of view, the above embodiments of the present application should be regarded as exemplary and non-restrictive, and the scope of the present application is defined by the attached claims rather than the above description, and therefore it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Obtaining an image to be identified and a standard image; Acquire a first text area in the image to be recognized; Acquire a second text region in the standard image, and extract a text window according to the second text region, wherein the text window includes a plurality of sub-windows; Based on the first text area and the text window, obtaining a target text area in the image to be recognized includes: intercepting a second text area in the standard image; matching the second text area with the first text area to find an area in the second text area that is most similar to the first text area; and selecting the area with the highest similarity in the first text area using the text window to obtain the target text area; Segmenting the target text region according to the multiple sub-windows to obtain a first text sub-region set, and segmenting the second text region according to the multiple sub-windows to obtain a second text sub-region set; Determining whether all first text subregions in the first text subregion set are identical to corresponding second text subregions in the second text subregion set, comprising: calculating the similarity between the first text subregion in the first text subregion set and the corresponding second text subregion in the second text subregion set to obtain a similarity set; determining whether each similarity in the similarity set is greater than or equal to a preset value; when each similarity in the similarity set is greater than or equal to the preset value, confirming that all first text subregions in the first text subregion set are identical to corresponding second text subregions in the second text subregion set; when there is a first text subregion in the similarity set having a similarity less than the preset value, confirming that there is a first text subregion in the first text subregion set that is not identical to the corresponding second text subregion in the second text subregion set; and When all first text sub-regions in the first text sub-region set are identical to corresponding second text sub-regions in the second text sub-region set, the image to be recognized is marked as a qualified image.

2. The image processing method according to claim 1, characterized in that: The method for obtaining the similarity set includes: Extracting a first feature value of each first text sub-region in the first text sub-region set by a preset classifier, and extracting a second feature value of each second text sub-region in the second text sub-region set by the preset classifier; and The similarity between the first eigenvalue and the second eigenvalue is calculated to obtain a similarity set.

3. The image processing method according to claim 2, characterized in that: The method for obtaining the similarity set further includes: Each first text sub-region in the first text sub-region set is input into the preset classifier to identify each first text sub-region.

4. The image processing method according to claim 1, wherein: The method further comprises: When there is a first text sub-region in the first text sub-region set that is different from a corresponding second text sub-region in the second text sub-region set, the image to be identified is marked as a defective image.

5. The image processing method according to claim 4, characterized in that: The method further includes preprocessing the image to be identified, wherein the preprocessing includes: Filter out noise in the image to be identified by using a filter; enhancing the first text area by using image enhancement technology; and The image to be identified is binarized.

6. The image processing method according to claim 5, characterized in that: The method further comprises: After marking the image to be identified as a qualified image, outputting prompt information to prompt that the image to be identified is a qualified image; or After marking the image to be identified as a defective image, a prompt message is output to prompt that the image to be identified is a defective image.

7. An electronic device, characterized in that: The electronic device comprises: Processor; and A memory, wherein a plurality of program modules are stored in the memory, and the plurality of program modules are loaded by the processor and execute the image processing method according to any one of claims 1 to 6.

8. A storage medium having at least one computer instruction stored thereon, characterized in that: The instructions are loaded by a processor and execute the image processing method according to any one of claims 1 to 6.

Images