Precise template construction method and device and template matching method and device

Abstract

The invention relates to the technical field of semiconductor detection, provides a semiconductor machine vision detection method, and particularly relates to a precise template construction method and device and a template matching method and device. According to the method, automatic modeling of the standard object is achieved through the image processing technology, manual operation is not needed in the modeling process, implementation is easy, modeling efficiency and accuracy are greatly improved, important basic conditions are provided for follow-up detection precision, the universality of the template construction method is good, and the method can adapt to modeling of standard objects of different models.

Description

Technical Field

[0001] The present application relates to the field of semiconductor detection technology, and is a semiconductor machine vision detection method, and specifically to a precision template construction method and device, and a template matching method and device. Background Art

[0002] IC Package Substrate, also known as IC package substrate or package substrate, is an important semiconductor electronic component. Its main functions are to protect, fix and carry semiconductor IC chips, and to serve as a bridge between the chip and the PCB to achieve signal transmission connection. However, due to the limitations of process and equipment in the production process of IC substrates, some defects may occur. These defects will not only affect the quality of IC substrates, but also directly affect the performance and stability of electronic equipment. Therefore, in order to ensure the finished product quality and shipment yield of IC substrates, it is necessary to effectively test the quality of IC substrates during the production process.

[0003] At present, for defect detection of IC substrates, the existing detection technologies of enterprises include traditional manual visual inspection. Manual detection methods are inefficient, slow, and low in precision, and are prone to missed detection or false detection, making it difficult to meet the detection needs of products. In addition, there are automatic detection methods based on image processing. Such methods mainly create standard template images of normal IC substrates and compare them with the detection substrate images to determine whether there are defects. However, due to the high complexity of the substrates and the cumbersome modeling process, manual operation of the modeling process is usually required. Manual operation is inefficient and is affected by subjective factors. The accuracy of the standard template cannot be guaranteed, and the needs of automated detection cannot be met. Summary of the invention

[0004] In order to solve the above problems, the present application provides a method and device for constructing a precision template, a method and device for template matching, and an apparatus for automatically and quickly modeling a standard IC substrate to provide an accurate standard template for the subsequent defect detection process, thereby improving detection efficiency and accuracy. In order to achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows:

[0005] In a first aspect, a method for constructing a precision template is provided, the method comprising: acquiring a standard image of a standard object, the standard image being obtained by performing image acquisition on the standard object through an image acquisition device; aligning the standard image with a design drawing, determining the correspondence between different physical functional areas on the design drawing in the standard image, and acquiring sub-template images corresponding to different physical functional areas, and constructing a basic precision template based on the sub-template images; the physical functional areas include a pad area, a green oil area, a circuit area, and a substrate area; determining a functional area with the least feature expression in each of the three non-collinear areas of the standard image as a positioning point template, each of the positioning point templates recording center point coordinate information and deflection angle information; and combining the three positioning point templates with the basic precision template to obtain a precision template.

[0006] In some specific implementations, the standard image is obtained by stitching at least four sub-region images, and the four sub-region images are obtained by capturing images of four different regions of the standard object using an image capture device having the same field of view.

[0007] In some specific implementations, the four different areas of the standard object include the upper left area, lower left area, upper right area and lower right area of ​​the standard object; stitching at least four of the sub-area images includes: respectively obtaining the relative offset between two adjacent areas, and stitching the images corresponding to the areas based on the relative offset to obtain a first area image and a second area image; obtaining the relative offset between the first area image and the second area image, and stitching the first area image and the second area image based on the relative offset to obtain the standard image.

[0008] In some specific implementations, the standard image and the design drawing are aligned to determine the correspondence between different physical functional areas on the design drawing in the standard image, including: selecting one of the physical functional areas on the design drawing as a target functional area, and using the target functional area as a functional template to perform template matching on the standard image to determine the coordinate offset between the design drawing and the standard image.

[0009] In some specific implementations, the functional area with the least feature expression in each of the three non-collinear areas of the standard image is determined as a positioning point template, including: obtaining all functional areas of the same type in each of the areas as reference functional areas, and determining the features of each of the reference functional areas, and clustering all the reference functional areas based on feature similarity to obtain multiple clustering groups; screening the clustering group with the least number of reference functional areas in the clustering group as a target clustering group, and using the reference functional areas in the target clustering group as the positioning point template; the positioning point template contains the center coordinate information and deflection angle information of the reference functional area.

[0010] In some specific implementations, before aligning the standard image and the design drawing, the method also includes: acquiring the two physical functional areas with the same characteristics on the same column on the standard image, and determining the center coordinates of the two physical functional areas on the standard image, and according to the angular relationship between the center coordinates and the column axis, obtaining the rotation angle for the standard image and rotating the standard image so that the standard image and the design drawing have the same spatial position relationship.

[0011] In some specific implementations, obtaining the two physical functional areas with the same features on the same column on the standard image includes: dividing the standard image into two upper and lower areas, randomly selecting any physical functional area of ​​any area, and matching this physical functional area with the physical functional area of ​​the other area until the matching result meets the matching threshold requirement.

[0012] In a second aspect, a precision template construction device is provided, the device comprising: an image acquisition module, for acquiring a standard image of a standard object, the standard image being acquired by an image acquisition device performing image acquisition on the standard object; a basic precision template construction module, for aligning the standard image with a design drawing, determining the correspondence between different physical functional areas on the design drawing in the standard image, and acquiring sub-template images corresponding to different physical functional areas, and constructing a basic precision template based on the sub-template images; a precision template construction module, for determining, in three non-collinear areas of the standard image, a functional area with the least feature expression in each area as a positioning point template, each of the positioning point templates recording center point coordinate information and deflection angle information; and combining the three positioning point templates with the basic precision template to obtain a precision template.

[0013] In a third aspect, a template matching method is provided, the method comprising: performing positioning point matching between a positioning point template and an image to be detected, and determining at least three non-collinear positioning points on the detection image; the positioning point template is constructed based on the method described in any one of the above items; based on at least three positioning points, a basic precision template is matched to the image to be detected through affine transformation; the basic precision template is constructed based on the method described in any one of the above items.

[0014] In a fourth aspect, a template matching device is provided, which includes: a positioning module, used to match the positioning points with the image to be detected through the positioning point template, and determine at least three non-collinear positioning points on the detection image; a matching module, used to match the basic precise template to the image to be detected through affine transformation based on at least three of the positioning points.

[0015] In the technical solution provided in the embodiment of the present application, automatic modeling of standard objects is achieved through image processing technology. No manual operation is required during the modeling process, which is simple to implement, greatly improving the modeling efficiency and accuracy, providing an important basic condition for subsequent detection accuracy, and the template construction method has good versatility and can adapt to the modeling of standard objects of different models. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] The methods, systems and / or programs in the accompanying drawings will be further described according to exemplary embodiments. These exemplary embodiments will be described in detail with reference to the drawings. These exemplary embodiments are non-limiting exemplary embodiments, wherein example numbers represent similar mechanisms in the various views of the accompanying drawings.

[0018] Figure 1 It is a flow chart of the precision template construction method provided in the embodiment of the present application.

[0019] Figure 2 It is a schematic diagram of an IC carrier in an embodiment of the present application.

[0020] Figure 3 It is a schematic diagram of the standard image construction method in the embodiment of the present application.

[0021] Figure 4 It is a schematic diagram of the structure of the precision template construction device provided in the embodiment of the present application.

[0022] Figure 5It is a schematic diagram of the template matching method flow provided in the embodiment of the present application.

[0023] Figure 6 It is a schematic diagram of the structure of the template matching device provided in the embodiment of the present application.

[0024] Figure 7 It is a schematic diagram of the server structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to better understand the above technical scheme, the technical scheme of the present application is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical scheme of the present application, rather than limitations on the technical scheme of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0026] In the following detailed description, numerous specific details are set forth by way of example in order to provide a comprehensive understanding of the relevant guidance. However, it will be apparent to those skilled in the art that the present application may be practiced without these details. In other cases, well-known methods, procedures, systems, compositions and / or circuits have been described at a relatively high level, without detail, in order to avoid unnecessarily obscuring aspects of the present application.

[0027] Flowcharts are used in the present application to illustrate the execution process performed by the system according to the embodiment of the present application. It should be clearly understood that the execution process of the flowchart may not be performed in order. On the contrary, these execution processes may be performed in reverse order or simultaneously. In addition, at least one other execution process may be added to the flowchart. One or more execution processes may be deleted from the flowchart.

[0028] Before further describing the embodiments of the present invention in detail, the nouns and terms involved in the embodiments of the present invention are described. The nouns and terms involved in the embodiments of the present invention are subject to the following interpretations.

[0029] (1) In response, it is used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed may be in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations executed are executed.

[0030] (2) Based on is used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed may be in real time or have a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations executed are executed.

[0031] The embodiment of the present application provides a method for constructing a precision template, and a template matching processing method based on the precision template, to achieve surface detection of semiconductor products. Among them, the semiconductor product in this embodiment is an IC substrate (ICPackage Substrate), which can also be referred to as an IC packaging substrate or packaging substrate in other embodiments. However, it is worth noting that the method in the embodiment of the present application is described with an IC substrate, but in other embodiments, this scheme can also be applied to template matching processing methods for other semiconductor products, including but not limited to semiconductor electronic devices such as wafer bare chips and packaged IC chips.

[0032] The precision template is a standard template image used in the template matching method, which is obtained by capturing images of the standard IC substrate. In the template matching test, the precision template is compared with the image of the substrate to be tested to determine whether there are defects. However, in the prior art, the modeling process is cumbersome due to the high complexity of the IC substrate, and is usually manually operated. However, manual operation is inefficient, and the accuracy of the constructed standard template cannot be guaranteed due to subjective factors, and cannot meet the needs of automated testing.

[0033] Therefore, in order to solve the above technical problem, the present application embodiment first provides a method for constructing a precision template to achieve standard template modeling for complex electronic products. Figure 1 , this method includes the following steps:

[0034] Step S11: Acquire a standard image of a standard object.

[0035] Step S12. Align the standard image with the design drawing, determine the correspondence between different physical functional areas on the design drawing in the standard image, obtain sub-template images corresponding to different physical functional areas, and construct a basic precision template based on the sub-template images.

[0036] Step S13: Determine, in the three non-collinear regions of the standard image, a functional region with the least feature expression in each region as a positioning point template, and combine the three positioning point templates with the basic precise template to obtain a precise template.

[0037] Among them, for each of the positioning point templates, the center point coordinate information and the deflection angle information are recorded. In step S11 of this embodiment, the standard object is a standard IC substrate, and the characteristics of the standard IC substrate are complete and normal. This standard object can also be called a golden IC substrate, and in other scenarios in semiconductor products, such as packaged chip scenarios, it can also be called a Golden Die. Correspondingly, the standard image is the soil system obtained by image acquisition of the standard object, and in other embodiments it can also be called a Golden Image.

[0038] The use of standard images is achieved through high-resolution cameras and lenses.

[0039] For the acquired standard image, it is used to record and express the image information of the standard IC carrier, and to match the IC carrier to be inspected, so as to determine whether the IC carrier to be inspected has defects and the type of corresponding defects through the differences between the image features. However, it is worth noting that for complex electronic products, especially for IC carriers, there are multiple different physical functional areas, such as pad areas, green oil areas, circuit areas and substrate areas, and different physical functional areas have different corresponding inspection standards. Therefore, in the actual inspection scenario, it is necessary to match different physical functional areas with corresponding inspection templates. Furthermore, for the establishment of a precise template, it is necessary to establish corresponding sub-templates according to different physical functional areas, and combine different sub-templates to obtain a precise template for the IC carrier.

[0040] The core of this process is how to determine the corresponding different functional areas on the image, which is mainly achieved through manual annotation in the prior art. In this embodiment, it is achieved by calling the standard object, that is, the design drawing of the IC carrier, and aligning it with the standard image obtained in step S11, that is, the design drawing with the same size is matched with the standard image one by one, and each physical functional area in the design drawing is correspondingly annotated on the standard image. In the design drawing of the IC carrier, there is label information of different physical functional areas. By aligning this design drawing with the standard image, the aligned area can be mapped by label to obtain information about the corresponding area on the standard image, so that this area image can be used as a sub-template image about this physical functional area.

[0041] However, in the actual process, because there is an offset between the design drawing and the standard image, it is necessary to adjust the spatial position of the design drawing with the standard image before positioning processing, so that the design drawing and the standard image can be accurately aligned.

[0042] Specifically, in this embodiment, the pad area is used as the physical functional area for explanation. The pad area is selected in the design drawing and a pad area matching map is established. Template matching is performed on the standard image through the pad area matching map to obtain the row and column coordinate offset between the standard image and the pad area matching map. This row and column coordinate offset is a two-dimensional offset, including an x-coordinate offset and a y-coordinate offset. The design drawing is transferred to the standard image based on this row and column coordinate offset, thereby realizing automatic drawing-image alignment, and each physical functional area on the standard image after alignment is saved as a sub-template map, and a basic precision template is constructed based on multiple sub-template maps.

[0043] Through step S11 and step S12, a basic precise template can be obtained, wherein corresponding sub-template images are configured in the basic precise template according to different physical functional areas.

[0044] In the template matching process, it is necessary not only to obtain templates corresponding to different physical functional areas, but also to configure positioning points on the templates. The process of performing template matching in the image to be detected through the positioning points is specifically to affine the precise template to the image to be detected through affine transformation, so as to use different detection rules to detect in different physical functional areas. For affine transformation processing, it is necessary to solve the affine matrix. The solution of the affine matrix requires the mapping relationship of at least three non-collinear points, so it is necessary to configure at least three positioning points with non-collinear relationships in the precise template.

[0045] During the detection process, template matching needs to be performed on the image. In order to prevent the template from matching other similar areas, the positioning points should be unique. In order to ensure that the selected positioning points are unique, in this embodiment, the positioning points are selected by clustering, and the clustering method can ensure that there are large differences between different cluster groups, so that each cluster is unique.

[0046] Specifically, the standard image is divided into multiple regions, and any region is selected as the target region for positioning point selection. Figure 2 As shown in the figure, the black box corresponds to an area. A complete standard image is composed of multiple identical areas (views) arranged and combined. When modeling, you only need to select one of the areas. The positioning points generated in the area can be used for detection of all areas.

[0047] Three non-collinear regions are determined on the target region map, wherein the concept of the three non-collinear regions is that the three regions are not in the same column. Specifically, in this embodiment, the upper left position, the lower left position and the lower right position in the target region map are the three non-collinear regions. Three positioning point search areas are generated in the above three non-collinear regions, and the positioning points of each area are determined in the above three positioning point search areas. Among them, the positioning points need to be matched with templates on the image to be detected in the subsequent detection process. In order to prevent the template from matching with other similar areas during matching, the positioning points must be unique as templates.

[0048] In this embodiment, in order to ensure the uniqueness of the positioning point, the most unique point in each area should be determined during the positioning point search. The specific processing method is to find the feature point with the least clustering items in each positioning point search area by clustering. Among them, the feature selected is the gold surface. Specifically, a clustering operation is performed on the gold surfaces in each positioning point search area, and the gold surfaces with similar structures are clustered into a group to obtain multiple cluster groups, and the number of gold surfaces contained in each cluster group is different. Then, the group with the least number of gold surfaces is selected from the cluster groups as the selected positioning group, and all gold surface areas in the positioning group are identified as positioning point areas.

[0049] Through the above processing, the center coordinates and deflection angle information of each positioning point corresponding to the three positioning point search areas can be obtained, and the three positioning points are combined with the basic precise template to obtain the final precise template.

[0050] In this embodiment, the positioning points obtained as above are precise positioning points, which are used to initially position the IC carrier and the standard image through the initial positioning points to achieve the initial position of the IC carrier.

[0051] For the initial positioning point, the pad area on the standard image is first divided into the boundary pad area and the middle pad area, and the gold surface area is selected in the boundary pad area from top to bottom, and the gold surface template is constructed using the gold surface image corresponding to the gold surface area. The gold surface area is expanded by 1000 pixels to obtain dila_roi, and the uniqueness of the gold surface is judged in dila_roi, specifically, template matching is performed in the area to find whether there are two gold surfaces with the same structure. When there is only one gold surface with the same structure, the gold surface area is the initial positioning ROI area, and this initial positioning point is the center point of the template. The initial positioning template is saved for subsequent detection; if there are two gold surfaces with the same structure, the gold surface cannot be identified as the initial positioning ROI, and the next gold surface area is selected, and the judgment is made according to the above process.

[0052] It is worth noting that, because the IC carrier may have a certain tilt angle during the image acquisition process, in order to ensure the accuracy of subsequent detection, the standard image needs to be rotated. The rotation process obtains the two physical functional areas with the same characteristics on the same column on the standard image, and determines the center coordinates of the two physical functional areas on the standard image, and obtains the rotation angle for the standard image according to the angular relationship between the center coordinates and the column axis, and rotates the standard image, so that the standard image has the same spatial position relationship with the design drawing.

[0053] Among them, the physical functional area in this embodiment is preferably selected as the gold surface area, the same characteristics refer to the same shape, color and area, and the same column refers to being in the same vertical line.

[0054] Specifically, for this process, two gold surface areas with the same structure are found in the upper and lower areas of the standard image respectively, and the two gold surface areas with the same structure are on the same vertical line on the IC carrier sheet, so as to calculate the center coordinates of the two gold surface structures, and calculate the angle with the y-axis according to the center coordinates to obtain the rotation angle.

[0055] The gold surface area is the characteristic expression of the pad area. The pad area on the standard image is obtained by binary segmentation of the standard image. The minimum bounding rectangle of the pad area is calculated, and the rectangle is morphologically eroded inward to make the vertical side of the rectangle intersect with the gold surface, and then the gold surface intersecting with the left vertical side of the rectangle is obtained, that is, all the gold surfaces on the same vertical line of the left vertical side of the rectangle are obtained.

[0056] The gold surface on the same vertical line obtained above is divided into two upper and lower regions with the y-axis center of the IC substrate, and the gold surfaces in the upper and lower regions are selected in turn for template matching to determine two gold surfaces with the same structure.

[0057] Specifically, first select the first gold surface based on the upper part, and use it to create a gold surface template, then select the gold surface in the lower area and match it with the gold surface template in the upper area. If the matching beam is greater than 0.9, it is considered that the matched gold surface and the gold surface in the upper area have the same structure; if it is less than 0.9, continue to select the next gold surface in the gold surface in the lower area for matching, and cycle in sequence until two gold surfaces with the same structure are found.

[0058] According to the matching results, the center coordinates Top(x,y) and Bottom(x,y) of the upper and lower gold surface structures are obtained respectively, and the two center points are connected into a straight line. The angle between the straight line and the y-axis, i.e., the rotation angle, is calculated, and the standard image is rotated based on the rotation angle.

[0059] In some other embodiments, because the accuracy required for defect detection is high and the size of the IC substrate is large, in order to obtain a high-precision standard image, it is not possible to obtain it in one acquisition due to the field of view limitation of the image acquisition device. It is necessary to acquire images of multiple areas of the standard object and splice the acquired sub-images to obtain the final standard image.

[0060] For more information about how to obtain the standard image, please refer to Figure 3 , including the following steps:

[0061] Step S31: collecting data from four different regions of the standard object based on the same field of view.

[0062] In this embodiment, the four different regions of the standard object are the upper left region, the upper right region, the lower left region and the lower right region of the standard object, i.e., the IC carrier. The image acquisition device acquires images of the above four regions respectively to obtain four corresponding sub-region images. It is worth noting that the four sub-region images are in an adjacent relationship, and there is a small amount of overlapping area between the four sub-region images. The combination of the four sub-region images can constitute a complete image of the standard object.

[0063] Step S32: stitching the four sub-region images.

[0064] The stitching of the above four sub-region images mainly includes obtaining stitching parameters, calculating the offset between the four sub-region images, and stitching the four sub-region images based on the offset.

[0065] Among them, for the stitching parameters, the overlapping area in any one of the two adjacent sub-area images is obtained, and a template area is selected in the overlapping area as the area template, and the center point coordinates P1 (join_row1, join_col1) of this area template are determined. For example, the upper left area image is obtained, and a template area is selected in the overlapping area of ​​the upper left area image, and a template is created using the template area and the center point coordinates of the template are determined.

[0066] The selected template area is translated along the X direction to the overlapping area in another sub-area image, and the area is morphologically expanded, and the expanded area is defined as the area to be searched. Template matching is performed in the area to be searched to obtain the coordinates of the center point after matching P2 (join_row2, join_col2).

[0067] Wherein, P1 and P2 are stitching parameters for image stitching in this embodiment.

[0068] The image offsets of the above two sub-region images are calculated for the obtained stitching parameters P1 and P2, where the offset in the X direction offset_x1 is: join_row1-join_row2, and the offset in the Y direction offset_y1 is: Width-offset / 2.0. Where Width is the width of a sub-region image, and the actual overlap width offset = Width-(join_col1-join_col2).

[0069] According to the obtained offset in the x and y directions, another sub-region image is offset to obtain a first stitched image after the two sub-region images are stitched together. Similarly, the same method is used for the remaining two sub-region images to obtain a second stitched image.

[0070] The above stitching method is also used for stitching between the first stitched image and the second stitched image. The corresponding stitching parameters of P1 and P2 are found in the overlapping area of ​​the two stitched images respectively, and the image offset is calculated according to P1 and P2, wherein the offset in the x direction offset_x is: offset_x:=join_col1-join_col2; the offset in the y direction offset_y is: Height-offset / 2.0, Height is the height of the TopImage image, offset:=Height-(join_row1-join_row2).

[0071] According to the offset in the xy direction obtained by the above processing, the first stitched image and the second stitched image are stitched together to obtain a complete standard image of the standard object.

[0072] The precision template construction method provided in the embodiment of the present application utilizes image processing technology to realize automatic modeling of IC substrates. No manual operation is required during the modeling process, and the implementation is simple. It greatly improves the modeling efficiency and accuracy, and provides an important basic condition for subsequent detection accuracy. In addition, the template construction method has good versatility and can adapt to the modeling of IC substrates of different models.

[0073] In order to better implement the above method, the embodiment of the present application also provides a precision template construction device, which can be integrated in an electronic device, and the electronic device can be a terminal, a server, etc. Among them, the terminal can be a mobile phone, a tablet computer, a smart Bluetooth device, a laptop, a personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers.

[0074] For example, in this embodiment, the method of the embodiment of the present application will be described in detail by taking the specific integration of the precision template construction device in an electronic device as an example.

[0075] For example, Figure 4 As shown, the precision template construction device 40 may include an image acquisition module 41, a basic precision template construction module 42 and a precision template construction module 43, wherein:

[0076] The image acquisition module 41 is used to acquire a standard image of a standard object. The standard image is acquired by performing image acquisition on the standard object through an image acquisition device.

[0077] In some specific embodiments, the standard image is obtained by stitching at least four sub-region images, and the four sub-region images are obtained by capturing images of four different regions of the standard object using an image capture device having the same field of view.

[0078] In some specific implementations, the four sub-region images are images corresponding to the upper left region, the lower left region, the upper right region, and the lower right region of the standard object, respectively.

[0079] In some specific embodiments, the four sub-region images are stitched together by respectively obtaining the relative offset between two adjacent regions and stitching the images corresponding to the regions based on the relative offset to obtain the first region image and the second region image; the relative offset between the first region image and the second region image is obtained, and the first region image and the second region image are stitched together based on the relative offset to obtain the standard image.

[0080] The basic precision template construction module 42 is used to align the standard image with the design drawing, determine the correspondence between different physical functional areas on the design drawing in the standard image, and obtain the sub-template images corresponding to different physical functional areas, and construct a basic precision template based on the sub-template images.

[0081] In some specific embodiments, the alignment processing is performed by selecting a physical functional area on the design drawing as a target functional area, and using the target functional area as a functional template to perform template matching on the standard image to determine the coordinate offset between the design drawing and the standard image.

[0082] In some specific embodiments, before aligning the standard image with the design drawing, the unit also obtains the two physical functional areas with the same characteristics on the same column on the standard image, and determines the center coordinates of the two physical functional areas on the standard image, and according to the angular relationship between the center coordinates and the column axis, obtains the rotation angle for the standard image and rotates the standard image so that the standard image and the design drawing have the same spatial position relationship.

[0083] Specifically, the two physical functional areas are obtained by dividing the standard image into two upper and lower areas, randomly selecting any physical functional area of ​​any area, and matching this physical functional area with the physical functional area of ​​the other area until the matching result meets the matching threshold requirement.

[0084] The precise template construction module 43 is used to determine the functional area with the least feature expression in each area of ​​the three non-collinear areas of the standard image as a positioning point template, each of which records the center point coordinate information and the deflection angle information; the three positioning point templates are combined with the basic precise template to obtain a precise template.

[0085] In some specific embodiments, the positioning point template is determined by obtaining all functional areas of the same type in each of the areas as reference functional areas, determining the characteristics of each of the reference functional areas, and clustering all the reference functional areas based on characteristic similarity to obtain multiple cluster groups; the cluster group with the least number of reference functional areas in the cluster group is selected as the target cluster group, and the reference functional areas in the target cluster group are used as the positioning point template; the positioning point template contains the center coordinate information and deflection angle information of the reference functional area.

[0086] In this embodiment, a template matching method of a precision template constructed based on the above method and device is also provided. For details about this method, please refer to Figure 5 , specifically including the following steps:

[0087] Step S51: performing positioning point matching between the positioning point template and the image to be detected to determine at least three non-collinear positioning points on the detection image.

[0088] Step S52: Based on at least three of the positioning points, the basic precise template is affine transformed to the image to be detected for matching.

[0089] Similarly, in this embodiment, a template matching device is also provided, which can be integrated in an electronic device, and the electronic device can be a terminal, a server, etc. The terminal can be a mobile phone, a tablet computer, a smart Bluetooth device, a laptop, a personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers.

[0090] For example, in this embodiment, the method of the embodiment of the present application will be described in detail by taking the template matching device specifically integrated in an electronic device as an example.

[0091] For example, Figure 6 As shown, the template matching device 60 includes:

[0092] The positioning module 61 is used to match the positioning points with the image to be detected through the positioning point template to determine at least three non-collinear positioning points on the detection image.

[0093] The matching module 62 is used to match the basic precision template to the image to be detected through affine transformation based on at least three of the positioning points.

[0094] The embodiment of the present application also provides an electronic device, which can be a terminal, a server, etc. The terminal can be a mobile phone, a tablet computer, a smart Bluetooth device, a laptop, a personal computer, etc. The server can be a single server or a server cluster composed of multiple servers, etc.

[0095] In some embodiments, the precision template construction device and template matching device can also be integrated into multiple electronic devices. For example, the precision template construction device can be integrated into multiple servers, and the precision template construction method of the present application can be implemented by multiple servers.

[0096] In this embodiment, the electronic device of this embodiment is a server as an example for detailed description, for example, Figure 7 As shown, it shows a schematic diagram of the structure of the server involved in the embodiment of the present application, specifically:

[0097] The server may include one or more processing core processors 701, one or more computer-readable storage media memories 702, a power supply 703, an input module 704, and a communication module 705. Those skilled in the art will appreciate that Figure 7 The server structure shown in the figure does not constitute a limitation on the server, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0098] The processor 701 is the control center of the server, and uses various interfaces and lines to connect various parts of the entire server. It executes various functions of the server and processes data by running or executing software programs and / or modules stored in the memory 702, and calling data stored in the memory 702. In some embodiments, the processor 701 may include one or more processing cores; in some embodiments, the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 701.

[0099] The memory 702 can be used to store software programs and modules. The processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the server, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 702 may also include a memory controller to provide the processor 701 with access to the memory 702.

[0100] The server also includes a power supply 703 for supplying power to various components. In some embodiments, the power supply 703 can be logically connected to the processor 701 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 703 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0101] The server may further include an input module 704, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0102] The server may further include a communication module 705. In some embodiments, the communication module 705 may include a wireless module. The server may perform short-range wireless transmission through the wireless module of the communication module 705, thereby providing wireless broadband Internet access for the user. For example, the communication module 705 may be used to help the user send and receive emails, browse web pages, and access streaming media.

[0103] Although not shown, the server may also include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 701 in the server will load the executable files corresponding to the processes of one or more application programs into the memory 702 according to the following instructions, and the processor 701 will run the application programs stored in the memory 702, thereby implementing the steps in the methods of the embodiments of the present application.

[0104] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0105] From the above, it can be seen that automatic modeling of standard objects can be achieved through image processing technology. No manual operation is required in the modeling process, which is simple to implement, greatly improving the modeling efficiency and accuracy, providing an important basic condition for subsequent detection accuracy. In addition, this template construction method has good versatility and can adapt to the modeling of standard objects of different models.

[0106] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0107] To this end, an embodiment of the present application provides a computer-readable storage medium, which stores a plurality of instructions, which can be loaded by a processor to execute the steps in any one of the precise template construction methods and template matching methods provided in the embodiments of the present application. For example, the instructions can execute the following steps:

[0108] Acquiring a standard image of a standard object, wherein the standard image is obtained by performing image acquisition on the standard object by an image acquisition device;

[0109] Performing alignment processing on the standard image and the design drawing, determining the correspondence between different physical functional areas on the design drawing and the standard image, and obtaining sub-template images corresponding to different physical functional areas, and constructing a basic precision template based on the sub-template images; the physical functional areas include pad areas, green oil areas, circuit areas and substrate areas;

[0110] Among the three non-collinear areas of the standard image, the functional area with the least feature expression in each area is determined as the positioning point template, and each of the positioning point templates records the center point coordinate information and deflection angle information; the three positioning point templates are combined with the basic precise template to obtain a precise template.

[0111] And, you can also perform the following steps:

[0112] Performing positioning point matching between the positioning point template and the image to be detected to determine at least three non-collinear positioning points on the detection image;

[0113] Based on at least three of the positioning points, the basic precise template is affine transformed to the image to be detected for matching.

[0114] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0115] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including a computer program / instruction, the computer program / instruction being stored in a computer-readable storage medium. A processor of an electronic device reads the computer program / instruction from the computer-readable storage medium, and the processor executes the computer program / instruction, so that the electronic device executes the method provided in various optional implementations provided in the above embodiments.

[0116] Since the instructions stored in the storage medium can execute the steps of any one of the precise template construction and template matching methods provided in the embodiments of the present application, the beneficial effects that can be achieved in any one of the precise template construction and template matching methods provided in the embodiments of the present application can be achieved. Please see the previous embodiments for details and will not be repeated here.

[0117] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0118] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0119] The above is a detailed introduction to a precision template construction method, device, template matching method and device, electronic device, storage medium and program product provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for constructing a precision template, characterized in that: The method comprises: Acquiring a standard image of a standard object, wherein the standard image is obtained by performing image acquisition on the standard object by an image acquisition device; Align the standard image with the design drawing, determine the correspondence between different physical functional areas on the design drawing and the standard image, obtain sub-template images corresponding to different physical functional areas, and construct a basic precision template based on the sub-template images; the physical functional areas include pad area, green oil area, circuit area and substrate area; Among the three non-collinear areas of the standard image, the functional area with the least feature expression in each area is determined as the positioning point template, and the three positioning point templates are combined with the basic precise template to obtain a precise template; each of the positioning point templates records the center point coordinate information and deflection angle information.

2. The method for constructing a precision template according to claim 1, characterized in that: The standard image is obtained by splicing at least four sub-region images, and the four sub-region images are obtained by performing image acquisition on four different regions of the standard object by an image acquisition device having the same field of view.

3. The method for constructing a precision template according to claim 2, characterized in that: The four different regions of the standard object include an upper left region, a lower left region, an upper right region, and a lower right region of the standard object; Stitching at least four of the sub-region images, including: respectively acquiring relative offsets between two adjacent regions, and stitching the images corresponding to the regions based on the relative offsets to obtain a first region image and a second region image; The relative offset between the first region image and the second region image is obtained, and the first region image and the second region image are spliced ​​based on the relative offset to obtain the standard image.

4. The method for constructing a precision template according to claim 1, characterized in that: The method of aligning the standard image with the design drawing to determine the correspondence between different physical functional areas on the design drawing in the standard image includes: selecting one physical functional area on the design drawing as a target functional area, and using the target functional area as a functional template to perform template matching on the standard image to determine the coordinate offset between the design drawing and the standard image.

5. The method for constructing a precision template according to claim 1, characterized in that: The method of determining the functional area with the least feature expression in each of the three non-collinear areas of the standard image as a positioning point template includes: obtaining all functional areas of the same type in each of the areas as reference functional areas, determining the features of each of the reference functional areas, and clustering all the reference functional areas based on feature similarity to obtain multiple clustering groups; screening the clustering group with the least number of reference functional areas in the clustering group as a target clustering group, and using the reference functional areas in the target clustering group as the positioning point template; the positioning point template contains the center coordinate information and deflection angle information of the reference functional area.

6. The method for constructing a precision template according to claim 1, characterized in that: Before aligning the standard image and the design drawing, the method also includes: acquiring the two physical functional areas with the same characteristics on the same column on the standard image, and determining the center coordinates of the two physical functional areas on the standard image, and according to the angular relationship between the center coordinates and the column axis, obtaining the rotation angle for the standard image and rotating the standard image so that the standard image and the design drawing have the same spatial position relationship.

7. The method for constructing a precision template according to claim 6, characterized in that: The obtaining of the two physical function areas with the same features on the same column of the standard image includes: dividing the standard image into two upper and lower areas, randomly selecting any physical function area of ​​any area, and matching this physical function area with the physical function area of ​​the other area until the matching result meets the matching threshold requirement.

8. A precision template construction device, characterized in that: The device comprises: An image acquisition module, used to acquire a standard image of a standard object, wherein the standard image is acquired by performing image acquisition on the standard object by an image acquisition device; A basic precision template construction module is used to perform alignment processing on the standard image and the design drawing, determine the correspondence between different physical function areas on the design drawing and the standard image, obtain sub-template images corresponding to different physical function areas, and construct a basic precision template based on the sub-template images; A precision template construction module is used to determine the functional area with the least feature expression in each of the three non-collinear areas of the standard image as a positioning point template, each of the positioning point templates records the center point coordinate information and deflection angle information; the three positioning point templates are combined with the basic precision template to obtain a precision template.

9. A template matching method, characterized in that: The method comprises: By matching the positioning points of the positioning point template with the image to be detected, at least three non-collinear positioning points on the detection image are determined; the positioning point template is constructed based on the method described in any one of claims 1 to 7; Based on at least three of the positioning points, the basic precise template is matched to the image to be detected through affine transformation; the basic precise template is constructed based on the method described in any one of claims 1-7.

10. A template matching device, characterized in that: The device comprises: A positioning module, used to match positioning points with the image to be detected through a positioning point template, and determine at least three non-collinear positioning points on the detection image; A matching module is used to match the basic precise template to the image to be detected through affine transformation based on at least three of the positioning points.

Images