Comparison template generation method, apparatus and device, and computer readable storage medium

Abstract

The invention discloses a comparison template generation method, device and equipment and a computer readable storage medium, and belongs to the field of image detection, and the method comprises the steps: analyzing a target file of a to-be-modeled product, so as to obtain a pre-modeled image and primitive contour information from the target file; a station image of the to-be-modeled product and the pre-modeling image are calibrated and aligned, and the station image is an image collected by a camera on the to-be-modeled product at a production station; marking a difference area and attribute information of the difference area based on the station image and the pre-modeling image after calibration and alignment; and repairing the difference region in the pre-modeling image based on the primitive contour information according to the attribute information of the difference region so as to generate a comparison template file fitting the actual state of the to-be-modeled product. The comparison template constructed by the method can meet the detection requirements of high precision and high efficiency.

Description

Technical Field

[0001] The present application generally relates to the field of image detection technology. More specifically, the present application relates to a method, device, equipment and storage medium for generating a comparison template. Background Art

[0002] PCB design file is a standard file format used for printed circuit board (PCB) manufacturing. It contains the layout information of the circuit board, including circuits, pads, component locations and other graphic elements. In the appearance inspection of PCB products, it is usually necessary to use a comparison template. The comparison template is generally generated by manual calibration on the initial PCB design file based on the actual photos of the product's workstation. However, there are many PCB product models and they change frequently, so the corresponding PCB design files are also frequently updated. Therefore, the manual method is difficult to meet the needs of template construction.

[0003] In view of this, there is an urgent need to provide a method for generating a comparison template to meet the needs of high-precision and high-efficiency detection. Summary of the invention

[0004] In order to at least solve one or more of the technical problems mentioned above, the present application proposes a method, device, equipment and computer-readable storage medium for generating a comparison template in multiple aspects.

[0005] In a first aspect, the present application provides a method for generating a comparison template, comprising: parsing a target file of a product to be modeled to obtain a pre-modeled image and primitive contour information from the target file; calibrating and aligning a workstation image of the product to be modeled with the pre-modeled image, wherein the workstation image is an image of the product to be modeled captured by a camera at a production workstation; marking a difference area and attribute information of the difference area based on the calibrated and aligned workstation image and the pre-modeled image; and repairing the difference area in the pre-modeled image based on the primitive contour information according to the attribute information of the difference area to generate a comparison template file that fits the actual state of the product to be modeled.

[0006] In some embodiments, parsing the target file of the product to be modeled to obtain the pre-modeling image and primitive contour information from the target file includes: responding to a file parsing instruction, splitting the target file to obtain the pre-modeling image and the primitives contained therein; extracting coordinate data of the primitives; and determining the primitive contour information based on the coordinate data.

[0007] In some embodiments, the calibrating and aligning the work station image of the product to be modeled with the pre-modeling image includes: extracting common feature points between the work station image and the pre-modeling image; and calibrating and aligning the work station image with the pre-modeling image based on the common feature points and the conversion relationship between the work station image and the pre-modeling image.

[0008] In some embodiments, the calibration and alignment of the work station image with the pre-modeled image based on the conversion relationship between the common feature points, the work station image and the pre-modeled image includes: acquiring initial coordinate data of the common feature points in the work station image; based on the conversion relationship, converting the initial coordinate data to target coordinate data in the coordinate system of the pre-modeled image; and aligning the common feature points of the work station image with the pre-modeled image based on the target coordinate data and the initial coordinate data.

[0009] In some embodiments, the marking of the difference area and the attribute information of the difference area based on the calibrated and aligned work station image and the pre-modeling image includes: comparing the distance between the work station image and the graphic contour of the pre-modeling image; recording the difference contour whose distance exceeds a preset threshold to mark the difference area; dividing the difference area into corresponding repair categories and recording the corresponding attribute information.

[0010] In some embodiments, dividing the difference area into corresponding repair categories and recording corresponding attribute information includes: determining a primitive repair category contained in the difference area; and recording the corresponding attribute information according to the primitive repair category.

[0011] In some embodiments, repairing the difference area in the pre-modeled image based on the primitive contour information according to the attribute information of the difference area includes: determining the corresponding primitive contour information according to the primitive repair category into which the difference area is divided; and adjusting the corresponding primitive contour in the pre-modeled image according to the attribute information and the primitive contour information to repair the difference area.

[0012] In a second aspect, the present application also provides a device for generating a comparison template, comprising: a parsing module, used to parse a target file of a product to be modeled, so as to obtain a pre-modeling image and primitive contour information from the target file; an alignment module, used to calibrate and align a workstation image of the product to be modeled with the pre-modeling image, wherein the workstation image is an image of the product to be modeled captured by a camera at a production workstation; a labeling module, used to label a difference area and attribute information of the difference area based on the calibrated and aligned workstation image and the pre-modeling image; a repair module, used to repair the difference area in the pre-modeling image based on the primitive contour information according to the attribute information of the difference area, so as to generate a comparison template file that fits the actual state of the product to be modeled.

[0013] In a third aspect, the present application also provides a device for generating a comparison template, comprising: a processor; and a memory storing program instructions for generating a comparison template, wherein when the program instructions are executed by the processor, the method described above is implemented.

[0014] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer program instructions for generating a comparison template, and when the computer program instructions are executed by a processor, the method as described above is implemented.

[0015] Through the generation method of the comparison template provided above, after parsing the pre-modeling image and primitive contour information of the product to be modeled, the workstation image and the pre-modeling image are calibrated and aligned. Then, based on the aligned workstation image and pre-modeling image, the difference area and the attribute information of the difference area are annotated. Then, the difference area is repaired in the pre-modeling image using the primitive contour information to generate a comparison template file that fits the actual state of the product to be modeled. Through the above method, the construction accuracy of the comparison template can be improved, so that it can better meet the requirements of high-precision and high-efficiency detection. Furthermore, in some embodiments, when calibrating and aligning the workstation image and the pre-modeling image, the conversion relationship between the camera coordinate system of the workstation image and the pre-modeling image is directly used. This process no longer needs to pass through the coordinate system of the calibration plate, which simplifies the calibration process. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become easy to understand. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein: Figure 1 An exemplary flow chart showing a method for generating a comparison template according to some embodiments of the present application is shown; Figure 2An exemplary flow chart showing a method for generating a comparison template according to other embodiments of the present application is shown; Figure 3 An exemplary flow chart showing a method for generating a comparison template according to some other embodiments of the present application is shown; Figure 4 An exemplary flow chart showing a method for generating a comparison template according to other embodiments of the present application is shown; Figure 5 A first schematic diagram showing a comparison of contour differences between a workstation image and a pre-modeled image according to an embodiment of the present application is shown; Figure 6 A second schematic diagram showing the difference between the contours of the workstation image and the pre-modeling image according to an embodiment of the present application is shown; Figure 7 A third schematic diagram showing the difference between the contours of the workstation image and the pre-modeling image according to the embodiment of the present application is shown; Figure 8 A fourth schematic diagram showing a comparison of contour differences between a workstation image and a pre-modeled image according to an embodiment of the present application is shown; Fig. 9 This is an exemplary structural diagram of a device for generating a comparison template according to an embodiment of the present application; Fig.10 This is an exemplary structural diagram of a device for generating a comparison template according to an embodiment of the present application. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0018] It should be understood that the terms "include" and "comprising" used in the specification and claims of the present application indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0019] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and claims, the singular forms of "a", "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this application specification and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.

[0020] As used in this specification and claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]", depending on the context.

[0021] The specific implementation of the present application is described in detail below with reference to the accompanying drawings.

[0022] Reference Figure 1 , to explain Figure 1 The method 10 for generating the comparison template shown in the figure comprises: at step S101, parsing the target file of the product to be modeled to obtain the pre-modeled image and the outline information of the primitive from the target file. In some embodiments, the product to be modeled may be a physical PCB board. The production of the PCB board needs to be carried out according to the pre-designed drawings, which are usually saved in the form of Gerber drawing files. In the actual production process of the PCB board, there are often slight differences between the actual PCB board and the drawings due to some processes, lighting, dust and other reasons. However, such differences usually do not affect the normal use of the PCB board. The PCB board produced needs to be inspected for appearance to determine whether the solder joints at various places in the board are intact, whether the line width and spacing are compliant, and whether the overall size and hole position meet the design specifications. The comparison template used for appearance inspection needs to be constructed according to the physical product of the PCB board, which is more in line with the actual state of the product.

[0023] Correspondingly, the target file can be a PCB design file such as a Gerber diagram of the product to be modeled. Taking the Gerber diagram as an example, the pre-modeled image obtained by parsing the target file is also the Gerber diagram of the product. The Gerber diagram describes the graphic information of the product to be modeled, which is composed of different types of primitives, such as lines, holes, pads, etc. The primitive contour information refers to the information obtained by subdividing the boundary or shape of the graphic, usually consisting of a series of line segments, arcs or other curves. Exemplarily, the primitive contour information of a rounded rectangle can be subdivided into four arcs and four line segments. Further, when receiving the parsing instruction, the target file of the product to be modeled can be parsed to obtain the corresponding pre-modeled image. The primitives in the pre-modeled image are subjected to edge detection, contour extraction and other steps to obtain the subdivided primitive contour information. The primitive contour information can be stored in the form of coordinate data for subsequent repair processes. It is worth noting that this storage method stores more information and is easier to process than the previous method of storing pixel image information to construct a comparison template.

[0024] Next, at step S102, the station image of the product to be modeled is calibrated and aligned with the pre-modeled image. In some embodiments, the station image refers to the image of the product to be modeled taken by the station camera on the production line, which can clearly display the graphics on the product to be modeled. The station camera can shoot the station image in a grid shooting or strip shooting manner. Among them, grid shooting is to divide the product to be modeled into multiple grid units, and shoot an image on each grid unit by the camera; strip shooting is to shoot continuously along the long strip area of ​​the product to be modeled. There will be a certain distortion in the station image taken by the camera, so it is necessary to calibrate and align the station image with the pre-modeled image. In the process of calibration and alignment, the common feature points between the two images can be extracted first, and then the transformation relationship between the image coordinate system of the camera image and the pre-modeled image coordinate system can be used to calculate the transformation parameters, and the common feature points between the two images are aligned by the transformation parameters. It should be noted that the transformation parameters can be in the form of coordinate data, and the corresponding transformation form includes but is not limited to translating, scaling, and rotating the image. After calibration and alignment, the pattern of the product to be modeled in the workstation image corresponds one-to-one with the pattern at the same position in the pre-modeling image.

[0025] For example, refer to Figure 5 , which shows the state of a certain primitive in the product to be modeled after calibration and alignment. The green line represents the outline of the primitive in the pre-modeling image, and the red line represents the actual outline of the primitive in the workstation image ( Figure 6 , Figure 7 and Figure 8 It can be seen that most of the red and green outlines in the figure are overlapping, but there are slight differences at some corners. The purpose of constructing the comparison template later is to make the outlines of each pattern in the pre-modeled image fit the actual outline of the production product.

[0026] It can be understood that after calibration and alignment, the workstation image and the pre-modeled image are registered, which facilitates the subsequent repair of the pre-modeled image using the previously stored primitive contour information.

[0027] Furthermore, at step S103, based on the calibrated and aligned workstation image and the pre-modeled image, the difference area and the attribute information of the difference area are marked. In some embodiments, in the pre-modeled image after calibration and alignment, there are still differences between the outline of each primitive and the outline of the corresponding primitive in the workstation image. These differences may be reflected in the differences between the same graphics in the workstation image, and the differences between the pattern and the same graphics in the pre-modeled image. The difference area between the workstation image and the pre-modeled image needs to be marked so that it can be repaired later. Accordingly, the evaluation indicators for the difference area that needs to be repaired in the figure can be divided into two types: one is the difference that appears after superimposing the outlines of the same pattern in the workstation image, and the reasons for determining the generation of such difference areas are often dust, bumps, etc.; the other is the difference that appears when the pattern in the workstation image is superimposed with the standard graphics of the same pattern in the pre-modeled image, and the reasons for determining the generation of such difference areas are usually process differences.

[0028] To this end, it is necessary to mark the difference area between the two and record the attribute information of the difference area so that the difference area can be repaired later. When marking the difference area, a contour spacing threshold (hereinafter referred to as the preset threshold) can be set in advance, and then the spacing between the contours of each graphic in the workstation image and the pre-modeling image is compared. If the spacing between the contours of a certain graphic exceeds the preset threshold, it means that the actual contour at that location is significantly different from the contour on the drawing, and then the difference contour that exceeds the preset threshold is recorded and marked as the difference area. The difference area can be divided into four categories. The first category is the difference caused by actual product defects, refer to Figure 6 , Figure 6 The area marked in the red rectangle is the difference area. It can be seen that these difference areas are mostly manifested as partial area missing or partial area overflow of the product. The second type is the defect caused by slight deformation of the local area. Figure 7 ,like Figure 7 The difference area marked in the red rectangle is the defect caused by slight deformation of the local area. Figure 8 The third type is defects caused by special areas such as arc surfaces, concave and convex areas, and corners. Figure 8 It can be seen that the difference areas marked by the red rectangle are distributed at the corners and arcs. The fourth category is small burr differences or small deformation differences. In the actual process of marking the difference areas, the differences can be divided and marked according to the above categories, so that the differences of the same category can be repaired uniformly in the future.

[0029] Furthermore, after the determined difference contours exceeding the preset threshold are divided into corresponding difference categories, the corresponding attribute information will also be recorded. In this process, the primitive repair category contained in the difference area will be determined first, that is, on which primitive the contour difference is distributed. Based on these primitive repair categories, the corresponding attribute information (such as length, width, angle gradient, etc.) can be recorded. In addition, it should be noted that the attribute information refers to the contour attribute information of the difference area in the workstation image. The difference contour and attribute information are recorded in the same document information. This is because the subsequent repair work on the difference area cannot guarantee that all positions can achieve 100% repair improvement, so it is necessary to record the entire repair process, including the difference contour, attribute information, etc., so that the difference comparison can be carried out in combination with the marked difference area and attribute information in the subsequent detection process.

[0030] At step S104, according to the attribute information of the difference area, the difference area is repaired in the pre-modeled image based on the primitive contour information to generate a comparison template file that fits the actual state of the product to be modeled. In some embodiments, the difference area obtained through the above steps needs to be repaired in the pre-modeled image, that is, the primitive contours of the difference area in the drawing are adjusted to make it more suitable for the actual state of the product to be modeled. The main logic of the repair is to focus on processing the position with obvious difference features (that is, the difference area) based on the previously marked attribute information, so that it is more suitable for the actual state of the product to be modeled.

[0031] It should be noted that the result obtained after repair is a comparison template file containing a comparison template image and JSON data. The JSON data contains the record of the entire repair process, including difference contours, attribute information, etc. In the subsequent detection process, the comparison template will first be compared with the repaired contour. If there is still a significant difference compared with the repaired contour, it will be compared again with the difference contour with annotated attribute information. If there is still a difference, the defect judgment will be made in combination with the annotated difference area.

[0032] In the above content, the construction process of the comparison template uses the pre-modeled image (Gerber image) of the product to be modeled as the carrier to achieve end-to-end processing and simplify the construction process. At the same time, the pre-modeled image is repaired using the primitive contour information containing more content, so that the constructed comparison template is more in line with the actual state of the production product, meeting the needs of high-precision and high-efficiency appearance inspection.

[0033] Reference Figure 2 , continue to explain as Figure 2 The generation method 20 of the comparison template shown. It should be noted that Figure 2 The generation method of the comparison template shown in the above text can be combined with Figure 1A specific implementation of step S101 in the method 10 described above, therefore, the above Figure 1 The description also applies to the following Figure 2 Description of each step.

[0034] At step S201, in response to the file parsing instruction, the target file is split to obtain the pre-modeled image and the primitives contained therein. In some embodiments, the file parsing instruction can be an instruction pre-defined by the developer, or can be directly selected from an existing program library. Its triggering form can be manual triggering or automatic program triggering. When the file parsing instruction is triggered, the content of the target file, that is, the pre-modeled image, will be read out. The pre-modeled image is split, and the position and shape of the primitives (such as line segments, arcs, polygons, etc.) can be parsed therefrom.

[0035] Furthermore, considering that the pre-modeled image will also be calibrated and aligned with the workstation image in the subsequent process, and the shooting method of the workstation image is divided into grid shooting or strip shooting of multiple workstations. Therefore, in a feasible implementation, the target file can be split into several pre-modeled sub-images of the corresponding workstations while retaining the image source signal. Exemplarily, if the workstation image adopts a grid shooting method, that is, a workstation image is composed of several grid images collected from different workstations. Then when splitting the target file, the pre-modeled image can also be split into several pre-modeled sub-images of the corresponding workstation in a one-to-one correspondence with the shooting station of the grid image. It can be understood that if the acquisition method of the workstation image is strip shooting, the splitting method of the pre-modeled image is the same as the above content. In the above implementation, by splitting the target file into pre-modeled images of the corresponding workstations, it is convenient to obtain more detailed primitive contour information. At the same time, it also makes the subsequent processing flow of constructing the comparison template more refined.

[0036] Further, at step S202, the coordinate data of the primitives are extracted. In some embodiments, the primitives in the pre-modeled image obtained by splitting usually belong to different layers (such as copper layer, silk screen layer, etc.), and the coordinate data of the primitives therefore need to be extracted from the corresponding layers. As a feasible implementation method, the layer information of each primitive can be identified, and then the path coordinates or arc data of the primitive can be extracted based on the layer information. It should be noted that these coordinate data all belong to the coordinate system of the pre-modeled image.

[0037] Finally, at step S203, the primitive contour information is determined based on the coordinate data. In some embodiments, the coordinate data of the extracted primitives will be used to represent the primitive contour information. For example, the contour information of a line segment is represented by two coordinate points (X1, Y1) and (X2, Y2); an arc is represented by the center coordinates (Cx, Cy), the radius R, and the start angle, end angle and other data; a polygon is represented by a set of continuous coordinate point data, etc. After that, the primitive contour information of each primitive will be stored. The primitive contour information stored here is more detailed and rich, and can further improve the precision of repairing the difference area through the primitive contour information in the subsequent process.

[0038] Figure 3 This is a flow chart of the method 30 for generating a comparison template of the present application. It should also be noted that: Figure 3 The generation method of the comparison template shown in the above text can be combined with Figure 1 A specific implementation of step S102 in the method 10 described above, therefore, the above Figure 1 The description also applies to the following Figure 3 Description of each step. Figure 3 At step S301, common feature points between the workstation image and the pre-modeled image are extracted. In some embodiments, the common feature points between the workstation image and the pre-modeled image can be understood as points with unique textures or shapes in the image. These points are relatively stable in the image and are not easily affected by geometric transformations such as rotation and scaling.

[0039] As a feasible implementation method, multi-scale feature points in the two images can be extracted respectively, and a vector representing the local area information around the feature point can be generated for each feature point. This vector can be used to compare the feature points at the same position in the two images. The next step is to match the feature points based on the feature points and corresponding vectors of the workstation image and the pre-modeled image. A pair of feature points that are successfully matched are common feature points, which are usually in the same pattern or the same primitive in the two images.

[0040] Next, the process proceeds to step S302. At this step, the workstation image and the pre-modeled image are calibrated and aligned based on the common feature points and the conversion relationship between the workstation image and the pre-modeled image. In some embodiments, after extracting the common feature points in the two images, it is necessary to calculate the geometric transformation parameters between the images according to the conversion relationship between the workstation image and the pre-modeled image, and calibrate and align based on the parameters. Here, it can be understood that the perspective of the workstation image is converted into the perspective of the pre-modeled image, so as to achieve the purpose of aligning and calibrating the two images.

[0041] Furthermore, in a feasible implementation, the transformation relationship between the workstation image and the pre-modeled image can be represented by a homography matrix. The homography matrix can be calculated from the coordinate data of at least 4 pairs of common feature points in the workstation image and the pre-modeled image, respectively, which describes the projection transformation from one plane to another. Subsequently, by obtaining the initial coordinate data of the common feature points in the workstation image, the homography matrix is ​​multiplied by the initial coordinate data to calculate the target coordinate data of the common feature points in the coordinate system of the pre-modeled image. Finally, based on the initial coordinate data and the target coordinate data obtained after mapping, the geometric transformation from the workstation image to the pre-modeled image is realized, and the workstation image is aligned with the pre-modeled image. Based on the aligned workstation image and pre-modeled image, it is more conducive to identifying the difference area between the two in the subsequent process, so as to repair the difference area.

[0042] Reference Figure 4 , to explain Figure 4 The generation method 40 of the comparison template shown. It can be understood that Figure 4 The generation method of the comparison template shown in the above text can be combined with Figure 1 A specific implementation of step S104 in the method 10 described above, therefore, the above Figure 1 The description also applies to the following Figure 4 Description of each step. In the method, at step S401, the corresponding primitive contour information is determined according to the primitive repair category divided by the difference area. In some embodiments, the corresponding primitive repair category can be divided by extracting the contour information of the difference area. For example, the same graphic that appears repeatedly in the workstation image can be classified into the same primitive repair category, and this category corresponds to the same repair project. In the subsequent process, the primitive contour information corresponding to the area to be repaired will be determined according to the same primitive repair category. In the subsequent process, the average contour of the primitive repair category will be determined based on the previously recorded attribute information, and then based on the average contour, the corresponding primitive contour in the pre-modeled image will be adjusted to make it more in line with the actual state of the product. Among them, the adjustment of the primitive contour is based on the corresponding primitive contour information. As a feasible implementation method, a corresponding model can be constructed by machine learning based on the evaluation indicators mentioned above, and the corresponding primitive repair category can be divided by the extracted contour information through the model.

[0043] Subsequently, at step S402, the corresponding primitive contour is adjusted in the pre-modeled image according to the attribute information and the primitive contour information to repair the difference area. In some embodiments, the attribute information includes information such as the length, width, and angle gradient of the primitive in the difference area. According to the attribute information, the position where the difference feature of the current primitive repair category is more obvious can be determined. Then, in the process of adjusting the corresponding primitive contour in the pre-modeled image according to the attribute information and the primitive contour information, the corresponding repair method can also be performed according to the difference category previously divided into the difference area. For the first type of difference, the green contour of the PCB board design file can be directly retained, the contour junction is smoothed, and this is used as the repaired comparison template; for the second type of difference, it can be first determined whether it is a position offset or a shape deformation, and the green contour of the PCB board design file is adjusted accordingly according to the judgment result, and this is used as the repaired comparison template; for the third type, the difference distribution area can be divided, the actual red contour of the product is retained, the contour junction is smoothed, and this is used as the repaired comparison template. Here, the average contour can be determined by sampling multiple groups of the same type of graphics in the product station image and taking the attribute information of the difference area of ​​these graphics as the benchmark. In a feasible embodiment, the connected areas of the contours in the difference area corresponding to different samples of such graphics can be superimposed to determine an average contour. It can be understood that the average contour is more in line with the actual status of most products. Then, based on the average contour and the primitive contour information, the contour points in the difference area are adjusted in the pre-modeled image. For the fourth category, it can be first determined whether the difference area is continuous. If it is a continuous area, the green contour of the PCB design file is adjusted as needed. The green contour of the PCB design file is directly retained in the burr area, and it is used as a comparison template after repair.

[0044] Furthermore, when adjusting the contour, as a feasible implementation method, the contour points of the difference area in the pre-modeled image can be adjusted based on the attribute information and the primitive contour information by image reconstruction, shape optimization or geometric transformation, so that it is basically consistent with the primitive target contour. Thereafter, the contour points can also be accurately adjusted by minimizing the contour error or other similarity metrics. Optionally, in another feasible implementation method, the contour can also be deepened by texture synthesis, interpolation methods, etc., so that the primitive contour repaired in the pre-modeled image is clearer. The result obtained after repair is a comparison template file containing a comparison template image and JSON data. The JSON data contains the record content of the entire repair process, including the difference contour, attribute information, etc.

[0045] To facilitate understanding of the technical solution of the current embodiment, the following Figure 7 The red lines represent the contours in the workstation image, and the green lines represent the standard contours in the pre-modeling image. Figure 7 As shown in the figure, it can be seen from the local part of the aligned workstation image and the pre-modeled image that there are several repeated patterns. For such repeated patterns, similar samples from multiple groups of workstation images can be collected, and the connected areas of the contours of the corresponding positions of these different samples can be superimposed to determine an average contour. Then, the pre-modeled image is repaired based on the average contour through the primitive contour information.

[0046] In the above implementation, by adjusting the contour points of the difference area in the pre-modeled image, the purpose of repairing the pre-modeled image can be achieved, thereby obtaining a comparison template that is more in line with the actual state of the product. The template construction process and construction results can meet the needs of high-efficiency and high-precision appearance inspection.

[0047] Reference Fig. 9 The embodiment of the present application also provides a device 90 for generating a comparison template, including: a parsing module 910, used to parse a target file of a product to be modeled, so as to obtain a pre-modeled image and primitive contour information from the target file; an alignment module 920, used to calibrate and align a workstation image of the product to be modeled with the pre-modeled image, wherein the workstation image is an image of the product to be modeled captured by a camera at a production workstation; a labeling module 930, used to label a difference area and attribute information of the difference area based on the calibrated and aligned workstation image and the pre-modeled image; a repairing module 940, used to repair the difference area in the pre-modeled image based on the primitive contour information according to the attribute information of the difference area, so as to generate a comparison template file that fits the actual state of the product to be modeled.

[0048] Fig.10 is a schematic diagram showing a system for generating a comparison template according to an embodiment of the present application. Fig.10 As shown, the system 100 may include a device 1001 for generating a comparison template according to an embodiment of the present application, as well as its peripheral devices and an external network, to achieve the aforementioned combination Figure 1 - Figure 4 The described method for generating a comparison template according to an embodiment of the present application.

[0049] like Fig.10As shown in , the generation device 1001 of the comparison template may include a CPU 1011, which may be a general-purpose CPU, a special-purpose CPU or other information processing and program execution unit. Further, the generation device 1001 of the comparison template may also include a large-capacity memory 1012 and a read-only memory ROM 1013, wherein the large-capacity memory 1012 may be configured to store various types of data. In the embodiment of the present application, it may include target files, primitive contour information, pre-modeled images, station images, comparison templates, coordinate data, etc. In addition, ROM 1013 may be configured to store the initialization of each functional module of the generation device 1001 of the comparison template, the basic input / output driver of the system, and the data required for booting the operating system.

[0050] Furthermore, the system 100 may also include other hardware platforms or components, such as the tensor processing unit (TPU) 1014, the image processing unit (GPU) 1015, the field programmable gate array (FPGA) 1016, and the machine learning unit (MLU) 1017. It is understood that although a variety of hardware platforms or components are shown in the system 100, they are merely exemplary and not restrictive, and those skilled in the art may add or remove corresponding hardware according to actual needs.

[0051] The device 1001 for generating the comparison template also includes a communication interface 1018, so that it can be connected to a local area network / wireless local area network (LAN / WLAN) 1005 through the communication interface 1018, and then connected to a local server 1006 or to the Internet ("Internet") 1005 through the LAN / WLAN. Alternatively or additionally, the device 1001 for generating the comparison template of the embodiment of the present application can also be directly connected to the Internet or a cellular network through the communication interface 1018 based on wireless communication technology, such as wireless communication technology based on the third generation ("3G"), the fourth generation ("4G") or the fifth generation ("5G") . In some application scenarios, the device 1001 for generating the comparison template can also access a server 1008 of an external network and a possible database 1009 as needed to obtain various known data such as target files, workstation images, parsing instructions, etc.

[0052] The peripheral devices of the generation device 1001 of the comparison template may include a display device 1002, an input device 1003 and a data transmission interface 1004. In one embodiment, the display device 1002 may, for example, include one or more speakers and / or one or more visual displays, which are configured to perform voice prompts and / or image video display on the computing process or final result of the generation device for the comparison template of the present application. The input device 1003 may include, for example, a keyboard, a capture camera, or other input buttons or controls. The data transmission interface 1004 may include, for example, a serial interface, a parallel interface or a universal serial bus interface ("USB"), a small computer system interface ("SCSI"), a serial ATA, a FireWire ("FireWire"), a PCI Express and a high-definition multimedia interface ("HDMI"), etc., which are configured for data transmission and interaction with other devices or systems. According to the solution of the present application, the data transmission interface 1004 can transmit coordinate data, etc. The CPU 1011, the mass storage 1012, the read-only memory ROM 1013, the TPU 1014, the GPU 1015, the FPGA 1016, the MLU 1017 and the communication interface 1018 of the device 1001 for generating the comparison template of the present application can be connected to each other through the bus 1019, and data interaction with peripheral devices can be achieved through the bus. In one embodiment, through the bus 1019, the CPU 1011 can control other hardware components and peripheral devices in the device 1001 for generating the comparison template.

[0053] It should also be understood that any module, unit, component, server, computer, terminal or device of the example execution instructions of the present application may include or otherwise access computer-readable media, such as storage media, computer storage media or data storage devices (removable and / or non-removable) such as disks, optical disks or tapes. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules or other data.

[0054] The present application also discloses a computer-readable storage medium, in which program instructions are stored, and the program instructions are suitable for being loaded and executed by a processor: parsing a target file of a product to be modeled to obtain a pre-modeled image and primitive contour information from the target file; calibrating and aligning the workstation image of the product to be modeled with the pre-modeled image, wherein the workstation image is an image of the product to be modeled captured by a camera at a production station; based on the calibrated and aligned workstation image and the pre-modeled image, marking a difference area and attribute information of the difference area; according to the attribute information of the difference area, repairing the difference area in the pre-modeled image based on the primitive contour information to generate a comparison template file that fits the actual state of the product to be modeled. Based on this, when the scheme of the present application is embodied in the form of a software product (computer-readable storage medium), the software product can be stored in a memory, which can include several instructions to enable a computer device (such as a personal computer, a server, etc.) to perform some or all steps of the method described in the embodiment of the present application. The aforementioned memory may include, but is not limited to, a USB flash drive, a flash drive, a read-only memory ROM, a random access memory ("Random Access Memory", abbreviated as RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0055] Although multiple embodiments of the present application have been shown and described herein, it is obvious to those skilled in the art that such embodiments are provided only by way of example. Those skilled in the art can think of many changes, modifications and alternatives without departing from the thought and spirit of the present application. It should be understood that in the process of practicing the present application, various alternatives to the embodiments of the present application described herein can be adopted. The attached claims are intended to limit the scope of protection of the present application, and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A method for generating a comparison template, characterized in that: include: Parsing a target file of a product to be modeled to obtain pre-modeled images and primitive contour information from the target file; Calibrate and align the workstation image of the product to be modeled with the pre-modeling image, wherein the workstation image is an image of the product to be modeled captured by a camera at a production station; Based on the calibrated and aligned workstation image and the pre-modeling image, marking the difference area and the attribute information of the difference area; According to the attribute information of the difference area, the difference area is repaired in the pre-modeling image based on the primitive contour information to generate a comparison template file that fits the actual state of the product to be modeled.

2. The method according to claim 1, characterized in that The step of parsing the target file of the product to be modeled to obtain the pre-modeled image and primitive contour information from the target file includes: In response to the file parsing instruction, split the target file to obtain the pre-modeled image and the primitives contained therein; Extracting coordinate data of the graphic element; The primitive contour information is determined according to the coordinate data.

3. The method according to claim 1, characterized in that: The calibrating and aligning the workstation image of the product to be modeled with the pre-modeled image comprises: Extracting common feature points between the workstation image and the pre-modeling image; Based on the common feature points, the conversion relationship between the workstation image and the pre-modeling image, the workstation image and the pre-modeling image are calibrated and aligned.

4. The method according to claim 3, characterized in that The calibrating and aligning the workstation image with the pre-modeled image based on the common feature point, the conversion relationship between the workstation image and the pre-modeled image comprises: Acquire initial coordinate data of the common feature point in the workstation image; Based on the conversion relationship, converting the initial coordinate data into target coordinate data in the coordinate system of the pre-modeled image; Based on the target coordinate data and the initial coordinate data, the common feature points of the workstation image and the pre-modeling image are aligned.

5. The method according to claim 1, characterized in that The marking of the difference area and the attribute information of the difference area based on the workstation image and the pre-modeling image after calibration and alignment includes: comparing the distance between the workstation image and the pre-modeled image graphic contour; Recording the difference contours whose spacing exceeds a preset threshold to mark the difference area; The difference area is divided into corresponding repair categories, and corresponding attribute information is recorded.

6. The method according to claim 5, characterized in that The dividing the difference area into corresponding repair categories and recording corresponding attribute information includes: Determining the primitive repair category contained in the difference area; According to the primitive repair category, the corresponding attribute information is recorded.

7. The method according to claim 1, characterized in that The repairing the difference area in the pre-modeled image based on the primitive contour information according to the attribute information of the difference area comprises: Determine the corresponding primitive contour information according to the primitive restoration category divided by the difference area; According to the attribute information and the primitive contour information, the corresponding primitive contour is adjusted in the pre-modeled image to repair the difference area.

8. A device for generating a comparison template, characterized in that: include: A parsing module, used for parsing a target file of a product to be modeled, so as to obtain a pre-modeling image and primitive contour information from the target file; An alignment module, used for calibrating and aligning the workstation image of the product to be modeled with the pre-modeling image, wherein the workstation image is an image of the product to be modeled captured by a camera at a production station; A marking module, used for marking the difference area and the attribute information of the difference area based on the workstation image and the pre-modeling image after calibration and alignment; A repair module is used to repair the difference area in the pre-modeled image according to the attribute information of the difference area and based on the primitive contour information, so as to generate a comparison template file that fits the actual state of the product to be modeled.

9. A device for generating a comparison template, characterized in that: include: processor; A memory storing program instructions for generating a comparison template, wherein when the program instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions for generating a comparison template. When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images