Product positioning method and device, equipment and storage medium

Abstract

The invention relates to a product positioning method and device, equipment and a storage medium, and the method comprises the steps: obtaining a positioning image of a product, and extracting an image contour of the product from the positioning image; converting image coordinates of the image contour into CAD coordinates to obtain a CAD contour of the product; generating a plurality of degree-of-freedom parameter groups, and transforming the position of the CAD contour through each degree-of-freedom parameter group to obtain a transformed contour corresponding to each degree-of-freedom parameter group; based on the transformation contour corresponding to each degree-of-freedom parameter group and a CAD (Computer Aided Design) gridding model pre-constructed for the product, calculating a coverage score corresponding to each degree-of-freedom parameter group; based on the coverage score corresponding to each degree-of-freedom parameter group, screening out a target degree-of-freedom parameter group; determining the position of the product based on the target degree-of-freedom parameter group; through the product positioning method provided by the invention, accurate positioning of the product without obvious characteristics can be realized, so that the product positioning precision is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a product positioning method, device, equipment and storage medium. Background Art

[0002] In the industrial application of machine vision, visual guidance is widely used in the process of product assembly, bonding, feeding, etc., and the core algorithm of visual guidance is image search and positioning. In the traditional search and positioning method, it is generally to obtain the training image of the product, and then select the obvious feature points on the training image to train the positioning model for search and positioning, and then use the trained positioning model to search and locate these obvious feature points on the real-time image of the product. In this search and positioning method, the product to be photographed needs to have obvious feature points, otherwise the positioning accuracy will not be high. In actual application, there are more and more product positioning scenarios without obvious features, such as car display screens and VR lenses. Therefore, how to achieve accurate positioning of products without obvious features is a technical problem that technical personnel need to solve urgently. Summary of the invention

[0003] In view of this, the present application proposes a product positioning method, device, equipment and storage medium, which can achieve accurate positioning of products without obvious features.

[0004] According to one aspect of the present application, a product positioning method is provided, comprising:

[0005] Acquire a positioning image of a product, and extract an image contour of the product from the positioning image;

[0006] Converting the image coordinates of the image contour into CAD coordinates to obtain the CAD contour of the product;

[0007] Generate a plurality of degree of freedom parameter groups, and transform the position of the CAD contour respectively by using each of the degree of freedom parameter groups to obtain a transformed contour corresponding to each of the degree of freedom parameter groups;

[0008] Calculating the coverage score corresponding to each of the groups of degree of freedom parameters based on the transformation profile corresponding to each of the groups of degree of freedom parameters and the CAD mesh model pre-built for the product;

[0009] Based on the coverage scores corresponding to the degree of freedom parameter groups, a target degree of freedom parameter group is screened out;

[0010] Determining the position of the product based on the target degree of freedom parameter group;

[0011] The degree of freedom parameter group includes scaling parameters, rotation parameters and translation parameters.

[0012] In one possible implementation, when generating multiple sets of degree of freedom parameters, it includes:

[0013] Acquire a zoom range and a zoom step of the zoom parameter, and calculate a traversal space of the zoom parameter based on the zoom range and the zoom step;

[0014] Acquire a rotation range and a rotation step of the rotation parameter, and calculate a traversal space of the rotation parameter based on the rotation range and the rotation step of the rotation parameter;

[0015] Determining a plurality of combinations of scaling parameters and rotation parameters based on the traversal space of the scaling parameters and the traversal space of the rotation parameters;

[0016] For each combination of scaling parameters and rotation parameters, determine the traversal space of the corresponding translation parameters;

[0017] Based on the traversal space of each combination of scaling parameter and rotation parameter and corresponding translation parameter, a plurality of degree-of-freedom parameter groups are generated.

[0018] In a possible implementation, when constructing the CAD mesh model of the product, the process includes:

[0019] Obtaining a CAD drawing of the product, deleting other information on the CAD drawing except the product outline, and obtaining an offline model of the product;

[0020] Calculating the bounding rectangle of the offline model, and meshing the inner area of ​​the bounding rectangle to obtain a CAD mesh model composed of a plurality of meshes;

[0021] Grids associated with the product contour are selected from the CAD grid model as valid grids, and matching scores of the valid grids are calculated.

[0022] In a possible implementation, calculating the coverage score corresponding to the current degree of freedom parameter group based on the grid position of each contour point of the current transformed contour in the CAD grid model includes:

[0023] Traversing the transformation profiles corresponding to the degree of freedom parameter groups;

[0024] For the current transformation contour corresponding to the traversed current degree of freedom parameter group, the grid position of each contour point of the current transformation contour in the CAD grid model is calculated, and based on the grid position of each contour point of the current transformation contour in the CAD grid model, the coverage score corresponding to the current degree of freedom parameter group is calculated;

[0025] After the traversal is completed, the coverage score corresponding to the group of freedom parameter groups is obtained.

[0026] In a possible implementation, calculating the coverage score corresponding to the current degree of freedom parameter group based on the grid position of each contour point of the current transformed contour in the CAD grid model includes:

[0027] Based on the grid position of each contour point of the current transformed contour in the CAD grid model, obtaining a grid matching score corresponding to each contour point of the current transformed contour;

[0028] Based on the grid matching scores corresponding to the contour points of the current transformed contour, the coverage score corresponding to the current degree of freedom parameter group is calculated. In a possible implementation, after the target degree of freedom parameter group is screened out, the method further includes:

[0029] Obtaining a target transformation profile corresponding to a target degree of freedom parameter group;

[0030] Calculate the closest point from each contour point in the target transformed contour to the CAD grid model;

[0031] Calculating adjustment values ​​of the target degree of freedom parameter group based on the calculated closest points;

[0032] The target degree of freedom parameter group is optimized using the adjustment value to obtain a final target degree of freedom parameter group, and the position of the product is determined based on the final target degree of freedom parameter group.

[0033] In a possible manner, the positioning image includes a plurality of local positioning images of the product;

[0034] When generating the CAD outline of the product based on each of the local positioning images, it includes

[0035] Extracting a local image contour of the product from each of the local positioning images;

[0036] Convert the image coordinates of each of the local image contours into CAD coordinates to obtain a local CAD contour of the product;

[0037] The local CAD contours are combined to obtain the CAD contour of the product.

[0038] According to a second aspect of the present application, a product positioning device is provided, comprising:

[0039] An image acquisition module, used to acquire a positioning image of a product and extract an image contour of the product from the positioning image;

[0040] An image processing module, used for converting the image coordinates of the image contour into CAD coordinates to obtain the CAD contour of the product;

[0041] A position transformation module, used to generate a plurality of degree of freedom parameter groups, and transform the position of the CAD contour respectively through each of the degree of freedom parameter groups to obtain a transformed contour corresponding to each of the degree of freedom parameter groups;

[0042] A coverage score calculation module, used for calculating the coverage score corresponding to each of the groups of degree of freedom parameters based on the transformation profile corresponding to each of the groups of degree of freedom parameters and the CAD mesh model pre-built for the product;

[0043] A degree of freedom parameter screening module, used for screening out a target degree of freedom parameter group based on the coverage scores corresponding to each of the degree of freedom parameter groups;

[0044] A positioning module, used for determining the position of the product based on the target degree of freedom parameter group;

[0045] The degree of freedom parameter group includes scaling parameters, rotation parameters and translation parameters.

[0046] According to a third aspect of the present application, a product positioning device is provided, comprising:

[0047] processor;

[0048] a memory for storing processor-executable instructions;

[0049] Wherein, the processor is configured to implement any one of the methods described in the first aspect of the present application when executing the executable instructions.

[0050] According to a fourth aspect of the present application, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, characterized in that when the computer program instructions are executed by a processor, the method described in any one of the first aspects of the present application is implemented.

[0051] The present application relates to a product positioning method, device, equipment and storage medium, the method comprising: obtaining a positioning image of a product, and extracting an image contour of the product from the positioning image; converting the image coordinates of the image contour into CAD coordinates to obtain the CAD contour of the product; generating multiple degrees of freedom parameter groups, and transforming the position of the CAD contour respectively through each degree of freedom parameter group to obtain the transformed contour corresponding to each degree of freedom parameter group; based on the transformed contour corresponding to each degree of freedom parameter group and the CAD grid model pre-built for the product, calculating the coverage score corresponding to each degree of freedom parameter group; based on the coverage score corresponding to each degree of freedom parameter group, screening out the target degree of freedom parameter group; based on the target degree of freedom parameter group, determining the position of the product; wherein the degree of freedom parameter group includes scaling parameters, rotation parameters and translation parameters. Through the method of the present application, accurate positioning of products without obvious features can be achieved, thereby improving the accuracy of product positioning.

[0052] Other features and aspects of the present application will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the present application.

[0054] Figure 1 A schematic diagram showing calculation of an off-line model circumscribed rectangle according to an embodiment of the present application is shown;

[0055] Figure 2 A schematic diagram of a grid model according to an embodiment of the present application is shown;

[0056] Figure 3 A schematic diagram of an effective grid according to an embodiment of the present application is shown;

[0057] Figure 4 A schematic diagram showing effective grid diffusion according to an embodiment of the present application is shown;

[0058] Figure 5 A schematic diagram showing storage of effective grid matching scores according to an embodiment of the present application is shown;

[0059] Figure 6 An algorithm flow chart showing a product positioning method according to an embodiment of the present application;

[0060] Figure 7 A schematic diagram showing calculation of the effective area in the X direction according to an embodiment of the present application is shown;

[0061] Figure 8 A schematic diagram showing the alignment between the converted contour of a product and the CAD meshed model according to an embodiment of the present application is shown;

[0062] Fig. 9 A peak detection flow chart of an embodiment of the present application is shown;

[0063] Fig.10 A schematic diagram showing the calculation of overlap ratio according to an embodiment of the present application is shown;

[0064] Fig.11 A schematic diagram of parabola interpolation calculation according to an embodiment of the present application is shown;

[0065] Fig.12 A schematic diagram showing a product positioning method according to another embodiment of the present application;

[0066] Fig.13 A schematic block diagram showing a product positioning device according to an embodiment of the present application;

[0067] Fig.14 A schematic block diagram of a product positioning device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0068] Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0069] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0070] In addition, in order to better illustrate the present application, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.

[0071] Before product positioning, it is necessary to build a CAD mesh model of the product. The constructed CAD mesh model is used for subsequent product positioning.

[0072] In one possible implementation, when constructing a CAD mesh model of a product, the following steps may be included:

[0073] First, obtain the CAD drawing of the product, delete all information except the product outline on the CAD drawing, and obtain the offline model of the product.

[0074] Normally, the CAD drawing of a product contains all the outlines of the product, product information text, and additional notes, borders and other information. However, in the product positioning process of this application, only the outline of the product is needed, so it is necessary to delete all the information except the outline on the product CAD drawing to obtain the offline model of the product, thereby avoiding the influence of other information except the outline on the product positioning. That is to say, the offline model of the product only includes the geometric elements that constitute the outline of the product, where the geometric elements can include at least one of line segments, circular arcs, elliptical arcs, polygons and sample curves.

[0075] Second, the bounding rectangle of the offline model is calculated, and the inner area of ​​the bounding rectangle is meshed to obtain a CAD meshed model composed of multiple meshes.

[0076] The bounding rectangle of the offline model is the union of the bounding rectangles of all geometric elements in the offline model. When calculating the bounding rectangle of the offline model, it can be done as follows: Figure 1 The following steps are included: the first step is to calculate the bounding rectangle of each geometric element in the offline model; the second step is to calculate the union of the bounding rectangles of all geometric elements to obtain the bounding rectangle of the offline model. On this basis, in order to prevent the mesh from being diffused outside the bounding rectangle of the offline model when the valid mesh is diffused later, after obtaining the union of the bounding rectangles of all geometric elements, the Figure 1 In the third step, the bounding rectangle after the union is expanded by a certain distance, and the rectangle obtained after the expansion is used as the bounding rectangle of the offline model.

[0077] The outward expansion distance is the capture distance multiplied by the grid size. The capture distance can be set according to the number of edge transition pixels of the product positioning image. For example, the capture distance can be set to a range of 1-16. Preferably, the capture distance can be set to 4. The grid size is the side length of each square grid obtained after the internal area of ​​the circumscribed rectangle of the offline model is gridded. The grid size can be set according to the actual physical distance of a single pixel of the product positioning image.

[0078] After obtaining the bounding rectangle of the offline model, the internal area of ​​the bounding rectangle of the offline model can be meshed according to the preset mesh size, thereby obtaining a CAD mesh model composed of multiple meshes. Figure 2 As shown in the left figure in the figure, the CAD mesh model obtained after meshing can be Figure 2 As shown in the right figure of . The number of rows and columns of the grid in the CAD grid model is determined according to the height h, width w of the circumscribed rectangle of the offline model and the preset grid size GridLength. Specifically, the calculation formula for the number of rows row of the grid in the CAD grid model is as follows:

[0079]

[0080] The calculation formula for the number of columns col in the CAD mesh model is as follows:

[0081]

[0082] Third, the meshes associated with the product contour are selected from the CAD mesh model as valid meshes, and the matching scores of each valid mesh are calculated.

[0083] In one possible implementation, Figure 3 As shown, the meshes through which geometric elements pass in the CAD meshing model can be regarded as valid meshes associated with the product contour, and the meshes through which no geometric elements pass can be regarded as invalid meshes.

[0084] In another possible implementation, in order to make the subsequent product positioning process more robust, such as Figure 4 As shown, the mesh through which geometric elements pass in the CAD mesh model can be used as the initial valid mesh (i.e. Figure 4 ); then diffuse the initial effective grid to obtain the diffused effective grid; then use the initial effective grid and the diffused effective grid as the effective grids associated with the product contour; finally, use the grids other than the effective grid as invalid grids. When diffusing the initial effective grid, an 8-field diffusion method is used, that is, the effective grids are diffused in the up, down, left, right and diagonal directions, and the diffusion size is the preset capture distance. After screening out the effective grids, the matching scores of the effective grids can be calculated. The calculation process of the matching scores of the various grids is the same. The following takes a grid as an example to illustrate the calculation process of the grid matching score.

[0085] First, the shortest distance from the grid to the offline model is calculated. Specifically, the distance from the grid to each geometric element in the offline model is calculated, and the minimum distance is selected as the shortest distance from the grid to the offline model. The calculation formula for the shortest distance can be shown as follows:

[0086]

[0087] Among them, MinDis is the shortest distance from the grid to the offline model, P is the coordinate of the grid P, and E j is the jth geometric element in the offline model, Dis(P→E j ) is the distance from the grid P to the jth geometric element in the offline model, and n is the total number of geometric elements included in the offline model.

[0088] Secondly, the matching score of the grid is calculated based on the shortest distance from the grid to the offline model and the preset grid size and capture distance. The calculation formula of the grid matching score can be as follows:

[0089]

[0090] Among them, CaptureDis is the capture distance, GridLength is the grid size, and Score is the matching score of the grid. After calculation by the above formula, the matching score calculated for a valid grid is greater than 0, and the matching score calculated for an invalid grid is equal to 0.

[0091] The matching score of each grid can be obtained by referring to the above method.

[0092] In the subsequent product positioning process, the grid matching score will be frequently queried based on the grid coordinates. Therefore, the grid matching score needs to be stored for subsequent queries. When storing, the scores of all grids can be stored in the order of grid coordinates. In this way, the grid matching score can be queried based on the grid coordinates. However, in actual applications, the number of grids will be very large, resulting in storage space far from enough to store all grid information. If only the matching scores of valid grids are stored, it is also necessary to query the storage location of the matching scores corresponding to the valid grid coordinates. The fastest way is to use binary search, but it cannot meet the efficiency requirements of queries.

[0093] In order to improve the efficiency of subsequent queries and take into account storage space, in a possible implementation, a hash-like storage + binary search method can be used to store and query the matching scores of valid grids. Specifically, when using a hash-like storage method to store the matching scores of valid grids, it includes: first creating a hash table with a size equal to the number of grid columns, each column in the hash table includes multiple storage units from top to bottom, which are used to sequentially store the matching scores of valid grids in the corresponding columns in the CAD mesh model. Specifically, for each column in the CAD mesh model, search for valid grids from top to bottom. After finding the valid grid, determine the number of rows of the valid grid in the CAD mesh model, and store the matching scores of the valid grids in the corresponding column storage units in the hash table in order from top to bottom, and use the row number of the valid grid as the index value of the storage unit. For example, for Figure 5The CAD grid model shown in FIG. 1 includes 121 grids in 11 rows and 11 columns. A hash table with 11 columns is first constructed for the CAD grid model. Then, for the first column in the CAD grid model, valid grids are searched from top to bottom (from row 0 to row 11). After searching for a valid grid located in row 3 of the first column, the matching score of the valid grid is stored in the first storage unit of the first column on the hash table, and the index of the storage unit is set to 3. Then, after searching for a valid grid located in row 4 of the first column, the matching score of the valid grid is stored in the second storage unit of the first column on the hash table, and the index of the storage unit is set to 4. Next, after searching for a valid grid located in row 5 of the first column, the matching score of the valid grid is stored in the third storage unit of the first column on the hash table, and the index of the storage unit is set to 5. This process is repeated until the matching scores of all valid grids in the first column of the CAD grid model are stored in the storage units of the first column on the hash table. The matching scores of the valid grids in other columns are stored in turn according to the storage method of the valid grid matching scores in the first column, until the storage of all valid grid matching scores in the CAD grid model is completed, and the hash table corresponding to the CAD grid model can be obtained.

[0094] After constructing the hash table corresponding to the CAD mesh model, when querying the matching score of the valid mesh in the subsequent query, the storage location of the valid mesh in the hash table can be queried by binary search, and the corresponding matching score can be read from the corresponding storage location for calculation. Specifically, after determining the grid coordinates of each contour point in the mesh model after transformation, each grid coordinate is rounded down to obtain the integer coordinates of each grid coordinate, and the corresponding column in the hash table is queried according to the column index of the integer coordinate. The row index of the valid matching grid score in the column is stored in order in the list, and then the corresponding row index is queried in the corresponding column according to the binary search. If the corresponding row index is queried, the corresponding stored matching score is extracted as the grid matching score corresponding to the transformed contour point; if the corresponding row index is not queried, the grid matching score corresponding to the transformed contour point is 0. For example, the coordinates of the transformed contour point after rounding down are [r, c]. First, find the column corresponding to the c-th column in the hash table, and then use the binary search method to find out whether the c-th column contains the index r. If not, the matching score of the contour point is 0. Otherwise, the matching score of the corresponding grid is extracted as the effective grid matching score of the contour point.

[0095] After completing the construction of the CAD grid model of the product and obtaining the matching score of each valid grid in the CAD grid model, the product positioning method in this application can be executed. Figure 6 An algorithm flow chart of a product positioning method according to an embodiment of the present application is shown as follows: Figure 6As shown, the method includes steps S1100-S1600.

[0096] S1100, obtaining a positioning image of the product, and extracting the image contour of the product from the positioning image. Specifically, when positioning the product, the product needs to be placed on a guide device (the guide device is used to guide the product to be assembled with other products on the guide device), and then an image of the product on the guide device is captured by a camera device, and the image is the positioning image of the product. After the positioning image of the product is obtained, the image contour of the product can be extracted from the positioning image.

[0097] In a possible implementation, when extracting the image contour of the product from the positioning image, the following steps may be included:

[0098] First, boundary point detection: Specifically, edge points in the positioning image are extracted using edge detection algorithms (such as Sobel, Prewitt, Laplacian, etc.).

[0099] Second, boundary point connection: Specifically, the extracted edge points are connected in sequence to obtain a boundary chain.

[0100] Third, delete weak boundary chains. Specifically, in the process of connecting edge points, some weak boundary chains may be generated. These weak boundary chains may be caused by noise or errors in the image processing process. In order to obtain more accurate contour information, these weak boundary chains need to be deleted to ensure that the extracted contour lines are clearer and more accurate.

[0101] Fourth, boundary chain sampling. Specifically, the remaining boundary chains are sampled to obtain the final image contour. By sampling, the number of contour points can be reduced while maintaining the basic shape and features of the contour, which helps to improve the speed and efficiency of subsequent processing.

[0102] After extracting the image contour of the product, step S1200 can be executed to convert the image coordinates of the image contour into CAD coordinates to obtain the CAD contour of the product. Specifically, for each contour point that constitutes the image contour, the image coordinates of each contour point are obtained, and then the image coordinates of each contour point are converted into CAD coordinates through calibration information to obtain the CAD contour of the product. Among them, the CAD coordinate system is equivalent to the physical coordinate system. The coordinate system takes the rotation center of the guide device as the origin, the left and right moving direction of the guide device as the X-axis, and the up and down moving direction as the Y-axis. The conversion between image coordinates and physical coordinates (i.e., CAD coordinates) can be realized through calibration information. How to realize the conversion between image coordinates and physical coordinates through calibration information is common knowledge in the field and will not be repeated here.

[0103] After obtaining the CAD contour of the product, step S1300 can be executed to generate multiple degrees of freedom parameter groups, and the position of the CAD contour is transformed by each degree of freedom parameter group to obtain the transformed contour corresponding to each degree of freedom parameter group. The degree of freedom parameter group includes scaling parameters, rotation parameters and translation parameters.

[0104] Before executing step S1300, multiple sets of degree of freedom parameters need to be generated. When generating multiple sets of degree of freedom parameters, steps S1310-S1350 may be included.

[0105] S1310, obtaining a scaling range and a scaling step of the scaling parameter, and calculating the traversal space of the scaling parameter based on the scaling range and the scaling step of the scaling parameter. The scaling range can be set according to a specific scenario. The scaling step can be calculated based on a preset grid size and the distance between the two farthest contour points in the CAD contour of the product. The calculation formula of the scaling step can be as follows:

[0106]

[0107] Where Δs is the scaling step, GridLength is the preset grid size, and MaxDis is the distance between the two farthest contour points in the product's CAD contour.

[0108] It should be noted that when determining the scaling step, it is required that after the product CAD contour undergoes a scaling step conversion, the distance between the new CAD contour point obtained and the corresponding original product CAD contour point cannot exceed the size of the offline model grid.

[0109] After determining the scaling range and scaling step of the scaling parameter, the traversal space of the scaling parameter can be calculated based on the scaling range and scaling step. The traversal space of the scaling parameter includes multiple scaling parameters. The calculation formula of each scaling parameter in the traversal space of the scaling parameter can be as follows:

[0110]

[0111] In the formula, s i is the i-th scaling parameter in the scaling traversal space, [s min ,s max ]s min is the lower limit of the scaling range, s max is the upper limit of the scaling range, and Δs is the scaling step size.

[0112] S1320, obtaining the rotation range and rotation step of the rotation parameter, and calculating the traversal space of the rotation parameter based on the rotation range and rotation step of the rotation parameter. The rotation range can be set according to the specific scenario. The rotation step can be calculated based on the preset grid size and the distance between the two farthest contour points in the CAD contour of the product. The calculation formula of the rotation step can be as follows:

[0113]

[0114] Where Δθ is the rotation step, GridLength is the preset grid size, and MaxDis is the distance between the two farthest contour points in the product's CAD contour.

[0115] It should be noted that when determining the rotation step, it is required that after the product CAD contour undergoes a rotation step transformation, the distance between the new CAD contour point obtained and the corresponding original product CAD contour point cannot exceed the size of the offline model grid.

[0116] After the rotation range and rotation step of the rotation parameter are determined, the traversal space of the rotation parameter can be calculated based on the rotation range and the rotation step. The traversal space of the rotation parameter includes multiple rotation parameters. The calculation formula of each rotation parameter in the traversal space of the rotation parameter can be as follows:

[0117]

[0118] In the formula, θ i is the i-th rotation parameter in the rotation traversal space, θ min is the lower limit of the scaling range, θ max is the upper limit of the scaling range, and Δθ is the rotation step size.

[0119] S1330: Determine a plurality of combinations of scaling parameters and rotation parameters based on the traversal space of scaling parameters and the traversal space of rotation parameters. For example, the traversal space of scaling parameters is (S1, S2, S3, ..., S m ), that is, the traversal space of the scaling parameters includes m scaling parameters, and the traversal space of the rotation parameters is (θ1, θ2, θ3, ..., θ n ), that is, the traversal space of the rotation parameter includes n rotation parameters. Combining all the obtained scaling parameter values ​​and all the rotation parameter values ​​respectively can obtain multiple combinations of scaling parameters and rotation parameters. For example, combining scaling parameter S1 with all the rotation parameters will obtain S1θ1, S1θ2, S1θ3..., S1θ nThese n combinations, continuing to combine the scaling parameter S2 with all the rotation parameters will result in S2θ1, S2θ2, S2θ3, ..., S2θ n These n combinations, and so on, can obtain the combination of each scaling parameter and all rotation parameters, and finally obtain m*n combinations of scaling parameters and rotation parameters.

[0120] S1340, for each combination of scaling parameters and rotation parameters, determine the traversal space of the corresponding translation parameters. The translation parameters include the translation parameters in the X direction and the translation parameters in the Y direction. The following takes a set of scaling parameters and rotation parameters as an example to illustrate the process of determining the traversal space of the corresponding translation parameters. Specifically, the following steps may be included:

[0121] First, each contour point on the product CAD contour is scaled and rotated according to the scaling parameter S and the rotation parameter θ, thereby obtaining a plurality of new contour points, and the plurality of new contour point sets constitute a new CAD contour.

[0122] Second, the circumscribed rectangle of the new CAD contour is calculated. For the specific calculation process of the circumscribed rectangle, refer to the calculation process of the circumscribed rectangle of the offline model, which will not be repeated here.

[0123] Third, based on the circumscribed moment of the new CAD contour and the CAD mesh model, the translation range in the X direction and the translation range in the Y direction corresponding to the combination of the scaling parameter and the rotation parameter are calculated.

[0124] Specifically, the bounding rectangle of the new CAD outline is translated in the X direction, and the minimum X-direction translation value and the maximum X-direction translation value that make the bounding rectangle of the new CAD outline overlap with the CAD grid model during the translation process are recorded. Based on the minimum X-direction translation value and the maximum X-direction translation value, the translation range in the X direction corresponding to the combination of the scaling parameter and the rotation parameter is determined.

[0125] In a possible implementation, when determining the translation range in the X direction corresponding to the combination of the scaling parameter and the rotation parameter based on the minimum X direction translation value and the maximum X direction translation value, the method includes:

[0126] First, the difference between the minimum X-direction translation value and the maximum X-direction translation value is calculated as the effective area in the X direction.

[0127] Secondly, based on the effective area in the X direction, determine the effective area in the X direction of each new contour point. Specifically, for each new contour point on the new CAD contour, the current X direction coordinate is used as the lower limit of the effective area in the X direction, and the sum of the current X direction coordinate and the effective area is used as the upper limit of the effective area in the X direction. The lower limit and upper limit of the effective area in the X direction constitute the effective area in the X direction of each new contour point.

[0128] In such Figure 7 In the example shown, the effective area in the X direction is w, then the lower limit value of the effective area in the X direction determined for the new contour point 1 is x1, the upper limit value is x1+w, and the area from x1 to x1+w is the effective area in the X direction corresponding to the new contour point 1. Similarly, the lower limit value of the effective area in the X direction determined for the contour point 2 is x2, the upper limit value is x2+w, and the area from x2 to x2+w is the effective area in the X direction corresponding to the new contour point 2. The lower limit value of the effective area in the X direction determined for the contour point 3 is x3, the upper limit value is x3+w, and the area from x3 to x3+w is the effective area in the X direction corresponding to the new contour point 3.

[0129] Next, the X-direction effective areas of each new contour point are merged to obtain multiple sub-effective areas, and the area score of each sub-effective area is calculated. The area score of each sub-effective area is equal to p / n, where p represents the effective area of ​​the sub-effective area including p new contour points, and n is the number of contour points that make up the product CAD contour.

[0130] like Figure 7 As shown in the figure, after merging the effective areas in the X direction corresponding to the new contour points 1 to 3, the following is obtained: Figure 7 There are five sub-valid areas shown, which are marked as the first sub-valid area, the second sub-valid area, the third sub-valid area, the fourth sub-valid area and the fifth sub-valid area from left to right, wherein the first sub-valid area includes the X-direction valid area of ​​the new contour point 1, so its corresponding area fraction is 1 / n; the second sub-valid area includes the X-direction valid area of ​​the new contour point 1 and the new contour point 2, so its corresponding area fraction is 2 / n; the third sub-valid area includes the X-direction valid area of ​​the new contour point 1, the new contour point 2 and the new contour point 3, so its corresponding area fraction is 3 / n; the fourth sub-valid area includes the X-direction valid area of ​​the new contour point 2 and the new contour point 3, so its corresponding area fraction is 2 / n; the fifth sub-valid area includes the X-direction valid area of ​​the new contour point 3, so its corresponding area fraction is 1 / n.

[0131] Finally, the sub-valid areas whose area scores are lower than the coverage score are deleted from the sub-valid areas, and the translation range in the X direction is determined based on the remaining sub-valid areas. Specifically, the minimum X coordinate in the remaining sub-valid areas is used as the lower limit of the translation range in the X direction, and the maximum X coordinate in the remaining sub-valid areas is used as the upper limit of the translation range in the X direction. At this point, the translation range in the X direction corresponding to the combination of the scaling parameter and the rotation parameter can be obtained.

[0132] Similarly, the translation range in the Y direction can be determined by referring to the method for determining the translation range in the X direction. The specific steps will not be repeated.

[0133] Fourth, based on the translation range in the X direction and the translation range in the Y direction, the traversal space in the X direction and the traversal space in the Y direction are determined.

[0134] Specifically, after determining the translation range in the X direction, the traversal space in the X direction can be determined based on the lower limit value and the upper limit value of the translation range in the X direction, and the X direction translation step length. Among them, the traversal space in the X direction includes multiple translation values ​​Txi in the X direction. The calculation process of the translation value Txi refers to the scaling parameter, which will not be repeated here. The X direction translation step length is equal to the preset grid size GridLength.

[0135] Similarly, after determining the translation range in the Y direction, the traversal space in the Y direction can be determined based on the lower limit and upper limit of the translation range in the Y direction, as well as the Y direction translation step length. Among them, the traversal space in the Y direction includes multiple Y direction translation values ​​Tyi. The calculation process of the translation value Tyi refers to the scaling parameter, which will not be repeated here. The Y direction translation step length is equal to the preset grid size GridLength.

[0136] By referring to the above steps, each combination of scaling parameters and rotation parameters can be obtained, and the corresponding traversal space in the X direction and the traversal space in the Y direction.

[0137] S1350: Generate multiple degree of freedom parameter groups based on the traversal space of each combination of scaling parameters and rotation parameters and the corresponding translation parameters. Specifically, for each combination of scaling parameters and rotation parameters, traverse and combine it with multiple Tx values ​​in the traversal space in the X direction and multiple Ty values ​​in the traversal space in the Y direction, and obtain the scaling parameter S, the rotation parameter θ, the translation value T in the X direction corresponding to each combination of scaling parameters and rotation parameters. x And the translation value T in the Y direction y Multiple parameter groups composed of

[0138] After obtaining multiple free parameter combinations, each degree of freedom parameter group can be used to transform the position of the CAD contour respectively to obtain the transformed contour corresponding to each degree of freedom parameter group. Specifically, for each group of degree of freedom parameter groups, the positions of the contour points constituting the CAD contour are transformed to obtain the contour points after position transformation. The contour points after position transformation constitute the transformed contour corresponding to each degree of freedom parameter group. Among them, the calculation formula for position transformation of contour points using degree of freedom parameter groups is as follows:

[0139]

[0140] In the formula, (P′ x ,P′ y) is the coordinate of the transformed CAD contour point obtained after position transformation by the freedom parameter group, (P x ,P y ) is the coordinate of the product CAD contour point, (s,θ,T x ,T y ) is the current degree of freedom parameter group.

[0141] After obtaining the transformation profile corresponding to each DOF parameter group, step S1400 may be executed to calculate the coverage score corresponding to each DOF parameter group based on the transformation profile corresponding to each DOF parameter group and the CAD mesh model pre-built for the product. Specifically, steps S1410-S1430 may be included.

[0142] S1410, traverse the transformation profile corresponding to each degree of freedom parameter group.

[0143] S1420, for the current transformation contour corresponding to the current degree of freedom parameter group traversed, calculate the grid position of each contour point of the current transformation contour in the CAD grid model, and calculate the coverage score corresponding to the current degree of freedom parameter group based on the grid position of each contour point of the current transformation contour in the CAD grid model.

[0144] Specifically, when calculating the grid position of each contour point of the current transformed contour in the CAD grid model, it can be implemented based on the following calculation formula:

[0145]

[0146] In the formula, (Q x ,Q y ) is the origin coordinate of the coordinate system where the CAD grid model is located, GridLength is the preset grid size, (P′ x ,P′ y ) is the coordinate of the contour point of the current transformed contour, and (u,v) is the grid position of the contour point of the current transformed contour in the CAD mesh model.

[0147] For example, for Figure 8 The CAD outline of the product shown in the middle left figure is obtained through the degree of freedom parameter group (s, θ, T x , T y ) will be transformed to obtain the corresponding transformed contour, calculate the grid position of each contour point in the transformed contour in the CAD grid model, and mark it to the corresponding CAD grid model, and you can get the following: Figure 8 The matching diagram is shown on the right.

[0148] After calculating the grid position of each contour point in the transformed contour in the CAD grid model, the grid matching score can be correspondingly calculated based on the grid position.

[0149] In a possible implementation, when calculating the coverage score corresponding to the current degree of freedom parameter group based on the grid positions of each contour point of the current transformed contour in the CAD grid model, the following steps may be included:

[0150] First, based on the grid positions of each contour point of the current transformed contour in the CAD grid model, the grid matching scores corresponding to each contour point of the current transformed contour are obtained. Specifically, for each contour point, the corresponding grid position coordinates are rounded down to obtain a grid index, and then the corresponding grid matching score is queried based on the grid index. For example, its grid position coordinates are (3.2, 2.2), and the grid index obtained after rounding down is (3, 2). Based on the grid index, the second column of the hash table is located, and a binary search is used in the second column of the hash table to find whether there is an element with a row index of 3. The row index with an element of 3 is found, and the corresponding stored matching score is extracted, which is the grid matching score corresponding to each contour point. If the row index with an element of 3 is not found, the grid matching score corresponding to the contour point is 0.

[0151] Second, based on the grid matching scores corresponding to each contour point of the current transformed contour, the coverage score corresponding to the current degree of freedom parameter group is calculated. Specifically, the calculation formula of the coverage score is as follows:

[0152]

[0153] Where Score(i) is the grid matching score corresponding to the i-th contour point, n is the number of contour points, and CoverageScore is the coverage score corresponding to the free parameter group. It should be noted that if the grid is not a valid grid, the matching score is 0.

[0154] S1430, after the traversal is completed, the coverage score corresponding to each degree of freedom parameter group can be obtained.

[0155] After calculating the coverage scores of each degree of freedom parameter group, in order to improve the search accuracy, it is necessary to perform peak detection on the coverage scores corresponding to each degree of freedom parameter group. Among them, the degree of freedom parameters for peak detection are only the scaling parameter and the rotation parameter. Fig. 9 As shown in the peak detection flow chart, the steps of peak detection include:

[0156] First, traverse each degree of freedom parameter group, and search for the adjacent degree of freedom parameter groups for the current degree of freedom parameter group. If the adjacent degree of freedom parameter groups are found, execute the following second step to calculate the overlap rate of the bounding rectangle of the transformed contour after the transformation of the adjacent degree of freedom parameter groups; if the adjacent degree of freedom parameter groups are not found, retain the current degree of freedom parameter group and retain the coverage score corresponding to the current degree of freedom parameter group.

[0157] The search method for finding the front and back degrees of freedom of scaling is a scaling step, which can be expressed as:

[0158] (s-Δs,θ,Tx1,Ty1)←(s,θ,Tx,Ty)→(s+Δs,θ,Tx2,Ty2) (12)

[0159] Similarly, the search method for finding the front and rear degrees of freedom of rotation is a rotation step, which can be expressed as:

[0160] (s,θ-Δθ,Tx1,Ty1)←(s,θ,Tx,Ty)→(s,θ+Δθ,Tx2,Ty2) (13)

[0161] Second, the CAD contour of the product is transformed using the adjacent DOF parameter groups to obtain the transformed contour (recorded as the adjacent transformed contour), and the bounding rectangle of the adjacent transformed contour is calculated; the CAD contour of the product is transformed using the current DOF parameter to obtain the transformed contour (recorded as the current transformed contour), and the bounding rectangle of the current transformed contour is calculated; finally, the overlap ratio of the bounding rectangle of the adjacent transformed contour and the bounding rectangle of the current transformed contour is calculated. Specifically, refer to Fig.10 Schematic diagram of the overlap rate, schematic diagram a is the circumscribed rectangle of the current transformed contour after the current degree of freedom parameter group conversion, schematic diagram b is the circumscribed rectangle of the adjacent transformed contour points after the adjacent degree of freedom parameter group conversion, and schematic diagram c is the calculation formula of the overlap rate between the circumscribed rectangle of the adjacent transformed contour and the circumscribed rectangle of the current transformed contour as shown below:

[0162]

[0163] Where Overlap is the overlap ratio between the bounding rectangle of the adjacent transformed contour and the bounding rectangle of the current transformed contour, d x is the distance between the center points of the two circumscribed rectangles in the X direction, d yis the distance between the center points of the two circumscribed rectangles in the Y direction; w1 is the distance from the center point of the circumscribed rectangle of the current transformed contour to the boundary in the X direction, and h1 is the distance from the center point of the circumscribed rectangle of the current transformed contour to the boundary in the Y direction; w2 is the distance from the center point of the circumscribed rectangle of the adjacent transformed contour to the boundary in the X direction, and h2 is the distance from the center point of the circumscribed rectangle of the adjacent transformed contour to the boundary in the Y direction.

[0164] Third, determine whether the calculated overlap rate is less than the preset overlap rate threshold. If it is less than the overlap rate threshold, delete the adjacent degree of freedom parameter group; if it is greater than the overlap rate threshold, perform parabolic interpolation calculation on the current degree of freedom parameter to obtain a new degree of freedom parameter group, and calculate the coverage score corresponding to the new degree of freedom parameter group. The overlap rate threshold can be set according to the specific scenario. For example, the overlap rate threshold can be set to 0.8.

[0165] Specifically, when performing parabolic interpolation calculation on the current degree of freedom parameter to obtain a new degree of freedom parameter group, and calculating the coverage score corresponding to the new degree of freedom parameter group, the calculation formula is as follows:

[0166] refer to Fig.11 , the new degree of freedom parameter MaxX after interpolation can be expressed as:

[0167]

[0168] The coverage score MaxY corresponding to the new degree of freedom parameter MaxX can be expressed as:

[0169]

[0170] Among them, x1 is the current degree of freedom parameter, y1 is the coverage score corresponding to the degree of freedom parameter group where the current degree of freedom parameter is located, x0 is the previous degree of freedom parameter adjacent to the current degree of freedom parameter, y0 is the coverage score corresponding to the degree of freedom parameter group where the previous degree of freedom parameter adjacent to the current degree of freedom parameter is located, x2 is the next degree of freedom parameter adjacent to the current degree of freedom parameter, and t2 is the coverage score corresponding to the degree of freedom parameter group where the next degree of freedom parameter adjacent to the current degree of freedom parameter is located.

[0171] For example, when performing peak detection of scaling parameters on each degree of freedom parameter group, the current scaling parameter in the traversed current degree of freedom parameter group (s, θ, Tx, Ty) is s, the scaling parameter in the previous degree of freedom parameter group (s-Δs, θ, Tx1, Ty1) adjacent to the current degree of freedom parameter group is s-Δs, and the scaling parameter in the next degree of freedom parameter group (s+Δs, θ, Tx2, Ty2) adjacent to the current degree of freedom parameter group is (s+Δs, θ, Tx2, Ty2). If it is calculated that the overlap rate of the bounding rectangle of any adjacent transformed contour after the previous degree of freedom parameter group (s-Δs, θ, Tx1, Ty1) or the next degree of freedom parameter group (s+Δs, θ, Tx2, Ty2) is greater than 0.8, then based on the current scaling parameter s( That is x1 in formula 15-16), the coverage score corresponding to the degree of freedom parameter group (s, θ, Tx, Ty) where the current scaling parameter s is located (that is y1 in formula 15-16), the previous scaling parameter s-Δs of the current scaling parameter s (that is x0 in formula 15-16), the coverage score corresponding to the degree of freedom parameter group (s-Δs, θ, Tx1, Ty1) where the previous scaling parameter s-Δs is located (that is y0 in formula 15-16), the next scaling parameter s+Δs of the current scaling parameter s (that is x2 in formula 15-16), the coverage score corresponding to the degree of freedom parameter group (s+Δs, θ, Tx2, Ty2) where the next scaling parameter s+Δs is located (that is y2 in formula 15-16), and the new scaling parameter interpolation value corresponding to the current scaling parameter s is calculated to be S′, which is MaxX in formula 15.

[0172] Fourth, for the calculated new degree of freedom parameters, their translation parameters also need to be updated.

[0173] It should be noted here that the rotation parameter corresponding to the new scale parameter interpolation MaxX is equal to the rotation parameter in the current degree of freedom parameter group (s,θ,Tx,Ty), which is still θ. However, in order to correctly search for the new scale parameter interpolation MaxX and the corresponding rotation parameter θ, the translation parameter corresponding to the new scale parameter interpolation MaxX and the corresponding rotation parameter θ needs to be updated, that is, each time the scale and rotation parameters are updated, the corresponding translation parameters also need to be updated in real time. When updating the translation parameters, the following steps can be included:

[0174] First, the coverage scores of all translation transformations in the 6*6 neighborhood of the current degree of freedom parameter group are calculated. The coverage score calculation method is the same as the coverage score calculation method above, and will not be described in detail here.

[0175] Second, find the location with the largest coverage score.

[0176] Finally, parabolic interpolation is performed on the X and Y directions of the position with the maximum coverage score, and the interpolation result is updated to the current result.

[0177] Specifically, take the example of parabolic interpolation of scaling parameters in a set of degree of freedom parameter groups to obtain new scaling parameters and calculate corresponding translation parameters. After parabolic interpolation of scaling parameters S in a set of degree of freedom parameter groups (S, θ, Tx, Ty), the new scaling parameters S′ are obtained, and the corresponding initial degree of freedom parameter group (S′, θ, Tx, Ty) is obtained. However, this result is not a good result, so it is necessary to update the translation parameters under the new S′ and θ. Since the interpolation change is small, the new translation result should be close to the original result, so the update is to search in the 6*6 neighborhood of the initial degree of freedom parameter group (S′, θ, Tx, Ty), that is, to find each degree of freedom parameter group in the range of (S′, θ, Tx-3, Ty-3) to (S′, θ, Tx+3, Ty+3). That is to say, for the degree of freedom parameter group S′,θ determined, the grid size is used as the translation step to search for the corresponding three adjacent translation parameters in the X direction and the three adjacent translation parameters in the Y direction, that is, for Tx, six adjacent corresponding translation parameters are found in the X direction, and for Ty, six adjacent corresponding translation parameters are found in the Y direction. The obtained translation parameters are arranged and combined in 6*6 to obtain 36 combinations of translation parameters (Tx, Ty), and these 36 combinations of translation parameters (Tx, Ty) are combined with the determined degree of freedom parameters S′,θ, and finally 36 new degree of freedom parameter groups are obtained, that is, 36 adjacent degree of freedom parameter groups within the 6*6 neighborhood of the initial degree of freedom parameter group are found. The coverage score corresponding to the contour after the transformation of each adjacent degree of freedom parameter group is calculated, and the degree of freedom parameter group corresponding to the largest coverage score is selected, that is, the selected degree of freedom parameter group is (S′,θ,Tx′,Ty′).

[0178] The translation parameters in the screened degree of freedom parameter group are respectively subjected to parabolic interpolation calculation in the X direction and the Y direction, that is, parabolic interpolation calculation is performed on Tx′, and parabolic interpolation calculation is also performed on Ty′. When performing parabolic interpolation calculation on Tx′, the adjacent degrees of freedom before and after Tx′, namely (S′,θ,Tx′-1,Ty′) and (S′,θ,Tx′+1,Ty′), are calculated. At the same time, the corresponding coverage scores after the three sets of degree of freedom parameters (S′,θ,Tx′,Ty′), (S′,θ,Tx′-1,Ty′) and (S′,θ,Tx′+1,Ty′) are calculated. The three sets of degree of freedom parameters and the corresponding coverage scores are used to perform parabolic interpolation calculation on the translation parameters in the X direction to obtain new translation parameters in the X direction, which are recorded as Tx". Similarly, the translation parameters in the Y direction are calculated by parabolic interpolation, which are recorded as Ty", and the final translation parameters Tx",Ty" are obtained. At this time, only the translation parameters Tx,Ty have changed, so it is called translation parameter update. The specific parabolic interpolation calculation method is consistent with the parabolic interpolation calculation above, and will not be described in detail here.

[0179] Finally, for the initial degree of freedom parameter group (S′, θ, Tx, Ty), the degree of freedom parameter group obtained after updating the translation parameters is (S′, θ, Tx", Ty").

[0180] Take the peak detection of the scaling parameter in a set of degree of freedom parameter groups as an example. For the current degree of freedom parameter group (s, θ, Tx, Ty) traversed, the peak value of the scaling parameter S in the current degree of freedom parameter group is detected.

[0181] First, the scaling parameter S searches for the adjacent degrees of freedom before and after according to the scaling step, and obtains two sets of adjacent degrees of freedom parameters. Among them, the previous adjacent degree of freedom parameter group is (s-Δs,θ,Tx1,Ty1), which is recorded as the first set of degree of freedom parameters. The next adjacent degree of freedom parameter group is (s+Δs,θ,Tx2,Ty2), which is recorded as the second set of degree of freedom parameters.

[0182] Secondly, the overlap ratio of the circumscribed rectangle of the contour point set after the transformation of the two sets of degree of freedom parameters and the circumscribed rectangle of the contour point set after the transformation of the current degree of freedom parameter group is calculated respectively. The adjacent degree of freedom parameters corresponding to the overlap ratio less than or equal to the threshold are discarded, and the current degree of freedom parameters are retained. The adjacent degree of freedom parameters corresponding to the overlap ratio greater than the threshold are retained, and the new degree of freedom parameters are obtained by parabolic interpolation calculation. Among them, the threshold is set to 0.8. In other words, the overlap ratio of the circumscribed rectangle of the contour point set after the transformation of the first set of degree of freedom parameter group (s-Δs,θ,Tx1,Ty1) and the circumscribed rectangle of the contour point set after the transformation of the current degree of freedom parameter group (s,θ,Tx,Ty) is calculated, and the overlap ratio of the circumscribed rectangle of the contour point set after the transformation of the second set of degree of freedom parameter group (s+Δs,θ,Tx2,Ty2) and the circumscribed rectangle of the contour point set after the transformation of the current degree of freedom parameter group (s,θ,Tx,Ty) is calculated.

[0183] Taking the calculation of the overlap rate of the circumscribed rectangle of the contour point set obtained by the first set of degree of freedom parameter transformation as an example, the calculated overlap rate is compared with the threshold, including the following two cases:

[0184] The first case: if the overlap rate between the circumscribed rectangle of the contour point set after the transformation of the current degree of freedom parameter group (s, θ, Tx, Ty) and the circumscribed rectangle after the transformation of the first degree of freedom parameter group (s-Δs, θ, Tx1, Ty1) is less than or equal to 0.8, the first degree of freedom parameter group (s-Δs, θ, Tx1, Ty1) is discarded and the current degree of freedom parameter group (s, θ, Tx, Ty) is retained.

[0185] The second case: When the overlap rate between the circumscribed rectangle of the contour point set after the transformation of the current degree of freedom parameter group (s, θ, Tx, Ty) and the circumscribed rectangle after the transformation of the first set of degree of freedom parameter group (s-Δs, θ, Tx1, Ty1) is greater than 0.8, parabolic interpolation calculation is performed on the first set of degree of freedom parameters and the current degree of freedom parameters. When performing the parabolic interpolation calculation, it is also necessary to find the second set of degree of freedom parameter group (the next adjacent degree of freedom parameter group) of the current degree of freedom parameters, and calculate the corresponding coverage score of the contour point set after the transformation of these three sets of degree of freedom parameter groups. The coverage score calculated after the current degree of freedom parameter transformation is recorded as M, the coverage score calculated after the first set of degree of freedom parameter transformation is recorded as M1, and the coverage score calculated after the second set of degree of freedom parameter transformation is recorded as M2. The coverage score calculation method is the same as the coverage score calculation method mentioned above, so it will not be described in detail here.

[0186] The three sets of values ​​obtained, namely, the current degree of freedom parameter S and the corresponding coverage fraction M, the first set of degree of freedom parameters s-Δs and the corresponding coverage fraction M1, and the second set of degree of freedom parameters s+Δs and the corresponding coverage fraction M2, are used to perform parabolic interpolation calculations with the above formula 15 to obtain new rotation parameters, which are recorded as S0. The corresponding translation parameters corresponding to the rotation parameter S0 updated by the above method for updating the translation parameters are recorded as (Tx0, Ty0), and a new set of degree of freedom parameter groups (S0, θ, Tx0, Ty0) are obtained. The current degree of freedom parameter group (s, θ, Tx, Ty), the first set of degree of freedom parameter group (s-Δs, θ, Tx1, Ty1), and the second set of degree of freedom parameter group (s+Δs, θ, Tx2, Ty2) are deleted, and only the new degree of freedom parameter group (S0, θ, Tx0, Ty0) is retained.

[0187] Finally, the peak value detection is performed on each group of rotation parameters and scaling parameters by the same method. The corresponding new degree of freedom parameter groups are obtained by performing peak detection screening on the scaling parameters, and the corresponding new degree of freedom parameter groups are also obtained by performing peak detection screening on the rotation parameters. Finally, the degree of freedom parameter groups screened out for the rotation parameters and scaling parameters are unified as the degree of freedom parameter groups for the conversion of the contour point set, and the corresponding coverage scores are calculated for the degree of freedom parameter groups obtained after the peak value detection.

[0188] S1500: Filter out a target degree of freedom parameter group based on the coverage scores corresponding to each degree of freedom parameter group.

[0189] Specifically, each calculated coverage score is compared with a threshold, and a coverage score greater than the threshold is screened, and the corresponding degree of freedom parameter group is the target degree of freedom parameter group.

[0190] In order to further improve the search accuracy, peak detection is performed on the target degree of freedom parameter group to obtain new degree of freedom parameter groups. The coverage scores corresponding to the new degree of freedom parameter groups are calculated, and the degree of freedom parameter group with the largest corresponding coverage score is selected as the final target degree of freedom parameter group. Specifically, the peak detection process is consistent with the peak detection process above, and the coverage score calculation method is consistent with the coverage score calculation method above, which will not be described in detail here. Among them, the threshold of the coverage score can be determined according to the quality of the acquired product positioning image. If the imaging quality is good, the coverage score threshold can be set to a larger value, generally set to 0.8. If the image quality is poor, a smaller value needs to be set, and generally should not be less than 0.4. The default threshold is generally 0.6.

[0191] In a possible implementation, after selecting the target degree of freedom parameter group, the following steps are further included:

[0192] Obtain a target transformation contour corresponding to a target degree of freedom parameter group; calculate the nearest point from each contour point in the target transformation contour to the CAD grid model; calculate an adjustment value of the target degree of freedom parameter group based on the calculated nearest points; optimize the target degree of freedom parameter group using the adjustment value to obtain a final target degree of freedom parameter group, and determine the position of the product based on the final target degree of freedom parameter group.

[0193] Specifically, the optimization process of the degree of freedom parameters can be divided into the following four cases according to the user's input:

[0194] (1) Scaling and rotation are not fixed

[0195] In this case, a linear optimization model is established, and the translation parameter T x 、T y , rotation parameter θ, scaling parameter s, the coordinates of the contour points before and after transformation can be described as:

[0196]

[0197] Can also be written as:

[0198]

[0199] in,

[0200]

[0201] T x =X3 (21)

[0202] T y =X4 (22)

[0203] If the coordinates of the contour point set after transformation are (x i ,y i )(i=1,2,3……,n), the coordinates of the nearest point set of each contour point corresponding to the offline model are (x′ i ,y′ i )(i=1,2,3,…,n), the corresponding relationship can be:

[0204]

[0205] The weight is obtained by multiplying the distance score by the angle score:

[0206] w i =wd i ·wr i (twenty four)

[0207]

[0208] wr i=|cos(r i -r′ i )| (26)

[0209] Among them, d i is the distance between the i-th point pair, r i is the gradient direction of the i-th contour point after transformation, r′ i is the gradient direction of the nearest point of the i-th contour point model. The above equation can be decomposed using SVD to find its least squares solution. After the calculation is completed, the above solution can be used to convert the average distance between the contour point and the offline model to define the fitting score. If the distance of a single point is e i (i=1,2,3...,n), then the RMS error is:

[0210]

[0211] The fitting score is:

[0212]

[0213] Among them, when finding the nearest point of the offline model, the nearest point on the offline model may not be the corresponding point of the contour, so it is necessary to gradually approach the contour point to the offline model through iteration. When the number of iterations reaches the maximum number of iterations or the results of two adjacent iterations are not much different, the loop is exited and the final result is output. The final result of the algorithm is the combination of the target degree of freedom parameter and the optimization result. If the result of the target degree of freedom parameter is:

[0214]

[0215] The optimization results are:

[0216]

[0217] The final result can be expressed as:

[0218]

[0219] So the final result degrees of freedom can be expressed as:

[0220] s=s f s r (33)

[0221] θ=θ f +θ r (34)

[0222] T x =T xf +T xr s f cosθ f -Tyr s r sinθ f (35)

[0223] T y =T yf +T xr s f sinθ f +T yr s r cos θ f (36)

[0224] (2) Scaling is not fixed, rotation is fixed

[0225] You can use the setting of situation (1) to establish the conversion relationship:

[0226]

[0227] Where s = X1, T x =X2,T y =X3, then the model equation can be written as:

[0228]

[0229] The SVD decomposition is also used to find the least squares solution, and the subsequent calculations are consistent with (1).

[0230] (3) Scaling is fixed, rotation is not fixed

[0231] When the scaling is fixed and the rotation is not fixed, a linear least squares model cannot be established, so only a nonlinear least squares model can be established, which is:

[0232]

[0233] The optimization parameters are θ, T x , T y , we can use the LM method to optimize it, and take partial derivatives of all parameters to get:

[0234] dx i =sx i cosθ-sy i sinθ+T x -x′ i (40)

[0235] dy i =sx i sinθ+sy i cosθ+T y -y′ i (41)

[0236]

[0237] (4) Scaling and rotation are fixed

[0238] In this case, we only need to optimize the translation parameters, which can be calculated directly:

[0239]

[0240] The obtained target degree of freedom parameter group is optimized according to the user's input to obtain the final target degree of freedom parameter group.

[0241] S1600: Based on the final target degree of freedom parameter group, determine the position of the product on the guiding device, and guide the product to be assembled.

[0242] It should be noted here that, in the physical coordinate system, to determine the position of the product on the guide device, that is, to determine the X, Y offset and rotation angle of the center point of the product relative to the rotation center of the guide device, and the center point of the CAD mesh model is equivalent to the rotation center of the guide device, so the X, Y offset and rotation angle of the product center point compared to the center point of the CAD mesh model, that is, the X, Y offset and rotation angle of the product center point compared to the rotation center of the guide device. After determining the X, Y offset and rotation angle of the product center point compared to the rotation center of the guide device, the position of the product on the guide device can be determined. Among them, the X, Y offset and rotation angle of the product center point compared to the center point of the CAD mesh model are the X, Y offset and rotation angle in the final target degree of freedom parameter group.

[0243] After determining the position of the product on the guide device, the assembly between different products can be guided. Specifically, the position of the product on the guide device is characterized by the final target degree of freedom parameter group. For product A and product B that are about to be assembled, the final target degree of freedom parameter group corresponding to product A and the final target degree of freedom parameter group corresponding to product B are determined by the above method, and then the translation parameter deviation and rotation parameter deviation in the final degree of freedom parameter groups of the two products are calculated respectively. According to the calculated translation parameter deviation and rotation parameter deviation, the two products can be guided for assembly.

[0244] If you need to obtain multiple positioning images of the product, refer to Fig.12, extract the image contours corresponding to the product from multiple product positioning images, convert the coordinates of each image contour of the product into CAD coordinates, obtain the CAD contours corresponding to the product, combine the CAD contours to obtain the CAD contour of the product, and then generate multiple degree of freedom parameter groups, and transform the position of the CAD contours through each degree of freedom parameter group to obtain the transformed contours corresponding to each degree of freedom parameter group; based on the transformed contours corresponding to each degree of freedom parameter group and the CAD grid model pre-built for the product, calculate the coverage score corresponding to each degree of freedom parameter group; based on the coverage score corresponding to each degree of freedom parameter group, screen out the target degree of freedom parameter group; based on the target degree of freedom parameter group, determine the position of the product. The specific processing method is consistent with the previous single image processing process, so I will not go into details here.

[0245] Therefore, the present application relates to a product positioning method, including: obtaining a positioning image of the product, and extracting the image contour of the product from the positioning image; converting the image coordinates of the image contour into CAD coordinates to obtain the CAD contour of the product; generating multiple degrees of freedom parameter groups, and transforming the position of the CAD contour respectively through each degree of freedom parameter group to obtain the transformed contour corresponding to each degree of freedom parameter group; based on the transformed contour corresponding to each degree of freedom parameter group and the CAD grid model pre-built for the product, calculating the coverage score corresponding to each degree of freedom parameter group; based on the coverage score corresponding to each degree of freedom parameter group, screening out the target degree of freedom parameter group; based on the target degree of freedom parameter group, determining the position of the product; wherein the degree of freedom parameter group includes scaling parameters, rotation parameters and translation parameters. Through the product positioning method of the present application, accurate positioning of products without obvious features can be achieved, thereby improving the accuracy of product positioning.

[0246] Further, according to another aspect of the present application, a product positioning device 100 is provided. Fig.13, including an image acquisition module 110, an image processing module 120, a position change module 130, a coverage score calculation module 140, a degree of freedom parameter screening module 150 and a positioning module 160. Among them, the image acquisition module 110 is used to acquire the positioning image of the product and extract the image contour of the product from the positioning image; the image processing module 120 is used to convert the image coordinates of the image contour into CAD coordinates to obtain the CAD contour of the product; the position transformation module 130 is used to generate multiple degree of freedom parameter groups, and transform the position of the CAD contour respectively through each degree of freedom parameter group to obtain the transformation contour corresponding to each degree of freedom parameter group; the coverage score calculation module 140 is used to calculate the coverage score corresponding to each degree of freedom parameter group based on the transformation contour corresponding to each degree of freedom parameter group and the CAD grid model pre-built for the product; the degree of freedom parameter screening module 150 is used to screen out the target degree of freedom parameter group based on the coverage score corresponding to each degree of freedom parameter group; the positioning module 160 is used to determine the position of the product based on the target degree of freedom parameter group.

[0247] Furthermore, according to another aspect of the present application, a product positioning device 200 is also provided. Fig.14 The product positioning device 200 of the embodiment of the present application includes a processor 210 and a memory 220 for storing executable instructions of the processor 210. The processor 210 is configured to implement any of the above product positioning methods when executing the executable instructions.

[0248] Here, it should be noted that the number of processors 210 can be one or more. At the same time, in the multi-camera image matching device 200 of the embodiment of the present application, an input device 230 and an output device 240 may also be included. Among them, the processor 210, the memory 220, the input device 230 and the output device 240 may be connected through a bus or in other ways, which are not specifically limited here.

[0249] The memory 220 is a computer-readable storage medium that can be used to store software programs, computer executable programs, and various modules, such as the program or module corresponding to the product positioning method of the embodiment of the present application. The processor 210 executes various functional applications and data processing of the product positioning device 200 by running the software programs or modules stored in the memory 220.

[0250] The input device 230 may be used to receive input numbers or signals. The signals may be key signals related to user settings and function control of the device / terminal / server. The output device 240 may include display devices such as display screens.

[0251] According to another aspect of the present application, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by the processor 210, any of the above product positioning methods is implemented.

[0252] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A product positioning method, characterized in that: include: Acquire a positioning image of a product, and extract an image contour of the product from the positioning image; Converting the image coordinates of the image contour into CAD coordinates to obtain the CAD contour of the product; Generate a plurality of degree of freedom parameter groups, and transform the position of the CAD contour respectively by using each of the degree of freedom parameter groups to obtain a transformed contour corresponding to each of the degree of freedom parameter groups; Calculating the coverage score corresponding to each of the groups of degree of freedom parameters based on the transformation profile corresponding to each of the groups of degree of freedom parameters and the CAD mesh model pre-built for the product; Based on the coverage scores corresponding to the degree of freedom parameter groups, a target degree of freedom parameter group is screened out; Determining the position of the product based on the target degree of freedom parameter group; The degree of freedom parameter group includes scaling parameters, rotation parameters and translation parameters.

2. The method according to claim 1, characterized in that When generating multiple sets of degree of freedom parameters, include: Acquire a zoom range and a zoom step of the zoom parameter, and calculate a traversal space of the zoom parameter based on the zoom range and the zoom step; Acquire a rotation range and a rotation step of the rotation parameter, and calculate a traversal space of the rotation parameter based on the rotation range and the rotation step of the rotation parameter; Determining a plurality of combinations of scaling parameters and rotation parameters based on the traversal space of the scaling parameters and the traversal space of the rotation parameters; For each combination of scaling parameters and rotation parameters, determine the traversal space of the corresponding translation parameters; Based on the traversal space of each combination of scaling parameter and rotation parameter and corresponding translation parameter, a plurality of degree-of-freedom parameter groups are generated.

3. The method according to claim 1, characterized in that When constructing the CAD mesh model of the product, it includes: Obtaining a CAD drawing of the product, deleting other information on the CAD drawing except the product outline, and obtaining an offline model of the product; Calculating the bounding rectangle of the offline model, and meshing the inner area of ​​the bounding rectangle to obtain a CAD mesh model composed of a plurality of meshes; Grids associated with the product outline are selected from the CAD grid model as valid grids, and matching scores of the valid grids are calculated.

4. The method according to claim 1, characterized in that When calculating the coverage score corresponding to each of the groups of degree of freedom parameters based on the transformation profile corresponding to each of the groups of degree of freedom parameters and the CAD meshing model pre-built for the product, it includes: Traversing the transformation profiles corresponding to the degree of freedom parameter groups; For the current transformation contour corresponding to the traversed current degree of freedom parameter group, the grid position of each contour point of the current transformation contour in the CAD grid model is calculated, and based on the grid position of each contour point of the current transformation contour in the CAD grid model, the coverage score corresponding to the current degree of freedom parameter group is calculated; After the traversal is completed, the coverage score corresponding to the group of freedom parameter groups is obtained.

5. The method according to claim 4, characterized in that Calculating a coverage score corresponding to the current degree of freedom parameter group based on a grid position of each contour point of the current transformed contour in the CAD grid model includes: Based on the grid position of each contour point of the current transformed contour in the CAD grid model, obtaining a grid matching score corresponding to each contour point of the current transformed contour; Based on the grid matching scores corresponding to the contour points of the current transformed contour, a coverage score corresponding to the current degree of freedom parameter group is calculated.

6. The method according to claim 1, characterized in that After filtering out the target degree of freedom parameter group, it also includes: Obtaining a target transformation profile corresponding to a target degree of freedom parameter group; Calculate the closest point from each contour point in the target transformed contour to the CAD grid model; Calculating adjustment values ​​of the target degree of freedom parameter group based on the calculated closest points; The target degree of freedom parameter group is optimized using the adjustment value to obtain a final target degree of freedom parameter group, and the position of the product is determined based on the final target degree of freedom parameter group.

7. The method according to claim 1, characterized in that The positioning image includes a plurality of local positioning images of the product; When generating the CAD outline of the product based on each of the local positioning images, it includes Extracting a local image contour of the product from each of the local positioning images; Convert the image coordinates of each of the local image contours into CAD coordinates to obtain a local CAD contour of the product; The local CAD contours are combined to obtain the CAD contour of the product.

8. A product positioning device, characterized in that: include: An image acquisition module, used to acquire a positioning image of a product and extract an image contour of the product from the positioning image; An image processing module, used for converting the image coordinates of the image contour into CAD coordinates to obtain the CAD contour of the product; A position transformation module, used to generate a plurality of degree of freedom parameter groups, and transform the position of the CAD contour respectively through each of the degree of freedom parameter groups to obtain a transformed contour corresponding to each of the degree of freedom parameter groups; A coverage score calculation module, used for calculating the coverage score corresponding to each of the groups of degree of freedom parameters based on the transformation profile corresponding to each of the groups of degree of freedom parameters and the CAD mesh model pre-built for the product; A degree of freedom parameter screening module, used for screening out a target degree of freedom parameter group based on the coverage scores corresponding to each of the degree of freedom parameter groups; A positioning module, used for determining the position of the product based on the target degree of freedom parameter group; The degree of freedom parameter group includes scaling parameters, rotation parameters and translation parameters.

9. A product positioning device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the method described in any one of claims 1 to 7 when executing the executable instructions.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images