Part free edge visual matching method and system based on model matching

Abstract

The invention provides a part free edge visual matching method and system based on model matching, and the method comprises the steps: analyzing an assembly three-dimensional model or a cutting two-dimensional model of a ship part, and building a part model database; determining a visual matching template database according to the two-dimensional contour of the part and the part features; collecting an image of a to-be-detected part; determining part features of the to-be-detected part; matching the part features of the to-be-tested part with the visual matching template database, and determining a matching result; if the matching result is successful matching, determining pixel coordinates of the to-be-detected part in the image of the to-be-detected part according to a visual matching template corresponding to the image of the to-be-detected part; and converting the pixel coordinates of the to-be-detected part in the image of the to-be-detected part into actual coordinates of a physical space, and determining the free edge of the to-be-detected part on the machine tool of the ship. According to the method and the device, the free edges of various parts of the ship are positioned, and the efficiency and the quality of ship manufacturing are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular, to a method and system for visual matching of free edges of parts based on model matching. Background Art

[0002] In the field of shipbuilding, the precise machining and positioning of sheet metal parts are crucial. Traditional methods for positioning ship sheet metal parts often rely on manual measurement, which has problems such as low accuracy, low efficiency, and large errors. With the continuous progress of technology, the demand for automated and intelligent production is increasing day by day.

[0003] Visual matching technology has been widely used in industrial production due to its advantages such as non-contact, high precision, and fast response. However, in the free edge positioning of ship sheet metal parts, due to the diversity of sheet metal shapes, the complexity of the environment, and various interference factors during the processing, existing visual matching methods face many challenges.

[0004] On the one hand, the free edges of ship sheet metal parts have irregular shapes, making it difficult to accurately identify and position them through simple geometric features. On the other hand, there are significant differences in dimensions, shapes, etc. among different types of ship sheet metal parts, and a general positioning method that can adapt to different parts is needed. In addition, factors such as lighting conditions and noise at the shipbuilding site also affect the accuracy and reliability of visual matching. Summary of the Invention

[0005] Aiming at the defects in the prior art, the purpose of the present disclosure is to provide a method and system for visual matching of free edges of parts based on model matching.

[0006] To achieve the above object, according to one aspect of the present disclosure, there is provided a method for visual matching of free edges of parts based on model matching, including:

[0007] Analyze the assembly three-dimensional model or cutting plate two-dimensional model of the parts of the ship, determine the two-dimensional contour and part features of the parts, and establish a part model database;

[0008] Generate a visual matching template based on the two-dimensional contour and part features of the parts, and determine a visual matching template database;

[0009] Collect an image of the part to be measured;

[0010] Perform feature extraction processing on the image of the part to be measured to determine the part features of the part to be measured;

[0011] Match the part features of the part to be measured with the visual matching template database to determine the matching result;

[0012] If the matching result is a successful match, determine the pixel coordinates of the part to be measured in the image of the part to be measured according to the visual matching template corresponding to the image of the part to be measured;

[0013] Convert the pixel coordinates of the part to be measured in the image of the part to be measured into the actual coordinates in the physical space, and determine the position of the free edge of the part to be measured on the machine tool of the ship.

[0014] Optionally, the generating a visual matching template according to the two-dimensional contour of the part and the part features and determining the visual matching template database includes:

[0015] Convert the two-dimensional contour of each part and the part features into a visual matching format to generate a visual matching template for each part;

[0016] The visual matching templates of each part form the visual matching template database.

[0017] Optionally, the method further includes:

[0018] Perform preprocessing on the image of the part to be measured to determine the preprocessed image of the part to be measured, and the preprocessing includes denoising processing, smoothing processing, and enhancing contrast processing.

[0019] Optionally, the part features of the part to be measured include the coding features of the part to be measured and / or the two-dimensional contour features of the part to be measured;

[0020] Optionally, the performing feature extraction processing on the image of the part to be measured to determine the part features of the part to be measured includes:

[0021] Perform coding recognition on the image of the part to be measured to determine the coding features of the part to be measured;

[0022] Perform edge detection processing on the image of the part to be measured to determine the two-dimensional contour features of the part to be measured.

[0023] Optionally, the matching the part features of the part to be measured with the visual matching template database to determine the matching result includes:

[0024] According to the coding features of the part to be measured and / or the two-dimensional contour features of the part to be measured, search for a visual matching template that matches the part to be measured in the visual matching template database;

[0025] If there is a visual matching template in the visual matching template database that matches the part to be measured, the matching result is a successful match, and determine the visual matching template corresponding to the image of the part to be measured;

[0026] If there is no visual matching template in the visual matching template database that matches the part to be measured, the matching result is a matching failure.

[0027] Optionally, if the matching result is a successful match, according to the visual matching template corresponding to the image of the part to be measured, determining the pixel coordinates of the part to be measured in the image of the part to be measured includes:

[0028] Using a preset algorithm to determine the similarity between each region of the image of the part to be measured and the visual matching template corresponding to the image of the part to be measured;

[0029] According to the similarity between each region of the image of the part to be measured and the visual matching template corresponding to the image of the part to be measured, determining the region with the highest similarity as the target region;

[0030] According to the pixel coordinates of the target region, determining the pixel coordinates of the part to be measured in the image of the part to be measured.

[0031] Optionally, converting the pixel coordinates of the part to be measured in the image of the part to be measured into the actual coordinates in the physical space, and determining the position of the free edge of the part to be measured on the ship's machine tool includes:

[0032] According to the preset calibration matrix conversion relationship of pixel-image-world, converting the pixel coordinates of the part to be measured into the image coordinates of the part to be measured, and then converting the image coordinates of the part to be measured into the actual coordinates of the part to be measured in the physical space, to determine the position information and attitude information of the part to be measured in the actual space;

[0033] According to the part features of the part to be measured and the position information and attitude information of the part to be measured in the actual space, determining the position of the free edge of the part to be measured on the ship's machine tool, where the free edge represents a non-assembly edge.

[0034] Optionally, the method further includes:

[0035] If the matching result is a successful match, setting the free edge of the part to be measured on the ship's machine tool as the grinding edge, and grinding the grinding edge;

[0036] If the matching result is a matching failure, end the grinding.

[0037] According to the second aspect of the present disclosure, there is provided a visual matching system for the free edge of a part based on model matching, including:

[0038] An analysis module for analyzing the assembled three-dimensional model or the cutting plate two-dimensional model of the parts of the ship, determining the two-dimensional contour and part features of the parts, and establishing a part model database;

[0039] A visual matching template database building module, configured to generate a visual matching template according to the two-dimensional contour of the part and the part features, and determine a visual matching template database;

[0040] An image acquisition module, configured to acquire an image of a part to be measured;

[0041] A feature extraction module, configured to perform feature extraction processing on the image of the part to be measured, and determine the part features of the part to be measured;

[0042] A model matching module, configured to match the part features of the part to be measured with the visual matching template database, and determine a matching result;

[0043] A pixel coordinate determination module, configured to, if the matching result is successful, determine the pixel coordinates of the part to be measured in the image of the part to be measured according to the visual matching template corresponding to the image of the part to be measured;

[0044] A free edge positioning module, configured to convert the pixel coordinates of the part to be measured in the image of the part to be measured into actual coordinates in the physical space, and determine the position of the free edge of the part to be measured on the machine tool of the ship.

[0045] According to a third aspect of the present disclosure, there is provided a part free edge visual matching system based on model matching, including: a server and a visual acquisition device;

[0046] The server includes a data storage system and a visual acquisition device interface, and the server is configured to store and process three-dimensional model data, image data, and free edge positioning information;

[0047] The visual acquisition device communicates with the server through the visual acquisition device interface, and the visual acquisition device is configured to acquire an image of a part to be measured.

[0048] Compared with the prior art, the embodiments of the present disclosure have at least one of the following beneficial effects:

[0049] Through the above technical solution, based on the three-dimensional model, the three-dimensional model of the assembly of the parts of the ship or the two-dimensional model of the cutting plate is intelligently parsed and converted into a visual matching template in a visual matching format, and a visual matching template database is determined. Without pre-collecting part images, a part model database can be established, reducing the manual workload and improving work efficiency. Based on visual recognition and measurement, the image of the part to be measured is automatically matched with the visual matching template database to obtain the position information and attitude information of the part to be measured in the actual space. There is a certain tolerance for identifying the position and attitude of the part, and the free edge of the part to be measured on the ship's machine tool is located, which is convenient for placing the part and improves the accuracy of the free edge positioning, solving the problems of small batch, multiple types, and difficult positioning of ship parts. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Other features, objects, and advantages of the present disclosure will become more apparent by reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0051] Figure 1 is a schematic flowchart of a method for visually matching the free edge of a part based on model matching according to an exemplary embodiment.

[0052] Figure 2 is a schematic flowchart of a method for establishing a part model database and a visual matching template database according to an exemplary embodiment.

[0053] Figure 3 is a schematic flowchart of a method for processing the image of the part to be measured according to an exemplary embodiment.

[0054] Figure 4 is a schematic flowchart of a method for positioning the free edge position of the part to be measured according to an exemplary embodiment.

[0055] Figure 5 is a schematic diagram of a preset calibration matrix conversion relationship between pixels, images, and the world according to an exemplary embodiment.

[0056] Figure 6 is a schematic flowchart of a method for determining the corner points and non-corner points of the part to be measured according to an exemplary embodiment.

[0057] Figure 7 is a block diagram of a system for visually matching the free edge of a part based on model matching according to an exemplary embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0058] The present disclosure will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present disclosure, but do not limit the present disclosure in any form. It should be noted that those of ordinary skill in the art can make several modifications and improvements without departing from the concept of the present disclosure. These all belong to the protection scope of the present disclosure.

[0059] Figure 1 is a schematic flowchart of a method for visual matching of free edges of parts based on model matching shown according to an exemplary embodiment. Figure 2 is a schematic flowchart of a method for establishing a part model database and a visual matching template database shown according to an exemplary embodiment.

[0060] As Figure 1 shown, the present disclosure provides a method for visual matching of free edges of parts based on model matching, including S11 to S17.

[0061] S11, parse the assembly three-dimensional model or the cutting plate two-dimensional model of the parts of the ship, determine the two-dimensional contour and features of the parts, and establish a part model database.

[0062] As Figure 2 shown, parse the assembly three-dimensional model of the parts of the ship to perform two-dimensional mapping on the three-dimensional model of the ship plate parts to generate a two-dimensional model of the parts. The two-dimensional model of the parts can also be further segmented and the areas refined to facilitate subsequent matching of local features of the parts and provide basic data; extract the two-dimensional contour and features of the parts of each two-dimensional model of the parts to establish a part model database. The part features may include the outer contour, category, size, grinding edge, shape, material, and thickness of the parts.

[0063] The part model database may include the two-dimensional contours and features of parts of various different types of ship plate parts. The part model database is used to retrieve part models and quickly locate different types of parts.

[0064] The two-dimensional contour of the parts of the present disclosure is generated based on the two-dimensional model of the parts parsed from the three-dimensional model, and has high precision and stability, and can adapt to different lighting conditions, perspective changes, and environmental interferences.

[0065] Among them, the cutting plate two-dimensional model is the part cutting layout diagram, and the two-dimensional contour and features of the parts can also be extracted according to the cutting plate two-dimensional model of each part to establish a part model database.

[0066] S12, generate a visual matching template according to the two-dimensional contour and features of the parts, and determine a visual matching template database.

[0067] As Figure 2As shown, the two-dimensional contour of the part can accurately present part features such as the shape, size, and corner points of the part. The visual matching template is used as a positioning template.

[0068] The visual matching database can include visual matching templates formed by various different types of ship plate parts.

[0069] S13, collect an image of the part to be measured.

[0070] Among them, an industrial vision camera can be used as the visual acquisition device to collect an image of the part to be measured.

[0071] S14, perform feature extraction processing on the image of the part to be measured to determine the part features of the part to be measured.

[0072] Among them, the part features of the part to be measured can include the coding features and / or two-dimensional contour features of the part to be measured, such as two-dimensional codes.

[0073] S15, match the part features of the part to be measured with the visual matching template database to determine the matching result.

[0074] S16, if the matching result is successful, determine the pixel coordinates of the part to be measured in the image of the part to be measured according to the visual matching template corresponding to the image of the part to be measured.

[0075] S17, convert the pixel coordinates of the part to be measured in the image of the part to be measured into the actual coordinates in the physical space to determine the free edge of the part to be measured on the ship's machine tool.

[0076] Among them, the free edge of the part to be measured on the ship's machine tool is a non-assembly edge.

[0077] Through the above technical solutions, based on the three-dimensional model-driven intelligent parsing of the assembly three-dimensional model or cutting plate two-dimensional model of the ship's parts, and converting it into a visual matching template in the visual matching format, determining the visual matching template database, a part model database can be established without pre-collecting part images, reducing the manual workload and improving work efficiency; based on visual recognition and measurement, automatically matching the image of the part to be measured and the visual matching template database, obtaining the position information and attitude information of the part to be measured in the actual space, having a certain tolerance for identifying the position and attitude of the part, positioning the free edge of the part to be measured on the ship's machine tool, facilitating part placement, and improving the accuracy of free edge positioning, solving the problems of small batch, multiple types, and difficult positioning of ship parts.

[0078] As Figure 2 shown, in a possible embodiment, S11, parse the assembly three-dimensional model or cutting plate two-dimensional model of the ship's parts, determine the part two-dimensional contour and part features, and establish a part model database, including:

[0079] As an example, parse the assembled 3D model of the parts of a ship, generate the 2D models of the parts, and based on the 2D models of the parts, obtain the 2D contours and features of the parts, such as the outer contours, categories, dimensions, grinding edges, shapes, materials, and thicknesses, and store them in the part model database to establish the part model database.

[0080] Among them, the material and thickness information of the parts can be modified through manual maintenance.

[0081] By automatically parsing the assembled 3D model or obtaining the grinding edge information of the parts based on the outer contours of the parts to determine the assembly edges, the errors caused by manual observation or simple measurement can be avoided.

[0082] For parts with complex shapes, parsing the assembled 3D model clearly presents each edge, improving the positioning accuracy of the free edges.

[0083] The present disclosure can batch process the parts of a ship, analyze and process the 3D models of multiple parts at the same time, identify the free edges of each part, establish the part model database, facilitate production and processing, and improve the production efficiency in the ship manufacturing process.

[0084] As another example, parse the 2D model of the cutting plate of the parts of a ship, that is, the cutting layout of the ship steel plate, and obtain the 2D contours and features of the parts, such as the outer contours, categories, dimensions, grinding edges, shapes, materials, and thicknesses, and store them in the part model database to establish the part model database.

[0085] Among them, the grinding edge information of the parts can be obtained based on the outer contours of the parts.

[0086] As Figure 2 shown, in a possible embodiment, S12, according to the 2D contours and features of the parts, generate a visual matching template and determine the visual matching template database, which may include S21 to S22.

[0087] S21, convert the 2D contours and features of each part into a visual matching format to generate a visual matching template for each part.

[0088] Among them, the visual matching template of the part is used for visual matching with the actually captured image.

[0089] S22, the visual matching templates of each part form the visual matching template database.

[0090] Among them, the visual matching templates in the visual matching template database can be 2D contour images of the parts.

[0091] In a possible embodiment, the visual matching template of each part, that is, the two-dimensional contour image of the part, is subjected to image enhancement processing to enhance the edges and features of the part and remove redundant details and redundant information, so as to make the visual matching template easier to match with the actual image and improve the efficiency of subsequent matching.

[0092] Figure 3 FIG. 4 is a schematic flow chart of preprocessing an image of a part to be measured according to an exemplary embodiment.

[0093] As Figure 3 shown, in a possible embodiment, a method for visual matching of free edges of parts based on model matching further includes S18.

[0094] S18: Preprocess the image of the part to be measured to determine the preprocessed image of the part to be measured.

[0095] Among them, the preprocessing includes denoising processing, smoothing processing, and contrast enhancement processing.

[0096] Through denoising processing, the noise points in the image of the part to be measured during the measurement process or the acquisition process are removed, the noise and interference elements of the two-dimensional contour in the image of the part to be measured are removed, the image quality is improved, and the noise interference is eliminated.

[0097] Through smoothing processing, the continuity and regularity of the two-dimensional contour edge of the part in the image of the part to be measured are optimized.

[0098] Through contrast enhancement processing, the features of the part are strengthened.

[0099] In a possible embodiment, the part features of the part to be measured include the coding feature of the part to be measured and / or the two-dimensional contour feature of the part to be measured.

[0100] In a possible embodiment, S14: Perform feature extraction processing on the image of the part to be measured to determine the part features of the part to be measured, including:

[0101] As an example, perform coding recognition on the image of the part to be measured to determine the coding feature of the part to be measured.

[0102] Among them, the coding feature of the part to be measured can be a two-dimensional code. Scanning the two-dimensional code of the part to be measured by a visual acquisition device can view the part number of the part to be measured.

[0103] As another example, perform edge detection processing on the image of the part to be measured to determine the two-dimensional contour feature of the part to be measured.

[0104] As another example, a feature point extraction algorithm can also be used to perform feature point extraction processing on the image of the part to be measured to determine the feature points of the part to be measured.

[0105] When extracting and selecting image features from the image of the part to be measured, that is, feature extraction processing, statistical pattern recognition, syntactic pattern recognition, and fuzzy pattern recognition methods can be used.

[0106] Among them, statistical pattern recognition obtains the raw data of the object to be recognized through sensors, extracts the feature vectors reflecting the essence of the pattern from it, screens out the most representative and discriminative features, and reduces the data dimension. Then, according to the application requirements and data characteristics, a suitable classifier algorithm is selected and trained with a large number of known class samples to determine its parameters. Finally, the sample to be recognized is input into the trained classifier, and the recognition result is output.

[0107] Syntactic pattern recognition decomposes complex patterns into simple primitives, and constructs a syntactic structure based on the structural relationship and grammar rules of the primitives. By learning known class samples, infer the grammar rules and establish a syntactic model. Extract primitives from the pattern to be recognized, perform syntactic analysis, match with the existing grammar rules, and determine the class according to the matching result.

[0108] Fuzzy pattern recognition transforms the raw data or feature values into elements of a fuzzy set, and describes their degree of belonging to fuzzy concepts by means of membership functions. Establish a fuzzy relation matrix based on the similarity or correlation between features to represent the fuzzy relationship of the pattern. Use fuzzy logic and inference rules to operate and infer this matrix to obtain the fuzzy membership degrees of the pattern to various classes, and then convert them into clear classification results according to the criteria.

[0109] Among them, classifier design rules include statistical recognition rules and error correction rules.

[0110] Statistical recognition rules ensure the comprehensiveness of classification by collecting a large amount of data related to the objects to be recognized covering various situations and features.

[0111] Error correction rules detect whether there are errors in the information or data to be recognized by performing preliminary processing and analysis on them and comparing with known correct patterns or rules, so as to ensure the reliability of classification.

[0112] After determining the part features of the part to be measured, use the designed classifier to make a classification decision to determine its part type in the part model database.

[0113] Figure 4 It is a schematic flowchart of positioning the free edge position of a part to be measured shown according to an exemplary embodiment.

[0114] As Figure 4 shown, identify the number of the part to be measured, call the corresponding template in the template database, trigger visual positioning, perform image processing on the image of the part to be measured, and perform matching based on the matching similarity between the image of the part to be measured and the visual matching template to determine the matching result.

[0115] If the matching result is successful, search for feature information within the matching region and give the positioning information of the free edge of the part.

[0116] If the matching result is a failure, the free edge positioning fails.

[0117] In a possible embodiment, in S15, match the part features of the part to be measured with the visual matching template database to determine the matching result, including: S31 to S33.

[0118] S31, according to the coding features of the part to be measured and / or the two-dimensional contour features of the part to be measured, search for a visual matching template that matches the part to be measured in the visual matching template database.

[0119] Among them, the part number corresponding to the part to be measured can be obtained by identifying the two-dimensional code of the part to be measured, compare the part number of the part to be measured with the part number corresponding to the visual matching template stored in the visual matching template database, or compare the two-dimensional contour features of the part to be measured with the two-dimensional contour features corresponding to the visual matching template in the visual matching template database, so as to search for a visual matching template and part data that match the part to be measured in the visual matching template database and achieve efficient visual matching.

[0120] S32, if there is a visual matching template that matches the part to be measured in the visual matching template database, the matching result is successful, and determine the visual matching template corresponding to the image of the part to be measured.

[0121] S33, if there is no visual matching template that matches the part to be measured in the visual matching template database, the matching result is a failure.

[0122] Steps S31 to S33 of the present disclosure can be executed by a pre-trained deep learning model, and the nearest neighbor search algorithm and the feature descriptor matching algorithm are used to match the part features of the part to be measured and the visual matching template database, train the neural network, learn the part features of the part and perform model parsing. For ship parts of the same category, using a pre-trained deep learning model can produce highly consistent and repeatable free edge determination results.

[0123] The matching result can be verified to judge the accuracy and reliability of the matching. According to the matching metrics, such as the number of matching features and the matching degree, evaluate the matching result. If the matching result is not ideal, adjust the part features of the part, the matching algorithm, and the parameters of the pre-trained deep learning model, or re-acquire the image of the part to be measured for matching, and optimize the model matching process according to the actual application requirements to improve the matching accuracy and the positioning accuracy.

[0124] Pre-trained deep learning models can also adopt parallel computing and acceleration algorithms to improve the matching speed, or increase the diversity and adaptability of the models to improve the accuracy and robustness of matching.

[0125] The pre-trained deep learning model uses deep learning technology, has the advantages of fast processing speed and high precision, improves the stability of ship part production quality, and improves the stability and accuracy of quality inspection in the production process, improves production efficiency, reduces the manual rework rate, reduces quality fluctuations. Moreover, the pre-trained deep learning model can continuously learn from new data, and its performance continues to improve over time, further optimizing the production process, reducing costs and shortening the production cycle.

[0126] In a possible embodiment, in S16, if the matching result is successful, according to the visual matching template corresponding to the image of the part to be measured, determine the pixel coordinates of the part to be measured in the image of the part to be measured, including S41 to S43.

[0127] S41, use a preset algorithm to determine the similarity between each region of the image of the part to be measured and the visual matching template corresponding to the image of the part to be measured.

[0128] S42, according to the similarity between each region of the image of the part to be measured and the visual matching template corresponding to the image of the part to be measured, determine the region with the highest similarity as the target region.

[0129] S43, according to the pixel coordinates of the target region, determine the pixel coordinates of the part to be measured in the image of the part to be measured.

[0130] Among them, take the pixel coordinates of the target region as the pixel coordinates of the part to be measured in the image of the part to be measured.

[0131] In a possible embodiment, in S17, convert the pixel coordinates of the part to be measured in the image of the part to be measured into the actual coordinates in the physical space, and determine the position of the free edge of the part to be measured on the ship's machine tool, including S51 to S52.

[0132] S51, according to the preset calibration matrix conversion relationship of pixel-image-world, convert the pixel coordinates of the part to be measured into the actual coordinates of the part to be measured in the physical space, and determine the position information and attitude information of the part to be measured in the actual space.

[0133] Figure 5 It is a schematic diagram showing a preset calibration matrix conversion relationship of pixel-image-world according to an exemplary embodiment.

[0134] Such as Figure 5As shown, the preset calibration matrix conversion relationship between pixel-image-world is to first obtain the image coordinates through the conversion from pixel coordinates to image coordinates, and then convert the image coordinates to world coordinates through the internal and external parameters of the camera.

[0135] The actual coordinates of the part to be measured in the physical space are the coordinates of the part to be measured relative to the grinding machine in the actual space, and the actual coordinates of the part to be measured in the physical space after conversion are calibrated and error-corrected to improve the positioning accuracy.

[0136] Figure 6 It is a schematic flowchart of a process for judging the corner points and non-corner points of a part to be measured shown according to an exemplary embodiment.

[0137] As Figure 6 shown, for each pixel point in the image of the part to be measured, calculate the square of the gradient and the product of the gradients. Taking each pixel point as the center, in a preset neighborhood, use a Gaussian window to perform weighted summation calculation on the square of the gradient and the product of the gradients of the pixel point to calculate the values of the four elements of matrix M, and use Gaussian smoothing filtering.

[0138] According to matrix M:

[0139]

[0140] Calculate the corner response value R corresponding to each pixel in the image of the part to be measured. The formula is:

[0141] R = det(M) - k(trace(M)) 2 ,

[0142] where det(M) = AB - C 2 represents the determinant of matrix M, trace(M) = A + B represents the trace of matrix M, and k represents an empirical constant.

[0143] Preset a threshold T, and compare the calculated corner response value R with the threshold T. If R ≥ T, it means that this point is a corner point; if R < T, it means that this point is a non-corner point.

[0144] The judgment result of the corner points of the part to be measured can be used as the part features of the part to be measured.

[0145] S52. According to the part features of the part to be measured, as well as the position information and attitude information of the part in the actual space, determine the position of the free edge of the part to be measured on the ship's machine tool.

[0146] Among them, the free edge represents a non-assembly edge.

[0147] The part features of the part to be measured include the two-dimensional contour features of the part to be measured, and may also include the category, material, and thickness of the part to be measured. Based on the position information and attitude information of the part to be measured, the assembly edge and free edge of the part to be measured are determined.

[0148] In a possible embodiment, a vision matching method for the free edge of a part based on model matching may further include S19 to S20.

[0149] S19. If the matching result is successful, set the free edge of the part to be measured on the ship machine tool as the grinding edge and grind the grinding edge.

[0150] In the present disclosure, all the free edges of the part to be measured with successful positioning can be regarded as grinding edges, or some free edges of the part to be measured with successful positioning can be regarded as grinding edges according to a preset rule.

[0151] S20. If the matching result is failed, end the grinding.

[0152] Among them, a failed match means that the positioning of the free edge of the part to be measured fails, and the grinding process ends.

[0153] Figure 7 It is a block diagram of a vision matching system for the free edge of a part based on model matching shown according to an exemplary embodiment.

[0154] Based on the same concept, the present disclosure also provides a vision matching system 100 for the free edge of a part based on model matching, as Figure 7 shown, including a parsing module 110, a vision matching template database building module 120, an image acquisition module 130, a feature extraction module 140, a model matching module 150, a pixel coordinate determination module 160, and a free edge positioning module 170.

[0155] The parsing module 110 is configured to parse the assembly three-dimensional model or the cutting plate two-dimensional model of the parts of the ship, determine the two-dimensional contour and part features of the parts, and establish a part model database;

[0156] The vision matching template database building module 120 is configured to generate a vision matching template according to the two-dimensional contour and part features of the parts, and determine the vision matching template database;

[0157] The image acquisition module 130 is configured to acquire an image of the part to be measured;

[0158] The feature extraction module 140 is configured to perform feature extraction processing on the image of the part to be measured to determine the part features of the part to be measured;

[0159] The model matching module 150 is configured to match the part features of the part to be measured with the vision matching template database to determine the matching result;

[0160] A pixel coordinate determination module 160, configured to, if the matching result is a successful match, determine the pixel coordinates of the part to be measured in the image of the part to be measured according to the visual matching template corresponding to the image of the part to be measured;

[0161] A free edge positioning module 170, configured to convert the pixel coordinates of the part to be measured in the image of the part to be measured into actual coordinates in the physical space, and determine the position of the free edge of the part to be measured on the ship's machine tool.

[0162] Through the above technical solutions, based on the three-dimensional model-driven intelligent parsing of the assembly three-dimensional model or the cutting plate two-dimensional model of the ship's parts, and converting it into a visual matching template in the visual matching format, determining the visual matching template database, it is possible to establish a part model database without pre-collecting part images, reduce the manual workload, and improve work efficiency; based on visual recognition and measurement, automatically match the image of the part to be measured and the visual matching template database, obtain the position information and attitude information of the part to be measured in the actual space, have a certain tolerance for identifying the position and attitude of the part, position the free edge of the part to be measured on the ship's machine tool, facilitate part placement, improve the accuracy of free edge positioning, and solve the problems of small batch, multiple types, and difficult positioning of ship parts.

[0163] Regarding the embodiments of the above system, the specific ways in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0164] In a possible embodiment, a visual matching system for the free edge of a part based on model matching is further provided, including a server and a visual acquisition device.

[0165] The server includes a data storage system and a visual acquisition device interface. The server is used to store and process three-dimensional model data, image data, and free edge positioning information.

[0166] The visual acquisition device communicates with the server through the visual acquisition device interface. The visual acquisition device is used to acquire the image of the part to be measured.

[0167] Among them, the visual acquisition device uses a visual sensor, such as an industrial vision camera, a laser scanner, or a depth camera.

[0168] During the process of positioning the free edge of the ship's parts, the transmission and processing of data are coordinated and managed by the server. The server also stores and processes the parsed two-dimensional model file of the part, the part model database including the part two-dimensional contour and part features, the visual matching template database including the visual matching template, the image data of the part to be measured captured by the industrial vision camera, and the positioning result information of the free edge, providing data support for production and quality control.

[0169] When parsing the assembled 3D model, the processor can comprehensively consider the mutual relationship between the edges of parts and adjacent edges, such as the connection method, included angle, etc. Judging whether an edge of a certain part is a free edge according to the mutual relationship between the edge of the part and the adjacent edge can effectively improve the accuracy of the judgment.

[0170] For ship parts with complex shapes and parts with special shapes, internal structures or multiple connection parts, a visual matching system for free edges of parts based on model matching provided by the present disclosure can also perform comprehensive and accurate analysis and make free edge positioning judgments, providing reliable technology for the judgment of free edges of various complex parts in shipbuilding. Accurately judge the welding edges and grinding edges connected to other parts.

[0171] Specifically, except for the assembly welding edges, the rest of the edges can be grinding edges; or the grinding edges can be judged according to the outer contour of the part to be measured.

[0172] The processor uses computer programs and algorithms to parse the assembled 3D model, and can quickly judge the free edges of a large number of ship parts. Compared with manually checking the free edges of parts one by one, the work efficiency is greatly improved, and time and labor costs are saved. In large-scale shipbuilding projects, the number of parts is numerous, and the advantage of high-efficiency model parsing is more obvious. At the same time, by accurately judging the free edges of the part model, subsequent processing processes and grinding paths and other production processes can be reasonably planned according to information such as the position and quantity of the free edges. For example, the movement trajectory and processing sequence of the grinding tool can be determined according to the distribution of the free edges, improving the processing efficiency and quality.

[0173] The accuracy of model parsing highly depends on the accuracy of the 3D model of ship parts. If there are errors in the modeling process, such as inaccurate dimension measurement, incomplete model construction or deviation from the actual parts, the judgment results of model parsing will also be affected. Considering that model parsing is mainly based on theoretical geometric models for analysis, some factors in the actual production process cannot be fully considered, such as part deformation, processing errors, assembly errors, etc. These actual working conditions may cause the actual feature situation of the part to be different from the model parsing result, thus affecting the accuracy of visual matching.

[0174] In production links with high requirements for free edge accuracy, it is necessary to combine actual measurement and inspection means to verify and correct the results of model parsing.

[0175] One or more specific regions can be extracted from the assembled 3D model of the part as templates, such as a part of the part's contour, a specific pattern, etc. In the actual image, a template matching algorithm is used to search for regions similar to the template, and the similarity between the template and each region in the image is calculated to determine the region with the highest similarity as the target region. To improve the accuracy of the matching, the template matching result can be optimized by extracting representative features from the 3D model, such as geometric shape features (edges, corner points, etc.), texture features, or other specific features. According to the characteristics and positioning requirements of the part, a feature extraction method is selected to extract features from the preprocessed image and match them with the visual matching template database.

[0176] Image features can be extracted using algorithms such as edge detection algorithms and feature point extraction, and the image features are matched with the visual matching template database. The matching algorithm can use nearest neighbor search and feature descriptor matching algorithms to search for the part of the image that is most similar to the visual matching template through the matching algorithm, and determine the position and pose of the part in the image.

[0177] Verify the matching result to judge the accuracy and reliability of the matching. The matching result can be evaluated by comparing indicators such as the number of matching features and the matching degree. If the matching result is not ideal, the parameters of the feature extraction and matching algorithms can be adjusted, or the image can be re-acquired for matching. Optimize the model matching process according to the actual application requirements, such as using technologies such as parallel computing and acceleration algorithms to improve the matching speed, or increasing the diversity and adaptability of the model to improve the accuracy and robustness of the matching.

[0178] Through the above model matching steps, the visual matching of the free edge of the part is realized, providing accurate position information for subsequent processing, inspection and other operations. Using the feature matching algorithm, the preprocessed image is matched with the visual matching template to determine the position and pose of the part in the image, calculate the similarity of the matching, judge the accuracy of the matching, and optimize the matching result to improve the positioning accuracy.

[0179] The camera signal and parameters of the industrial vision camera can be selected according to the shooting requirements of the part to be measured. Before shooting the part to be measured, installation and modulation are required to ensure stable shooting of a clear image of the part.

[0180] The parts for which the industrial vision camera is used to collect the images to be measured need to have sufficient resolution and clarity and be able to accurately reflect the appearance characteristics of the parts to be measured. The industrial vision camera can be used to collect multiple images of the parts to be measured from different angles and under different lighting conditions to improve the accuracy and robustness of the visual matching.

[0181] The visual acquisition device communicates with the server through the visual acquisition device interface and transmits the collected images of the parts to be measured to the server for processing.

[0182] A vision recognition and positioning system is provided in the vision acquisition device to accurately recognize and position the position information and attitude information of the part to be measured.

[0183] The vision recognition and positioning system uses an industrial vision camera to scan the QR code on the part to be measured or match with the vision matching template in the vision matching template database to identify the part number of the part to be measured and obtain its outer contour. Taking the center point of the longest side of the outer contour of the part to be measured as the coordinate origin, calculate the coordinates of the part to be measured, and use the counterclockwise direction of the longest side as the positive direction to determine the rotation angle of the part to be measured, so as to achieve accurate positioning of the position information and attitude information of the part to be measured.

[0184] During the process of identifying the part to be measured, the industrial vision camera first takes a picture of the area of the part to be measured, and then performs QR code recognition or matches with the vision matching template in the vision matching template database to identify the part number of the part to be measured. QR code recognition involves using an industrial vision camera to capture the image of the area of the part to be measured, identifying the QR code in the image of the area of the part to be measured to obtain the part number of the part to be measured, and retrieving the matching part data in the part model database. For those parts that have not been maintained with data, the system will prohibit the grinding operation.

[0185] Contour matching includes using an industrial vision camera to capture the image of the part area, identifying the part number of the part to be measured in the image, and obtaining the outer contour information of the part to be measured. Then, the system will search for the part data that matches the outer contour information in the part model database. Similarly, for parts that have not been maintained with data, the system will prohibit grinding.

[0186] The system performs a matching search in the vision matching template database according to the QR code or outer contour information of the current part to be measured. If no matching item is found, the positioning process ends; if the matching is successful, the system will obtain the corresponding category, outer contour dimensions, material and thickness information, match the corresponding grinding parameters, and finally count the number of grinding edges.

[0187] By adopting a vision matching system for free edges of parts based on model matching provided by the present disclosure, it is possible to establish a model library of parts, adopt advanced image processing and template recognition technologies, achieve rapid and accurate positioning of the free edges of ship plate parts, and improve the efficiency and quality of shipbuilding.

[0188] Based on the same concept as above, in another embodiment of the present disclosure, an electronic device is further provided, including a memory, a processor, and a computer program stored on the memory and capable of running on the processor. When the processor executes the program, it is used to execute the vision matching method for free edges of parts based on model matching.

[0189] Optionally, a memory for storing programs; the memory may include volatile memory (e.g., random-access memory, such as static random-access memory (SRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM), etc.); the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs and functional modules for implementing the above methods), computer instructions, etc. The above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories. And the above computer programs, computer instructions, data, etc. can be called by the processor.

[0190] The above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories. And the above computer programs, computer instructions, data, etc. can be called by the processor.

[0191] A processor for executing the computer programs stored in the memory to implement each step in the method related to the above embodiments. For details, reference can be made to the relevant descriptions in the foregoing method embodiments.

[0192] The processor and the memory can be of independent structure or integrated structure. When the processor and the memory are of independent structure, the memory and the processor can be coupled and connected through a bus.

[0193] In the embodiments of the present disclosure, a non-transitory computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of a method for visual matching of free edges of parts based on model matching in any of the above embodiments are implemented.

[0194] Those skilled in the art should understand that the embodiments of the present disclosure can be provided as a method, a system, or a computer program product. Therefore, the present disclosure can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present disclosure can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0195] This disclosure is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the disclosure. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce a means for implementing the functions specified in the Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0196] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including an instruction means that implements the functions specified in the Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0197] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operational steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0198] Although the preferred embodiments of the disclosure have been described, additional changes and modifications can be made to these embodiments by those skilled in the art once they learn of the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments and all changes and modifications that fall within the scope of the disclosure.

[0199] Obviously, those skilled in the art can make various changes and variations to the disclosure without departing from the spirit and scope of the disclosure. Thus, if these modifications and variations of the disclosure fall within the scope of the claims of the disclosure and their equivalent technologies, the disclosure is also intended to include these changes and variations.

Claims

1. A method for visual matching of free edges of parts based on model matching, characterized in that: include: Analyze the assembled 3D model or cut-out 2D model of the ship's parts, determine the 2D contours and features of the parts, and establish a parts model database; Generate a visual matching template according to the two-dimensional contour of the part and the part features, and determine a visual matching template database; Collect images of the parts to be tested; Performing feature extraction processing on the image of the part to be tested to determine the part features of the part to be tested; Matching the part features of the part to be tested with the visual matching template database to determine a matching result; If the matching result is a successful match, determining the pixel coordinates of the part to be tested in the image of the part to be tested according to the visual matching template corresponding to the image of the part to be tested; The pixel coordinates of the part to be measured in the image of the part to be measured are converted into actual coordinates in a physical space, and the position of the free edge of the part to be measured on the machine tool of the ship is determined.

2. The method according to claim 1, characterized in that The step of generating a visual matching template according to the two-dimensional contour of the part and the part feature and determining a visual matching template database comprises: Convert the part two-dimensional contour and the part features of each part into a visual matching format to generate a visual matching template for each part; The visual matching template of each of the parts constitutes the visual matching template database.

3. The method according to claim 1, characterized in that The method further comprises: The image of the part to be tested is preprocessed to determine the preprocessed image of the part to be tested, wherein the preprocessing includes denoising, smoothing, and contrast enhancement.

4. The method according to claim 1, characterized in that: The part features of the part to be measured include the coding features of the part to be measured and / or the two-dimensional contour features of the part to be measured; The step of performing feature extraction processing on the image of the part to be measured to determine the part features of the part to be measured includes: Performing coding recognition on the image of the part to be tested to determine the coding features of the part to be tested; Perform edge detection processing on the image of the part to be measured to determine the two-dimensional contour features of the part to be measured.

5. The method according to claim 4, characterized in that The step of matching the part feature of the part to be tested with the visual matching template database to determine a matching result includes: Searching, in the visual matching template database, for a visual matching template matching the part to be tested according to the coding feature of the part to be tested and / or the two-dimensional contour feature of the part to be tested; If there is a visual matching template matching the part to be tested in the visual matching template database, the matching result is a successful match, and the visual matching template corresponding to the image of the part to be tested is determined; If there is no visual matching template matching the part to be tested in the visual matching template database, the matching result is a matching failure.

6. The method according to claim 5, characterized in that If the matching result is a successful match, determining the pixel coordinates of the part to be tested in the image of the part to be tested according to the visual matching template corresponding to the image of the part to be tested, comprises: Using a preset algorithm to determine the similarity between each area of ​​the image of the part to be tested and the visual matching template corresponding to the image of the part to be tested; According to the similarity between each area of ​​the image of the part to be tested and the visual matching template corresponding to the image of the part to be tested, determining the area with the highest similarity as the target area; The pixel coordinates of the part to be measured in the image of the part to be measured are determined according to the pixel coordinates of the target area.

7. The method according to claim 1, characterized in that The step of converting the pixel coordinates of the part to be tested in the image of the part to be tested into actual coordinates in a physical space and determining the position of the free edge of the part to be tested on the machine tool of the ship comprises: According to the preset pixel-image-world calibration matrix conversion relationship, the pixel coordinates of the part to be tested are converted into the image coordinates of the part to be tested, and then the image coordinates of the part to be tested are converted into the actual coordinates of the part to be tested in the physical space, so as to determine the position information and posture information of the part to be tested in the actual space; According to the part features of the part to be measured and the position information and posture information of the part to be measured in the actual space, the position of the free edge of the part to be measured on the machine tool of the ship is determined, and the free edge represents the non-assembly edge.

8. The method according to claim 1, characterized in that The method further comprises: If the matching result is successful, the free edge of the part to be tested on the ship machine tool is set as a grinding edge, and the grinding edge is ground; If the matching result is a matching failure, the polishing is terminated.

9. A part free edge visual matching system based on model matching, characterized in that: include: The parsing module is used to parse the assembled three-dimensional model or the cut-plate two-dimensional model of the ship's parts, determine the two-dimensional contour and features of the parts, and establish a parts model database; A visual matching template database establishment module is used to generate a visual matching template according to the two-dimensional contour of the part and the part features, and determine a visual matching template database; An image acquisition module, used for acquiring images of the parts to be tested; A feature extraction module is used to perform feature extraction processing on the image of the part to be tested to determine the part features of the part to be tested; A model matching module, used to match the part features of the part to be tested with the visual matching template database to determine a matching result; A pixel coordinate determination module, for determining the pixel coordinates of the part to be tested in the image of the part to be tested according to a visual matching template corresponding to the image of the part to be tested if the matching result is a successful match; The free edge positioning module is used to convert the pixel coordinates of the part to be tested in the image of the part to be tested into actual coordinates in the physical space, and determine the position of the free edge of the part to be tested on the machine tool of the ship.

10. A part free edge visual matching system based on model matching, characterized in that: include: Server and visual acquisition device; The server includes a data storage system and a visual acquisition device interface, and the server is used to store and process three-dimensional model data, image data, and free edge positioning information; The visual acquisition device communicates with the server via the visual acquisition device interface, and the visual acquisition device is used to acquire images of the parts to be tested.

Images