Part image processing method and system based on multi-camera computer vision

Abstract

The invention relates to the technical field of image processing, and particularly discloses a part image processing method and system based on multi-camera computer vision, and the method comprises the following steps: setting three 2D cameras based on a preset position relation, obtaining an image of a part based on the 2D cameras, recording the image as a sub-image, and generating a sub-image set; acquiring a subimage set of the part in any posture, and grouping the subimage set; and obtaining an undetermined image set, obtaining a group F where the undetermined image set is located, and judging the quality of the part based on the group F. According to the invention, the quality of parts can be efficiently and accurately detected.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method and system for processing component images based on multi-camera computer vision. Background Art

[0002] With the gradual systematization and modularization of the supporting supply of automotive component manufacturers to vehicle manufacturers, the industry's requirements for the quality inspection of components are also increasing day by day. Geometric tolerances on different reference planes are important parameters in length measurement and need to meet high measurement accuracy.

[0003] However, due to differences in production processes and quality control capabilities, the quality of components in the same batch may vary, which not only affects the economic benefits of component manufacturing enterprises but also impacts subsequent vehicle production. Therefore, it is necessary to conduct full inspection of automotive components to ensure that each product meets production requirements. Currently, 3D imaging instruments or coordinate measuring equipment are generally used for multi-reference measurement. However, such standardized equipment is often large in size, and due to its characteristics as a standard device, it is difficult to be flexibly integrated or installed in non-standard equipment, so it is subject to certain limitations in practical applications. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and system for processing component images based on multi-camera computer vision to solve the following technical problems:

[0005] 3D imaging instruments or coordinate measuring equipment are generally used for multi-reference measurement. However, such standardized equipment is often large in size, and due to its characteristics as a standard device, it is difficult to be flexibly integrated or installed in non-standard equipment, so it is subject to certain limitations in practical applications.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] A method for processing component images based on multi-camera computer vision includes the following steps:

[0008] Set three 2D cameras based on a preset positional relationship, obtain images of the component based on the 2D cameras, record them as sub-images, and generate a sub-image set;

[0009] Obtain the sub-image set of the component in any posture and group the sub-image set;

[0010] Obtain the sub-images of the component to be detected, generate a sub-image set, record it as a pending image set, obtain the group F where the pending image set is located, and judge the quality of the component based on the group F.

[0011] As a further solution of the present invention: The process of generating the sub-image set includes:

[0012] Name the two 2D cameras as 2D camera a1 and 2D camera a2 respectively. Then the sub-image set JH = (A1, A2, A3). The sub-image set is an ordered set, and A1, A2, and A3 respectively represent the sub-images captured by the 2D camera a1, the 2D camera a2, and the 2D camera a3.

[0013] As a further solution of the present invention: Grouping the sub-image set includes:

[0014] Obtain the sub-image at the b-th position in the sub-image set, denoted as the comparison image. Perform grayscale processing on the comparison image to obtain a grayscale image;

[0015] Obtain the grayscale values of the pixel points in the same row of the grayscale image. Number the pixel points in order from left to right to generate coordinate points (i, Di), where Di represents the grayscale value of pixel point i, and i represents the number of the pixel point corresponding to Di. Remove the coordinate points with grayscale values greater than the preset grayscale value threshold, and perform fitting on the remaining coordinate points to obtain a fitting curve f(x), where x represents the number of the pixel point;

[0016] Obtain the fitting curves corresponding to the pixel points in the same row of the grayscale images x1 and x2, denoted as comparison curves. Obtain the Euclidean distance between the comparison curves, and calculate the total Euclidean distance as the judgment value;

[0017] If the judgment value is less than the preset judgment value threshold, it is determined that the comparison images X1 and X2 corresponding to the grayscale images x1 and x2 are similar;

[0018] If the comparison images at the same positions in the sub-image sets J1 and J2 are all similar, then put the sub-image sets J1 and J2 into the same group.

[0019] As a further solution of the present invention: Obtain the group F where the to-be-determined image set is located. Judging the quality of the component based on the group F includes:

[0020] Obtain the standard set in the group, and the total judgment value between the standard set and the remaining sub-image sets in the group is the smallest;

[0021] Obtain the judgment value between the to-be-determined image set and the standard set, denoted as the screening value. Take the group corresponding to the minimum value in the screening values as the group F where the to-be-determined image set is located;

[0022] Obtain the judgment values between the to-be-determined image set and all the sub-image sets in the group F as the target values, and obtain the Euclidean distance corresponding to the target values as the judgment distance;

[0023] Count the proportion of the determined distances that are greater than or equal to C1. If the proportion is greater than 0.4, it is determined that the quality of the component is unqualified, where C1 represents a preset determination threshold.

[0024] As a further solution of the present invention: Based on a preset positional relationship, three 2D cameras are set up, including:

[0025] The 2D camera a1 and the 2D camera a2 are in the same plane;

[0026] The connection line between the projection point of the position where the 2D camera a3 is located on this plane and the position where the 2D camera a3 is located is perpendicular to this plane;

[0027] The distances between the 2D camera a1 and the 2D camera a2, between the 2D camera a2 and the 2D camera a3, and between the 2D camera a1 and the 2D camera a3 are all preset values.

[0028] As a further solution of the present invention: When two or more screening values are the same and the smallest, the group with the smallest number of corresponding sub-image sets is taken as the group F.

[0029] As a further solution of the present invention: Before generating the sub-image sets, the sub-images are denoised and enhanced.

[0030] A component image processing system based on multi-camera computer vision includes:

[0031] A layout module: Based on a preset positional relationship, three 2D cameras are set up, and images of components are obtained based on the 2D cameras, denoted as sub-images, and sub-image sets are generated;

[0032] A grouping module: Obtain the sub-image sets of the components in any posture and group the sub-image sets;

[0033] A judgment module: Obtain the sub-images of the component to be detected, generate a sub-image set, denoted as a pending image set, obtain the group F where the pending image set is located, and judge the quality of the component based on the group F.

[0034] The beneficial effects of the present invention: Compared with the prior art:

[0035] 1) In traditional visual inspection, in order to obtain unified and comparable image information, it is often necessary to manually place and position the workpieces accurately. Otherwise, due to the rotation or translation of the parts, the images will vary greatly, affecting the inspection results. In this solution, by obtaining sub-images of the parts under multiple angles and multiple cameras and using the similarity of gray curves for comparative analysis, reliable inspection results can still be obtained even when the parts are not strictly aligned. This significantly reduces the dependence on the accuracy of manual placement, reduces the additional positioning time-consuming and the difficulty of manual intervention, making the entire inspection process more flexible and efficient;

[0036] 2) After allocating similar sub-image sets to the same group, the sub-image set generated by the parts to be inspected only needs to confirm which group it belongs to first, rather than comparing with all sub-images in all groups one by one. This can significantly save the number of comparison calculations, avoid "all-to-all" comparisons among a large number of sub-images, and thus greatly improve the operation efficiency of the system. Especially when there are many types or quantities of parts in the actual production environment, the role of this grouping method in saving resources is more obvious;

[0037] 3) When initially grouping, the strategy of "removing pixels with high gray values" is mainly adopted because the parts are often in the center of the image and their colors are usually not too bright, and the corresponding gray values in the gray space will be relatively low; while high-gray-value pixels generally correspond to the background or highlight areas, which are not the main information of the parts. By removing these high-gray pixels with little comparative significance and only retaining the pixel regions that are representative and have differences for inspecting the parts, the fitting curve can focus more on the characteristics of the parts themselves, thereby improving the accuracy and efficiency of the fitting curve calculation. Since the number of curve points is reduced and the data distribution is relatively concentrated, the fitting model is simpler and more stable. Therefore, the calculation of the Euclidean distance between subsequent comparison curves is faster and the results are more reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The present invention will be further described below with reference to the accompanying drawings.

[0039] Figure 1 is a schematic flowchart of the method for processing images of parts based on multi-camera computer vision of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0041] Please refer toFigure 1 As shown in Figure 1 , the present invention is a method for processing part images based on multi-camera computer vision, including the following steps:

[0042] Set three 2D cameras based on a preset positional relationship, obtain images of parts based on the 2D cameras, record them as sub-images, and generate a set of sub-images;

[0043] In a preferred embodiment of the present invention, before generating the set of sub-images, denoise and enhance the sub-images;

[0044] In a preferred embodiment of the present invention, the process of generating the set of sub-images includes:

[0045] Name two of the 2D cameras as 2D camera a1 and 2D camera a2 respectively, then the set of sub-images JH = (A1, A2, A3), the set of sub-images is an ordered set, and A1, A2, and A3 respectively represent the sub-images taken by 2D camera a1, 2D camera a2, and 2D camera a3;

[0046] In a preferred case of this embodiment, setting three 2D cameras based on a preset positional relationship includes:

[0047] 2D camera a1 and 2D camera a2 are in the same plane;

[0048] The connection line between the projection point of the position of 2D camera a3 on this plane and the position of 2D camera a3 is perpendicular to this plane;

[0049] The distances between 2D camera a1 and 2D camera a2, between 2D camera a2 and 2D camera a3, and between 2D camera a1 and 2D camera a3 are all preset values;

[0050] It should be noted that 2D camera a1 and 2D camera a2 are arranged on the same horizontal plane, so that their central axes are generally parallel to this plane; a preset horizontal distance is maintained between them to form a certain baseline length in this plane, so as to obtain plane images at different angles, and a certain degree of stereo vision correction can be performed as needed;

[0051] 2D camera a3 is above the plane where the 2D cameras are located, and the connection line between the projection point of its position on this plane and the position of 2D camera a3 itself is perpendicular to this plane. That is to say, from a side view, 2D camera a3 will be higher than the plane where the 2D cameras are located to obtain three-dimensional information or depth data of the parts;

[0052] A light source can be set on the other side or near the opposite side of the 2D camera and the 2D camera a3 (diffuse reflection or multi-directional light source can be used according to actual requirements) to ensure the lighting effect of the components and avoid obvious shadows or overexposed areas; and the distances between the 2D camera a1 and the 2D camera a2, between the 2D camera a2 and the 2D camera a3, and between the 2D camera a1 and the 2D camera a3 are all set to preset values that have been debugged and fixed in advance in experiments or on the production line (for example, determined after comprehensive evaluation based on the size of the components, the required resolution, the camera field of view angle, etc.);

[0053] Obtain a set of sub-images of the component in any pose and group the set of sub-images;

[0054] In another preferred embodiment of the present invention, grouping the set of sub-images includes:

[0055] Obtain the sub-image at the b-th position in the set of sub-images, denoted as the comparison image, perform gray-scale processing on the comparison image to obtain a gray-scale image;

[0056] Obtain the gray-scale values of the pixel points in the same row of the gray-scale image, number the pixel points in order from left to right to generate coordinate points (i, Di), where Di represents the gray-scale value of pixel point i and i represents the number of the pixel point corresponding to Di. Remove the coordinate points with gray-scale values greater than the preset gray-scale value threshold, and perform fitting on the remaining coordinate points to obtain a fitting curve f(x), where x represents the number of the pixel point;

[0057] Obtain the fitting curves corresponding to the pixel points in the same row of the gray-scale images x1 and x2, denoted as the comparison curves, obtain the Euclidean distance between the comparison curves, and calculate the total Euclidean distance as the judgment value;

[0058] If the judgment value is less than the preset judgment value threshold, it is determined that the comparison images X1 and X2 corresponding to the gray-scale images x1 and x2 are similar;

[0059] If the comparison images at the same positions in the set of sub-images J1 and the set of sub-images J2 are all similar, then put the set of sub-images J1 and the set of sub-images J2 into the same group;

[0060] It should be noted that, by way of example, first, the first image in J1 is converted into a grayscale image. Assuming the image size is 640×480 pixels, in order to extract the features of the part, we select a certain middle row, such as the 240th row. Along this row, each pixel point is numbered sequentially from left to right, with the number i ranging from 1 to 640, and the corresponding grayscale value is obtained. Then, a grayscale threshold is set, for example, 200, with the aim of filtering out those pixel points with higher grayscale values, because these pixel points may belong to the background or the highlighted area, while the part itself is usually located in the center of the image and has a lower grayscale. Thus, for the 240th row, if the grayscale value of a certain pixel is greater than 200, that point is discarded, and the remaining pixel points form a set of data points, such as (45, 120), (46, 118), (47, 122), etc. Then, we fit these remaining data points to obtain a fitting curve f(x), which reflects the grayscale change trend of the part edge or the main features in this row; next, the same processing is repeated for the corresponding position image in J2 to obtain the fitting curve g(x). After obtaining f(x) and g(x), the Euclidean distance between the two curves at the same pixel position is calculated to obtain the judgment value D; when all the images at the corresponding positions in the two sub-image sets satisfy this similarity condition, it is considered that the two sets are consistent in overall features and thus classified into the same group. This grouping method uses the sub-images obtained in any pose as the standard (each pose corresponds to a sub-image set). Subsequently, for a new part, it only needs to compare the generated sub-image set with the corresponding set in the standard library to quickly determine whether it is qualified;

[0061] Obtain the sub-images of the parts to be detected, generate a sub-image set, denoted as the to-be-determined image set, obtain the group F where the to-be-determined image set is located, and judge the quality of the parts based on the group F;

[0062] In a preferred case of this embodiment, obtaining the group F where the to-be-determined image set is located and judging the quality of the parts based on the group F includes:

[0063] Obtain the standard set in the group, where the total judgment value between the standard set and the remaining sub-image sets in the group is the smallest;

[0064] Obtain the judgment value between the to-be-determined image set and the standard set, denoted as the screening value, and take the group corresponding to the minimum value in the screening values as the group F where the to-be-determined image set is located;

[0065] Obtain the judgment values between the to-be-determined image set and all the sub-image sets in the group F as the target values, and obtain the Euclidean distance corresponding to the target values as the judgment distance;

[0066] Count the proportion of the determined distances greater than or equal to C1. If the proportion is greater than 0.4, it is determined that the quality of the component is unqualified, where C1 represents a preset determination threshold;

[0067] It should be noted that first, within each group, calculate the total determination value between the sub-image sets within the group (i.e., the difference value obtained by the gray-scale line fitting and Euclidean distance methods described above). For example, if there are three sub-image sets J1, J2, and J3 in a group, we calculate the determination values between J1 and J2, J1 and J3, and J2 and J3 respectively, and then accumulate the determination values of each set with other sets. The set with the smallest total determination value is selected as the standard set for that group because it has the highest similarity with other sets within the group and can represent the standard image features of the part in that pose;

[0068] After that, compare the to-be-determined image set with the respective standard sets in each group to obtain corresponding screening values. For example, assume that the determination values of the to-be-determined image set with the standard sets in Group 1, Group 2, and Group 3 are 40, 35, and 50 respectively. Then the group corresponding to the smallest screening value of 35 is determined as the group F to which the to-be-determined image set belongs;

[0069] After determining Group F, it is further necessary to quantify the difference between the to-be-determined image set and all sub-image sets in Group F. After calculating the determination values of the to-be-determined image set with each sub-image set in Group F respectively, a series of target values are obtained, and then these target values are converted into Euclidean distances. For example, if the Euclidean distances between the to-be-determined image set and four sub-image sets in Group F are 30, 45, 55, and 20 respectively, then these distance values are taken out and compared with the preset determination threshold C1. Assume that C1 is 50, then only the value of 55 is greater than or equal to C1, and the proportion is 1 / 4, that is, 0.25. If in the statistical result, the proportion of the Euclidean distances greater than or equal to C1 exceeds 0.4, it can be determined that the quality of the component is unqualified; otherwise, the quality of the component is considered qualified;

[0070] Using the sub-image sets in each pose that have been collected as standards, first select the standard set that is closest to other sets within the group, and then compare the to-be-determined image set with each standard set, so as to quickly locate the most matching group F. Then, within this group, conduct a detailed comparison of all sub-image sets, quantify and judge the difference between the to-be-determined image set and the standard, and judge whether the quality of the component meets the requirements through the preset threshold and proportion standard. The entire process not only ensures that the image data collected for the component in any pose can participate in the determination, but also enables the quality inspection of new parts without requiring the component to be placed correctly, but relying on the sub-image sets corresponding to each pose in the standard library for comparison, thereby improving the detection efficiency and accuracy;

[0071] It should be noted that when two or more screening values are the same and the smallest, the group with the smallest number of corresponding sub-image sets is taken as the group F.

[0072] A component image processing system based on multi-camera computer vision, comprising:

[0073] A layout module: setting three 2D cameras based on a preset positional relationship, obtaining images of components based on the 2D cameras, recording them as sub-images, and generating a sub-image set;

[0074] A grouping module: obtaining the sub-image set of the component in any posture and grouping the sub-image set;

[0075] A judgment module: obtaining the sub-images of the component to be detected, generating a sub-image set, recording it as a pending image set, obtaining the group F where the pending image set is located, and judging the quality of the component based on the group F.

[0076] The above has described an embodiment of the present invention in detail, but the above content is only a preferred embodiment of the present invention and cannot be considered as limiting the scope of implementation of the present invention. All equivalent changes and improvements made according to the scope of the application of the present invention should still fall within the scope covered by the patent of the present invention.

Claims

1. A method for processing part images based on multi-camera computer vision, characterized in that, Including the following steps: Set three 2D cameras based on a preset positional relationship, obtain images of the components based on the 2D cameras, record them as sub-images, and generate a sub-image set; Obtain the sub-image set of the components in any pose, and group the sub-image set; Obtain the sub-images of the components to be detected, generate a sub-image set, record it as the pending image set, obtain the group F where the pending image set is located, and judge the quality of the components based on the group F.

2. The method for processing component image based on multi-camera computer vision according to claim 1, wherein, The process of generating the sub-image set includes: Name the 2D cameras as 2D camera a1, 2D camera a2, and 2D camera a3 respectively. Then the sub-image set JH = (A1, A2, A3). The sub-image set is an ordered set, and A1, A2, and A3 respectively represent the sub-images taken by the 2D camera a1, the 2D camera a2, and the 2D camera a3.

3. The method for processing component image based on multi-camera computer vision according to claim 1, wherein Grouping the sub-image set includes: Obtain the sub-image at the b-th position in the sub-image set, record it as the comparison image, perform grayscale processing on the comparison image to obtain a grayscale image; Obtain the grayscale values of the pixel points in the same row of the grayscale image, number the pixel points in order from left to right to generate coordinate points (i, Di), where Di represents the grayscale value of pixel point i, and i represents the number of the pixel point corresponding to Di. Remove the coordinate points with grayscale values greater than the preset grayscale value threshold, and fit the remaining coordinate points to obtain a fitting curve f(x), where x represents the number of the pixel point; Obtain the fitting curves corresponding to the pixel points in the same row of the grayscale images x1 and x2, record them as comparison curves, obtain the Euclidean distance between the comparison curves, and calculate the total Euclidean distance as the judgment value; If the judgment value is less than the preset judgment value threshold, it is determined that the comparison images X1 and X2 corresponding to the grayscale images x1 and x2 are similar; If the comparison images at the same positions in the sub-image set J1 and the sub-image set J2 are all similar, then put the sub-image set J1 and the sub-image set J2 into the same group.

4. The method for processing component image based on multi-camera computer vision according to claim 3, characterized in that Obtain the group F where the pending image set is located, and judge the quality of the components based on the group F includes: Obtain the standard set in the group, where the total judgment value between the standard set and the remaining sub-image sets in the group is the smallest; Obtain the judgment value between the pending image set and the standard set, record it as the screening value, and take the group corresponding to the minimum value in the screening value as the group F where the pending image set is located; Obtain the judgment values between the pending image set and all sub-image sets in the group F as the target values, and obtain the Euclidean distance corresponding to the target values as the judgment distance; Count the proportion of the judgment distances greater than or equal to C1. If the proportion is greater than 0.4, it is determined that the quality of the components is unqualified, and C1 represents the preset judgment threshold.

5. The component image processing method based on multi-camera computer vision according to claim 2, wherein, Setting three 2D cameras based on a preset positional relationship includes: The 2D camera a1 and the 2D camera a2 are in the same plane; The line connecting the projection point of the position of the 2D camera a3 on the plane and the position of the 2D camera a3 is perpendicular to the plane; The distances between the 2D camera a1 and the 2D camera a2, between the 2D camera a2 and the 2D camera a3, and between the 2D camera a1 and the 2D camera a3 are all preset values.

6. The component image processing method based on multi-camera computer vision according to claim 4, wherein, When two or more screening values are the same and the smallest, the group with the smallest number of corresponding sub-image sets is taken as the group F.

7. The method for processing component image based on multi-camera computer vision according to claim 1, wherein Before generating the sub-image set, denoise and enhance the sub-images.

8. A component image processing system based on multi-camera computer vision, characterized in that, Including: Layout module: Set three 2D cameras based on a preset positional relationship, obtain an image of a component based on the 2D cameras, record it as a sub-image, and generate a sub-image set; Grouping module: Obtain the sub-image set of the component in any posture, and group the sub-image set; Judgment module: Obtain the sub-image of the component to be detected, generate a sub-image set, record it as a pending image set, obtain the group F where the pending image set is located, and judge the quality of the component based on the group F.

Images