Image recognition method and device, equipment and medium
By combining watershed segmentation and mask enhancement with image moment feature classification, the accuracy problem of identifying highly reflective metal parts was solved, achieving high-precision segmentation and recognition in complex industrial scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-25
- Publication Date
- 2026-04-10
AI Technical Summary
In existing technologies, it is difficult to recognize stacked metal parts, especially to accurately segment and recognize highly reflective metal parts. Traditional image processing methods are easily affected by reflections, while depth camera solutions are costly and have insufficient resolution.
Bright spot regions are extracted using watershed segmentation and mask enhancement, and bright spot types are classified by combining image moment features. Target regions are generated for identification by using a preset region growth similarity criterion and line fitting.
It improves the segmentation accuracy and recognition robustness of images of stacked metal parts in complex backgrounds, reduces costs, and accurately restores the geometry and spatial position of metal parts under conditions of occlusion, overlap and reflective interference.
Smart Images
Figure CN121837702A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of image recognition, and in particular to an image recognition method, device, equipment and medium. BACKGROUND
[0002] In the field of industrial automation detection, the recognition of the stacking scene of metal parts has always been a technical difficulty, especially the accurate segmentation and recognition of high-reflective metal parts (such as chrome-plated bolts and shaft parts). The image processing method in the related art mainly includes two categories: the first category is a method based on traditional image processing, such as a watershed algorithm combined with feature matching. This kind of method performs well on non-reflective objects, but is sensitive to metal high-reflective areas and is easily disturbed by reflection, resulting in over-segmentation or mismatching. The second category is a three-dimensional point cloud scheme based on a depth camera. Although three-dimensional information can partially overcome the reflection problem, it has defects such as high hardware cost and insufficient resolution for small-sized parts. Therefore, there is an urgent need for a low-cost and high-robustness image recognition method that can effectively utilize the metal reflection characteristics rather than avoiding them, thereby realizing accurate segmentation and recognition of stacked metal part images under monocular vision conditions. SUMMARY
[0003] The present application provides an image recognition method, device, equipment and medium to solve the technical problem of poor accuracy of recognition of stacked metal part images in the related art.
[0004] In a first aspect, an image recognition method is provided, comprising: performing a preprocessing operation on a collected stacked metal part image to obtain a bright spot area, and performing connected domain labeling on the bright spot area to obtain a bright spot connected domain; wherein the preprocessing operation at least includes a watershed segmentation processing and a mask enhancement processing; classifying the bright spot connected domain based on image moment features to obtain a bright spot feature set containing point-like bright spots, line-like bright spots and strip-like bright spots; wherein the image moment features include a bright spot area parameter, a bright spot center of gravity coordinate, a bright spot principal axis direction angle and a bright spot length; grouping the bright spot feature set according to a preset region growing similarity criterion to obtain an initial bright spot grouping, and performing straight line fitting based on the initial bright spot grouping to obtain a target grouping including several groups of similar metal parts and target axis parameters corresponding to the similar metal parts; generating a target region according to the target axis parameters and corresponding prior sizes, and performing recognition on the stacked metal part image in the target region to obtain a recognition result.
[0005] In a second aspect, an image recognition device is provided, comprising: The marking module is configured to perform a preprocessing operation on the collected stacked metal part image to obtain a highlight region, and perform connected domain marking on the highlight region to obtain a highlight connected domain; wherein the preprocessing operation at least includes a watershed segmentation processing and a mask enhancement processing. The classification module is configured to classify the highlight connected domain based on image moment features to obtain a highlight feature set including point-like highlights, line-like highlights and band-like highlights; wherein the image moment features include a highlight area parameter, a highlight barycenter coordinate, a highlight principal axis direction angle and a highlight length. The grouping module is configured to group the highlight feature set according to a preset region growing similarity criterion to obtain an initial highlight grouping, and perform straight line fitting based on the initial highlight grouping to obtain a target grouping including a plurality of groups of similar metal parts and target axis parameters corresponding to the similar metal parts. The recognition module is configured to generate a target region according to the target axis parameters and corresponding prior dimensions, and perform recognition on the stacked metal part image in the target region to obtain a recognition result.
[0006] In a third aspect, a computer device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the above image recognition method when executing the computer program.
[0007] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program implements the steps of the above image recognition method when executed by a processor.
[0008] The above-mentioned image recognition method, apparatus, computer equipment, and storage medium implement a scheme in which the method includes: preprocessing the acquired image of stacked metal parts to obtain bright spot regions, and marking the bright spot regions with connected components to obtain bright spot connected components; wherein the preprocessing operation includes at least watershed segmentation and mask enhancement; classifying the bright spot connected components based on image moment features to obtain a set of bright spot features including point-like bright spots, line-like bright spots, and strip-like bright spots; wherein the image moment features include bright spot area parameters, bright spot centroid coordinates, bright spot principal axis direction angle, and bright spot length; grouping the bright spot feature set according to a preset region growth similarity criterion to obtain initial bright spot groups, and performing line fitting based on the initial bright spot groups to obtain target groups including several groups of similar metal parts and target axis parameters corresponding to similar metal parts; generating target regions according to the target axis parameters and corresponding prior dimensions, and recognizing the image of stacked metal parts within the target regions to obtain recognition results. In this invention, by introducing watershed segmentation and mask enhancement in the preprocessing operation, the bright spot regions are accurately highlighted in complex backgrounds. Furthermore, image moment features can be used to achieve high-precision classification of point-like, line-like, and strip-like bright spots. Based on a preset region growth similarity criterion combined with straight-line fitting, the geometric shape and spatial position of similar metal parts can be accurately restored even in the presence of occlusion, overlap, and reflective interference. In addition, target regions are generated using prior dimensions and target axis parameters, effectively eliminating background noise and achieving spatially independent separation of multiple targets. This enables the recognition of stacked metal parts images within the target region, improving recognition accuracy. This method significantly improves the segmentation accuracy and recognition robustness of stacked metal part images in complex industrial scenarios while maintaining the low cost of monocular vision. Attached Figure Description
[0009] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0010] Figure 1 This is a schematic diagram of an application environment for an image recognition method according to an embodiment of the present invention; Figure 2 This is a schematic flowchart of an image recognition method according to an embodiment of the present invention; Figure 3 This is a scene diagram of the bright spot area in step S10 provided by the present invention; Figure 4 yes Figure 2 A schematic diagram of a specific implementation method for step S20; Figure 5 is a scene schematic diagram of linear bright spots and strip bright spots provided by the present application; Figure 6 is a scene schematic diagram of initial bright spot grouping in step S30 provided by the present application; Figure 7 is a scene schematic diagram of a target region in step S40 provided by the present application; Figure 8 is another scene schematic diagram of a target region in step S40 provided by the present application; Figure 9 is a scene schematic diagram of recognition results in step S40 provided by the present application; Figure 10 is a structure schematic diagram of an image recognition device in an embodiment of the present application; Figure 11 is a structure schematic diagram of a computer device in an embodiment of the present application; Figure 12 is another structure schematic diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION
[0011] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.
[0012] The image recognition method provided by the embodiments of the present application can be applied in, for example, Figure 1In an application environment of the application, the client communicates with the server through a network. The server can perform a preprocessing operation on the collected stacked metal part image through the client to obtain a highlight area, and perform connected domain labeling on the highlight area to obtain a highlight connected domain. The preprocessing operation at least includes watershed segmentation processing and mask enhancement processing. Based on image moment features, the highlight connected domain is classified to obtain a highlight feature set containing point-like highlights, line-like highlights and strip-like highlights. The image moment features include highlight area parameters, highlight barycenter coordinates, highlight principal axis direction angle and highlight length. A preset region growing similarity criterion is used to group the highlight feature set to obtain an initial highlight grouping, and based on the initial highlight grouping, straight line fitting is performed to obtain a target grouping including several groups of similar metal parts and target axis parameters corresponding to the similar metal parts. According to the target axis parameters and corresponding prior dimensions, a target region is generated, and the stacked metal part image is recognized in the target region to obtain a recognition result. In the application, by introducing watershed segmentation and mask enhancement in the preprocessing operation, the highlight area can be accurately highlighted in a complex background. Further, the image moment features can be used to realize high-precision classification of point-like highlights, line-like highlights and strip-like highlights, and based on the preset region growing similarity criterion combined with straight line fitting, the geometric shape and spatial position of similar metal parts can be accurately restored in the presence of occlusion, overlap and reflection interference. In addition, by generating a target region through prior dimensions and target axis parameters, background noise can be effectively removed and spatial independent separation of multiple targets can be realized, so that the stacked metal part image can be recognized in the target region, improving the recognition accuracy. The method significantly improves the segmentation accuracy and recognition robustness of the stacked metal part image in a complex industrial scene while maintaining the low cost of monocular vision. The client can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices. The server can be implemented by an independent server or a server cluster composed of multiple servers. The application will be described in detail below through specific embodiments.
[0013] Referring to Figure 2 as shown, Figure 2 A flowchart of an image recognition method provided by an embodiment of the application is shown. The method includes the following steps: S10: Perform a preprocessing operation on the collected stacked metal part image to obtain a highlight area, and perform connected domain labeling on the highlight area to obtain a highlight connected domain.
[0014] The preprocessing operation at least includes watershed segmentation processing and mask enhancement processing.
[0015] In some embodiments, the preprocessing operation on the collected stacked metal part image to obtain the highlight area comprises: performing noise suppression on the stacked metal part image by Gaussian filtering to obtain a noise-suppressed image; enhancing the contrast of the noise-suppressed image by histogram equalization to obtain an enhanced image; performing gamma correction on the enhanced image to obtain a preprocessed image; and sequentially performing a watershed segmentation processing and a mask enhancement processing on the preprocessed image to obtain the highlight area.
[0016] It should be noted that the stacked metal part image is an original image collected in an industrial scene by an imaging device. The original image can include a plurality of metal parts with high light reflection characteristics, and there can be occlusion and stacking relationship between the metal parts, resulting in more background interference and blurred target edges.
[0017] For example, the stacked metal part image can be subjected to Gaussian filtering, histogram equalization and gamma correction, which are not limited in the present application. It should be understood that the Gaussian filtering can suppress the noise of the stacked metal part image and highlight the effective information of the edges of the stacked metal part image; the histogram equalization can enhance the contrast of the stacked metal part image and highlight the details of the stacked metal part image, thereby providing a better basis for subsequent image processing. The gamma correction can retain the distinction of dark details and brightness of the stacked metal part image, thereby reducing the interference of the background of the stacked metal part image. The gamma correction formula can specifically include:
[0018] wherein Gray represents the gray image obtained after gamma correction; R, G and B represent the pixel values of the red, green and blue channels of the stacked metal part image respectively; and γ=2.2.
[0019] The above embodiments can suppress the strong light reflection of the stacked metal part image and enhance the contrast by performing Gaussian filtering, histogram equalization and gamma correction on the stacked metal part image.
[0020] Please refer to Figure 3 , Figure 3 is a scene schematic diagram of the highlight area in step S10 provided by the present application. On the basis of the above embodiments, the stacked metal part image can be further subjected to a watershed segmentation processing and a mask enhancement processing. Specifically, the stacked metal part image can be first decomposed into a plurality of regions with different pixel characteristics to preliminarily separate the foreground region and the background region; and then the highlight of the light reflection on the surface of the metal part is visually highlighted by enhancing the target brightness characteristics and suppressing the background interference, i.e. obtaining the highlight area, thereby facilitating subsequent feature extraction.
[0021] Further, after obtaining the bright spot region, the bright spot region can be labeled in a connected domain to obtain a bright spot connected domain. It should be noted that the application does not limit the way of labeling the connected domain, for example, the bright spot region can be labeled in a connected domain by a seed filling algorithm, that is, a bright spot pixel higher than a set threshold is taken as a seed point, adjacent bright spot pixels in its eight-neighbor domain are recursively searched and classified, and all bright spot pixel regions are labeled until the labeling is completed. A scanning labeling method can also be used, for example, a two-pass scanning method, in which a temporary label is assigned according to the adjacency relationship of the pixels in the first scanning, and the equivalent label relationship is recorded, and in the second scanning, the equivalent labels are merged to obtain the final connected domain labeling result; the bright spot pixels can also be regarded as nodes in a graph structure based on a depth-first search or a breadth-first search method, and the adjacent pixels are gradually traversed and classified into corresponding connected domains; the union-find set data structure can also be used to dynamically merge the pixel sets having an adjacency relationship in the scanning process to achieve fast labeling.
[0022] In addition, for large-size images or scenes requiring efficient processing, a block parallel labeling strategy can also be used, the image is divided into a plurality of sub-blocks for connected domain analysis, and the labels are merged at the sub-block boundaries, so as to improve the processing efficiency while ensuring accuracy.
[0023] Through the above connected domain labeling processing, a plurality of bright spot connected domains that do not overlap and have independent identification can be obtained, which provides accurate pixel range division for subsequent feature classification and grouping processing.
[0024] As shown in FIG. 10, the step S10, that is, the labeling of the bright spot region in a connected domain to obtain a bright spot connected domain, includes the following steps: Figure 4 S11: determining each unmarked first bright spot pixel in the bright spot region as a seed point to obtain a current seed set containing a plurality of seed points.
[0025] S12: searching for unmarked second bright spot pixels in the eight-neighbor domain of each seed point in the seed set, and determining a plurality of the first seed points and the corresponding second bright spot pixels as a set of adjacent bright spot pixels.
[0026] S13: updating the current seed set based on the set of adjacent bright spot pixels to obtain a target seed set.
[0027] S14: determining the region corresponding to the target seed set as the bright spot connected domain.
[0028] For example, first, as shown in step S11, each unmarked highlight pixel in the highlight region can be determined as a first highlight pixel and added to the current seed set as a seed point. Then, according to step S12, for each seed point in the current seed set, it is detected whether there is an unmarked second highlight pixel within the eight-neighbor range, and if so, these second highlight pixels are classified into the adjacent highlight pixel set together with the corresponding first highlight pixel, thereby establishing the connectivity of the pixels in the local range.
[0029] Further, as shown in step S13, the current seed set can be replaced and updated by the adjacent highlight pixel set, so that the search range is continuously expanded to the adjacent region until no new highlight pixel is found, and a complete target seed set is obtained. Finally, according to step S14, the region corresponding to the target seed set is determined as an independent highlight connected domain. Through the above iteration and classification processing, the highlight region can be divided into multiple independent connected domains, which provides an accurate pixel range basis for subsequent feature classification and grouping.
[0030] S20: Classifying the highlight connected domain based on image moment features to obtain a highlight feature set containing point-like highlights, line-like highlights and strip-like highlights.
[0031] The image moment features include a highlight area parameter, a highlight barycenter coordinate, a highlight principal axis direction angle and a highlight length.
[0032] Please refer to Figure 5 , Figure 5 is a scene diagram of the line-like highlights and the strip-like highlights provided by the application. As shown in Figure 5 , the highlight connected domain can be classified based on image moment features to obtain a highlight feature set containing point-like highlights, line-like highlights and strip-like highlights. The point-like highlights are usually screw thread interval reflections, irregular independent pixel blocks; the line-like highlights are usually bolt head corner reflections, jagged short lines, vertical axis; the strip-like highlights are usually continuous highlight reflection highlights combined by the distance between the light source and the camera and the parts on the top of the cylindrical surface, and have greater length and width than the line-like highlight features.
[0033] It should be noted that the image moment features are used to describe the geometric shape and spatial distribution features of the highlight connected domain, including the highlight area parameter, the highlight barycenter coordinate, the highlight principal axis direction angle and the highlight length, and other information.
[0034] For example, the bright spot area parameter can be calculated by the zeroth moment to reflect the number of pixels of the bright spot; the bright spot gravity center coordinates can be calculated by the first moment to determine the position of the bright spot in the image plane; the bright spot principal axis direction angle can be calculated by the second moment to reflect the main extension direction of the bright spot; and the bright spot length can be calculated based on the extreme coordinates of the bright spot pixel points and in combination with Chebyshev distance or the like to reflect the linear or strip-shaped extension characteristics of the bright spot.
[0035] In addition, it needs to be noted that the classification method of the bright spot connected domain is not limited in the present application. For example, a rule-based threshold classification method can be adopted, that is, the bright spot connected domain is divided into three categories of point-like bright spot, linear bright spot and strip-like bright spot according to the numerical range of the bright spot area parameter, bright spot gravity center coordinates, bright spot principal axis direction angle and bright spot length and the like; a clustering analysis-based method can also be adopted to realize automatic classification by unsupervised clustering of the bright spot feature vector; and a machine learning or deep learning model can also be combined to train a classifier using labeled samples to perform pattern recognition on the bright spot feature set.
[0036] Through the above-mentioned various methods, the optimal classification strategy can be flexibly selected in different application scenarios to ensure the accuracy and stability of the bright spot feature set and provide reliable input data for subsequent grouping and axis fitting.
[0037] In some embodiments, the classification of the bright spot connected domain based on the image moment features to obtain the bright spot feature set containing point-like bright spots, linear bright spots and strip-like bright spots comprises: determining the zeroth moment, first moment and second moment of the bright spot connected domain respectively to obtain the bright spot area parameter, bright spot gravity center coordinates and bright spot principal axis direction angle in sequence; determining the extreme coordinates among the pixel point coordinates of the bright spot connected domain and determining the maximum distance between the extreme coordinates as the bright spot length; and classifying the bright spot connected domain according to the bright spot area parameter, bright spot gravity center coordinates, bright spot principal axis direction angle and bright spot length to obtain the bright spot feature set.
[0038] For example, the zeroth moment, first moment and second moment can be determined by the following formula to obtain the bright spot area parameter, bright spot gravity center coordinates and bright spot principal axis direction angle:
[0039]
[0040] wherein, M 00 represents the zeroth moment, that is, the bright spot area parameter; (M 01 , M 10 ) represents the first moment; (x0, y0) represents the bright spot gravity center coordinates; (M 11 , M 02 and M20 ) represents the second moment, and represents the direction angle of the main axis of the bright spot; μ pq represents the center distance; and pq represents the normalized center distance; x represents the horizontal coordinate of a pixel in the bright spot connected domain; y represents the vertical coordinate of the pixel in the bright spot connected domain; and 00 represents the zero-order center distance; a represents the stretching degree of the bright spot connected domain in the x direction; b represents the covariance of the bright spot connected domain; c represents the stretching degree of the bright spot connected domain in the y direction; p represents the moment order controlling the x direction; q represents the moment order controlling the y direction; and r represents the normalization index.
[0041] Further, the extreme values (X min , max Y min , max ) in the pixel coordinates (X, Y) in the bright spot connected domain can be determined, and four coordinate points having the extreme value coordinates are obtained according to the determined extreme values, the distances between the four points are calculated according to the Chebyshev distance, and the maximum distance between the extreme value coordinates is the length of the bright spot.
[0042] Referring to Figure 6 , Figure 6 is a scene schematic diagram of initial bright spot grouping in step S30 provided by the application. For example, the classification manner is not limited in the application, and a threshold division method based on rules can be used, for example, bright spots with small area parameters and short lengths are classified as point-like bright spots, bright spots with lengths and main axis direction angles meeting requirements are classified as linear bright spots, and bright spots with large area parameters and lengths are classified as strip-like bright spots. For example, a significance analysis can be performed based on the statistical distribution of the zero-order moment, the bright spots are divided into three intervals to realize coarse discrimination, and the bright spots are further finely discriminated in combination with the lengths or widths, wherein the region with a large zero-order moment but a small length is classified as a linear bright spot reflecting the position of a bolt head, so that efficient hierarchical identification and classification of bright spot features are realized.
[0043] For another example, bright spots with area parameters less than 20 pixels and lengths less than 15 pixels are classified as point-like bright spots, bright spots with lengths greater than 15 pixels and less than 50 pixels and main axis direction angle changes of no more than ±10° are classified as linear bright spots, and bright spots with area parameters greater than 50 pixels and lengths greater than 50 pixels are classified as strip-like bright spots.
[0044] The bright spot feature set obtained through classification can ensure the accuracy and applicability of the bright spot feature set, and provide reliable basic data for subsequent grouping and straight line fitting.
[0045] S30: grouping the set of bright spot features according to a preset region growing similarity criterion to obtain an initial bright spot grouping, and performing straight line fitting based on the initial bright spot grouping to obtain a target grouping including a plurality of groups of similar metal parts and target axis parameters corresponding to the similar metal parts.
[0046] It should be noted that the preset region growing similarity criterion can be determined based on the spatial position and morphological features of the bright spots. For example, when the center of gravity coordinate chessboard distance between the bright spots does not exceed 10 pixels and the difference in the main axis direction angle does not exceed ±10°, it can be considered that they belong to the same metal part. Further, when performing grouping, point-like bright spots can be preferentially selected as initial growing points to avoid grouping errors caused by excessive number of pixels in the strip-like bright spots. By gradually merging the bright spots that meet the similarity criterion until no new bright spot is added to the grouping, an initial bright spot grouping set can be formed, and each grouping corresponds to a group of bright spot features that may belong to the same metal part.
[0047] Further, since the metal part occlusion can cause the bright spot features of the same axis type part to be broken into two parts, if the center distance exceeds a preset threshold, the initial bright spot grouping will have a misjudgment as two independent target groups, thereby affecting the accuracy of the grouping. Therefore, straight line fitting can be performed based on the initial bright spot grouping to extract the geometric axis parameters of the metal part.
[0048] For example, the least squares method can be used to fit all bright spot pixel coordinates in the grouping to obtain the slope, intercept and endpoint coordinates of the fitted straight line. For the fitted straight lines with similar slope and intercept, merging can be performed to correct the axis segmentation problem caused by occlusion or breakage. Finally, each target grouping will correspond to a complete geometric axis, and the endpoint coordinates, center point coordinates and direction angle will be used as the basic parameters for generating the target region subsequently, thereby ensuring that the grouping result is highly consistent with the actual posture of the metal part.
[0049] Specifically, the following formula can be used for illustration: For bright spot pixel coordinates (x i ,y i ), assuming that the fitting model is y=f(x), the core idea of the least squares method is to find the f(x) function parameters when the target function L takes the minimum value. The target function L is shown in the following formula, that is, the fitting criterion is to minimize the square sum of the distance between y i and f(x i ):
[0050] For linear fitting of pixel points, the fitting model f(x)=kx+b, that is, to find the k value and b value that minimize L. The target function L in linear fitting is shown in the following formula:
[0051] After simplification and elimination operations, the values of k and b can be obtained as follows:
[0052] wherein L is a target function; k is an intercept; and b is an intercept.
[0053] It should be noted that the least square method is used to fit the straight line of the bright point pixel coordinates, and in the calculation process, attention should be paid to the difference between the image coordinate system (the origin is located at the upper left corner, and the Y axis is downward) and the conventional Cartesian coordinate system. Therefore, when calculating the direction angle of the main shaft, the negative value of the slope of the straight line is taken and the arctangent operation is performed. By merging the fitting straight line groups with similar slopes and intercepts, the fine grouping of the bright point features of the lower layer shaft type parts is realized, and the pixel coordinates at both ends of the fitting straight line are used to limit the interval and correct the precision of the straight line parameters. Based on this, the center axis of each target is constructed, the center point position (Xc, Yc) is calculated through the endpoint coordinates (X1, Y1) and (X2, Y2), and the angle between the axis and the positive direction of the image X axis (defined in the counterclockwise direction, the angle range is [0°, 180°]) is taken as the direction angle of the main shaft. Finally, through normalization processing, it is ensured that the angle value conforms to the specified range, so as to completely describe the spatial position and attitude features of the target.
[0054] In some embodiments, grouping the set of bright point features according to the preset region growing similarity criterion to obtain an initial bright point grouping comprises: taking a point bright point in the set of bright point features as a starting growing point, determining a bright point barycenter coordinate and a bright point main shaft direction angle of each starting growing point; determining a spatial distance and a difference value of the bright point main shaft direction angle between each starting growing point and a neighboring bright point; wherein the spatial distance is determined based on the bright point barycenter coordinates corresponding to the starting growing point and the neighboring bright point respectively; if the spatial distance is less than or equal to a first preset threshold and the difference value is less than or equal to a second preset threshold, the starting growing point and the neighboring bright point are merged into the same group to obtain a candidate bright point group; the other bright point features in the set of bright point features except the point bright point are grouped, and the grouping result is combined with the candidate bright point group to obtain the initial bright point grouping.
[0055] For example, point-like bright spots can be used as starting growth points, and the centroid coordinates and angle of the principal axis direction of each starting growth point can be determined. Further, the spatial distance between each starting growth point and its adjacent bright spots can be calculated based on the centroid coordinates. This spatial distance is measured using checkerboard distance or Euclidean distance to determine the proximity between bright spots. It should be noted that the initial bright spots are point-like bright spots with relatively few local pixels, which avoids grouping errors caused by excessive data volume during the region growth process of strip-shaped bright spots. Adjacent bright spots are bright spot features that satisfy certain spatial proximity relationships in the image. By comparing the spatial distance with a first preset threshold and the difference in the principal axis direction angle with a second preset threshold, it can be determined whether adjacent bright spots satisfy the region growth similarity criterion with the starting growth point.
[0056] It should be noted that the first preset threshold is used to measure the spatial proximity between the starting growth point and adjacent bright spots, usually based on the checkerboard distance or Euclidean distance calculated from the centroid coordinates of the bright spots. For example, if the first preset threshold is set to 10 pixels, the region growth criterion is only met when the distance between the centroid coordinates of two bright spots is less than or equal to 10 pixels, thus avoiding grouping failure caused by incorrect merging of bright spots that are too far apart. The second preset threshold is used to control the similarity of the main axis direction of the bright spots and is the standard for judging whether the difference in the direction of the bright spots is acceptable. For example, the second preset threshold can include 5°, 10°, and 15°, etc. The angle difference between the fitted straight line direction of the starting growth point and the adjacent bright spots is calculated and compared with the second preset threshold. If the angle difference is less than or equal to the second preset threshold, the directions are considered similar and can be merged into the same candidate bright spot group; otherwise, merging is not performed. The second preset threshold ensures the directional consistency of bright spots in the same group, thereby reflecting the linear arrangement characteristics of shaft parts along the main axis direction, which helps to eliminate noisy bright spots that do not match the direction of the target shaft parts.
[0057] Furthermore, based on the above determination results, the initial growth points that meet the spatial distance threshold and direction difference threshold conditions are merged with adjacent bright spots to form candidate bright spot groups. Further, a similarity merging operation can be performed on other bright spot features in the bright spot feature set, excluding point-like bright spots, and the resulting grouping results are integrated with the candidate bright spot groups to finally generate initial bright spot groups.
[0058] It should be understood that the initial bright spot grouping reflects the bright spot feature group belonging to the same shaft type parts. The grouping process utilizes the features of bright spots being close in distance, having similar main axis directions, and having approximately collinear pixels. This helps to accurately identify shaft type parts located in the upper layer of the scene that are not occluded, and to achieve reliable classification of bright spot features in the image and subsequent identification of shaft type parts.
[0059] S40: Generate a target region based on the target axis parameters and the corresponding prior dimensions, and identify the image of the stacked metal parts within the target region to obtain the identification result.
[0060] In some embodiments, the generating the target region according to the target axis parameter and the corresponding prior size, and identifying the stacked metal part image in the target region to obtain an identification result, comprises: determining the vertex coordinates of the target region according to the target axis parameter and the prior size; generating a rectangular mask with the target axis as the symmetry axis according to the vertex coordinates, and determining the region corresponding to the rectangular mask as the target region; extracting a target sub-image in the stacked metal part image in the target region to identify the stacked metal part image to obtain an identification result.
[0061] Referring to Figures 7 to 9 , Figure 7 is a scene schematic diagram of the target region in step S40 provided by the application; Figure 8 is another scene schematic diagram of the target region in step S40 provided by the application; Figure 9 is a scene schematic diagram of the identification result in step S40 provided by the application.
[0062] For example, the target axis parameter and the corresponding prior size information can be obtained respectively, wherein the target axis parameter includes the slope, intercept, endpoint coordinates and center point coordinates of the fitted axis, and the prior size includes the diameter and length of the shaft part. Further, based on the axis parameter and the prior size, the diameter and the direction angle can be mapped to the image coordinate system through geometric calculation to obtain the four corner vertex coordinates of the rectangular target region.
[0063] It should be noted that the rectangular target region takes the target axis as the long symmetry axis and the center point as the symmetry center, and the vertex coordinates of the target region calculated by the formula can ensure that the rectangular mask and the actual shaft part in the image are highly matched in spatial position, direction and size, thereby providing a structured basis for subsequent target separation and identification. The vertex coordinates can be determined by vector transformation and rotation matrix method, ensuring that the rectangular side length is consistent with the diameter and length of the shaft part, and the rectangular direction is consistent with the direction of the target axis.
[0064] Specifically, the following formula can be used for illustration:
[0065] wherein D is the diameter; a is the slope; P1, P2, P3, P4 are the vertex coordinates of the target region; (x h ,y h ) are the center coordinates of the "head" end point (such as the head of a bolt) of the target region; (x1, y1) are the center coordinates of the "tail" end point (such as the tail of a bolt) of the target region Thus, the image region corresponding to the rectangular mask can be determined as the target region. Further, the sub-image in the stacked metal part image can be extracted in the target region, realizing accurate positioning and separation of the target region. By performing a bit operation between the rectangular mask and the original image, background interference can be removed, the spatial independent extraction of a single target can be realized, and reliable input for multi-target image recognition can be provided. It should be noted that the method uses parameterized geometric modeling to map the physical size information to a structured mask in image coordinates, and the high coincidence of the rectangular mask and the actual shaft type part is ensured by the center symmetry and the consistency of the long axis direction, so as to ensure the accuracy and reliability of the recognition result in the target region, and realize the spatial independence and accuracy of the multi-target recognition of the stacked parts.
[0066] For example, after obtaining the target region, the efficiency and accuracy can be improved by a local fine recognition strategy: for ordinary shaft type parts that do not need to distinguish the direction (such as optical shaft), the shaft segment region is directly taken as the positioning result; and for bolt type parts that need to distinguish the head and tail, the symmetric rectangular sub-region is extracted along the axis at both ends based on the prior head size (such as Figure 8 shown), and the head positioning is realized by comparing the local features at both ends. The specific comparison method is: if most of the target region is point-like bright pixel, it is considered that the end is adjacent to another target, and there is no bolt head; if the target region is a linear bright feature, it is considered that the end belongs to the target, which is the position of the bolt head.
[0067] Further, by the above method, the recognition result as shown in Figure 9 can be obtained. Among them, the red is the upper layer metal part, and the yellow is the metal part, which proves that the method proposed in the application realizes the improvement of the accuracy of the stacked metal part image.
[0068] As can be seen, in the above scheme, by introducing watershed segmentation and mask enhancement in the preprocessing operation, the bright spot region is accurately highlighted in the complex background. Further, the image matrix feature can be used to realize high-precision classification of point-like bright spots, linear bright spots and strip-like bright spots, and based on the preset region growing similarity criterion combined with straight line fitting, the geometric shape and spatial position of the same type of metal part can be accurately restored in the presence of shielding, overlapping and reflection interference. In addition, by generating a target region based on prior size and target axis parameters, background noise can be effectively removed and multi-target spatial independent separation can be realized, so that the stacked metal part image can be recognized in the target region, and the accuracy of the recognition is improved. The method keeps the low cost of monocular vision, and significantly improves the segmentation accuracy and recognition robustness of the stacked metal part image in complex industrial scenes.
[0069] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0070] In one embodiment, an image recognition device is provided, which corresponds one-to-one with the image recognition methods described in the above embodiments. For example... Figure 10 As shown, the image recognition device includes a labeling module 101, a classification module 102, a grouping module 103, and a recognition module 104. Detailed descriptions of each functional module are as follows: The labeling module 101 is used to perform preprocessing operations on the acquired images of stacked metal parts to obtain bright spot regions, and to label the bright spot regions with connected components to obtain bright spot connected components; wherein, the preprocessing operations include at least watershed segmentation processing and mask enhancement processing; The classification module 102 is used to classify the bright spot connected components based on image moment features to obtain a bright spot feature set including point bright spots, line bright spots, and band bright spots; wherein, the image moment features include bright spot area parameters, bright spot centroid coordinates, bright spot principal axis direction angle, and bright spot length; Grouping module 103 is used to group the set of bright spot features according to a preset region growth similarity criterion to obtain an initial bright spot group, and to perform line fitting based on the initial bright spot group to obtain a target group including several groups of similar metal parts and the target axis parameters corresponding to the similar metal parts. The recognition module 104 is used to generate a target region based on the target axis parameters and the corresponding prior dimensions, and to recognize the image of the stacked metal parts within the target region to obtain a recognition result.
[0071] The marking module 101 is used to suppress noise in the image of the stacked metal parts by Gaussian filtering to obtain a noise-suppressed image; enhance the contrast of the noise-suppressed image by histogram equalization to obtain an enhanced image; perform gamma correction on the enhanced image to obtain a preprocessed image; and perform watershed segmentation and mask enhancement processing on the preprocessed image in sequence to obtain the bright spot region.
[0072] The marking module 101 is used to determine each unmarked first bright pixel in the bright spot region as a seed point to obtain a current seed set containing a plurality of seed points; search for unmarked second bright pixels in the eight neighborhoods of each seed point in the seed set, and determine a plurality of first seed points and corresponding second bright pixels as a set of adjacent bright pixels; update the current seed set based on the set of adjacent bright pixels to obtain a target seed set; and determine the region corresponding to the target seed set as the bright spot connected region.
[0073] The classification module 102 is used to determine the zeroth moment, first moment, and second moment of the bright spot connected region, respectively, to obtain the bright spot area parameter, the bright spot centroid coordinates, and the bright spot principal axis direction angle in sequence; and to determine the extreme coordinates in the pixel coordinates of the bright spot connected region, and to determine the maximum distance between the extreme coordinates as the bright spot length; and to classify the bright spot connected region according to the bright spot area parameter, the bright spot centroid coordinates, the bright spot principal axis direction angle, and the bright spot length to obtain the bright spot feature set.
[0074] Grouping module 103 is used to take the point-like bright spots in the bright spot feature set as the starting growth point, determine the bright spot centroid coordinates and the angle of the bright spot principal axis direction of each starting growth point; determine the difference between the spatial distance between each starting growth point and the adjacent bright spot and the angle of the bright spot principal axis direction; wherein, the spatial distance is determined based on the bright spot centroid coordinates corresponding to the starting growth point and the adjacent bright spot respectively; if the spatial distance is less than or equal to a first preset threshold and the difference is less than or equal to a second preset threshold, then the starting growth point and the adjacent bright spot are merged into the same group to obtain a candidate bright spot group; other bright spot features in the bright spot feature set except for the point-like bright spots are merged and grouped, and the grouping results are merged with the candidate bright spot group to obtain the initial bright spot group.
[0075] Grouping module 103 is used to perform least-squares line fitting on the highlight feature pixels of each group in the initial highlight grouping result to obtain the initial axis parameters corresponding to each group; wherein, the initial axis parameters include slope and intercept; the group with slope difference lower than a third preset threshold and intercept difference lower than a fourth preset threshold within the initial axis parameters is determined as the target group; the axis parameters corresponding to the target group are determined as the target axis parameters.
[0076] The recognition module 104 is used to determine the vertex coordinates of the target region based on the target axis parameters and the prior dimensions; generate a rectangular mask with the target axis as the axis of symmetry based on the vertex coordinates, and determine the region corresponding to the rectangular mask as the target region; extract the target sub-image from the image of the stacked metal parts within the target region to recognize the image of the stacked metal parts and obtain the recognition result.
[0077] The application provides an image recognition device, which introduces watershed segmentation and mask enhancement in a preprocessing operation, so that a bright spot region can be accurately highlighted in a complex background. Further, image matrix features can be used to realize high-precision classification of point-like bright spots, line-like bright spots and strip-like bright spots, and based on a preset region growing similarity criterion combined with straight line fitting, the geometric shape and spatial position of similar metal parts can be accurately restored in the presence of occlusion, overlap and reflection interference. In addition, a target region is generated by using prior dimensions and target axis parameters, background noise is effectively removed, and spatial independent separation of multiple targets is realized, so that the recognition of stacked metal part images can be realized in the target region, and the recognition accuracy is improved. The method significantly improves the segmentation accuracy and recognition robustness of stacked metal part images in a complex industrial scene while maintaining the low cost of monocular vision.
[0078] The specific limitations of the image recognition device can refer to the limitations of the image recognition method in the foregoing, which will not be repeated here. Each module in the image recognition device described above can be realized by software, hardware and a combination thereof in whole or in part. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to call and execute the operations corresponding to the above-mentioned modules by the processor.
[0079] In one embodiment, a computer device is provided, which can be a server, and an internal structure diagram thereof can be as shown in Figure 11 The computer device includes a processor, a memory, a network interface and a database connected by a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with the external client through the network connection. The computer program is executed by the processor to realize the functions or steps of the image recognition method server side.
[0080] In one embodiment, a computer device is provided, which can be a client, and an internal structure diagram thereof can be as shown in Figure 12As shown in the figure. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is used to communicate with the external server through the network connection. The computer program is executed by the processor to realize the function or step of the image recognition method client side In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor executing the computer program to implement the following steps: The collected stacked metal part image is preprocessed to obtain a highlight area, and the highlight area is labeled with a connected domain to obtain a highlight connected domain; wherein the preprocessing operation at least includes a watershed segmentation processing and a mask enhancement processing; Classify the highlight connected domain based on image moment features to obtain a highlight feature set containing point-like highlights, line-like highlights and strip-like highlights; wherein the image moment features include highlight area parameters, highlight barycenter coordinates, highlight principal axis direction angle and highlight length; Group the highlight feature set according to a preset region growing similarity criterion to obtain an initial highlight grouping, and perform straight line fitting based on the initial highlight grouping to obtain a target grouping including several groups of similar metal parts and target axis parameters corresponding to the similar metal parts; Generate a target region according to the target axis parameters and corresponding prior size, and identify the stacked metal part image in the target region to obtain an identification result.
[0081] In one embodiment, a computer readable storage medium is provided, which stores a computer program, the computer program is executed by a processor to implement the following steps: The collected stacked metal part image is preprocessed to obtain a highlight area, and the highlight area is labeled with a connected domain to obtain a highlight connected domain; wherein the preprocessing operation at least includes a watershed segmentation processing and a mask enhancement processing; Classify the highlight connected domain based on image moment features to obtain a highlight feature set containing point-like highlights, line-like highlights and strip-like highlights; wherein the image moment features include highlight area parameters, highlight barycenter coordinates, highlight principal axis direction angle and highlight length; Group the bright spot feature set according to a preset region growing similarity criterion to obtain an initial bright spot grouping, and perform straight line fitting based on the initial bright spot grouping to obtain a target grouping including a plurality of groups of similar metal parts and target axis parameters corresponding to the similar metal parts; Generate a target region according to the target axis parameters and corresponding prior dimensions, and perform recognition on the stacked metal part image in the target region to obtain a recognition result.
[0082] It should be noted that the functions or steps described above with respect to the computer readable storage medium or the computer device can correspond to the related descriptions of the server side and the client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.
[0083] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0084] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is exemplified. In actual applications, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.
[0085] The above-described embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.
Claims
1. An image recognition method, characterized in that, The method includes: The acquired images of stacked metal parts are preprocessed to obtain bright spot regions, and the bright spot regions are then labeled with connected components to obtain bright spot connected components; wherein, the preprocessing operation includes at least watershed segmentation and mask enhancement processing; Based on image moment features, the bright spot connected components are classified to obtain a bright spot feature set containing point-like bright spots, line-like bright spots, and band-like bright spots; wherein, the image moment features include bright spot area parameters, bright spot centroid coordinates, bright spot principal axis direction angle, and bright spot length; The set of bright spot features is grouped according to a preset region growth similarity criterion to obtain initial bright spot groups. Based on the initial bright spot groups, a straight line fit is performed to obtain target groups including several groups of similar metal parts and target axis parameters corresponding to the similar metal parts. A target region is generated based on the target axis parameters and the corresponding prior dimensions, and the image of the stacked metal parts is identified within the target region to obtain the identification result.
2. The method according to claim 1, characterized in that, The preprocessing operation on the acquired images of stacked metal parts to obtain bright spot areas includes: The image of the stacked metal parts is subjected to noise suppression by Gaussian filtering to obtain a noise-suppressed image. Histogram equalization is used to enhance the contrast of the noise-suppressed image, resulting in an enhanced image. The enhanced image is then subjected to gamma correction to obtain a preprocessed image; The preprocessed image is then subjected to watershed segmentation and mask enhancement processes in sequence to obtain the bright spot region.
3. The method according to claim 1, characterized in that, The step of performing connected component labeling on the bright spot region to obtain the bright spot connected component includes: Each unmarked first bright pixel in the bright area is identified as a seed point, resulting in a current seed set containing several seed points; Search for unmarked second bright pixels in the eight neighborhoods of each seed point in the seed set, and determine a set of adjacent bright pixels by a number of first seed points and the corresponding second bright pixels. The current seed set is updated based on the set of adjacent bright pixels to obtain the target seed set; The region corresponding to the target seed set is determined as the bright spot connected region.
4. The method according to claim 1, characterized in that, The classification of the bright spot connected components based on image moment features yields a bright spot feature set containing point-like bright spots, line-like bright spots, and band-like bright spots, including: The zeroth, first, and second moments of the connected components of the bright spot are determined respectively, thereby obtaining the bright spot area parameters, the bright spot centroid coordinates, and the bright spot principal axis direction angles; and, Determine the extreme coordinates of the pixel coordinates in the bright spot connected region, and determine the maximum distance between the extreme coordinates as the bright spot length; The bright spot connected components are classified according to the bright spot area parameter, the bright spot centroid coordinate, the bright spot principal axis direction angle, and the bright spot length to obtain the bright spot feature set.
5. The method according to claim 1, characterized in that, The step of grouping the bright spot feature set according to a preset region growth similarity criterion to obtain initial bright spot groups includes: Using the point-like bright spots in the set of bright spot features as the starting growth points, determine the centroid coordinates and the angle of the principal axis direction of each starting growth point. The difference between the spatial distance between each initial growth point and its adjacent bright spot and the angle of the principal axis direction of the bright spot is determined; wherein, the spatial distance is determined based on the centroid coordinates of the bright spots corresponding to the initial growth point and its adjacent bright spots respectively; If the spatial distance is less than or equal to a first preset threshold and the difference is less than or equal to a second preset threshold, then the starting growth point and the adjacent bright spots are merged into the same group to obtain a candidate bright spot group. The bright spot features other than the dot-shaped bright spots in the bright spot feature set are merged and grouped, and the grouping results are merged with the candidate bright spot group to obtain the initial bright spot group.
6. The method according to claim 1, characterized in that, The process of performing linear fitting based on the initial bright spot grouping yields target groups comprising several groups of similar metal parts and target axis parameters corresponding to the similar metal parts, including: The feature pixels of each group in the initial bright spot grouping result are fitted with a least-squares line to obtain the initial axis parameters corresponding to each group; wherein, the initial axis parameters include slope and intercept. The group whose slope difference within the initial axis parameters is lower than the third preset threshold and whose intercept difference is lower than the fourth preset threshold is determined as the target group; The axis parameters corresponding to the target group are determined as the target axis parameters.
7. The method according to claim 1, characterized in that, The process of generating a target region based on the target axis parameters and corresponding prior dimensions, and recognizing the stacked metal parts image within the target region to obtain a recognition result includes: The vertex coordinates of the target region are determined based on the target axis parameters and the prior dimensions. A rectangular mask with the target axis as the axis of symmetry is generated based on the vertex coordinates, and the area corresponding to the rectangular mask is determined as the target area; The target sub-image is extracted from the image of the stacked metal parts within the target area to identify the image of the stacked metal parts and obtain the identification result.
8. An image recognition device, characterized in that, include: The labeling module is used to preprocess the acquired images of stacked metal parts to obtain bright spot regions, and to label the bright spot regions with connected components to obtain bright spot connected components; wherein, the preprocessing operation includes at least watershed segmentation processing and mask enhancement processing; The classification module is used to classify the bright spot connected components based on image moment features to obtain a bright spot feature set including point bright spots, line bright spots, and band bright spots; wherein, the image moment features include bright spot area parameters, bright spot centroid coordinates, bright spot principal axis direction angle, and bright spot length; The grouping module is used to group the set of bright spot features according to a preset region growth similarity criterion to obtain an initial bright spot group, and to perform line fitting based on the initial bright spot group to obtain a target group including several groups of similar metal parts and the target axis parameters corresponding to the similar metal parts; The recognition module is used to generate a target region based on the target axis parameters and the corresponding prior dimensions, and to recognize the image of the stacked metal parts within the target region to obtain the recognition result.
9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the image recognition method as described in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the image recognition method as described in any one of claims 1 to 7.