A method and system for detecting defects in the leaves of a simulated Christmas tree

Abstract

This application relates to the field of image processing technology, and specifically relates to a method and system for detecting defects in the leaves of a simulated Christmas tree. The method includes: collecting an image of the leaves of a simulated Christmas tree; dividing the leaf image into regions, obtaining an edge overlap index based on the difference between the edges and the fitted straight lines in the regions, and splicing the regions into an edge overlap region; obtaining the maximum circumscribed ellipse of the edge lines in the edge overlap region to obtain the complexity level of the pine branches; obtaining the intersection region and the shadow region of the edge lines in the edge overlap region, and obtaining the light and shadow feature parameters according to the pixel values of the shadow region in the intersection region and the pixel differences between the shadow regions, and forming a scatter plot; obtaining the light and shadow complexity according to the differences between the horizontal and vertical coordinates of the scatter plot, and calculating the edge blur parameter accordingly; adjusting the standard deviation of the Gaussian filter based on the edge blur parameter and then performing smoothing and denoising on the image, and performing defect detection on the denoised image. This application improves the accuracy and robustness of leaf edge detection.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and particularly to a method and system for detecting defects in the leaves of artificial Christmas trees. Background Art

[0002] Christmas trees are important elements of festival decorations and are widely used in places such as homes and shopping malls. With the increasing requirements of consumers for appearance quality, the production process of artificial Christmas trees has been continuously upgraded. The leaf part is prone to problems such as morphological defects, uneven color, and material damage due to its high simulation degree and complex details, and these defects directly affect the product quality and market competitiveness.

[0003] Existing artificial Christmas trees are composed of fixing multiple layers of artificial branches on a skeleton structure. The Christmas tree skeleton is formed by winding two iron wires, and the leaves (artificial pine needles) are inserted and screwed between the skeletons to form a dense and complex structure. Since there are intersections between the branches and the leaves, and the leaf layers on the same branch may overlap and block each other, edge overlap phenomena are likely to occur during existing image segmentation. This overlap will affect subsequent defect detection based on the edges of the leaves, thereby reducing the detection accuracy. Summary of the Invention

[0004] In order to solve the technical problem of low defect detection accuracy caused by edge overlap, this application provides a method and system for detecting defects in the leaves of artificial Christmas trees. The specific technical solutions adopted are as follows:

[0005] In the first aspect, this application proposes a method for detecting defects in the leaves of artificial Christmas trees, and the method includes the following steps:

[0006] Collect images of the leaves of artificial Christmas trees;

[0007] Perform edge detection on the images of the leaves of artificial Christmas trees to obtain an edge image; divide the edge image into several window regions evenly, perform linear fitting on the edge lines in the window regions, calculate the goodness of fit between the fitting straight line and the edge line, calculate the edge overlap index of the window region according to the goodness of fit and the number of edge lines in the window region; splice multiple window regions into an edge overlap region based on the edge overlap index;

[0008] Obtain the maximum circumscribed ellipse of all the edge lines in the edge overlapping region, and use the standard deviation of the diameter of the maximum circumscribed ellipse as the pine branch level complexity of the edge overlapping region; obtain the intersection points and edge angles of all the edge lines in the edge overlapping region to form an intersection region with the edge lines forming the intersection points as the boundary; segment the intersection region to obtain the shadow region; draw the angular bisector of the edge angle, and record the perpendicular line to the belonging angular bisector as the tangent line of the intersection region; obtain the first light and shadow feature index according to the pixel values of the shadow region among all the tangent lines in the intersection region; divide the shadow region into two regions based on the angular bisector, and obtain the second light and shadow feature index according to the difference in pixel values between the two regions; use the sum of the first light and shadow feature index and the second light and shadow feature index of the intersection region as the light and shadow feature parameter of the intersection region; map all the intersection regions to a scatter plot according to their spatial positions and light and shadow feature parameters; obtain the light and shadow complexity of the intersection region according to the horizontal and vertical coordinate differences between the discrete points in the scatter plot and the surrounding discrete points; calculate the edge blur parameter of the intersection region according to the light and shadow feature parameter and the light and shadow complexity of the intersection region;

[0009] Adjust the maximum standard deviation of the Gaussian filter based on the edge blur parameter to obtain the standard deviation of the Gaussian filter, and smooth the simulated Christmas tree leaf image based on the Gaussian filter and then smooth it again through edge detection. Detect defects in the finally smoothed image through a network.

[0010] In the above solution, the present application first uses a traditional edge detection algorithm to extract all the edge curve contours in the leaf surface image, and further identifies and extracts the edge dense overlapping regions; then, by analyzing these edge dense overlapping regions, it identifies the overlapping situations of different types of pine needle leaves, and further extracts all the leaf edge intersection regions in the image. Classify the intersection regions and identify the leaf edge intersections at different levels. Evaluate the degree to which the intersection region is affected by the light and shadow of other regions by measuring the density of the leaf edge intersections at different levels around the intersection region; finally, combine the evaluation results of the light and shadow influence to determine the filtering and smoothing scale for different intersection regions to eliminate the influence of the overlapping region on the edge detection result, thereby improving the accuracy and robustness of the leaf edge detection.

[0011] In one embodiment, the method for linearly fitting the edge lines in the window region, calculating the goodness of fit between the fitted line and the edge line, and calculating the edge overlapping index of the window region according to the goodness of fit and the number of edge lines in the window region is as follows:

[0012] Obtain the optimal fitted line of each edge line in the window region through the linear fitting algorithm, and calculate the goodness of fit between each edge line in the window region and the optimal fitted line; obtain the mean value of the goodness of fit between all the edge lines in the window region and their optimal fitted lines, and record it as the mean goodness of fit of the window region;

[0013] Count the number of edge lines within the statistical window area, and record the number of edge lines as the discontinuity index of the window area;

[0014] Calculate the edge overlap index of the window area according to the mean goodness of fit of the window area and the discontinuity index of the window area;

[0015] The edge overlap index has a positive correlation with discontinuity and a negative correlation with the mean goodness of fit.

[0016] In one embodiment, the method of splicing multiple window areas into an edge overlap area based on the edge overlap index is as follows:

[0017] Mark the window areas with an edge overlap index greater than a preset empirical value as edge overlap areas, and consider the spatial splicing of adjacent edge overlap areas in terms of spatial position as the same area.

[0018] In one embodiment, the method of obtaining the intersection points and edge angles of all edge lines in the edge overlap area to form an intersection area with the edge lines forming the intersection points as the boundary is as follows:

[0019] Extract the intersection points of all edge lines in the edge overlap area through Hough transform. Use the two edge lines forming the intersection point as the boundary. The intersection point divides the edge line into multiple segments. Take the angles corresponding to the two longest segments as the edge angles and spread inward from the edge angles. When spreading, use the intersection point of the two edge lines as the origin, and form a pixel data set with all adjacent points of the origin within the edge angle. Then, put the adjacent points of all data points in the data set into the pixel data set as the spreading pixel points until the number of pixel points in the pixel data set reaches the preset number or spreading is no longer possible. The area obtained after stopping spreading is recorded as the intersection area.

[0020] In one embodiment, the method of obtaining the first light and shadow feature index according to the pixel values of the shadow area among all the tangent lines in the intersection area is as follows:

[0021] Calculate the mean value of the pixel values in each tangent line of the intersection area, and calculate the absolute value of the difference between the mean values of the pixel points between adjacent tangent lines in the intersection area, and use it as the first light and shadow feature index of the intersection area.

[0022] In one embodiment, the method of dividing the shadow area into two areas based on the bisector and obtaining the second light and shadow feature index according to the difference in pixel values between the two areas is as follows:

[0023] The bisector divides the intersection area into two areas; mark the shadow areas in the two areas as the first shadow area and the second shadow area respectively; calculate the mean square error of the pixel values of all pixel points in the first shadow area and the second shadow area, and use the mean square error as the second light and shadow feature index of the intersection area.

[0024] In one embodiment, the method of mapping all intersection regions into a scatter plot according to their spatial positions and light and shadow feature parameters, and obtaining the light and shadow complexity of the intersection regions based on the horizontal and vertical coordinate differences between the discrete points in the scatter plot and the surrounding discrete points is as follows:

[0025] Taking any one intersection region as the central region, the abscissa of the scatter plot is the spatial position, the ordinate is the light and shadow feature parameter, and the spatial position is the distance between all intersection regions and the central region;

[0026] For the central region, taking the corresponding discrete point as the center, draw a circle with a diameter of 1 / 10 of the size of the scatter plot in the scatter plot as the action range of the central region; calculate the average value of the abscissa difference and the average value of the ordinate difference between the central region and all intersection regions within its action range; the abscissa difference is the absolute value of the difference between the abscissa of the central region and the abscissa of the intersection region, and the ordinate difference is the absolute value of the difference between the ordinate of the central region and the ordinate of the intersection region;

[0027] Calculate the ratio of the average value of the ordinate difference to the average value of the abscissa difference of the central region as the light and shadow complexity of the central region.

[0028] In one embodiment, the method of calculating the edge blur parameter of the intersection region based on the light and shadow feature parameter and the light and shadow complexity of the intersection region is as follows:

[0029] The edge blur parameter of the intersection region has a positive correlation with the light and shadow complexity of the intersection region and a positive correlation with the light and shadow feature parameter of the intersection region.

[0030] In one embodiment, the method of adjusting the maximum standard deviation of Gaussian filtering based on the edge blur parameter to obtain the standard deviation of Gaussian filtering is as follows:

[0031] , represents the pine branch level complexity of the intersection region; represents the edge blur parameter of the intersection region, represents the maximum standard deviation of Gaussian filtering, represents the adjusted standard deviation of Gaussian filtering.

[0032] In a second aspect, an embodiment of the present application further provides a simulation Christmas tree leaf defect detection system, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of the above-mentioned simulation Christmas tree leaf defect detection method.

[0033] The beneficial effects of the present application are as follows:

[0034] This application first uses traditional edge detection algorithms to extract all the edge curve contours in the leaf surface image, and further identifies and extracts the edge-dense overlapping regions. Then, by analyzing these edge-dense overlapping regions, different types of pine needle leaf overlapping situations are identified, and then all the leaf edge intersection regions in the image are extracted. The intersection regions are classified, and different levels of leaf edge intersections are identified. By measuring the density of different levels of leaf intersections around the intersection region, the degree of influence of the intersection region by the light and shadow of other regions is evaluated. Finally, combining the evaluation results of the light and shadow influence, the filtering and smoothing scales for different intersection regions are determined to eliminate the influence of the overlapping region on the edge detection result, thereby improving the accuracy and robustness of the leaf edge detection. Description of the Drawings

[0035] To more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0036] Figure 1 Flowchart of a method for detecting defects in the leaves of a simulated Christmas tree provided by an embodiment of the present application;

[0037] Figure 2 Schematic diagram of an edge image;

[0038] Figure 3 Schematic diagram of a scatter plot. Detailed Embodiments

[0039] To further elaborate on the technical means and effects adopted by the present application to achieve the intended invention purpose, the following will, in combination with the drawings and preferred embodiments, describe in detail the specific embodiments, structures, features, and effects of a method and system for detecting defects in the leaves of a simulated Christmas tree proposed according to the present application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present application belongs.

[0041] An embodiment of a method and system for detecting defects in the leaves of a simulated Christmas tree:

[0042] The following will specifically describe the specific solutions of a method and system for detecting defects in the leaves of a simulated Christmas tree provided by the present application in combination with the drawings.

[0043] Please refer to Figure 1 , which shows a flowchart of a method for detecting defects in the leaves of a simulated Christmas tree provided by an embodiment of the present application. The method includes the following steps:

[0044] Step S001, collect images of the leaves of the simulated Christmas tree.

[0045] Deploy an industrial camera directly above the leaves of the simulated Christmas tree. Set the camera exposure frequency to 1 s / time. The industrial camera used is a high-resolution industrial camera. Place an LED light source beside the leaves of the simulated Christmas tree to ensure image quality. Based on this, collect images of the leaves of the simulated Christmas tree.

[0046] So far, images of the leaves of the simulated Christmas tree have been obtained.

[0047] Step S002, perform edge detection on the leaf image to divide regions, obtain an edge overlap index based on the difference between the edges and the fitted straight line in the region, and splice the regions into an edge overlap region.

[0048] The central processing unit is used to retrieve the images of the leaves of the simulated Christmas tree. Labeling tools such as LabelMe and VGG Image Annotator are used to perform pixel-level annotation on the image dataset to calibrate the leaf structure region; the annotated image dataset is trained by U-Net semantic segmentation. The images of the leaves of the simulated Christmas tree are input into the trained U-Net network. The leaf region is segmented from the background and branches, reducing the interference of ornaments and branches on defect detection and focusing on the accurate identification of defects on the leaf surface, so as to obtain interference-free images of the leaves of the simulated Christmas tree and update the images of the leaves of the simulated Christmas tree.

[0049] Since the simulated leaves are inserted between the skeletons, a dense and complex structure is formed. Due to the intersection between the branches and the leaves, and the layers of leaves on the same branch may overlap and block each other, edge aliasing is likely to occur during existing image segmentation. Therefore, edge detection needs to be performed on the images of the leaves of the simulated Christmas tree to obtain an edge overlap region.

[0050] Use an edge detection algorithm to extract all the edge lines in the images of the leaves of the simulated Christmas tree. In this embodiment, the edge detection method used is the Canny edge detection algorithm to obtain an edge image. The image after edge extraction is as Figure 2 shown.

[0051] Divide the image into regions. Specifically, construct a window region with a size of 5*5 (unit: millimeters of the image). Starting from the upper left corner of the image, from left to right, from top to bottom, until the lower right corner of the image, the edge image is thus divided into several local window regions.

[0052] Count the number of edge lines within the statistical window area, and record the number of curves as the discontinuity index of the window area; due to the regularity of window area division, the window area may be set between two individual pine needle leaves, resulting in multiple edge lines in the window area, which is manifested as edge discontinuity.

[0053] Since the continuous edges in the edge overlap area are combinations between different individual pine needle leaves, the angles of their edge lines are different, resulting in multiple inflection points in the continuous edge segments within the area.

[0054] Obtain the optimal fitting line of each edge line within the window area through the least squares method and linear fitting, and calculate the goodness of fit between each edge line within the window area and the optimal fitting line; obtain the mean value of the goodness of fit between all edge lines within the window area and their optimal fitting lines, and record it as the mean goodness of fit of the window area; the smaller the mean goodness of fit, the greater the difference between the edge lines within the window area and the optimal fitting line, that is, the more inflection points of the edge lines.

[0055] Calculate the edge overlap index of the window area based on the discontinuity of the window area and the mean goodness of fit.

[0056] The edge overlap index has a positive correlation with discontinuity and a negative correlation with the mean goodness of fit.

[0057] It should be noted that positive correlation means that when one variable increases, the other variable also increases, and the change directions of the two variables are the same. When one variable changes from large to small or from small to large, the other variable also changes from large to small or from small to large; the specific relationship is determined by actual applications, and this application does not make special restrictions.

[0058] It should be noted that negative correlation means that when one variable increases, the other variable decreases accordingly, and the change directions of the two variables are opposite. When one variable changes from large to small or from small to large, the other variable also changes from small to large or from large to small; the specific relationship is determined by actual applications, and this application does not make special restrictions.

[0059] Preferably, in this embodiment, the ratio of the discontinuity of the window area to the mean goodness of fit of the window area is normalized as the edge overlap index of the window area. The larger this value, the greater the possibility that there is an edge overlap area in the window area. The normalization method used in this embodiment is linear normalization.

[0060] Obtain the edge overlap indices of all window areas, mark the window areas with edge overlap indices greater than the preset empirical value as edge overlap areas, and simply splice adjacent edge overlap areas in space as the same area.

[0061] Thus, multiple edge overlap areas in the edge image are obtained.

[0062] Step S003: Obtain the maximum circumscribed ellipse of the edge lines in the edge overlapping region to obtain the complexity level of the pine branches; obtain the intersection region and the shadow region of the edge lines in the edge overlapping region, and obtain the light and shadow feature parameters according to the pixel values of the shadow region in the intersection region and the pixel differences between the shadow regions to form a scatter plot; obtain the light and shadow complexity according to the differences between the horizontal and vertical coordinates of the scatter plot, and calculate the edge blur parameter based on this.

[0063] The edge dense region is essentially formed by the interlacing of individual pine needles and leaves at different levels and branches, resulting in the inability to accurately segment the individual differences of the regional edge contour. Therefore, it is necessary to further distinguish the edge pixels of the leaves at different levels in the edge overlapping region according to the light and shadow effects of the individual pine needles at different levels.

[0064] The edge overlapping region is divided into two categories. The first category of components is composed of pine needle structures at different levels on the same branch. In the edge image, the maximum diameters of the circumscribed ellipses of different edge contours are different. The second category of components is composed of pine needle structures at the same or different levels on different branches. There will be a large number of intersections in the dense edge structures of pine needles at different levels, resulting in different light and shadow effects. The shadow of the upper-layer pine needles covers the edges of the lower-layer pine needles, causing different degrees of blurring of their edges.

[0065] Perform leaf detection through YOLO, and combine the SORT target tracking algorithm to realize the automatic detection and tracking of leaves, and then judge which leaves belong to the same leaf branch. Mark the edge dense regions belonging to the same branch as Class A overlapping regions, and mark the edge dense regions belonging to different branches as Class B overlapping regions.

[0066] Mark the first category as Class A overlapping region; obtain the maximum circumscribed ellipse of all edge lines in the edge overlapping region and its maximum diameter data set, calculate the standard deviation of the maximum diameter data set, and record the standard deviation as the complexity level of the pine branches in the edge overlapping region.

[0067] Mark the second category as Class B overlapping region; extract the intersections of all edge lines in the edge overlapping region through the Hough transform, use the two edge lines forming the intersection as the boundaries. The intersection divides the edge line into multiple segments. Take the angles corresponding to the longest two segments as the edge angles, and spread inward from the edge angles until there are a preset number of pixel points in the region. The spread will not spread to the edge line. If the region spread is not enough for the preset number, the spread will also stop, and the intersection region is obtained. In this embodiment, the preset number is 100*100.

[0068] Specifically, during diffusion, taking the intersection point of two edge lines as the origin, all adjacent points within the angle and at the origin form a pixel dataset. Then, the adjacent points of all data points in the dataset are put into the pixel dataset as the diffused pixel points until the number of pixel points in the pixel dataset reaches a preset number or diffusion is no longer possible. After stopping diffusion, the obtained area is denoted as the intersection area.

[0069] Under a fixed light background, the intersection areas formed by combinations of pine needle leaves at different levels often exhibit different light and shadow effects. For the intersection areas, the Otsu algorithm is used to obtain the optimal segmentation threshold for the intersection areas. Pixel points with pixel values less than the segmentation threshold are regarded as shadow pixel points, and the entire intersection area is traversed to obtain the shadow area.

[0070] First, quantify the light and shadow characteristics of all intersection areas. The intersection areas are composed of two different pine needle edges, and the composition methods of their pine needle composition structures include the interlacing of pine needles at the same level and the interlacing of pine needles at different levels, and their corresponding shadow characteristics also have significant differences.

[0071] For each intersection area, draw the angle bisector corresponding to the angle of the intersection area through the intersection point, and record the perpendicular line of the angle bisector as the tangent line of the intersection area.

[0072] For the characteristic of the interlacing of pine needles at the same level in the intersection area, the pixel distribution of the shadow part of the area is relatively stable. Draw multiple tangent lines of the intersection area, where the distances between the tangent lines are the same. In this embodiment, 10 tangent lines of the intersection area are obtained. Calculate the mean value of the pixel values in each tangent line of the intersection area, and calculate the absolute value of the difference between the mean values of the pixel points between adjacent tangent lines in the intersection area, and use it as the first light and shadow feature index of the intersection area.

[0073] For the characteristic of the interlacing of pine needles at different levels in the intersection area, the distribution of the shadow part of the area is relatively disordered. Draw the angle bisector corresponding to the angle of the intersection area. The angle bisector not only divides the angle into two equal angles, but also divides the intersection area into two areas. Denote the shadow areas in the two areas as the first shadow area and the second shadow area respectively. Calculate the mean square error of the pixel values of all pixel points in the first shadow area and the second shadow area. The larger the mean square error, the more significant the difference between the shadows on both sides of the intersection area, and this value is also different as the layer difference of the pine needle edges corresponding to the intersection area is different. Use the mean square error as the second light and shadow feature index of the intersection area;

[0074] Take the sum of the first light and shadow feature index and the second light and shadow feature index of the intersection area as the light and shadow feature parameter of the intersection area. The larger the light and shadow feature parameter, the more significantly the intersection area is affected by light and shadow.

[0075] Obtain the light and shadow feature parameters of all intersection regions of the simulated Christmas tree leaf images according to the above steps. Map all the intersection regions to a scatter plot according to their spatial positions and light and shadow feature parameters, as Figure 3 shown.

[0076] Specifically, taking any one intersection region as the central region, the abscissa of the scatter plot is the spatial position, the ordinate is the light and shadow feature parameter, and the spatial position is the distance between all intersection regions and the central region. The farther away from the central region, the larger the abscissa, and the closer to the central region, the smaller the abscissa. In this embodiment, the size of the scatter plot is 10*10.

[0077] For the central region, taking the corresponding discrete point as the center, draw a circle with a diameter of 1 / 10 of the scatter plot size in the scatter plot as the action range of the central region. Calculate the average value of the abscissa difference and the average value of the ordinate difference between the central region and all intersection regions within its action range. The abscissa difference is the absolute value of the difference between the abscissa of the central region and the abscissa of the intersection region, and the ordinate difference is the absolute value of the difference between the ordinate of the central region and the ordinate of the intersection region.

[0078] Calculate the ratio of the average value of the ordinate difference to the average value of the abscissa difference of the central region as the light and shadow complexity of the central region. The larger this value is, it means that there are a large number of intersection regions with significant light and shadow differences around the central region, which may have a greater impact on the light and shadow effect (such as the degree of edge blurring) of the central region.

[0079] Therefore, obtain the edge blurring parameters of the intersection regions based on the light and shadow complexity and light and shadow feature parameters of each intersection region.

[0080] The edge blurring parameter of the intersection region has a positive correlation with the light and shadow complexity of the intersection region and a positive correlation with the light and shadow feature parameter of the intersection region.

[0081] Preferably, in this embodiment, the product of the light and shadow complexity of the intersection region and the light and shadow feature parameter of the intersection region is used as the edge blurring parameter of the intersection region.

[0082] So far, the edge blurring parameters of the intersection regions have been obtained.

[0083] Step S004, adjust the standard deviation of the Gaussian filter based on the edge blurring parameter and then perform smoothing denoising on the image, and perform defect detection on the denoised image.

[0084] In the shaded area of pine needle leaves, the light and shadow of the upper-layer pine needles may cause severe coverage of the lower-layer pine needles, resulting in the blurring of the edges of the lower layer, and thus the loss of edge information of surface defects. The traditional Gaussian filtering method may not be able to accurately eliminate such blurred pseudo-edge pixel points. Therefore, for areas with different layers of pine needle leaves and severely blurred local edges, the smoothing scale of Gaussian filtering can be adaptively adjusted to reduce the blurring effect and more accurately restore edge details.

[0085] For Gaussian filtering windows of different sizes, they have theoretically different maximum standard deviations. In this embodiment, the maximum standard deviation of any Gaussian filtering window is obtained.

[0086] Based on the edge blurring parameter of the intersection area and the complexity of the pine branch layers in the intersection area, the maximum standard deviation of Gaussian filtering is adjusted to obtain the standard deviation of Gaussian filtering. The expression is:

[0087] , represents the complexity of the pine branch layers in the intersection area; represents the edge blurring parameter of the intersection area, represents the maximum standard deviation of Gaussian filtering, represents the standard deviation of Gaussian filtering after adjustment.

[0088] Based on the standard deviation of Gaussian filtering, a set of weights is generated, and based on this, the pixel values in the simulated Christmas tree leaf image are weighted and averaged to obtain a smoothed image.

[0089] The gradient magnitude and direction of each pixel point in the smoothed image are calculated using the Sobel operator, and non-maximum suppression is performed in the gradient direction to accurately locate the edges. Finally, through edge intensity thresholding, the edge points are divided into strong edges and weak edges;

[0090] The detected edges are smoothed to obtain the final smoothed image, thereby reducing false edges caused by noise. Common methods include morphological operations (such as erosion, dilation, opening operation, closing operation), etc.

[0091] Images with different defects are classified and marked, and images with multiple defects are used for network training. In this embodiment, the neural network used is the CNN convolutional neural network, and the loss function is the cross-entropy loss function. The final image is input into the trained model to identify the defects in the final smoothed image.

[0092] Based on the same inventive concept as the above method, an embodiment of the present invention further provides a simulation Christmas tree leaf defect detection system, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of any one of the above-described simulation Christmas tree leaf defect detection methods are implemented.

[0093] It should be noted that the above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present application, and should all be included in the protection scope of the present application.

[0094] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. The key point of each embodiment is to illustrate the differences from other embodiments.

Claims

1. A method for detecting defects in the leaves of a simulated Christmas tree, characterized in that, The method comprises the following steps: Collect an image of the leaves of an artificial Christmas tree; Perform edge detection on the image of the leaves of the artificial Christmas tree to obtain an edge image; evenly divide the edge image into a plurality of window regions, perform linear fitting on the edge lines in the window regions, and calculate the goodness of fit between the fitting line and the edge lines. Calculate the edge overlap index of the window region according to the goodness of fit and the number of edge lines in the window region; splice a plurality of window regions into an edge overlap region based on the edge overlap index. The edge overlap region is divided into two categories. The first type of component is composed of pine needle structures at different levels on the same branch, and the second type of component is composed of pine needle structures at the same or different levels on different branches; Obtain the maximum circumscribed ellipse of all edge lines in the edge overlap region, and use the standard deviation of the diameter of the maximum circumscribed ellipse as the pine branch level complexity of the edge overlap region; obtain the intersection points and edge angles of all edge lines in the edge overlap region, and form an intersection region with the edge lines forming the intersection points as the boundary. Among them, the intersection points divide the edge lines into multiple segments, and take the angles corresponding to the two longest segments as the edge angles; segment the intersection region to obtain a shadow region; draw the bisector of the edge angle, and record the perpendicular line to the bisector as the tangent of the intersection region; obtain the first light and shadow feature index according to the pixel values of the shadow region among all the tangents in the intersection region; divide the shadow region into two regions based on the bisector, and obtain the second light and shadow feature index according to the difference in pixel values between the two regions; take the sum of the first light and shadow feature index and the second light and shadow feature index of the intersection region as the light and shadow feature parameter of the intersection region; map all the intersection regions to a scatter plot according to their spatial positions and light and shadow feature parameters; obtain the light and shadow complexity of the intersection region according to the horizontal and vertical coordinate differences between the discrete points in the scatter plot and the surrounding discrete points; calculate the edge blurring parameter of the intersection region according to the light and shadow feature parameter and the light and shadow complexity of the intersection region; Adjust the maximum standard deviation of Gaussian filtering based on the edge blurring parameter to obtain the standard deviation of Gaussian filtering. , represents the complexity of the pine branch levels in the intersection area; represents the edge blurring parameter of the intersection area, represents the maximum standard deviation of Gaussian filtering, represents the standard deviation of Gaussian filtering after adjustment; Smooth the simulated Christmas tree leaf image with the standard deviation of Gaussian filtering after adjustment, and then smooth it again through edge detection after Gaussian filtering. Detect defects in the finally smoothed image through the network.

2. The method for detecting defects of the leaves of an artificial Christmas tree according to claim 1, characterized in that, The method for performing linear fitting on the edge lines in the window region, calculating the goodness of fit between the fitting line and the edge lines, and calculating the edge overlap index of the window region according to the goodness of fit and the number of edge lines in the window region is as follows: Obtain the optimal fitting line of each edge line in the window region through a linear fitting algorithm, and calculate the goodness of fit between each edge line in the window region and the optimal fitting line; obtain the mean value of the goodness of fit between all edge lines in the window region and their optimal fitting lines, which is denoted as the mean goodness of fit of the window region; Count the number of edge lines in the window region, and denote the number of edge lines as the discontinuity index of the window region; Calculate the edge overlap index of the window region according to the mean goodness of fit of the window region and the discontinuity index of the window region; The edge overlap index has a positive correlation with discontinuity and a negative correlation with the mean goodness of fit.

3. The method for detecting defects of artificial Christmas tree leaves according to claim 1, characterized in that, The method for splicing a plurality of window regions into an edge overlap region based on the edge overlap index is as follows: Denote the window regions with an edge overlap index greater than a preset empirical value as edge overlap regions, and consider the spatial splicing of adjacent edge overlap regions as the same region.

4. The method for detecting defects of artificial Christmas tree leaves according to claim 1, characterized in that, The method for obtaining the intersection points and edge angles of all edge lines in the edge overlapping region to form an intersection region with the edge lines forming the intersection points as boundaries is as follows: Extract the intersection points of all edge lines in the edge overlapping region through Hough transformation. Use the two edge lines forming the intersection point as boundaries and diffuse into the interior of the edge angle. When diffusing, take the intersection point of the two edge lines as the origin, and form a pixel data set with all adjacent points of the origin within the edge angle. Then, put the adjacent points of all data points in the data set into the pixel data set as the diffused pixel points until the number of pixel points in the pixel data set reaches the preset number or diffusion is no longer possible. Stop diffusion, and the obtained region is recorded as the intersection region.

5. The method for detecting defects of artificial Christmas tree leaves according to claim 1, characterized in that, The method for obtaining the first light and shadow feature index based on the pixel values of the shadow region among all the tangents in the intersection region is as follows: Calculate the average value of the pixel values of each tangent in the intersection region, and calculate the absolute value of the difference between the average values of the pixel points between adjacent tangents in the intersection region, which is used as the first light and shadow feature index of the intersection region.

6. The method for detecting defects of artificial Christmas tree leaves according to claim 1, characterized in that, The method for dividing the shadow region into two regions based on the bisector and obtaining the second light and shadow feature index according to the difference in pixel values between the two regions is as follows: The bisector divides the intersection region into two regions; denote the shadow regions in the two regions as the first shadow region and the second shadow region respectively; calculate the mean square error of the pixel values of all pixel points in the first shadow region and the second shadow region, and use the mean square error as the second light and shadow feature index of the intersection region.

7. The method for detecting defects of artificial Christmas tree leaves according to claim 1, wherein The method for mapping all the intersection regions into a scatter plot according to their spatial positions and light and shadow feature parameters, and obtaining the light and shadow complexity of the intersection region according to the horizontal and vertical coordinate differences between the discrete points in the scatter plot and the surrounding discrete points is as follows: Take any intersection region as the central region, with the horizontal coordinate of the scatter plot as the spatial position and the vertical coordinate as the light and shadow feature parameter, and the spatial position is the distance between all intersection regions and the central region; For the central region, take the corresponding discrete point as the center of the circle, and draw a circle with a diameter of 1 / 10 of the size of the scatter plot in the scatter plot as the action range of the central region; calculate the average value of the horizontal coordinate difference and the average value of the vertical coordinate difference between the central region and all intersection regions within its action range; the horizontal coordinate difference is the absolute value of the difference between the horizontal coordinates of the central region and the intersection region, and the vertical coordinate difference is the absolute value of the difference between the vertical coordinates of the central region and the intersection region; Calculate the ratio of the average value of the vertical coordinate difference to the average value of the horizontal coordinate difference of the central region as the light and shadow complexity of the central region.

8. The method for detecting defects of artificial Christmas tree leaves according to claim 1, characterized in that, The method for calculating the edge blurring parameter of the intersection region according to the light and shadow feature parameter and the light and shadow complexity of the intersection region is as follows: The edge blurring parameter of the intersection region has a positive correlation with the light and shadow complexity of the intersection region and a positive correlation with the light and shadow feature parameter of the intersection region.

9. A simulation Christmas tree leaf defect detection system, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of a method for detecting defects in the leaves of a simulated Christmas tree as described in any one of claims 1-8.

Images