Simulated Christmas tree leaf defect detection method and system

Abstract

The invention relates to the technical field of image processing, in particular to a simulation Christmas tree leaf defect detection method and system. The method comprises the following steps: acquiring a simulation Christmas tree leaf image; dividing regions of the leaf image, obtaining an edge overlapping index based on the difference between the edge in the region and the fitting straight line, and splicing the region as an edge overlapping region; obtaining the maximum circumscribed ellipse of the edge line in the edge overlapping region so as to obtain the pine branch level complexity; obtaining an intersection point area and shadow areas of edge lines of the edge overlapping area, obtaining light and shadow characteristic parameters according to pixel values of the shadow areas in the intersection point area and pixel differences between the shadow areas, and forming a scatter diagram; acquiring shadow complexity according to the horizontal and vertical coordinate difference of the scatter diagram so as to calculate an edge blurring parameter; and adjusting the standard deviation of Gaussian filtering based on the edge blurring parameter, then carrying out smooth denoising on the image, and carrying out defect detection on the denoised image. According to the invention, the accuracy and robustness of leaf edge detection are improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and particularly to a method and system for detecting defects in the leaves of artificial Christmas trees. Background Art

[0002] Christmas trees are important elements of festival decorations and are widely used in places such as homes and shopping malls. With the increasing requirements of consumers for appearance quality, the production process of artificial Christmas trees has been continuously upgraded. The leaf part, due to its high simulation degree and complex details, is prone to problems such as morphological defects, uneven color, and material damage, which directly affect the product quality and market competitiveness.

[0003] Existing artificial Christmas trees are composed of fixing multiple layers of artificial branches on a skeleton structure. The Christmas tree skeleton is formed by winding two iron wires, and the leaves (artificial pine needles) are inserted and screwed between the skeletons to form a dense and complex structure. Due to the intersection between the branches and the leaves, and the leaf layers on the same branch may overlap and block each other, edge overlapping is likely to occur during existing image segmentation. This overlap will affect subsequent defect detection based on the leaf edges, thereby reducing the detection accuracy. Summary of the Invention

[0004] To solve the technical problem of low defect detection accuracy caused by edge overlap, this application provides a method and system for detecting defects in the leaves of artificial Christmas trees. The specific technical solutions adopted are as follows: In the first aspect, this application proposes a method for detecting defects in the leaves of artificial Christmas trees. The method includes the following steps: Collect images of the leaves of artificial Christmas trees; Perform edge detection on the images of the leaves of artificial Christmas trees to obtain edge images; evenly divide the edge images into several window areas, perform linear fitting on the edge lines in the window areas, calculate the goodness of fit between the fitted straight lines and the edge lines, calculate the edge overlap index of the window areas according to the goodness of fit and the number of edge lines in the window areas; splice multiple window areas into edge overlap areas based on the edge overlap index; Obtain the maximum circumscribed ellipse of all edge lines in the edge overlapping region, and use the standard deviation of the diameter of the maximum circumscribed ellipse as the pine branch level complexity of the edge overlapping region; obtain the intersection points and edge angles of all edge lines in the edge overlapping region to form an intersection region with the edge lines forming the intersection points as the boundary; segment the intersection region to obtain the shadow region; draw the bisector of the edge angle, and record the perpendicular line to the bisector as the tangent of the intersection region; obtain the first light and shadow feature index according to the pixel values of the shadow region among all the tangents in the intersection region; divide the shadow region into two regions based on the bisector, and obtain the second light and shadow feature index according to the difference in pixel values between the two regions; take the sum of the first light and shadow feature index and the second light and shadow feature index of the intersection region as the light and shadow feature parameter of the intersection region; map all the intersection regions to a scatter plot according to their spatial positions and light and shadow feature parameters; obtain the light and shadow complexity of the intersection region according to the horizontal and vertical coordinate differences between the discrete points in the scatter plot and the surrounding discrete points; calculate the edge blurring parameter of the intersection region according to the light and shadow feature parameter and the light and shadow complexity of the intersection region; Adjust the maximum standard deviation of Gaussian filtering based on the edge blurring parameter to obtain the standard deviation of Gaussian filtering, and smooth the simulated Christmas tree leaf image based on Gaussian filtering and then smooth it again through edge detection, and detect defects in the finally smoothed image through a network.

[0005] In the above solution, the present application first uses a traditional edge detection algorithm to extract all edge curve contours in the leaf surface image, and further identifies and extracts the edge dense overlapping regions; then, by analyzing these edge dense overlapping regions, it identifies different types of pine needle leaf overlapping situations, and further extracts all leaf edge intersection regions in the image. Classify the intersection regions and identify different levels of leaf edge intersections. Evaluate the degree to which the intersection region is affected by the light and shadow of other regions by measuring the density of different levels of leaf intersections around the intersection region; finally, combine the evaluation results of the light and shadow influence to determine the filtering and smoothing scale for different intersection regions to eliminate the influence of the overlapping region on the edge detection result, thereby improving the accuracy and robustness of the leaf edge detection.

[0006] In one embodiment, the method for linearly fitting the edge lines in the window region, calculating the goodness of fit between the fitted line and the edge lines, and calculating the edge overlap index of the window region according to the goodness of fit and the number of edge lines in the window region is as follows: Obtain the optimal fitted line of each edge line in the window region through a linear fitting algorithm, and calculate the goodness of fit between each edge line in the window region and the optimal fitted line; obtain the mean value of the goodness of fit between all edge lines in the window region and their optimal fitted lines, which is recorded as the mean goodness of fit of the window region; Count the number of edge lines in the window region, and record the number of edge lines as the discontinuity index of the window region; Calculate the edge overlap index of the window area according to the mean goodness of fit of the window area and the discontinuity index of the window area; The edge overlap index is positively correlated with the discontinuity and negatively correlated with the mean goodness of fit.

[0007] In one embodiment, the method of stitching multiple window areas into an edge overlap area based on the edge overlap index is as follows: Mark the window areas with edge overlap indices greater than the preset empirical value as edge overlap areas, and consider the spatial stitching of adjacent edge overlap areas in terms of spatial position as the same area.

[0008] In one embodiment, the method of obtaining the intersection points and edge angles of all edge lines in the edge overlap area to form an intersection area with the edge lines forming the intersection points as the boundaries is as follows: Extract the intersection points of all edge lines in the edge overlap area through Hough transform. Take the two edge lines forming the intersection points as the boundaries. The intersection points divide the edge lines into multiple segments. Take the angles corresponding to the two longest segments as the edge angles and spread inward from the edge angles. When spreading, use the intersection point of the two edge lines as the origin. Form a pixel data set with all adjacent points of the origin within the edge angles. Then, put the adjacent points of all data points in the data set into the pixel data set as the spreading pixel points until the number of pixel points in the pixel data set reaches the preset number or until no further spreading is possible. The area obtained after stopping the spreading is recorded as the intersection area.

[0009] In one embodiment, the method of obtaining the first light and shadow feature index according to the pixel values of the shadow area among all the tangents in the intersection area is as follows: Calculate the mean of the pixel values of each tangent in the intersection area, and calculate the absolute value of the difference between the means of the pixel points between adjacent tangents in the intersection area, and take it as the first light and shadow feature index of the intersection area.

[0010] In one embodiment, the method of dividing the shadow area into two areas based on the bisector and obtaining the second light and shadow feature index according to the difference in pixel values of the two areas is as follows: The bisector divides the intersection area into two areas; mark the shadow areas in the two areas as the first shadow area and the second shadow area respectively; calculate the mean square error of the pixel values of all pixel points in the first shadow area and the second shadow area, and take the mean square error as the second light and shadow feature index of the intersection area.

[0011] In one embodiment, the method of mapping all the intersection areas into a scatter plot according to their spatial positions and light and shadow feature parameters; and obtaining the light and shadow complexity of the intersection area according to the horizontal and vertical coordinate differences between the discrete points in the scatter plot and the surrounding discrete points is as follows: Taking any intersection area as the central area, the abscissa of the scatter plot is the spatial position, the ordinate is the light and shadow feature parameter, and the spatial position is the distance between all intersection areas and the central area; For the central area, taking its corresponding discrete point as the center, draw a circle with a diameter of 1 / 10 of the size of the scatter plot in the scatter plot as the action range of the central area; calculate the average value of the abscissa difference and the average value of the ordinate difference between the central area and all intersection areas within its action range; the abscissa difference is the absolute value of the difference between the abscissa of the central area and the abscissa of the intersection area, and the ordinate difference is the absolute value of the difference between the ordinate of the central area and the ordinate of the intersection area; Calculate the ratio of the average value of the ordinate difference to the average value of the abscissa difference of the central area as the light and shadow complexity of the central area.

[0012] In one embodiment, the method for calculating the edge blur parameter of the intersection area according to the light and shadow feature parameter and the light and shadow complexity of the intersection area is: The edge blur parameter of the intersection area is positively correlated with the light and shadow complexity of the intersection area and is positively correlated with the light and shadow feature parameter of the intersection area.

[0013] In one embodiment, the method for adjusting the maximum standard deviation of Gaussian filtering based on the edge blur parameter to obtain the standard deviation of Gaussian filtering is: , represents the pine branch level complexity of the intersection area; represents the edge blur parameter of the intersection area, represents the maximum standard deviation of Gaussian filtering, represents the standard deviation of Gaussian filtering after adjustment.

[0014] In a second aspect, the embodiments of the present application further provide a simulation Christmas tree leaf defect detection system, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of the above-mentioned simulation Christmas tree leaf defect detection method.

[0015] The beneficial effects of the present application are: First, this application uses traditional edge detection algorithms to extract all edge curve contours in the leaf surface image, and further identifies and extracts edge-dense overlapping regions. Then, by analyzing these edge-dense overlapping regions, different types of pine needle leaf overlapping situations are identified, and then all leaf edge intersection regions in the image are extracted. The intersection regions are classified, and different levels of leaf edge intersections are identified. By measuring the density of different levels of leaf intersections around the intersection region, the degree of influence of the intersection region by light and shadow from other regions is evaluated. Finally, combining the evaluation results of the light and shadow influence, the filtering and smoothing scales for different intersection regions are determined to eliminate the influence of the overlapping region on the edge detection result, thereby improving the accuracy and robustness of leaf edge detection. Description of the Drawings

[0016] To more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0017] Figure 1 Flowchart of a method for detecting defects in the leaves of a simulated Christmas tree provided by an embodiment of the present application; Figure 2 Schematic diagram of an edge image; Figure 3 Schematic diagram of a scatter plot. Detailed Embodiments

[0018] To further elaborate on the technical means and effects adopted by the present application to achieve the intended invention purpose, the following will, in combination with the drawings and preferred embodiments, detail the specific embodiments, structures, features, and effects of a method and system for detecting defects in the leaves of a simulated Christmas tree proposed according to the present application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art belonging to the technical field of the present application.

[0020] An embodiment of a method and system for detecting defects in the leaves of a simulated Christmas tree: The following will specifically describe the specific solutions of a method and system for detecting defects in the leaves of a simulated Christmas tree provided by the present application with reference to the drawings.

[0021] Please refer to Figure 1, which shows a flowchart of a method for detecting defects in the leaves of a simulated Christmas tree provided by an embodiment of the present application. The method includes the following steps: Step S001, collect an image of the leaves of the simulated Christmas tree.

[0022] Deploy an industrial camera directly above the leaves of the simulated Christmas tree. Set the camera exposure frequency to 1 s / time. The industrial camera used is a high-resolution industrial camera. Place an LED light source beside the leaves of the simulated Christmas tree to ensure image quality. Based on this, collect an image of the leaves of the simulated Christmas tree.

[0023] So far, an image of the leaves of the simulated Christmas tree has been obtained.

[0024] Step S002, perform edge detection on the leaf image to divide regions, obtain an edge overlap index based on the difference between the edges and the fitted straight lines in the regions, and splice the regions into an edge overlap region.

[0025] The central processing unit is used to retrieve the image of the leaves of the simulated Christmas tree. Labeling tools such as LabelMe and VGG Image Annotator are used to perform pixel-level annotation on the image dataset to calibrate the leaf structure region; the annotated image dataset is subjected to U-Net semantic segmentation training. The image of the leaves of the simulated Christmas tree is input into the trained U-Net network, and the leaf region is segmented from the background and branches, reducing the interference of ornaments and branches on defect detection and focusing on the accurate identification of defects on the leaf surface, so as to obtain an interference-free image of the leaves of the simulated Christmas tree and update the image of the leaves of the simulated Christmas tree.

[0026] Since the simulated leaves are inserted between the skeletons, a dense and complex structure is formed. Due to the intersection between the branches and the leaves, and the leaf layers on the same branch may overlap and block each other, edge aliasing is likely to occur during existing image segmentation. Therefore, edge detection needs to be performed on the image of the leaves of the simulated Christmas tree to obtain an edge overlap region.

[0027] Use an edge detection algorithm to extract all the edge lines in the image of the leaves of the simulated Christmas tree. In this embodiment, the edge detection method used is the Canny edge detection algorithm to obtain an edge image. The image after edge extraction is as Figure 2 shown.

[0028] Divide the image into regions. Specifically, construct a window region with a size of 5*5 (unit: millimeter of the image). Starting from the upper left corner of the image, from left to right, from top to bottom, until the lower right corner of the image, the edge image is thus divided into several local window regions.

[0029] Count the number of edge lines within the statistical window area, and record the number of curves as the discontinuity index of the window area; due to the regularity of window area division, the window area may be set between two individual pine needle leaves, resulting in multiple edge lines in the window area, which is manifested as edge discontinuity.

[0030] Since the continuous edges in the edge overlap area are combinations between different individual pine needle leaves, the angles of their edge lines are different, resulting in multiple inflection points in the continuous edge segments within the area.

[0031] Obtain the optimal fitting line of each edge line within the window area through the least squares method and linear fitting, and calculate the goodness of fit between each edge line within the window area and the optimal fitting line; obtain the mean value of the goodness of fit between all edge lines within the window area and their optimal fitting lines, and record it as the mean goodness of fit of the window area; the smaller the mean goodness of fit, the greater the difference between the edge line within the window area and the optimal fitting line, that is, the more inflection points of the edge line.

[0032] Calculate the edge overlap index of the window area based on the discontinuity of the window area and the mean goodness of fit.

[0033] The edge overlap index has a positive correlation with discontinuity and a negative correlation with the mean goodness of fit.

[0034] It should be noted that positive correlation means that when one variable increases, the other variable also increases, and the change directions of the two variables are the same. When one variable changes from large to small or from small to large, the other variable also changes from large to small or from small to large; the specific relationship is determined by actual applications, and this application does not make special restrictions.

[0035] It should be noted that negative correlation means that when one variable increases, the other variable decreases, and the change directions of the two variables are opposite. When one variable changes from large to small or from small to large, the other variable also changes from small to large or from large to small; the specific relationship is determined by actual applications, and this application does not make special restrictions.

[0036] Preferably, in this embodiment, the ratio of the discontinuity of the window area to the mean goodness of fit of the window area is normalized as the edge overlap index of the window area. The larger this value, the greater the possibility that there is an edge overlap area in the window area. The normalization method used in this embodiment is linear normalization.

[0037] Obtain the edge overlap indices of all window areas, mark the window areas with edge overlap indices greater than the preset empirical value as edge overlap areas, and simply splice adjacent edge overlap areas in space as the same area.

[0038] Thus, multiple edge overlap areas in the edge image are obtained.

[0039] Step S003: Obtain the maximum circumscribed ellipse of the edge lines in the edge overlapping region to obtain the complexity level of pine branches; obtain the intersection region and the shadow region of the edge lines in the edge overlapping region, and obtain the light and shadow feature parameters according to the pixel values of the shadow region in the intersection region and the pixel differences between the shadow regions to form a scatter plot; obtain the light and shadow complexity according to the differences between the horizontal and vertical coordinates of the scatter plot, and calculate the edge blur parameter accordingly.

[0040] The edge dense region is essentially formed by the interlacing of individual pine needles and leaves at different levels and branches, resulting in the inability to accurately segment the individual differences of the regional edge contour. Therefore, it is necessary to further distinguish the edge pixel of the leaves at different levels in the edge overlapping region according to the light and shadow effects of the individual pine needles at different levels.

[0041] The edge overlapping region is divided into two categories. The first category of components is composed of pine needle structures at different levels on the same branch. In the edge image, the maximum diameters of the circumscribed ellipses of different edge contours are different. The second category of components is composed of pine needle structures at the same or different levels on different branches. There will be a large number of intersections in the dense edge structures of pine needles at different levels, resulting in different light and shadow effects. The shadow of the upper-layer pine needles covers the edge of the lower-layer pine needles, causing different degrees of blurring of their edges.

[0042] Perform leaf detection through YOLO, and combine the SORT object tracking algorithm to realize the automatic detection and tracking of leaves, and then judge which leaves belong to the same leaf branch. The edge dense regions belonging to the same branch are recorded as type A overlapping regions, and the edge dense regions belonging to different branches are recorded as type B overlapping regions.

[0043] Record the first category as type A overlapping region; obtain the maximum circumscribed ellipse of all edge lines in the edge overlapping region and its maximum diameter data set, calculate the standard deviation of the maximum diameter data set, and record the standard deviation as the complexity level of pine branches in the edge overlapping region.

[0044] Record the second category as type B overlapping region; extract the intersections of all edge lines in the edge overlapping region through the Hough transform. Take the two edge lines forming the intersection as the boundaries. The intersection divides the edge line into multiple segments. Take the angles corresponding to the two longest segments as the edge angles, and spread inward from the edge angles until there are a preset number of pixel points in the region. The spread will not spread to the edge line. If the number of pixels in the region is less than the preset number after the spread is completed, the spread will also stop to obtain the intersection region. In this embodiment, the preset number is 100 * 100.

[0045] Specifically, during diffusion, taking the intersection point of the two edge lines as the origin, all adjacent points of the origin within the angle form a pixel data set. Then, the adjacent points of all data points within the data set are put into the pixel data set as the diffused pixel points until the number of pixel points in the pixel data set reaches a preset number or diffusion is no longer possible. After stopping diffusion, the obtained area is recorded as the intersection area.

[0046] Under a fixed light background, the intersection areas formed by the combination of pine needle leaves at different levels often exhibit different light and shadow effects. For the intersection area, the Otsu algorithm is used to obtain the optimal segmentation threshold of the intersection area. Pixel points with pixel values less than the segmentation threshold are regarded as shadow pixel points, and the entire intersection area is traversed to obtain the shadow area.

[0047] First, quantify the light and shadow characteristics of all intersection areas. The intersection area is composed of two different pine needle edges, and the composition methods of its pine needle composition structure include the interlacing of pine needles at the same level and the interlacing of pine needles at different levels, and there are also significant differences in their corresponding shadow characteristics.

[0048] For each intersection area, draw the angular bisector of the corresponding angle of the intersection area through the intersection point, and record the perpendicular line of the angular bisector as the tangent line of the intersection area.

[0049] For the interlacing characteristics of pine needles at the same level in the intersection area, the pixel distribution of the shadow part of the area is relatively stable. Draw multiple tangent lines of the intersection area, where the distances between the tangent lines are the same. In this embodiment, 10 tangent lines of the intersection area are obtained. Calculate the mean value of the pixel values in each tangent line of the intersection area, and calculate the absolute value of the difference between the mean values of the pixel points between adjacent tangent lines in the intersection area, which is used as the first light and shadow feature index of the intersection area.

[0050] For the interlacing characteristics of pine needles at different levels in the intersection area, the distribution of the shadow part of the area is relatively disordered. Draw the angular bisector of the corresponding angle of the intersection area. The angular bisector not only divides the angle into two equal angles, but also divides the intersection area into two areas. The shadow areas in the two areas are respectively recorded as the first shadow area and the second shadow area. Calculate the mean square error of the pixel values of all pixel points in the first shadow area and the second shadow area. The larger the mean square error, the more significant the difference between the shadows on both sides of the intersection area, and this value is also different with the different layer differences of the corresponding pine needle edges forming the intersection area. Take the mean square error as the second light and shadow feature index of the intersection area; Take the sum of the first light and shadow feature index and the second light and shadow feature index of the intersection area as the light and shadow feature parameter of the intersection area. The larger the light and shadow feature parameter, the more significantly the intersection area is affected by light and shadow.

[0051] Obtain the light and shadow feature parameters of all intersection regions of the simulated Christmas tree leaf images according to the above steps. Map all the intersection regions to a scatter plot according to their spatial positions and light and shadow feature parameters, as Figure 3 shown.

[0052] Specifically, taking any one intersection region as the central region, the abscissa of the scatter plot is the spatial position, the ordinate is the light and shadow feature parameter, and the spatial position is the distance between all intersection regions and the central region. The farther away from the central region, the larger the abscissa, and the closer to the central region, the smaller the abscissa. In this embodiment, the size of the scatter plot is 10*10.

[0053] For the central region, taking the corresponding discrete point as the center, draw a circle with a diameter of 1 / 10 of the size of the scatter plot in the scatter plot as the action range of the central region. Calculate the average value of the abscissa difference and the average value of the ordinate difference between the central region and all intersection regions within its action range. The abscissa difference is the absolute value of the difference between the abscissa of the central region and the abscissa of the intersection region, and the ordinate difference is the absolute value of the difference between the ordinate of the central region and the ordinate of the intersection region.

[0054] Calculate the ratio of the average value of the ordinate difference to the average value of the abscissa difference of the central region as the light and shadow complexity of the central region. The larger this value is, it means that there are a large number of intersection regions with significant light and shadow differences around the central region, which may have a greater impact on the light and shadow effect (such as the degree of edge blurring) of the central region.

[0055] Therefore, obtain the edge blurring parameters of the intersection regions based on the light and shadow complexity and light and shadow feature parameters of each intersection region.

[0056] The edge blurring parameter of the intersection region is positively correlated with the light and shadow complexity of the intersection region and is positively correlated with the light and shadow feature parameter of the intersection region.

[0057] Preferably, in this embodiment, the product of the light and shadow complexity of the intersection region and the light and shadow feature parameter of the intersection region is used as the edge blurring parameter of the intersection region.

[0058] So far, the edge blurring parameters of the intersection regions have been obtained.

[0059] Step S004, adjust the standard deviation of Gaussian filtering based on the edge blurring parameter and then perform smoothing denoising on the image, and perform defect detection on the denoised image.

[0060] In the shaded area of the pine needle leaves, the light and shadow of the upper-layer pine needles may cause serious coverage of the lower-layer pine needles, resulting in blurring of the lower-layer edges and thus loss of edge information of surface defects. The traditional Gaussian filtering method may not be able to accurately eliminate such blurred pseudo-edge pixel points. Therefore, for the area where there are individual pine needle leaves at different levels and the local edge blurring is serious, the smoothing scale of Gaussian filtering can be adaptively adjusted to reduce the blurring effect and more accurately restore the edge details.

[0061] For Gaussian filtering windows of different sizes, they have theoretically different maximum standard deviations. In this embodiment, the maximum standard deviation of any Gaussian filtering window is obtained.

[0062] Based on the edge blurring parameter of the intersection area and the complexity of the pine branch levels in the intersection area, the maximum standard deviation of Gaussian filtering is adjusted to obtain the standard deviation of Gaussian filtering. The expression is: , represents the complexity of the pine branch levels in the intersection area; represents the edge blurring parameter of the intersection area, represents the maximum standard deviation of Gaussian filtering, represents the standard deviation of Gaussian filtering after adjustment.

[0063] Based on the standard deviation of Gaussian filtering, a set of weights is generated, and based on this, the pixel values in the simulated Christmas tree leaf image are weighted and averaged to obtain a smoothed image.

[0064] The Sobel operator is used to calculate the gradient magnitude and direction of each pixel point in the smoothed image, and non-maximum suppression is performed in the gradient direction to accurately locate the edges. Finally, through edge intensity thresholding, the edge points are divided into strong edges and weak edges; The detected edges are smoothed to obtain the final smoothed image, thereby reducing the false edges caused by noise. Common methods include morphological operations (such as erosion, dilation, opening operation, closing operation), etc.

[0065] Images with different defects are classified and marked, and multiple images with defects are used for network training. In this embodiment, the neural network used is the CNN convolutional neural network, and the loss function is the cross-entropy loss function. The final image is input into the trained model to identify the defects in the final smoothed image.

[0066] Based on the same inventive concept as the above method, an embodiment of the present invention also provides a simulated Christmas tree leaf defect detection system, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above-described simulated Christmas tree leaf defect detection methods.

[0067] It should be noted that the above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present application, and should all be included within the protection scope of the present application.

[0068] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. The key point of each embodiment is to illustrate the differences from other embodiments.

Claims

1. A method for detecting defects in leaves of a simulated Christmas tree, characterized in that: The method comprises the following steps: Collect images of simulated Christmas tree leaves; Perform edge detection on the simulated Christmas tree leaf image to obtain an edge image; divide the edge image into several window areas, perform linear fitting on the edge lines in the window areas, and calculate the goodness of fit between the fitting straight line and the edge line, and calculate the edge overlap index of the window area according to the goodness of fit and the number of edge lines in the window area; splice multiple window areas into edge overlap areas based on the edge overlap index; Obtain the maximum circumscribed ellipse of all edge lines in the edge overlapping area, and use the standard deviation of the diameter of the maximum circumscribed ellipse as the pine branch layer complexity of the edge overlapping area; obtain the intersection points and edge angles of all edge lines in the edge overlapping area, and form an intersection area with the edge lines constituting the intersection as the boundary; divide the intersection area to obtain the shadow area; make a bisector of the edge angle, and record the perpendicular line of the bisector as the tangent of the intersection area; obtain the first light and shadow feature index according to the pixel value of the shadow area in all tangents in the intersection area; divide the shadow area into two areas based on the bisector, and obtain the second light and shadow feature index according to the difference in pixel values ​​of the two areas; use the sum of the first light and shadow feature index and the second light and shadow feature index of the intersection area as the light and shadow feature parameter of the intersection area; map all intersection areas into a scatter plot according to their spatial positions and light and shadow feature parameters; obtain the light and shadow complexity of the intersection area according to the difference in horizontal and vertical coordinates between the discrete points in the scatter plot and the surrounding discrete points; calculate the edge blurring parameter of the intersection area according to the light and shadow feature parameters and light and shadow complexity of the intersection area; The maximum standard deviation of the Gaussian filter is adjusted based on the edge blur parameter to obtain the standard deviation of the Gaussian filter, and the simulated Christmas tree leaf image is smoothed based on the Gaussian filter and then smoothed again through edge detection, and the final smoothed image is passed through the network for defect detection.

2. A method for detecting defects in artificial Christmas tree leaves as claimed in claim 1, characterized in that: The method of performing linear fitting on the edge lines in the window area, calculating the goodness of fit between the fitted straight line and the edge lines, and calculating the edge overlap index of the window area according to the goodness of fit and the number of edge lines in the window area is: The best fitting straight line for each edge line in the window region is obtained by a linear fitting algorithm, and the goodness of fit between each edge line in the window region and the best fitting straight line is calculated; the average of the goodness of fit between all edge lines in the window region and their best fitting straight lines is obtained, and recorded as the average goodness of fit of the window region; Count the number of edge lines in the window area, and record the number of edge lines as a discontinuity index of the window area; The edge overlap index of the window area is calculated according to the goodness-of-fit mean of the window area and the discontinuity index of the window area; The edge overlap index is positively correlated with discontinuity and negatively correlated with the mean goodness of fit.

3. A method for detecting defects in leaves of a simulated Christmas tree as claimed in claim 1, characterized in that: The method for splicing multiple window areas into edge overlapping areas based on edge overlapping indicators is: The window area with edge overlap index greater than the preset empirical value is recorded as the edge overlap area, and the adjacent edge overlap areas are spatially spliced ​​and regarded as the same area.

4. A method for detecting defects in artificial Christmas tree leaves as claimed in claim 1, characterized in that: The method for obtaining the intersection points and edge angles of all edge lines in the edge overlap area and forming an intersection area with the edge lines forming the intersection points as the boundaries is: The intersection of all edge lines in the edge overlapping area is extracted by Hough transform, and the two edge lines constituting the intersection are used as the boundary. The intersection divides the edge line into multiple segments, and the angles corresponding to the two longest segments are used as edge angles, which are diffused into the inner side of the edge angle. During the diffusion, the intersection of the two edge lines is used as the origin, and all the adjacent points of the origin in the edge angle constitute a pixel data set. Then, the adjacent points of all data points in the data set are put into the pixel data set as diffused pixel points, until the number of pixel points in the pixel data set reaches the preset number, or it cannot diffuse, and the area obtained after stopping the diffusion is recorded as the intersection area.

5. A method for detecting defects in leaves of a simulated Christmas tree as claimed in claim 1, characterized in that: The method for obtaining the first light and shadow feature index according to the pixel values ​​of the shadow area in all tangent lines in the intersection area is: The mean value of the pixel value in each tangent line of the intersection area is calculated, and the absolute value of the difference between the mean values ​​of the pixel points between adjacent tangent lines in the intersection area is calculated, which is used as the first light and shadow feature index of the intersection area.

6. A method for detecting defects in leaves of a simulated Christmas tree as claimed in claim 1, characterized in that: The method of dividing the shadow area into two areas based on the bisector and obtaining the second light and shadow feature index according to the difference in pixel values ​​of the two areas is: The bisector divides the intersection area into two areas; the shadow areas in the two areas are respectively recorded as the first shadow area and the second shadow area; the mean square error of the pixel values ​​of all pixels in the first shadow area and the second shadow area is calculated, and the mean square error is used as the second light and shadow feature indicator of the intersection area.

7. A method for detecting defects in leaves of a simulated Christmas tree as claimed in claim 1, characterized in that: The method of mapping all the intersection areas into a scatter plot according to their spatial positions and light and shadow feature parameters; and obtaining the light and shadow complexity of the intersection area according to the difference between the horizontal and vertical coordinates of the discrete points in the scatter plot and the surrounding discrete points is: Taking any intersection area as the central area, the horizontal axis of the scatter plot is the spatial position, the vertical axis is the light and shadow characteristic parameter, and the spatial position is the distance between all intersection areas and the central area; For the central area, take its corresponding discrete point as the center, and draw a circle with a diameter of 1 / 10 of the size of the scatter plot in the scatter plot as the scope of action of the central area; calculate the mean difference of the horizontal coordinates and the mean difference of the vertical coordinates between the central area and all the intersection areas within its scope of action; the horizontal coordinate difference is the absolute value of the difference between the horizontal coordinates of the central area and the intersection area, and the vertical coordinate difference is the absolute value of the difference between the vertical coordinates of the central area and the intersection area; The ratio of the mean difference of the ordinate in the central area to the mean difference of the abscissa is calculated as the light and shadow complexity of the central area.

8. A method for detecting defects in leaves of a simulated Christmas tree as claimed in claim 1, characterized in that: The method for calculating the edge blur parameters of the intersection area according to the light and shadow characteristic parameters and light and shadow complexity of the intersection area is: The edge blur parameters of the intersection area are positively correlated with the light and shadow complexity of the intersection area, and are positively correlated with the light and shadow characteristic parameters of the intersection area.

9. A method for detecting defects in leaves of a simulated Christmas tree as claimed in claim 1, characterized in that: The method for adjusting the maximum standard deviation of the Gaussian filter based on the edge blur parameter to obtain the standard deviation of the Gaussian filter is: , The pine branch level complexity representing the intersection area; Indicates the edge blur parameter of the intersection area, represents the maximum standard deviation of Gaussian filtering, Represents the standard deviation after Gaussian filter adjustment.

10. A system for detecting defects in leaves of a simulated Christmas tree, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of a method for detecting defects in leaves of a simulated Christmas tree as described in any one of claims 1-9 are implemented.

Images