Visual inspection methods and systems for crop growth environment

Abstract

This application relates to the field of image data processing technology, specifically to a visual detection method and system for crop growth environments. The method includes: acquiring standard *Sonchus oleraceus* images from multiple angles; for each angle, constructing a yellow distribution factor for the standard *Sonchus oleraceus* image; constructing a segmentation factor for the standard *Sonchus oleraceus* image, adaptively determining the number of superpixel blocks to obtain the superpixel blocks corresponding to the standard *Sonchus oleraceus* image; constructing petal saliency values ​​for each superpixel block; constructing *Sonchus oleraceus* saliency factors for each superpixel block; determining salient regions based on the *Sonchus oleraceus* saliency factors; matching salient feature points in the image to be detected with those in the standard *Sonchus oleraceus* image; and obtaining the detection result of the image to be detected based on the matching results. This application improves the accuracy of detecting *Sonchus oleraceus* in the corn seedling growth environment by analyzing the unique characteristics of *Sonchus oleraceus* in the corn seedling growth environment.

Description

Technical Field

[0001] This application relates to the field of image data processing technology, specifically to a visual inspection method and system for crop growth environment. Background Technology

[0002] Yellow-flowered black nightshade, also known as spiny nightshade, is a significant invasive and noxious weed worldwide. It typically grows in open, disturbed habitats such as fields and farmland. Due to its strong reproductive capacity, adaptability, and competitive nature, yellow-flowered black nightshade readily establishes itself in new environments, competing with existing crops and weeds for sunlight, water, nutrients, and space, severely inhibiting crop growth.

[0003] Due to its strong reproductive capacity, the yellow-flowered nightshade (Solanum nigrum) severely inhibits the growth of maize seedlings in farmland. Therefore, detecting yellow-flowered nightshade in the maize seedling growing environment and promptly removing infested areas based on the detection results is crucial for the healthy growth of maize seedlings and other crops. However, the traditional FLANN matching algorithm for detecting yellow-flowered nightshade in the maize seedling growing environment ignores unique characteristics of this environment, thus reducing the accuracy of detection. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides a visual inspection method and system for crop growth environments to solve the existing issues.

[0005] The visual inspection method and system for crop growth environment in this application adopts the following technical solution:

[0006] In a first aspect, embodiments of this application provide a visual detection method for crop growth environment, the method comprising the following steps:

[0007] Acquire standard images of yellow-flowered Solanum in the corn seedling growth environment from multiple angles;

[0008] For each angle of the standard yellow aster image, yellow pixels in the standard yellow aster image are selected according to the preset hue threshold range, and the yellow distribution factor of the standard yellow aster image is constructed according to the saturation and brightness ratio distribution of the yellow pixels.

[0009] The standard yellow-flowered prickly eggplant image is converted to grayscale to obtain a yellow-flowered prickly eggplant grayscale image. The texture distribution factor of the standard yellow-flowered prickly eggplant image is constructed based on the differences between pixel values ​​in the yellow-flowered prickly eggplant grayscale image. The segmentation factor of the standard yellow-flowered prickly eggplant image is constructed by combining the yellow distribution factor.

[0010] Based on the segmentation factor, superpixel blocks of a standard Solanum nigrum image are obtained using a superpixel segmentation algorithm.

[0011] Obtain all petal edges in each superpixel block based on the complexity of the edges in each superpixel block; construct the petal saliency value of each superpixel block based on the shape and size of the region enclosed by all petal edges in each superpixel block;

[0012] The spike salience value of each superpixel block is constructed based on the distribution ratio of corner points in each superpixel block, and the salience factor of each superpixel block is constructed by combining the petal salience value of each superpixel block to obtain the salient region.

[0013] A feature point detection algorithm is used to obtain salient feature points in the salient regions of the standard Solanum nigrum image. A matching algorithm is then used to match the salient feature points in the salient regions of the image to be detected and the standard Solanum nigrum image. Based on the matching results, the visual detection results of Solanum nigrum in the image to be detected are obtained.

[0014] Preferably, obtaining the yellow pixels in the standard yellow nightshade image and constructing the yellow distribution factor of the standard yellow nightshade image based on the proportional distribution of the saturation and brightness of the yellow pixels includes:

[0015] Create a mask that preserves yellow pixels according to a preset hue threshold range;

[0016] The standard *Helianthus alba* image is converted to the HSV color space. The ratio of the number of yellow pixels with saturation greater than a preset saturation threshold to the total number of yellow pixels in the standard *Helianthus alba* image is recorded as the saturation distribution factor of the standard *Helianthus alba* image; the ratio of the number of yellow pixels with brightness within a preset brightness range to the total number of yellow pixels in the standard *Helianthus alba* image is recorded as the brightness distribution factor of the standard *Helianthus alba* image; and the ratio of the total number of yellow pixels to the total number of pixels in the standard *Helianthus alba* image is recorded as the hue distribution factor of the standard *Helianthus alba* image.

[0017] The yellow distribution factor of the standard Solanum nigrum image is determined based on the saturation distribution factor, brightness distribution factor, and hue distribution factor. The yellow distribution factor of the standard Solanum nigrum image is the result of fusing the saturation distribution factor, brightness distribution factor, and hue distribution factor.

[0018] Preferably, the process of constructing the segmentation factors of the standard Solanum nigrum image is as follows:

[0019] The gray-level co-occurrence matrices of the Solanum nigrum grayscale image are obtained in the horizontal and vertical directions respectively. The product of the contrast and energy of the gray-level co-occurrence matrix in the horizontal direction is used as the feature value in the horizontal direction; the product of the contrast and energy of the gray-level co-occurrence matrix in the vertical direction is used as the feature value in the vertical direction; and the sum of the feature values ​​in the horizontal and vertical directions is used as the texture distribution factor of the standard Solanum nigrum image.

[0020] The segmentation factor of the standard yellow-flowered solanum image is the result of fusing the yellow distribution factor and texture distribution factor of the standard yellow-flowered solanum image.

[0021] Preferably, the process of obtaining superpixel blocks from the standard Solanum nigrum image is as follows:

[0022] The product of the segmentation factor of the standard Solanum nigrum image and the preset segmentation number is calculated and denoted as the segmentation quantity of the standard Solanum nigrum image.

[0023] The number of initial seed points in the superpixel segmentation process is obtained by rounding up the segmentation value of the standard Solanum nigrum image. The superpixel segmentation algorithm is then used to segment the standard Solanum nigrum image into superpixel blocks.

[0024] Preferably, the method for obtaining all petal edges in each superpixel block is as follows:

[0025] In each superpixel block, measure the length of each edge; calculate the mean length of all edges in the superpixel block, and denote it as the feature standard value; edges with a length greater than the feature standard value are taken as petal edges in the superpixel block.

[0026] Preferably, the process for constructing the petal saliency values ​​of each superpixel block is as follows:

[0027] Calculate the area of ​​the region enclosed by all petal edges in each superpixel block, and the perimeter of the region enclosed by all petal edges;

[0028] The ratio of the area to the square of the perimeter is recorded as the salient feature value of each superpixel block. The salient feature value is then multiplied by 4... The product of these is denoted as the near-circularity of each superpixel block;

[0029] The ratio of the number of yellow pixels whose hue falls within the hue threshold range to the total number of pixels in each superpixel block is denoted as the significance ratio of each superpixel block.

[0030] The petal saliency value of each superpixel block is determined based on the near-circularity and saliency ratio, and the petal saliency value of each superpixel block is the result of fusing the near-circularity and saliency ratio of each superpixel block.

[0031] Preferably, the spiking significance value of each superpixel block is the ratio of the number of corner points in each superpixel block to the average number of corner points in all superpixel blocks in the standard Solanum nigrum image.

[0032] Preferably, the construction of the salience factor of *Solanum lyratum* for each superpixel block to obtain salient regions includes:

[0033] The significance factor of Solanum nigrum in each superpixel block is the result of fusing the significance values ​​of the petals and the significance values ​​of the spikes in each superpixel block;

[0034] The salient factors of all superpixel blocks obtained from the segmentation of the standard Solanum nigrum image are arranged in descending order to form a salient factor sequence. The quartiles of all salient factors of Solanum nigrum in the salient factor sequence are calculated, and the superpixel blocks corresponding to the elements in the salient factor sequence that are before the first quartile are recorded as salient regions.

[0035] Preferably, the step of using a matching algorithm to match salient feature points in salient regions of the image to be detected and the standard Solanum nigrum image, and obtaining the visual detection result of Solanum nigrum in the image to be detected based on the matching result, includes:

[0036] Feature points of significant regions in standard Solanum nigrum images at each angle were obtained using a feature point extraction algorithm and used as significant feature points of the standard Solanum nigrum images at each angle.

[0037] RGB images of the growing area of ​​the corn seedling to be detected are acquired using a CMOS camera and recorded as the image to be detected;

[0038] The significant feature points in the image to be detected are obtained according to the method for obtaining significant feature points in the standard Solanum nigrum image;

[0039] A matching algorithm is used to match the salient feature points in the standard Solanum nigrum image at each angle with the salient feature points in the image to be detected to obtain matching salient feature point pairs;

[0040] The similarity ratio at each angle is calculated as the ratio of the number of salient feature point pairs that match between the image to be detected and the standard Solanum nigrum image at each angle to the number of pixels in the image to be detected.

[0041] The ratio of the similarity ratio at each angle to the mean distance between all the matched significant feature point pairs at each angle is used as the input to the normalization function. The product of the output of the normalization function and the coefficient of variation of the similarity ratio at all angles is used as the similarity between the image to be detected and the standard Solanum nigrum image.

[0042] If the similarity is greater than a preset value, the image to be detected contains Solanum nigrum; otherwise, the image to be detected does not contain Solanum nigrum.

[0043] Secondly, embodiments of this application also provide a visual inspection system for crop growth environment, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described visual inspection methods for crop growth environment.

[0044] This application has at least the following beneficial effects:

[0045] First, this application constructs a texture distribution factor for a standard *Solanum lyratum* image based on the differences in pixel values ​​between grayscale images of *Solanum lyratum*. Second, it constructs a segmentation factor for the standard *Solanum lyratum* image by combining the yellow distribution factor of the standard *Solanum lyratum* image with the segmentation factor. Based on the segmentation factor, it adaptively determines the number of superpixel blocks to be segmented in the standard *Solanum lyratum* image. This eliminates the influence of a fixed number on the feature segmentation of the standard *Solanum lyratum* image at different angles and makes the subsequent feature point extraction results of the standard *Solanum lyratum* image at different angles more consistent with the characteristics of *Solanum lyratum*. Then, based on the distribution ratio of corner points in each superpixel block and the petal saliency value of each superpixel block, it constructs a saliency factor for *Solanum lyratum* in each superpixel block and determines the salient region of the standard *Solanum lyratum* image. Finally, it uses a matching algorithm to determine the similarity between the image to be detected and the standard *Solanum lyratum* image based on the matching results of salient feature points in the salient region of the standard *Solanum lyratum* image at different angles. This takes into account the characteristic that the generation direction of *Solanum lyratum* in the corn seedling growth environment is not unique, thus realizing the visual detection of *Solanum lyratum* in the corn seedling growth environment. The visual detection method of this application takes into account the unique features of Solanum nigrum and eliminates the influence of other factors, thereby improving the detection accuracy of Solanum nigrum. The method calculates the number of adaptive segmentation regions by statistical features of the image, and then selects significant regions by the significance value of Solanum nigrum. Feature points are extracted only in significant regions, which strengthens the connection between feature points and Solanum nigrum and improves the accuracy of detecting Solanum nigrum in the growth environment of corn seedlings. Attached Figure Description

[0046] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A flowchart illustrating the steps of the visual inspection method and system for crop growth environment provided in the embodiments of this application;

[0048] Figure 2 This is a schematic diagram of the standard Solanum nigrum image segmentation process provided in the embodiments of this application;

[0049] Figure 3 This is a flowchart illustrating the process of obtaining significant regions as provided in an embodiment of this application. Detailed Implementation

[0050] To further illustrate the technical means and effects adopted by this application to achieve the intended inventive purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the visual inspection method and system for crop growth environment proposed in this application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0052] The following description, in conjunction with the accompanying drawings, details the specific scheme of the visual inspection method and system for crop growth environment provided in this application.

[0053] Please see Figure 1 The diagram illustrates a flowchart of a visual detection method for crop growth environment provided in one embodiment of this application, the method comprising the following steps:

[0054] Step S1: Collect standard images of yellow nightshade in the growth environment of corn seedlings and perform preprocessing.

[0055] In this embodiment, the yellow-flowered prickly eggplant in the growth environment of corn seedlings is used as the detection object. First, a CMOS camera is used to collect standard yellow-flowered prickly eggplant images from multiple angles in the growth environment of corn seedlings. This is because the growth angle of yellow-flowered prickly eggplant is not fixed in the growth environment of crops, i.e., the growth environment of corn seedlings. By collecting images from multiple angles and performing segmentation and matching, accurate visual detection results can be obtained.

[0056] Secondly, a filtering algorithm is used to denoise the standard Solanum nigrum image, eliminating noise and some external interference to avoid its impact on subsequent feature extraction.

[0057] It should be noted that there are many image denoising algorithms. As an embodiment of this application, this embodiment uses the Gaussian filtering algorithm to detect the growth environment of corn seedlings, that is, to denoise the above-mentioned standard yellow-flowered solanum image. Implementers may also use other denoising algorithms for image denoising, and this application does not impose any special restrictions.

[0058] Among them, the Gaussian filtering algorithm is a well-known technique, and its specific denoising principle will not be elaborated here.

[0059] Step S2: Construct the texture distribution factor of the standard yellow-flowered prickly eggplant image based on the differences between pixel values ​​in the standard yellow-flowered prickly eggplant grayscale image, and construct the segmentation factor of the standard yellow-flowered prickly eggplant image by combining the yellow distribution factor of the standard yellow-flowered prickly eggplant image.

[0060] In the growth environment of corn seedlings, corn seedlings constitute the majority of the area, while the remaining area may include green weeds or yellow-flowered solanum. Therefore, in a standard yellow-flowered solanum image, corn seedlings and green weeds appear green, while the flowers of the yellow-flowered solanum appear yellow overall. When using the traditional FLANN matching algorithm to match images, typically only the SIFT feature point detection algorithm is used to detect feature points in the image, and then the feature points in the two images are matched. This approach does not consider the unique color characteristics of yellow-flowered solanum, resulting in poor detection performance for yellow-flowered solanum in the corn seedling growth environment. Therefore, this application considers extracting feature points of the yellow-flowered solanum region in the standard yellow-flowered solanum image based on the distinctive salient features of yellow-flowered solanum in the corn seedling growth environment. Then, the image to be detected is matched with standard yellow-flowered solanum images from multiple angles to detect yellow-flowered solanum in the corn seedling growth environment. Specifically:

[0061] For a standard Solanum nigrum image at each angle, taking any one angle as an example, considering the different information content and complexity of different regions within the standard Solanum nigrum image, information-rich regions should be divided into smaller superpixel blocks to ensure sufficient and more accurate edge information is obtained; flat regions with less information should be clustered into larger superpixel blocks, which can speed up the segmentation process and avoid the generation of isolated, small superpixel blocks.

[0062] Therefore, based on the above analysis, the first step is to determine the number of segmented regions in the image. The more segmented regions there are, the smaller the size of each region, and the more image details can be captured. However, this also means processing more data and extracting more feature points. To balance the ability to capture image details and computational efficiency, the number of segmented regions can be adaptively set based on the statistical characteristics of the image. Specific steps are as follows... Figure 2 As shown, steps S201, S202, S203, S204, S205, and S206 are included:

[0063] S201: Convert the standard yellow aster image to the HSV color space.

[0064] The biggest color difference between Solanum nigrum and corn seedlings, other plants, or the background in their growing environment is that Solanum nigrum is mainly yellow or a similar color. The number of superpixel blocks can be determined based on the distribution ratio of yellow pixels in a standard Solanum nigrum image. More yellow pixels indicate a larger area belonging to Solanum nigrum in the standard image, and the image should be divided into more superpixel blocks to capture more feature points. Conversely, if there are few areas belonging to Solanum nigrum in the standard image, the number of superpixel blocks should be reduced to decrease the number of detected feature points and improve computational efficiency.

[0065] Converting a standard yellow nightshade image to the HSV color space is crucial for better color feature extraction. In the RGB color space, color is determined by three channel values, and changes in lighting conditions can affect the intensity of these channels, leading to color distortion and hindering the calculation of yellow pixel distribution. In the HSV color space, color is determined by three channels: hue, saturation, and lightness. This makes color analysis more intuitive and convenient, and changes in lighting conditions do not affect hue. Therefore, extracting the distribution features of yellow pixels in the HSV color space is more stable and accurate.

[0066] S202: Extract the region where the yellow-flowered prickly eggplant is located based on the color distribution of the standard yellow-flowered prickly eggplant image.

[0067] To more accurately detect *Solanum lyratum* in the growth environment of maize seedlings, the standard *Solanum lyratum* image was divided into tonal regions, specifically as follows:

[0068] Based on the color of *Aster tataricus*, a preset hue threshold range of 20° to 90° is defined. A mask is created using this preset hue threshold range, and pixels with hues falling within this range are designated as yellow pixels. Furthermore, the brightness and saturation histograms of the standard *Aster tataricus* image are calculated after conversion to the HSV color space. In the HSV color space, the hue of yellow is 60°, and a color range close to yellow is selected to represent the color of the *Aster tataricus* flower. Pixels of other colors are filtered out using the preset hue threshold range to reduce interference from other colors. The distribution characteristics of the yellow pixels after conversion are analyzed using the brightness and saturation histograms.

[0069] It should be noted that the preset hue threshold range is set manually. As an embodiment of this application, the preset hue threshold range is set to 20°~90°. Implementers can set it themselves according to specific circumstances, and this application does not impose any special restrictions.

[0070] In image processing, a mask is a binary image used to filter specific regions. The specific process of filtering is a well-known technique and will not be elaborated here.

[0071] S203: Construct a yellow distribution factor based on the distribution of yellow pixels in a standard Solanum nigrum image.

[0072] To better analyze the distribution characteristics of yellow pixels in standard yellow nightshade images, hue, brightness, and saturation are combined to analyze the distribution of yellow pixels. Specifically:

[0073] After converting a standard yellow aster image to the HSV color space, the ratio of the number of yellow pixels with saturation greater than a preset saturation threshold to the total number of yellow pixels in the standard yellow aster image is recorded as the saturation distribution factor of the standard yellow aster image; the ratio of the number of yellow pixels with brightness within a preset brightness range to the total number of yellow pixels in the standard yellow aster image is recorded as the brightness distribution factor of the standard yellow aster image; and the ratio of the total number of yellow pixels to the total number of pixels in the standard yellow aster image is recorded as the hue distribution factor of the standard yellow aster image.

[0074] The yellow distribution factor of the standard Solanum nigrum image is determined based on the saturation distribution factor, brightness distribution factor, and hue distribution factor. The yellow distribution factor of the standard Solanum nigrum image is the result of fusing the saturation distribution factor, brightness distribution factor, and hue distribution factor.

[0075] It should be understood that fusion refers to combining two or more indicators through addition or multiplication to obtain a comprehensive indicator, thereby providing a more comprehensive and accurate assessment of a phenomenon or problem. This fusion method is not limited to simple arithmetic operations and can also include more complex statistical models and analytical methods. Implementers can choose according to specific circumstances, and this embodiment does not impose any special restrictions.

[0076] Preferably, as an embodiment of this application, the yellow distribution factor is the result of multiplying the saturation distribution factor, the brightness distribution factor, and the hue distribution factor.

[0077] It should be noted that the preset saturation range and preset brightness range are set manually. In this embodiment, the preset saturation threshold is taken as an empirical value of 0.7, and the preset brightness range is 0.3~0.7. This value is chosen because yellow is a highly saturated color, so a relatively high saturation threshold needs to be set. Since the brightness of yellow varies greatly, a relatively large brightness range is also set. Implementers can also set other thresholds according to the specific circumstances of the image to ensure more accurate color distribution calculations.

[0078] S204: Convert a standard image of Solanum nigrum to a grayscale image.

[0079] In a standard *Sonchus oleraceus* image, not only does the color distribution affect the number of superpixel blocks, but the texture features of the image also influence the number of superpixel blocks. Images with rich texture features require more superpixel blocks to extract more feature points, resulting in more accurate matching. Conversely, a smaller number of superpixel blocks and fewer extracted feature points are needed to accelerate visual detection. Therefore, the standard *Sonchus oleraceus* image is converted to a grayscale image, denoted as the *Sonchus oleraceus* grayscale image.

[0080] S205: Determine the texture distribution factor of the standard Solanum nigrum image based on the texture features in the horizontal and vertical directions of the Solanum nigrum grayscale image.

[0081] Since the flowers in the standard Solanum nigrum image are arranged horizontally and the color of the flowers becomes darker from top to bottom, in order to analyze Solanum nigrum more in detail, the grayscale image of Solanum nigrum is scanned in both the horizontal and vertical directions, and the grayscale co-occurrence matrix of Solanum nigrum grayscale image in the horizontal and vertical directions is obtained respectively.

[0082] Furthermore, the product of contrast and energy of the gray-level co-occurrence matrix in the horizontal direction is taken as the eigenvalue A1 in the horizontal direction; the product of contrast and energy of the gray-level co-occurrence matrix in the vertical direction is taken as the eigenvalue A2 in the vertical direction; and the sum of the eigenvalues ​​A1 in the horizontal direction and A2 in the vertical direction is taken as the texture distribution factor of the standard Solanum nigrum image. The larger the value of the texture distribution factor, the richer the texture features of the image, and vice versa.

[0083] The calculation process of the gray-level co-occurrence matrix is ​​a well-known technique, and the specific process will not be described in detail here.

[0084] S206: Construct segmentation factors for the standard yellow-flowered sow thistle image based on texture distribution factors and yellow distribution factors; obtain superpixel blocks of the standard yellow-flowered sow thistle image based on the segmentation factors.

[0085] When segmenting a standard Solanum nigrum image, an increased number of segmented regions leads to more refined image analysis, but also increases the complexity of data processing and feature point extraction. Therefore, a segmentation factor is calculated to adaptively set the number of segmented regions, ensuring that necessary image details are captured without excessively increasing the computational burden. The specific process for constructing the segmentation factor is as follows:

[0086] The segmentation factor of the standard Solanum nigrum image is determined based on the yellow distribution factor and the texture distribution factor. The segmentation factor of the standard Solanum nigrum image is the result of fusing the yellow distribution factor and the texture distribution factor. The segmentation factor is positively correlated with the yellow distribution factor and the texture distribution factor, respectively.

[0087] Preferably, as an embodiment of this application, the segmentation factor is the product of the yellow distribution factor and the texture distribution factor.

[0088] In practical applications, as another implementation method, the segmentation factor can be an exponential function value with the natural constant as the base and the sum of the yellow distribution factor and the texture distribution factor as the independent variable.

[0089] Further, the product of the segmentation factor and the preset segmentation number is calculated and denoted as the segmentation amount of the standard Solanum nigrum image. The result of rounding up the segmentation amount of the standard Solanum nigrum image is used as the initial seed point number set in the Simple Linear Iterative SLIC algorithm. The standard Solanum nigrum image is used as input, and the superpixel blocks obtained by segmenting the standard Solanum nigrum image are obtained by using the Simple Linear Iterative SLIC algorithm.

[0090] It should be noted that the preset number of segments is set manually. In this embodiment of the application, the preset number of segments is 128. The implementer can set it according to the specific situation. This application does not impose any special restrictions. In addition, the SLIC superpixel segmentation algorithm is a well-known technology, and the specific process will not be described in detail.

[0091] The more superpixel blocks there are, the more feature points need to be extracted, resulting in more accurate matching results. However, this requires more computational resources and is slower. The above steps, which calculate an adaptive number of superpixel segments, ensure sufficient feature points are extracted for matching, guaranteeing the final recognition effect, while also accelerating computation and saving computational resources.

[0092] Step S3: Filter the petal edges in each superpixel block based on the length features of the edges in each superpixel block; construct the petal salience value of each superpixel block according to the shape and size of the petal edges in each superpixel block; construct the spike salience value of each superpixel block according to the distribution ratio of the corner points in each superpixel block; combine the petal salience value of each superpixel block to construct the yellow thorny solanum salience factor of each superpixel block to obtain the salient region.

[0093] Different superpixel blocks are used to represent different regions in a standard *Solanum nigrum* image. Features of *Solanum nigrum* within each region are obtained. If the saliency factor of *Solanum nigrum* within a region ranks in the top third of all regions, that region is designated as a salient region, and feature points are extracted from it. Non-salient regions are not used for feature point extraction. This approach ensures that the extracted feature points are specific to *Solanum nigrum*, rather than other parts of the image, while also reducing the number of feature points that need to be extracted. This not only guarantees a high success rate for feature point matching and enhances the recognition effect but also speeds up the recognition process. The process for obtaining salient regions is as follows: Figure 3 As shown, steps S301, S302, and S303 are included:

[0094] S301: Uses an edge detection algorithm to extract edges from all superpixel blocks.

[0095] Edge detection algorithms are used to detect edges in each superpixel block. Morphological operations are used to make the edges clearer, and polygon approximation algorithms are used to simplify the edge lines and form closed contours.

[0096] It should be noted that there are many commonly used edge detection algorithms. As an embodiment of this application, the Canny edge detection algorithm is used to extract the edge contours of each superpixel block. Implementers may also use other edge detection algorithms, and this embodiment does not impose any special restrictions. In addition, morphological operations and polygon approximation algorithms are used in this embodiment to enhance and simplify the edge contours. Implementers may also use other methods to achieve the purpose of enhancing and simplifying the edge contours.

[0097] The Canny edge detection algorithm, morphological operations, and polygon approximation algorithm are well-known technologies, and their specific processes will not be elaborated here.

[0098] S302: Obtain the petal edges in each superpixel block based on the length features of the edges in each superpixel block; construct the petal saliency value of each superpixel block based on the shape and size of the petal edges in each superpixel block.

[0099] Within each superpixel block, the length of each edge is measured; the mean length of all edges in the superpixel block is calculated and denoted as the feature standard value; edges with lengths greater than the feature standard value are designated as petal edges in the superpixel block. Further, based on the shape and size of the petal edges in each superpixel block, the petal saliency value of each petal in each superpixel block is calculated, specifically as follows:

[0100] Analyze the near-circularity of the regions enclosed by the edges of all petals in each superpixel block. Specifically: analyze the area and perimeter of the regions enclosed by the edges of all petals in each superpixel block; measure the ratio of the area to the square of the perimeter of the regions enclosed by the petal edges, and record it as the salient feature value of each superpixel block; then, multiply the salient feature value by 4... The product of these factors is denoted as the near-circularity of each superpixel block.

[0101] The ratio of the number of yellow pixels whose hue falls within the hue threshold range to the total number of pixels in each superpixel block is denoted as the significance ratio of each superpixel block.

[0102] The petal saliency value of each superpixel block is determined based on the near-circularity and saliency ratio. The petal saliency value of each superpixel block is the result of fusing the near-circularity and saliency ratio of each superpixel block.

[0103] It should be noted that fusion refers to combining two or more indicators through addition or multiplication to obtain a comprehensive indicator, thereby providing a more comprehensive and accurate assessment of a phenomenon or problem. This fusion method is not limited to simple arithmetic operations and can also include more complex statistical models and analytical methods. Implementers can choose according to specific circumstances, and this embodiment does not impose any special restrictions.

[0104] Preferably, as an embodiment of this application, the petal saliency value of each petal in each superpixel block is the product of the near-circularity and saliency ratio of each petal in each superpixel block.

[0105] Typically, each flower of the yellow nightshade has 5 petals. After segmenting the image into superpixel blocks, only a few superpixel blocks contain petals. Based on the closed shape formed by the edges of the petals, only when the length of each petal edge in the entire superpixel block is greater than the average length of all edges can it be classified as a petal outline. The closed shape formed by the petal edges is generally close to a circle. The probability that the petal edge belongs to a petal outline is characterized by calculating the near-circularity of the closed shape formed by the edges of each petal. The larger the value, the closer the closed shape is to a circle, and the more likely it is to be a petal outline. The probability of it being a petal outline is further determined by the proportion of yellow pixels in the superpixel block. The more yellow pixels in the superpixel block, the more likely it is to contain petals.

[0106] The above steps calculate the petal saliency value of each petal in each superpixel block. The larger the petal saliency value, the greater the probability that the superpixel block contains a petal of Solanum nigrum, and the more likely feature points should be extracted from it. Conversely, the smaller the petal saliency value, the less likely the superpixel block contains a petal of Solanum nigrum, and the probability of extracting feature points should be reduced.

[0107] S303: Construct the spike salience value of each superpixel block based on the distribution ratio of corner points in each superpixel block; combine the petal salience value of each superpixel block to construct the yellow thorn solanum salience factor of each superpixel block, and determine the salient region of the standard yellow thorn solanum image.

[0108] Besides its yellow petals, the most prominent feature of the yellow nightshade is its numerous sharp spines. Therefore, for superpixels that belong to the yellow nightshade rather than petals, corner detection can be used to obtain the number of corner points to further determine whether a superpixel belongs to the yellow nightshade. Only superpixels with a spine significance value greater than a preset threshold can be considered to belong to the yellow nightshade. The process of constructing the spine significance value is as follows:

[0109] Corner detection algorithms are used to detect corners in each superpixel block. The ratio of the number of corners in each superpixel block to the average number of corners in all superpixel blocks in the standard Solanum nigrum image is calculated and recorded as the spiky significance value of each pixel block.

[0110] It should be noted that in this embodiment, the preset threshold is set to an empirical value of 1. In actual applications, as other implementation methods, the preset threshold can be other suitable values, and this application does not impose any special restrictions on it.

[0111] Furthermore, by combining the petal salience values ​​of each petal in each superpixel block, a salience factor and salience region for *Solanum nigrum* are constructed, specifically:

[0112] The salience factor of Solanum nigrum for each superpixel block is determined based on the petal salience value and the spike salience value. The salience factor of Solanum nigrum for each superpixel block is the result of fusing the petal salience value and the spike salience value of each superpixel block.

[0113] Preferably, in this embodiment, the salience factor of each superpixel block is the sum of the petal salience value and the salience value of each superpixel block. In practical applications, as another implementation, the salience factor of each superpixel block is the product of the petal salience value and the salience value of each superpixel block.

[0114] For Solanum fulva, the petals and other parts must be considered. A superpixel block may contain only petals or other parts. Therefore, the petal salience value and the spike salience value are added together to represent the probability that a superpixel block contains Solanum fulva. The larger the value, the greater the probability that it contains Solanum fulva, and vice versa.

[0115] Arrange the saliency factors of all superpixel blocks in descending order to form a saliency factor sequence. Calculate the quartiles of all saliency factors of the saliency factors in the saliency factor sequence. The superpixel blocks corresponding to the elements in the saliency factor sequence that are before the first quartile are recorded as saliency regions.

[0116] Step S4: Use a matching algorithm to match salient feature points in the salient regions of the image to be detected and the standard Solanum nigrum image, and obtain the visual detection result of Solanum nigrum in the image to be detected based on the matching result.

[0117] Specifically, feature points are extracted from salient regions and feature point matching is performed. By selecting salient regions, the influence of feature points in superpixel blocks that do not contain Solanum nigrum is eliminated, which not only makes the detection results more accurate, but also speeds up the calculation.

[0118] As an example, the SIFT feature point detection algorithm is used on the salient regions of the standard Solanum nigrum image, and the detected feature points are used as the salient feature points of the standard Solanum nigrum image.

[0119] SIFT is a well-known technology in this field, and the specific process will not be described in detail.

[0120] Further, based on the above process, salient feature points in the salient regions of the standard *Solanum nigrum* image at each angle are obtained. Next, an RGB image of the corn seedling growth area to be detected is acquired using a CMOS camera, designated as the image to be detected. Following the above process, salient feature points in the image to be detected are obtained. Then, the FLANN algorithm is used to match the salient feature points in the salient regions of the standard *Solanum nigrum* image at each angle with the salient feature points in the image to be detected. Two matching salient feature points in the two images are designated as a matching salient feature point pair. Based on the matching results of the salient feature points in the salient regions of the standard *Solanum nigrum* image at all angles with the salient feature points in the image to be detected, the similarity between the image to be detected and the standard *Solanum nigrum* image is determined, specifically:

[0121] The expression for the similarity between the image to be detected and the standard Solanum nigrum image is:

[0122] In the formula, V represents the similarity between the image to be detected and the standard Solanum nigrum image; norm() represents the normalization function; N represents the number of angles from which the standard Solanum nigrum image was collected. This represents the ratio of the number of significant feature point pairs matched between the image to be detected and the standard Solanum nigrum image at the j-th angle to the number of pixels in the image to be detected. This represents the coefficient of variation for all of the stated ratios; Let represent the mean distance between all matching salient feature point pairs between the image to be detected and the standard Solanum nigrum image at the j-th angle. These are parameter tuning constants used to prevent the denominator from being zero. The size is taken as an empirical value of 0.001.

[0123] Among them, the more identical image information containing *Solanum nigrum* (a type of orchid) is between the image to be detected and the standard *Solanum nigrum* image at the j-th angle, the more salient feature point pairs are matched between the two images, and the smaller the matching distance. The larger the value, the larger the region of *Solanum nigrum* contained in the image to be detected. Therefore, the number of salient matching point pairs that can be successfully matched between the salient regions of the image to be detected and the standard *Solanum nigrum* images at N different angles will not differ significantly. Thus, the ratio of the number of salient feature point pairs matched between the image to be detected and the standard *Solanum nigrum* images at N different angles to the number of pixels in the image to be detected is close to the value of the salient feature point pairs matched between the image to be detected and the standard *Solanum nigrum* images at N different angles. The smaller the value of , the larger the value of V.

[0124] If the similarity between the image to be detected and the standard Solanum nigrum image is greater than a preset value, then the image to be detected contains Solanum nigrum; otherwise, the image to be detected does not contain Solanum nigrum. The FLANN algorithm is a well-known technique, and its specific steps will not be described in detail here.

[0125] It should be noted that the preset value in this embodiment is 0.7. Implementers may also set it according to their specific circumstances. This application does not impose any special restrictions.

[0126] Thus, this embodiment obtains standard images of Solanum nigrum from different angles in the corn seedling growth environment. Based on the significant differences in color space and texture between Solanum nigrum and other objects in the specific growth environment of the corn seedling, and combined with a feature point extraction algorithm, significant feature points in the standard Solanum nigrum images are obtained. Furthermore, following the method for extracting significant feature points, significant feature points in the image to be detected are extracted. Combined with a matching algorithm, Solanum nigrum in the image to be detected is detected, thereby improving the accuracy of visual detection of Solanum nigrum in the local growth environment of corn seedlings.

[0127] Based on the same inventive concept as the above methods, embodiments of this application also provide a visual inspection system for crop growth environment, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above-described visual inspection methods for crop growth environment.

[0128] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments of this specification have been described above. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0129] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0130] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them; modifications to the technical solutions described in the foregoing embodiments, or equivalent substitutions of some of the technical features, do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for visual inspection of a crop growing environment, characterized by, The method comprises the following steps: Obtaining standard Solanum lycopersicum images in corn seedling growth environment under multiple angles; For each standard Solanum lycopersicum image under each angle, yellow pixels in the standard Solanum lycopersicum image are obtained by screening according to a preset hue threshold range, and a yellow distribution factor of the standard Solanum lycopersicum image is constructed according to the proportion distribution of saturation and brightness of the yellow pixels; The standard Solanum lycopersicum image is grayscaled to obtain a Solanum lycopersicum grayscale image, a texture distribution factor of the standard Solanum lycopersicum image is constructed according to the difference between pixel values in the Solanum lycopersicum grayscale image, and a segmentation factor of the standard Solanum lycopersicum image is constructed in combination with the yellow distribution factor; According to the segmentation factor, a superpixel block of the standard Solanum lycopersicum image is obtained in combination with a superpixel segmentation algorithm; All petal edges in each superpixel block are obtained according to the complexity of edges in each superpixel block, and a petal saliency value of each superpixel block is constructed according to the shape and size of a region surrounded by all petal edges in each superpixel block; A thorn saliency value of each superpixel block is constructed according to the distribution proportion of corner points in each superpixel block, and a Solanum lycopersicum saliency factor of each superpixel block is constructed in combination with the petal saliency value of each superpixel block to obtain a salient region; Significant feature points in the salient region of the standard Solanum lycopersicum image are obtained by using a feature point detection algorithm, the significant feature points in the salient region of the standard Solanum lycopersicum image and the significant feature points in a to-be-detected image are matched by using a matching algorithm, and a visual detection result of Solanum lycopersicum in the to-be-detected image is obtained based on a matching result; The construction process of the petal saliency value of each superpixel block is as follows: The area of a region surrounded by all petal edges in each superpixel block and the circumference of the region surrounded by all petal edges are calculated; The ratio of the area to the square of the perimeter is denoted as a salient feature value of each superpixel block, and the product of the salient feature value and 4 is denoted as the near-circularity of each superpixel block. The ratio of the number of yellow pixels with a hue in a hue threshold range to the total number of pixels in each superpixel block is analyzed, and is recorded as a saliency proportion of each superpixel block; The petal saliency value of each superpixel block is determined based on the near-circular degree and the saliency proportion, and the petal saliency value of each superpixel block is the fusion result of the near-circular degree and the saliency proportion of each superpixel block; The thorn saliency value of each superpixel block is the ratio of the number of corner points in each superpixel block to the average number of corner points in all superpixel blocks of the standard Solanum lycopersicum image.

2. The visual inspection method for a crop growing environment of claim 1, wherein, The obtaining of the yellow pixels in the standard Solanum lycopersicum image and the construction of the yellow distribution factor of the standard Solanum lycopersicum image according to the proportion distribution of saturation and brightness of the yellow pixels comprises the following steps: A mask is created to reserve yellow pixels according to a preset hue threshold range; The standard Solanum lycopersicum image is converted into an HSV color space, the ratio of the number of yellow pixels with a saturation greater than a preset saturation threshold to the total number of yellow pixels in the standard Solanum lycopersicum image is recorded as a saturation distribution factor of the standard Solanum lycopersicum image, the ratio of the number of yellow pixels with a brightness in a preset brightness range to the total number of yellow pixels in the standard Solanum lycopersicum image is recorded as a brightness distribution factor of the standard Solanum lycopersicum image, and the ratio of the total number of yellow pixels to the total number of pixels in the standard Solanum lycopersicum image is recorded as a hue distribution factor of the standard Solanum lycopersicum image. Determine a yellow distribution factor of the standard withered leaf petunia image based on the saturation distribution factor, the brightness distribution factor and the hue distribution factor, and the yellow distribution factor of the standard withered leaf petunia image is a result of fusion of the saturation distribution factor, the brightness distribution factor and the hue distribution factor.

3. The visual inspection method for a crop growing environment of claim 1, wherein, The construction process of the segmentation factor of the standard withered leaf petunia image is: Obtain a gray level co-occurrence matrix of the withered leaf petunia gray level image in a horizontal direction and a vertical direction respectively, and take a product of contrast and energy of the gray level co-occurrence matrix in the horizontal direction as a characteristic value in the horizontal direction; Take a product of contrast and energy of the gray level co-occurrence matrix in the vertical direction as a characteristic value in the vertical direction; and take a sum of the characteristic value in the horizontal direction and the characteristic value in the vertical direction as a texture distribution factor of the standard withered leaf petunia image; The segmentation factor of the standard withered leaf petunia image is a result of fusion of the yellow distribution factor and the texture distribution factor of the standard withered leaf petunia image.

4. The visual inspection method for a crop growing environment of claim 1, wherein, The obtaining process of the super-pixel block of the standard withered leaf petunia image is: Calculate a product of the segmentation factor of the standard withered leaf petunia image and a preset segmentation number, and mark the product as a segmentation quantity of the standard withered leaf petunia image; Take an integer greater than the segmentation quantity of the standard withered leaf petunia image as a number of initial seed points in a super-pixel segmentation process, and segment the standard withered leaf petunia image into super-pixel blocks by using a super-pixel segmentation algorithm.

5. The visual inspection method for a crop growing environment of claim 1, wherein, The obtaining method of all petal edges in each super-pixel block is: In each super-pixel block, measure a length of each edge; calculate a mean value of lengths of all edges in the super-pixel block, and mark the mean value as a characteristic standard value; and take an edge with a length greater than the characteristic standard value as a petal edge in the super-pixel block.

6. The visual inspection method for a crop growing environment of claim 1, wherein, The construction of the withered leaf petunia saliency factor of each super-pixel block to obtain a saliency region includes: The withered leaf petunia saliency factor of each super-pixel block is a result of fusion of the petal saliency value and the thorn saliency value of each super-pixel block; Arrange the withered leaf petunia saliency factors of all super-pixel blocks segmented from the standard withered leaf petunia image in a descending order to form a saliency factor sequence, calculate a quartile of all withered leaf petunia saliency factors in the saliency factor sequence, and mark a super-pixel block corresponding to an element before the first quartile in the saliency factor sequence as a saliency region.

7. The visual inspection method for a crop growing environment of claim 1, wherein, The matching algorithm is used to match the salient feature points in the saliency region in the standard withered leaf petunia image and the saliency region in the to-be-detected image, and a visual detection result of the withered leaf petunia in the to-be-detected image is obtained based on a matching result, and the visual detection result includes: Obtain the salient feature points of the saliency region in the standard withered leaf petunia image under each angle by using the feature point extraction algorithm, as the salient feature points in the standard withered leaf petunia image under each angle; Collect an RGB image in a to-be-detected corn seedling growth region by using a CMOS camera, and mark the image as a to-be-detected image; Obtain the salient feature points in the to-be-detected image according to the obtaining method of the salient feature points in the standard withered leaf petunia image; Match the salient feature points in the standard withered leaf petunia image under each angle and the salient feature points in the to-be-detected image by using the matching algorithm to obtain a matching salient feature point pair; The ratio of the number of matched significant feature point pairs between the to-be-detected image and the standard S. surattense image at each angle to the number of pixel points in the to-be-detected image is calculated as a similarity ratio at each angle; The ratio of the similarity ratio at each angle to the average distance between all the matched significant feature point pairs at each angle is calculated as an input of a normalization function, and the product of the output of the normalization function and the coefficient of variation of the similarity ratios at all angles is taken as the similarity between the to-be-detected image and the standard S. surattense image; If the similarity is greater than a preset value, the to-be-detected image contains S. surattense; otherwise, the to-be-detected image does not contain S. surattense.

8. A visual inspection system for a crop growing environment comprising a memory, a processor and a computer program stored in the memory and running on the processor, characterised in that, The processor implements the steps of the method for visual detection of a crop growth environment according to any one of claims 1-7 when executing the computer program.

Images