Grassland obstacle visual detection and recognition method for mowing robot

Abstract

This invention relates to a visual detection and recognition method for obstacles in lawn mowing robots, belonging to the field of image recognition technology. The method first uses a traditional neural network to identify and remove obstacles from lawn images during mowing operations, obtaining a preliminary screening image. Then, based on differences in image texture, edge direction, and color distribution between inconspicuous obstacles and the lawn background, it multidimensionally characterizes the possibility of inconspicuous obstacles in different sub-regions of the preliminary screening image. This allows for adaptive and precise adjustment of the parameters of the lightweight neural network, ensuring that the adjusted network avoids losing weak and sparse feature signals of inconspicuous obstacles when facing preliminary screening images that may contain them. This achieves a balance between recognition efficiency and accuracy in identifying inconspicuous obstacles. Ultimately, this method significantly improves the accuracy of obstacle recognition when using a lightweight neural network to identify obstacles in lawn mowing operations.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method for visual detection and recognition of obstacles in lawns for lawn mowing robots. Background Technology

[0002] Vision detection of obstacles in grasslands is a core capability for lawnmower robots to achieve fully autonomous operation, playing a crucial role in perceiving the surrounding environment and identifying potential risks. Existing technologies typically utilize computer vision and deep learning methods to automatically learn feature representations of obstacles from a large amount of grassland image data by training a lightweight neural network model. Based on this trained lightweight neural network model, the lawnmower robot acquires images of the grassland area ahead using onboard visual sensors during operation, enabling rapid and accurate determination of the presence and location of obstacles.

[0003] Because existing lightweight neural networks are typically pre-trained on general datasets containing a large number of regular man-made objects, their shallow feature extractors become overly reliant on such highly salient features (such as regular textures and bright colors). Therefore, although they can maintain high recognition accuracy when detecting obstacles with similar features, the structural compression adopted by lightweight networks for efficiency results in limited feature capacity and transmission ability for low-salience obstacles. This makes it difficult to effectively capture the already weak and sparse feature signals of these obstacles in complex grassland backgrounds, ultimately leading to missed detections of such targets.

[0004] Therefore, current lawn mowing robots cannot fully and accurately identify obstacles in the grass, and there is a technical problem of insufficient accuracy in identifying obstacles in the grass during operation. Summary of the Invention

[0005] In view of this, the present invention provides a visual detection and recognition method for grass obstacles for lawn mowing robots, so as to solve the technical problem of insufficient accuracy in identifying grass obstacles during the current operation of lawn mowing robots.

[0006] The present invention provides a method for visual detection and recognition of obstacles in a lawn mowing robot, comprising:

[0007] The system acquires images of the grass field during the operation of the lawnmower robot, inputs the grass field images into a trained lightweight neural network for obstacle recognition, and removes the recognized obstacles from the grass field images to obtain preliminary screening images.

[0008] The image is traversed through a preset window to determine the image texture complexity and edge pixel direction disorder of any traversal window. The obstacle feature salience of any traversal window is determined by the image texture complexity and the edge pixel direction disorder.

[0009] Obtain the color distribution similarity between the image under any traversal window and the grassland barrier-free image, correct the obstacle feature salience of the any traversal window with the color distribution similarity, and determine whether the any traversal window is an obstacle window with the corrected obstacle feature salience.

[0010] The dynamic adjustment factor is determined by the proportion of obstacle windows in all traversal windows. The width multiplier and resolution multiplier in the lightweight neural network are adjusted by the dynamic adjustment factor to obtain the adjusted neural network. The adjusted neural network is then used to complete the obstacle recognition of the initial screened image.

[0011] Further, determining the image texture complexity includes:

[0012] Calculate the value of the local binary mode of each pixel in the grayscale image of any traversal window, and count the occurrence probability of each local binary mode value. Use the information entropy corresponding to the occurrence probability of all local binary mode values ​​as the image texture complexity of the any traversal window.

[0013] Further, determining the disorder of the edge pixel orientation includes:

[0014] Edge detection is performed on the image under any traversal window to determine edge pixels. Then, the gray-level gradient direction angle of each edge pixel is calculated. The range of the gray-level gradient direction angle of the edge pixels is divided into a first set number of intervals. The ratio of the number of edge pixels falling into each gray-level gradient direction interval to the total number of edge pixels in any traversal window is determined. The information entropy of the ratio of the number of all gray-level gradient direction intervals is used as the edge pixel direction disorder of any traversal window.

[0015] Furthermore, determining the salience of obstacle features in any traversal window includes:

[0016] The saliency of the obstacle features in any traversal window is inversely proportional to the image texture complexity of the traversal window and directly proportional to the disorder of the edge pixel orientation of the traversal window.

[0017] Furthermore, obtaining the color distribution similarity between the image in any traversal window and the grassland accessibility image includes:

[0018] Obtain the H channel value of each pixel in the grassland accessibility image in the HSV color space, divide the range of H channel values ​​into a second preset number of intervals, and determine the ratio of the number of pixels whose H channel values ​​fall into each H channel value interval to the total number of pixels in the grassland accessibility image as the pixel proportion of the grassland accessibility image in the corresponding H channel value interval.

[0019] Obtain the H channel value of each pixel in the image under any traversal window in the HSV color space, and determine the ratio of the number of pixels whose H channel values ​​fall into each H channel value interval to the total number of pixels in the image under any traversal window as the pixel proportion of the image under any traversal window in the corresponding H channel value interval.

[0020] Calculate the product of the percentage of pixels in the same H channel value range between the grassland accessibility image and the image under any traversal window, and use the sum of the products corresponding to all H channel value ranges as the color distribution similarity between the image under any traversal window and the grassland accessibility image.

[0021] Furthermore, the correction of the obstacle feature salience of any traversal window includes:

[0022] The difference between the color distribution similarity and the preset color distribution similarity benchmark is used as the color consistency adjustment factor. The color consistency adjustment factor is summed with a constant 1 and then multiplied by the obstacle feature salience of any traversal window to obtain the corrected obstacle feature salience.

[0023] Furthermore, determining whether any traversal window is an obstacle window includes:

[0024] The traversal window whose corrected obstacle feature salience is greater than the preset obstacle feature salience threshold is recorded as the obstacle window.

[0025] Furthermore, determining the dynamic adjustment factor based on the proportion of obstacle windows in all traversed windows includes:

[0026] The ratio of the number of obstacle windows to the total number of traversed windows is taken as the proportion of obstacle windows, and the ratio of the number of non-obstacle windows to the total number of traversed windows is taken as the proportion of non-obstacle windows. The difference between the proportion of obstacle windows and the proportion of non-obstacle windows is taken as the dynamic adjustment factor.

[0027] Furthermore, the step of adjusting the width multiplier and resolution multiplier in the trained lightweight neural network using a dynamic adjustment factor includes:

[0028] The product of the dynamic adjustment factor and the width multiplier adjustment coefficient is used as the width multiplier adjustment term, and the sum of the width multiplier adjustment term and the width multiplier base value is used as the adjusted width multiplier.

[0029] The product of the dynamic adjustment factor and the resolution multiplier adjustment coefficient is used as the resolution multiplier adjustment term, and the sum of the resolution multiplier adjustment term and the base value of the resolution multiplier is used as the adjusted resolution multiplier.

[0030] Furthermore, the width multiplier adjustment coefficient is less than the resolution multiplier adjustment coefficient.

[0031] The advantages of this invention compared to the prior art are:

[0032] This invention first performs a preliminary screening of grassland images generated during the operation of a lawnmower robot to obtain preliminary screening images. This initial screening identifies and removes obvious obstacles. Then, based on the differences in image texture complexity, edge pixel orientation, and color distribution between the preliminary screening images and the grassland background (which may contain less obvious obstacles), the probability of less obvious obstacles being present in each sub-region of the preliminary screening image is quantitatively characterized from multiple dimensions. This allows for adaptive and precise adjustment of the parameters of a lightweight neural network based on the presence of less obvious obstacles in each sub-region. The adjusted lightweight neural network avoids losing weak and sparse feature signals of less obvious obstacles when facing the current preliminary screening image that may contain them, thus achieving less obvious obstacle recognition that balances efficiency and accuracy. Finally, based on the preliminary screening of obvious obstacles and the targeted recognition of less obvious obstacles, a significant improvement in recognition accuracy is achieved when using a lightweight neural network to identify obstacles in grasslands. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This is a flowchart illustrating a method for visual detection and recognition of obstacles in a lawn mowing robot, as provided in Embodiment 1 of the present invention.

[0035] Figure 2 This is a schematic diagram of grayscale gradient direction interval division provided in Embodiment 1 of the present invention. Detailed Implementation

[0036] To further illustrate the technical solution of the present invention, specific embodiments are described below.

[0037] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a particular feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. Furthermore, a particular feature, structure, or characteristic in one or more embodiments may be combined in any suitable form, and the terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless otherwise specifically emphasized.

[0038] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0039] Method Implementation Examples:

[0040] See Figure 1 This is a flowchart illustrating a method for visual detection and recognition of obstacles in a lawn mowing robot, as provided in Embodiment 1 of the present invention. Figure 1 As shown, the identification method may include the following steps:

[0041] S101: Acquire images of the grass field during the operation of the lawnmower robot, input the grass field images into a trained lightweight neural network for obstacle recognition, and remove the recognized obstacles from the grass field images to obtain a preliminary screening image.

[0042] This invention aims to improve the accuracy of obstacle recognition during lawnmower robot operations. Therefore, the first step is to acquire images of the lawn during robot operation. These images can be acquired using a fixed vision sensor or a vision sensor mounted on the lawnmower robot. This embodiment preferably utilizes a global shutter RGB camera (resolution ≥1920×1080, frame rate ≥30fps) mounted on the lawnmower robot to capture images of the lawn scene within a typical operating height range of 0.5-2 meters. An integrated three-axis mechanical gimbal stabilization system ensures image clarity during movement.

[0043] Meanwhile, considering that the grassland environment is outdoors and its light intensity varies, this embodiment preferably equips the lawnmower robot with an adaptive exposure control module to cope with sudden changes in the grassland environment's light intensity, providing stable visual data input for subsequent obstacle recognition. Specifically, based on real-time data from the ambient light intensity sensor, a piecewise gamma correction algorithm is used to dynamically adjust the image brightness distribution. Overexposed areas (brightness > 200) are nonlinearly compressed, while underexposed areas (brightness < 50) are enhanced using an S-curve, ensuring the discernibility of obstacle details across the entire dynamic range.

[0044] Furthermore, to improve the accuracy of subsequent obstacle recognition, noise reduction and image enhancement were performed on the acquired grassland images: a hybrid denoising strategy of guided filtering and bilateral filtering was adopted to smooth repetitive grass texture noise while preserving the edge features of potential obstacles. The local contrast of the image was enhanced through the multi-scale Retinex algorithm, effectively improving the visual salience of low-contrast obstacles.

[0045] Considering that this invention aims to identify obstacles with weak features, in order to improve the accuracy of the identification of such obstacles, the grassland image is further preprocessed to enhance details: adaptive histogram equalization is performed on the saturation component in the HSV color space to enhance color differentiation; and unsharpening masking technology is used on the brightness component to enhance edge details, thereby highlighting the obstacle features while suppressing background interference.

[0046] Finally, based on the preprocessed grassland image, a lightweight neural network trained using traditional training methods is first employed for obstacle recognition. Obstacles with high saliency are identified and removed, eliminating unnecessary interference for subsequent analysis of difficult-to-identify obstacles. Specifically, a pre-trained lightweight obstacle detection neural network is used to perform initial obstacle recognition on the preprocessed grassland image, obtaining the bounding boxes of all detected obstacles. These are considered salient obstacle regions and directly excluded from the image. The remaining image region, after excluding salient obstacles, contains both the grassland background and any low-saliency obstacles that might not have been captured by the initial detection model. Next, preferably, morphological closing operations are used to fill in holes and discontinuous boundaries within the remaining image region, forming a complete outline of the target area, which is recorded as the initial screening image. This provides the target and accurate basis for subsequent feature analysis of inconspicuous obstacles.

[0047] S102, traverse the initially screened images with a preset window, determine the image texture complexity and edge pixel direction disorder of any traversal window, and determine the obstacle feature salience of any traversal window with the image texture complexity and the edge pixel direction disorder.

[0048] After initial detection and exclusion of significant obstacle regions, the preliminary images mainly contain two types of content: first, the real grassland background area; and second, potentially low-salience obstacle areas with weak features and high similarity to the background. Because these obstacles have colors highly similar to the grassland background and weak texture features, traditional detection networks have inherent limitations in processing them. Their fixed network structure and downsampling strategies further weaken the already sparse feature signals, leading to the loss of detailed information. Therefore, these types of obstacles are very likely to be missed in the initial detection.

[0049] To address this, this embodiment proposes to accurately locate regions of weakly marked obstacles through multi-dimensional feature quantification analysis, and then design a dynamic parameter adjustment strategy for a lightweight neural network based on the region classification results. This strategy coordinates the width multiplier and resolution multiplier to enhance the ability to preserve details when identifying obstacles in sparse initial screening images, thereby comprehensively improving the detection performance of low-salience obstacles.

[0050] Specifically, in this step, to accurately identify the extent to which non-significant obstacles may exist in the initial screening image and thus accurately adjust the neural network parameters, we first set the traversal window: assuming the area of ​​the initial screening image is... Set the size to The preset window size should be related to the size of the initial screening image. The formula for calculating the window size is as follows:

[0051]

[0052] in, The scaling factor is used to balance computational efficiency and feature analysis accuracy. This embodiment provides a preferred value. =16. This size design should follow the principle of controlling the number of sliding windows within a reasonable range while fully covering the size of typical obstacles, meeting the timeliness requirements of real-time detection for lawnmower robots. In practical applications, the size of the preset window can be dynamically adjusted according to the scale of the detection target.

[0053] After determining the preset window size for traversal, the image can be initially screened using the preset window to obtain the corresponding number of traversal windows. Then, the image texture complexity and edge pixel direction disorder under each traversal window are analyzed.

[0054] The reason for constructing image texture complexity and edge pixel direction disorder is that traditional obstacle detection methods have significant limitations when dealing with low-salience targets with weak features: single-feature dimension analysis is insufficient to fully capture the essential characteristics of the target. Therefore, this step in this embodiment aims to construct a collaborative discrimination mechanism based on texture sparsity and edge direction consistency. The theoretical basis is that the grass background has complex texture due to the dense distribution of grass blades, and because the grass blades grow in roughly the same direction (e.g., vertical growth), the edge direction in the grass background region has strong consistency. In contrast, low-salience obstacles, due to their smooth surfaces or simple structures, typically exhibit sparse texture, meaning significantly lower texture complexity. Furthermore, because they lack the uniform growth direction of grass blades, their edge direction is disordered. Based on this, this step, through quantitative analysis of texture sparsity and edge direction characteristics, can effectively filter out suspicious areas (i.e., potential obstacles) with low feature salience from the grass background.

[0055] Based on the above analysis, the preferred method for determining image texture complexity is as follows: calculate the value of the local binary mode of each pixel in the grayscale image of any traversal window, and count the occurrence probability of each local binary mode value. Use the information entropy corresponding to the occurrence probability of all local binary mode values ​​as the image texture complexity of the any traversal window.

[0056] LBP (Local Binary Pattern) features effectively capture local texture patterns by comparing the grayscale values ​​of the center pixel with its eight neighboring pixels. Therefore, the grayscale image corresponding to any given traversal window is first determined, and then LBP features are extracted from the grayscale image to obtain the LBP value of each pixel. In this embodiment, the preferred feature is the eight neighboring pixels of each pixel, so the final LBP value of each pixel ranges from [0, 255]. Then, based on the LBP values ​​of each pixel in the current traversal window, a quantitative index reflecting texture complexity, i.e., image texture complexity, is constructed, and its formula is as follows:

[0057]

[0058] in, The information entropy represents the probability of occurrence of the Local Binary Pattern (LBP) value corresponding to all pixels in the current traversal window, which is also the image texture complexity of the current traversal window. This represents the probability that a pixel with an LBP value equal to i will appear within the currently traversed window. This quantifies the degree of disorder in texture distribution within the entire window, reflecting the sparsity of the texture. A higher value indicates a more complex and disordered texture distribution, making it more likely to be a grassland background area; a lower value indicates a simpler and more uniform texture, making it more likely to be a potential (low-salience) obstacle area. In other words, the physical mechanism is that grassland backgrounds with regular structures exhibit diverse LBP responses and higher information entropy values; while low-salience suspicious areas have relatively smooth surface textures, with LBP responses concentrated in a few patterns and lower information entropy values.

[0059] Based on the aforementioned analysis, the preferred method for determining the disorder of edge pixel direction is as follows: edge detection is performed on the image under any traversal window to determine edge pixels, then the gray-level gradient direction angle of each edge pixel is calculated, the range of the gray-level gradient direction angle of the edge pixels is divided into a first set number of intervals, the ratio of the number of edge pixels falling into each gray-level gradient direction interval to the total number of edge pixels in any traversal window is determined, and the information entropy of the ratio of the number corresponding to all gray-level gradient direction intervals is used as the disorder of edge pixel direction in any traversal window.

[0060] In this process, edge detection is performed on the image under any traversal window. Specifically, edge detection algorithms (such as the Canny algorithm) are used to obtain edge pixels in the grayscale image corresponding to the traversal window, and then the grayscale gradient direction angle at each edge pixel is calculated. The gray-scale gradient direction of a pixel originally ranges from 0° to 360°. However, since taking the absolute value of the gradient direction is more effective for feature representation and statistics, this embodiment also preferably takes the absolute value of the gray-scale gradient direction angle, corresponding to a value range of 0° to 180°.

[0061] Then, in order to successfully analyze the degree of inconsistency in gradient directions and improve computational efficiency, this embodiment preferably adopts the following approach: Figure 2 As shown in the schematic diagram of the gray-level gradient direction interval division, the gray-level gradient direction angle of the edge pixel is divided into 4 intervals (D1, D2, D3, D4), that is, the value of the first set number is preferably 4. Of course, in other embodiments, the first set number can be other values, which will not be elaborated here.

[0062] Based on the defined gray-level gradient direction intervals, the number of edge pixels within each direction interval is counted, and the ratio of the number of edge pixels in each interval is calculated. Then based on the obtained quantity ratio The edge pixel direction disorder of any traversal window is determined by the following formula:

[0063]

[0064]

[0065] in, This represents the number of edge pixels within the current traversal window whose grayscale gradient direction is intersected by the i-th grayscale gradient direction interval. This represents the total number of edge pixels within the currently traversed window. This represents the ratio of the number of elements corresponding to the i-th gray-level gradient direction interval. This value represents the disorder of edge pixel orientation in the current traversal window. It uses information entropy to reflect the degree of disorder in edge direction distribution: a higher value indicates a more uniform edge direction distribution (i.e., each direction has a similar proportion, such as 0.25, 0.25, 0.25, 0.25), and a more disordered edge direction distribution, consistent with the characteristics of low-salience obstacles; a lower value indicates a more concentrated edge direction distribution (i.e., some directions have a significantly higher proportion than others, such as 0.7, 0.1, 0.1, 0.1), and a more consistent edge direction distribution, consistent with the characteristics of a grassland background. The physical mechanism is that grass blades in a grassland background usually exhibit a consistent directionality, with edge directions concentrated in a few dominant directions, resulting in a lower entropy value; while the edge directions of low-salience suspicious areas are irregular, uniformly distributed, and have a higher entropy value.

[0066] Thus, the salience of obstacle features in any traversal window can be determined by the image texture complexity and the disorder of edge pixel direction. Since a higher image texture complexity indicates that the corresponding traversal window is more likely to be a grassy background, and a higher disorder of edge pixel direction indicates that the corresponding traversal window is more likely to be an obstacle, it can be concluded that: the salience of obstacle features in any traversal window is inversely proportional to the image texture complexity of the any traversal window and directly proportional to the disorder of edge pixel direction of the any traversal window.

[0067] In a preferred embodiment, the saliency of the obstacle features in any traversal window is:

[0068]

[0069] in, This indicates the salience of obstacle features in the current traversal window. This indicates normalization, such as linear normalization or norm normalization, which maps the corresponding value to the interval [0,1]. The balancing weight is used to balance the contribution of image texture complexity and edge pixel orientation disorder in the comprehensive evaluation. In this embodiment, a balancing weight value of 0.5 is preferably given, indicating that the two features are equally important. When the F value approaches 1, it indicates that the window has both obvious sparse texture distribution and edges with low directional consistency, which is consistent with the characteristics of low saliency obstacles; when the F value approaches 0, it indicates that the window has complex texture and consistent edge direction, which is consistent with the characteristics of grassland background. This comprehensive index overcomes the limitations of single feature dimension analysis through linear weighted fusion, and significantly improves the ability to identify low saliency obstacles in complex grassland environments.

[0070] S103, obtain the color distribution similarity between the image under the any traversal window and the grassland barrier-free image, correct the obstacle feature salience of the any traversal window with the color distribution similarity, and determine whether the any traversal window is an obstacle window with the corrected obstacle feature salience.

[0071] Although the above steps obtain the saliency of each traversal window with respect to obstacle features based on texture distribution and edge direction, the acquisition process is based on the grayscale image under the traversal window. The grayscale image discards the color information of the original image, while the color distribution information contained in the image under the traversal window can also characterize whether it is a window containing obstacles. Therefore, by comparing the color distribution information, the accuracy of determining whether a window contains non-saliency obstacle features can be further improved.

[0072] Therefore, this step obtains the color distribution similarity between the image under any traversal window and the grassland accessibility image, and corrects the obstacle feature salience of the traversal window obtained in the previous step. Obtaining the color distribution similarity between the image under any traversal window and the grassland accessibility image includes:

[0073] Obtain the H channel value of each pixel in the grassland accessibility image in the HSV color space, divide the range of H channel values ​​into a second preset number of intervals, and determine the ratio of the number of pixels whose H channel values ​​fall into each H channel value interval to the total number of pixels in the grassland accessibility image as the pixel proportion of the grassland accessibility image in the corresponding H channel value interval.

[0074] Obtain the H channel value of each pixel in the image under any traversal window in the HSV color space, and determine the ratio of the number of pixels whose H channel values ​​fall into each H channel value interval to the total number of pixels in the image under any traversal window as the pixel proportion of the image under any traversal window in the corresponding H channel value interval.

[0075] Calculate the product of the percentage of pixels in the same H channel value range between the grassland accessibility image and the image under any traversal window, and use the sum of the products corresponding to all H channel value ranges as the color distribution similarity between the image under any traversal window and the grassland accessibility image.

[0076] Specifically, firstly, a color histogram of a typical grassland, i.e., an accessible grassland image, is plotted: the H channel of the accessible grassland image in the HSV color space is selected, where the H channel represents the core color information of the image, representing the hue information of each pixel in the image, with a value range of 0°-360°. To consider color distribution, this embodiment preferably divides the H value range into 16 intervals, that is, the second preset number of values ​​mentioned above is preferably 16. A schematic diagram of the H value range is provided below. Figure 2 A schematic diagram of the grayscale gradient direction interval division is not shown here. Then, the percentage of pixels whose H channel values ​​of all pixels in the grassland accessibility image fall into each interval is calculated. That is, the ratio of the number of pixels whose H channel values ​​of all pixels in the grassland accessibility image fall into each H channel value interval to the total number of pixels in the grassland accessibility image is determined as the percentage of pixels in the grassland accessibility image under the corresponding H channel value interval. With each H channel value interval as the horizontal axis and the percentage of pixels corresponding to each H channel value interval as the vertical axis, the normalized histogram corresponding to the grassland accessibility image can be obtained.

[0077] Then, the above-mentioned normalized histogram is also obtained for the image under any traversal window to obtain the normalized histogram corresponding to any traversal window. This gives the ratio of the number of pixels whose H channel values ​​fall into each H channel value interval to the total number of pixels in the image under any traversal window.

[0078] Based on the normalized histograms corresponding to the two images, the color distribution similarity between the image under the traversal window and the grassland accessibility image can be determined by calculating the similarity of the histograms:

[0079]

[0080] in, This indicates the color distribution similarity between the image in the current traversal window and the grassland accessibility image. This represents the percentage of pixels in the k-th H-channel value interval of the grassland accessibility image, which is also the corresponding value of the normalized histogram of the grassland accessibility image in the k-th interval. This represents the percentage of pixels in the k-th H-channel value interval of the image within the current traversal window, which is also the corresponding value of the normalized histogram of the image within the current traversal window in the k-th interval. This color distribution similarity calculation references the principle of Bach distance; the higher the similarity of the normalized histograms of two images, that is, the closer the color distributions of the two images, the higher the similarity. The higher the result value, the greater the similarity of the color distribution.

[0081] It is easy to understand that, and The values ​​of are all in the range of [0,1]. Therefore, the more similar the normalized histograms corresponding to the two figures above are, that is, the more similar the color distribution, the better the values ​​of are. The larger the values ​​of , the larger the final value of S, because and Since the values ​​of S are all 1, we know that the range of S is [0,1].

[0082] Based on the obtained color distribution similarity, it can be compared with the preset benchmark value to determine whether the color distribution between the image under the traversal window and the grassland barrier-free image is relatively similar or not similar.

[0083] Specifically, a preset color distribution similarity benchmark value is set based on the color distribution similarity. This benchmark value can be specifically set according to the obstacle detection accuracy requirements. In this embodiment, a value of 0.5 is preferred. Then, the difference between the color distribution similarity and the preset color distribution similarity benchmark value can be used to construct a color consistency adjustment factor.

[0084]

[0085] in, This is the color consistency adjustment factor corresponding to the current traversal window, which is subsequently used to correct the saliency of obstacle features in the current traversal window. Its value range is [-0.5, 0.5]. In the process of constructing the color consistency adjustment factor, the purpose of subtracting 0.5 from the color distribution similarity is to set a neutral benchmark point and form a two-way adjustment mechanism: when the similarity is greater than 0.5, the color similarity between the traversal window and the grassland barrier-free image is high, μ is a positive value, which enhances the saliency score of the window and thus increases the detection rate of low feature saliency areas with similar colors to the background; when the similarity is less than 0.5, the color similarity between the traversal window and the grassland barrier-free image is low, μ is a negative value, which suppresses the saliency score of the window and thus reduces the possibility of misclassifying these areas as low feature saliency areas.

[0086] Based on the color consistency adjustment factor corresponding to the current traversal window, the salience of obstacle features in the current traversal window is corrected:

[0087] The difference obtained by subtracting the preset color distribution similarity reference value from the color distribution similarity is used as the color consistency adjustment factor. After summing the color consistency adjustment factor with the constant 1 and multiplying it by the obstacle feature saliency of any traversal window, the corrected obstacle feature saliency is obtained.

[0088] The formulaic expression of the correction process is as follows:

[0089]

[0090] where is the corrected obstacle feature saliency corresponding to the current traversal window.

[0091] After that, an obstacle feature saliency threshold T1 is set to classify each traversal window according to the corrected obstacle feature saliency of each traversal window: when ≥T1, the current traversal window is a low feature saliency area, which is denoted as an obstacle window, that is, a traversal window containing an obstacle; when <T1, the current traversal window is a grassland background area, which is marked as a non-obstacle window. This scheme gives a reference threshold T1 = 0.6, which can be fine-tuned according to detection requirements in actual applications: if it is necessary to improve the detection rate of low feature saliency obstacles, the threshold can be appropriately reduced; if it is necessary to reduce false alarms, the threshold can be appropriately increased.

[0092] S104. Determine the dynamic adjustment factor based on the proportion of obstacle windows among all traversal windows, and adjust the width multiplier and resolution multiplier in the lightweight neural network with the dynamic adjustment factor to obtain an adjusted neural network, and use the adjusted neural network to complete the obstacle recognition of the pre-screened image.

[0093] After determining the type of each traversal window, the classification results of all traversal windows in the pre-screened image can be traversed, and the proportion of obstacle windows and non-obstacle windows among all traversal windows can be counted and non-obstacle window proportion , and construct a dynamic adjustment factor:

[0094]

[0095] When it means that the low feature saliency area dominates in the pre-screened image, and the feature extraction ability of the network needs to be enhanced. Increasing the width multiplier can improve the feature capacity and expand the network channels to capture weak texture and color differences; increasing the resolution multiplier can retain more spatial details and prevent fine obstacle features from being lost during the downsampling process.

[0096] When When the initial screening image shows that the grassland background area is dominant, the network complexity can be appropriately reduced to optimize computational efficiency. Reducing the width multiplier can reduce the number of channels and decrease computation; reducing the resolution multiplier can increase the downsampling rate and improve processing speed.

[0097] Therefore, based on the obtained dynamic adjustment factor, the width multiplier is adjusted. Sum of resolution multipliers To achieve precise dynamic control:

[0098]

[0099]

[0100] in The basic setting parameters are indicated. In this embodiment, it is preferred that the basic setting parameters of the width multiplier and the resolution multiplier are the same. In other embodiments, they can be set differently. The basic setting parameters of the width multiplier and the resolution multiplier can be manually given values, or they can be the current width multiplier and resolution multiplier values ​​of the trained lightweight neural network used in step S101. This is the width multiplier adjustment coefficient. This is the resolution multiplier adjustment coefficient. This embodiment preferably uses... =0.4、 =0.5. In the detection of obstacles with low feature saliency, low resolution directly leads to the loss of subtle contours, while a moderate reduction in the number of channels can still be compensated for by contextual information. Therefore, setting it to 0.5 is preferred. This ensures that the resolution is more sensitive to the response of ∆, prioritizing the preservation of spatial details.

[0101] Based on basic parameters Let's take a value of 0.7 as an example to analyze and explain the effect of regulation:

[0102] when =0.4 is a positive value, meaning that regions with low feature significance dominate the overall initial screening image: =0.7 + 0.4 × 0.4 = 0.86 =0.7 + 0.5 × 0.4 = 0.9. When low feature saliency regions dominate, the network simultaneously gains stronger feature extraction capabilities and richer detail preservation, significantly improving the detection sensitivity of hidden obstacles.

[0103] when = -0.4 is a negative value, meaning that the grassland background area dominates in the overall initial screening image: =0.7 + 0.4 × (-0.4) = 0.54 =0.7 + 0.5 × (-0.4) = 0.5. When the grassland background area dominates, the network automatically reduces complexity and optimizes the allocation of computing resources to ensure the system's real-time requirements.

[0104] This allows for the development of a lightweight neural network more suitable for the current initial screening images. This network balances accuracy and efficiency in identifying inconspicuous obstacles in the initial screening images, achieving final detection and identification of obstacles in the grassland. The specific identification process involves first resizing the initial screening images according to the optimized resolution multiplier, then inputting them into a neural network that reconstructs the number of channels according to the width multiplier. Utilizing the multi-level feature extraction capabilities of the neural network, the enhanced detail capture advantage is fully leveraged in low-salience regions to accurately identify low-salience obstacles. Finally, the detection head simultaneously outputs the bounding box position and category confidence of the obstacles. The entire detection process fully considers the unique characteristics of the grassland environment. Front-end salience analysis guides back-end network optimization, significantly improving the detection rate of low-salience obstacles while maintaining real-time performance. The final output includes complete obstacle information containing position, category, and confidence, providing a reliable perception foundation for the autonomous obstacle avoidance and path planning of the lawnmower robot.

[0105] This invention first performs a preliminary screening of the grass images generated during the lawnmower robot's operation to obtain preliminary screening images, thereby identifying and removing obvious obstacles. Then, based on the differences in image texture complexity, edge pixel orientation, and color distribution between the preliminary screening images and the grass background (potentially including inconspicuous obstacles), the likelihood of inconspicuous obstacles in different sub-regions of the preliminary screening images is quantified from multiple dimensions. This allows for adaptive and precise adjustment of the neural network parameters according to the presence of inconspicuous obstacles in each sub-region, enabling the adjusted neural network to achieve both efficiency and accuracy in recognizing inconspicuous obstacles in the current preliminary screening image. Ultimately, this process of preliminary screening of obvious obstacles and targeted recognition of inconspicuous obstacles significantly improves the accuracy of obstacle recognition in the grass.

[0106] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for visual detection and recognition of obstacles in a lawn mowing robot, characterized in that, The method includes: The system acquires images of the grass field during the operation of the lawnmower robot, inputs the grass field images into a trained lightweight neural network for obstacle recognition, and removes the recognized obstacles from the grass field images to obtain preliminary screening images. The image is traversed through a preset window to determine the image texture complexity and edge pixel direction disorder of any traversal window. The obstacle feature salience of any traversal window is determined by the image texture complexity and the edge pixel direction disorder. Obtain the color distribution similarity between the image under any traversal window and the grassland barrier-free image, correct the obstacle feature salience of the any traversal window with the color distribution similarity, and determine whether the any traversal window is an obstacle window with the corrected obstacle feature salience. The dynamic adjustment factor is determined by the proportion of obstacle windows in all traversal windows. The width multiplier and resolution multiplier in the lightweight neural network are adjusted by the dynamic adjustment factor to obtain the adjusted neural network. The adjusted neural network is then used to complete the obstacle recognition of the initial screened image.

2. The method for visual detection and recognition of obstacles in a lawn mowing robot according to claim 1, characterized in that, Determining the image texture complexity includes: Calculate the value of the local binary mode of each pixel in the grayscale image of any traversal window, and count the occurrence probability of each local binary mode value. Use the information entropy corresponding to the occurrence probability of all local binary mode values ​​as the image texture complexity of the any traversal window.

3. The method for visual detection and recognition of obstacles in a lawn mowing robot according to claim 1, characterized in that, Determining the disorder of the edge pixel orientation includes: Edge detection is performed on the image under any traversal window to determine edge pixels. Then, the gray-level gradient direction angle of each edge pixel is calculated. The range of the gray-level gradient direction angle of the edge pixels is divided into a first set number of intervals. The ratio of the number of edge pixels falling into each gray-level gradient direction interval to the total number of edge pixels in any traversal window is determined. The information entropy of the ratio of the number of all gray-level gradient direction intervals is used as the edge pixel direction disorder of any traversal window.

4. The method for visual detection and recognition of obstacles in a lawn mowing robot according to any one of claims 1 to 3, characterized in that, Determining the saliency of obstacle features in any given traversal window includes: The saliency of the obstacle features in any traversal window is inversely proportional to the image texture complexity of the traversal window and directly proportional to the disorder of the edge pixel orientation of the traversal window.

5. The method for visual detection and recognition of obstacles in a lawn mowing robot according to claim 1, characterized in that, The step of obtaining the color distribution similarity between the image under any traversal window and the grassland accessibility image includes: Obtain the H channel value of each pixel in the grassland accessibility image in the HSV color space, divide the range of H channel values ​​into a second preset number of intervals, and determine the ratio of the number of pixels whose H channel values ​​fall into each H channel value interval to the total number of pixels in the grassland accessibility image as the pixel proportion of the grassland accessibility image in the corresponding H channel value interval. Obtain the H channel value of each pixel in the image under any traversal window in the HSV color space, and determine the ratio of the number of pixels whose H channel values ​​fall into each H channel value interval to the total number of pixels in the image under any traversal window as the pixel proportion of the image under any traversal window in the corresponding H channel value interval. Calculate the product of the percentage of pixels in the same H channel value range between the grassland accessibility image and the image under any traversal window, and use the sum of the products corresponding to all H channel value ranges as the color distribution similarity between the image under any traversal window and the grassland accessibility image.

6. The method for visual detection and recognition of obstacles in a lawn mowing robot according to claim 1 or 5, characterized in that, The correction of the obstacle feature salience in any traversal window includes: The difference between the color distribution similarity and the preset color distribution similarity benchmark is used as the color consistency adjustment factor. The color consistency adjustment factor is summed with a constant 1 and then multiplied by the obstacle feature salience of any traversal window to obtain the corrected obstacle feature salience.

7. The method for visual detection and recognition of obstacles in a lawn mowing robot according to claim 1, characterized in that, Determining whether any traversal window is an obstacle window includes: The traversal window whose corrected obstacle feature salience is greater than the preset obstacle feature salience threshold is recorded as the obstacle window.

8. The method for visual detection and recognition of obstacles in a lawn mowing robot according to claim 1, characterized in that, The determination of the dynamic adjustment factor based on the proportion of obstacle windows in all traversed windows includes: The ratio of the number of obstacle windows to the total number of traversed windows is taken as the proportion of obstacle windows, and the ratio of the number of non-obstacle windows to the total number of traversed windows is taken as the proportion of non-obstacle windows. The difference between the proportion of obstacle windows and the proportion of non-obstacle windows is taken as the dynamic adjustment factor.

9. The method for visual detection and recognition of obstacles in a lawn mowing robot according to claim 8, characterized in that, The method of adjusting the width multiplier and resolution multiplier in the trained lightweight neural network using dynamic adjustment factors includes: The product of the dynamic adjustment factor and the width multiplier adjustment coefficient is used as the width multiplier adjustment term, and the sum of the width multiplier adjustment term and the width multiplier base value is used as the adjusted width multiplier. The product of the dynamic adjustment factor and the resolution multiplier adjustment coefficient is used as the resolution multiplier adjustment term, and the sum of the resolution multiplier adjustment term and the base value of the resolution multiplier is used as the adjusted resolution multiplier.

10. The method for visual detection and recognition of obstacles in a lawn mowing robot according to claim 9, characterized in that, The width multiplier adjustment coefficient is less than the resolution multiplier adjustment coefficient.

Images