Citrus picking and obstacle detection method and system

Abstract

The invention relates to the technical field of agricultural automatic intellectualization, in particular to a citrus picking and obstacle detection method and system, and the method comprises the following steps: building a detection image evaluation model, and carrying out the evaluation and analysis of original citrus detection image data, and obtaining citrus target detection image information; determining analysis characteristic parameters of the citrus detection image according to citrus picking and obstacle detection requirements, and further constructing a citrus detection image multi-level analysis system; setting calculation prediction models of different feature analysis layers in the orange detection image multi-level analysis system according to the analysis feature parameters, and analyzing the target detection image information by combining the multi-level analysis system and the calculation prediction models to obtain a multi-level analysis result of the orange detection image; on the basis of a multi-level analysis result and citrus target detection image information, citrus fruit types and obstacle conditions are analyzed, and different citrus picking schemes are formulated. According to the invention, automation of citrus picking and obstacle detection is realized, and agricultural intelligent development is facilitated.

Description

Technical Field

[0001] The present invention relates to the field of agricultural automation and intelligent technology, and specifically to a citrus picking and obstacle detection method and system. Background Art

[0002] During the citrus picking process, the orchard environment is complex and changeable, with various types of obstacles such as branches, weeds, and other crops. If the fruit type and obstacle conditions cannot be detected in a timely and effective manner, it is easy to cause equipment damage and increase maintenance costs, thereby affecting the fruit picking progress and work efficiency.

[0003] Currently, orchards primarily rely on manual observation and sensor monitoring for fruit picking and obstacle detection. Manual observation is subject to significant subjectivity, and different observers' experience and judgment criteria vary, resulting in inconsistent and inaccurate results for fruit identification and obstacle detection. Furthermore, while sensor monitoring can provide certain data, its detection range and information accuracy are limited by factors such as sensor type, performance, and installation location. Consequently, existing information collection methods struggle to comprehensively and timely detect all obstacles, nor can they accurately determine fruit type and obstacle location, failing to meet the demands of the actual citrus picking process.

[0004] With the continuous development of intelligent technologies, intelligent agriculture has become an inevitable trend in the agricultural sector. Optimizing citrus picking and obstacle detection technology can improve agricultural production efficiency, reduce production costs, and ensure the quality of agricultural products. This can better meet the actual needs of citrus picking and enhance the adaptability and intelligence of agricultural technology. Summary of the Invention

[0005] In view of the shortcomings of existing methods and the needs of practical applications, in order to achieve more efficient, accurate and reliable fruit picking and inspection operations, it is necessary to comprehensively use multi-sensor data, artificial intelligence algorithms and multi-dimensional indicator evaluation to dynamically optimize obstacle detection and fruit picking strategies, and be able to adjust picking plans according to different orchard environments and picking needs. On the one hand, the present invention provides a citrus picking and obstacle detection method, which includes the following steps: obtaining citrus original detection image data, establishing a detection image evaluation model to evaluate and analyze the citrus original detection image data, and obtaining citrus target detection image information; determining analysis feature parameters of the citrus detection image according to citrus picking and obstacle detection requirements, and constructing a citrus detection image multi-level analysis system based on the analysis feature parameters; setting a computational prediction model of different feature analysis layers in the citrus detection image multi-level analysis system based on the analysis feature parameters, analyzing the citrus target detection image information in combination with the citrus detection image multi-level analysis system and the computational prediction model, and obtaining a multi-level analysis result of the citrus detection image; analyzing citrus fruit type and obstacle conditions based on the multi-level analysis result and the citrus target detection image information, and formulating different citrus picking plans based on the citrus fruit type and obstacle conditions. The multi-level analysis result and citrus target detection image information of the present invention are conducive to accurately analyzing citrus fruit type and obstacle conditions, thereby formulating personalized picking plans and improving citrus yield and quality.

[0006] Optionally, the obtaining of original citrus detection image data, establishing a detection image evaluation model to evaluate and analyze the original citrus detection image data, and obtaining citrus target detection image information includes: obtaining structural distribution information of the citrus detection area based on the original citrus detection image data; obtaining the actual detection area of ​​the citrus detection image according to the structural distribution information; establishing a detection image evaluation model in combination with the target detection area and the actual detection area; evaluating and analyzing the original citrus detection image data through the detection image evaluation model to obtain the structural similarity of the original citrus detection image; and screening and optimizing the original citrus detection image data according to the structural similarity to obtain citrus target detection image information. The present invention evaluates and analyzes image data through the detection image evaluation model, can remove detection images with incomplete structure, blur or severe interference, improves the quality of image data, and provides an information basis for subsequent analysis of citrus fruit types and obstacle conditions.

[0007] Optionally, establishing the detection image evaluation model by combining the target detection area and the actual detection area includes: obtaining a center matrix and a normalization factor corresponding to the target detection area based on the target detection area; obtaining a center matrix and a normalization factor corresponding to the actual detection area based on the actual detection area; and establishing the detection image evaluation model by combining the center matrix and the normalization factor; The detection image evaluation model satisfies the following relationship: in, Indicates the structural similarity between the target detection area and the actual detection area in the citrus detection image, represents the balance parameter of the citrus detection image, represents the center alignment matrix of the target detection area in the citrus detection image, express The corresponding normalization factor, represents the center alignment matrix of the actual detection area in the citrus detection image, express The corresponding normalization factor.

[0008] The present invention combines the central matrix and the normalization factor to establish a detection image evaluation model, which fully considers the structure and feature information of the target detection area and the actual detection area, so that the model can more accurately evaluate the similarity between different monitoring areas, and provides a scientific and reasonable analysis basis for image screening optimization.

[0009] Optionally, determining the analysis feature parameters of the citrus detection image according to the requirements of citrus picking and obstacle detection, and constructing a multi-level analysis system for the citrus detection image based on the analysis feature parameters includes: determining the analysis feature parameters of the citrus detection image according to the requirements of citrus picking and obstacle detection, the analysis feature parameters including citrus geometric features, citrus texture features, citrus color features, spatial relationship features, citrus occlusion degree and citrus head and tail morphological features; constructing a multi-level analysis system for citrus detection images based on the citrus geometric features, the citrus texture features, the citrus color features, the spatial relationship features, the citrus occlusion degree and the citrus head and tail morphological features, the citrus detection image multi-level analysis system including a geometric feature analysis layer, a texture feature analysis layer, a color feature analysis layer, a spatial feature analysis layer, an occlusion degree analysis layer and a head and tail morphological feature analysis layer.

[0010] The analysis layers in the multi-level analysis system of the present invention are independent but correlated with each other. Each analysis layer can independently perform feature analysis and processing. At the same time, the analysis results between the layers can verify and supplement each other, which helps to obtain more comprehensive and accurate citrus comprehensive analysis results.

[0011] Optionally, the computational prediction models of different feature analysis layers are set in the multi-level analysis system of citrus detection images according to the analysis feature parameters, the citrus target detection image information is analyzed in combination with the multi-level analysis system of citrus detection images and the multi-level analysis results of the citrus detection images are obtained, including: analyzing the citrus target detection image information according to the geometric feature analysis layer and the computational prediction model to obtain the circularity, aspect ratio and convex hull rate of the citrus detection image; analyzing the citrus target detection image information based on the texture feature analysis layer and the computational prediction model to obtain the texture features, image contrast and Image entropy; analyzing the citrus target detection image information according to the color feature analysis layer and the computational prediction model to obtain the channel hue distribution and histogram similarity of the citrus detection image; using the spatial feature analysis layer and the computational prediction model to analyze the citrus target detection image information to obtain the Euclidean distance of the citrus detection image; analyzing the citrus target detection image information through the occlusion degree analysis layer and the computational prediction model to obtain the occlusion area of ​​the citrus detection image; combining the head and tail morphological feature analysis layer and the computational prediction model to analyze the citrus target detection image information to obtain the head and tail brightness difference and the head and tail curvature radius of the citrus detection image.

[0012] The analysis system of the present invention allocates different types of feature analysis to different analysis layers, and each analysis layer can perform parallel calculations and processing, thereby improving analysis efficiency and reducing analysis time. It can be applied to the citrus detection and picking planning process in large-scale orchards.

[0013] Optionally, the setting of computational prediction models of different feature analysis layers in the multi-level analysis system for citrus detection images based on the analysis feature parameters includes: setting a circularity analysis model, an aspect ratio analysis model, and a convex hull analysis model in the geometric feature analysis layer of the multi-level analysis system for citrus detection images; and setting a texture feature analysis model, an image contrast analysis model, and an image entropy analysis model in the texture feature analysis layer of the multi-level analysis system for citrus detection images. The texture feature analysis model satisfies the following relationship: in, Represents the center pixel of the citrus detection image The uniform LBP value of Indicates the total number of neighborhood pixels minus 1, represents the comparison function, Represents the neighborhood pixels of the citrus detection image The gray value of Represents the neighborhood pixels of the citrus detection image The gray value of Represents the index encoding of the neighborhood pixels of the citrus detection image, b represents the number of transitions in binary code. Indicates other situations.

[0014] The computational prediction model of the present invention can quickly calculate the corresponding characteristic parameters, enabling the analysis system to respond to citrus testing needs in a timely manner and providing technical support for subsequent decision-making.

[0015] Optionally, the computational prediction models for setting different feature analysis layers in the multi-level analysis system for citrus image detection based on the analysis feature parameters include: setting a channel hue distribution analysis model and a histogram similarity analysis model in the color feature analysis layer of the multi-level analysis system for citrus image detection; setting a Euclidean distance analysis model in the spatial feature analysis layer of the multi-level analysis system for citrus image detection; and setting an occlusion area analysis model in the occlusion degree analysis layer of the multi-level analysis system for citrus image detection. The present invention comprehensively considers information from different color channels and histogram similarity, reduces the impact of illumination changes on color analysis to a certain extent, and ensures the reliability of fruit color analysis results.

[0016] Optionally, setting the computational prediction models of different feature analysis layers in the multi-level analysis system for citrus detection images according to the analysis feature parameters includes: setting a head-tail brightness difference analysis model and a head-tail curvature radius analysis model in the head-tail morphological feature analysis layer of the multi-level analysis system for citrus detection images; The head-tail brightness difference analysis model satisfies the following relationship: in, represents the brightness difference between the head and tail of the citrus detection image, express The weight of represents the average grayscale value of the top area of ​​the citrus detection image, express The weight of represents the average grayscale value of the bottom area of ​​the citrus detection image, represents the minimum parameter; The head and tail curvature radius analysis model satisfies the following relationship: in, represents the head and tail curvature radius of the citrus detection image, Indicates the citrus detection image The coordinates of the next contour point, Indicates the citrus detection image The coordinates of the previous contour point, Represents the coordinates of the current citrus contour point in the citrus detection image.

[0017] The head-tail brightness difference analysis model and the head-tail curvature radius analysis model of the present invention can analyze the head-tail morphology of citrus from different angles. The two complement each other and can more comprehensively and accurately describe the head-tail characteristics of citrus, more accurately judge the morphology, quality and growth status of citrus, and reduce the errors and uncertainties brought by single model analysis.

[0018] Optionally, analyzing the citrus fruit type and obstacle conditions based on the multi-level analysis results and the citrus target detection image information, and formulating picking plans for different citrus fruits based on the citrus fruit type and the obstacle conditions includes: constructing a fruit type and obstacle recognition mechanism; identifying and judging the citrus fruit type and obstacle conditions by combining the fruit type and obstacle recognition mechanism, the multi-level analysis results, and the citrus target detection image information; and formulating picking plans for different citrus fruits based on the identification and judgment results of the citrus fruit type and obstacle conditions. The present invention constructs a fruit type and obstacle recognition mechanism, and integrates the multi-level analysis results with the citrus target detection image information, so as to identify and judge the citrus fruit type and obstacle conditions from multiple angles and dimensions.

[0019] In a second aspect, in order to efficiently execute the citrus picking and obstacle detection method provided by the present invention, the present invention also provides a citrus picking and obstacle detection system, comprising a processor, an input device, an output device, and a memory, wherein the processor, input device, output device, and memory are interconnected, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is configured to call the program instructions to execute the citrus picking and obstacle detection method as described in the first aspect of the present invention. The citrus picking and obstacle detection system of the present invention has a compact structure and stable performance, and can stably execute the citrus picking and obstacle detection method provided by the present invention, thereby improving the overall applicability and practical application capabilities of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a flow chart of the citrus picking and obstacle detection method of the present invention; Figure 2 This is a structural diagram of the citrus picking and obstacle detection system of the present invention. DETAILED DESCRIPTION

[0021] See also Figure 1 In order to ensure the accuracy and reliability of fruit type identification and obstacle detection, and to formulate fruit picking plans and obstacle avoidance plans based on the complex environment and fruit conditions of different orchards, the present invention provides a citrus picking and obstacle detection method, which includes the following steps: S1. Obtaining original citrus detection image data, establishing a detection image evaluation model to evaluate and analyze the original citrus detection image data, and obtaining citrus target detection image information. The specific implementation steps and contents are as follows; First, obtain the original citrus detection image data.

[0022] In this embodiment, an image information acquisition network covering the entire orchard is constructed by deploying intelligent cameras and data monitoring equipment throughout the orchard. The cameras capture raw citrus image data in real time, while the data monitoring equipment acquires orchard environmental information. These data are then transmitted wirelessly to the cloud server of the citrus picking and obstacle detection system for storage, providing technical support for subsequent intelligent big data analysis. The captured raw citrus image data is then associated with orchard meteorological data (such as temperature, humidity, and light) and soil data (such as pH and nutrient content), forming a comprehensive dataset of citrus growth environment and image features, providing comprehensive data support for fruit picking and decision-making.

[0023] Then, the structural distribution information of the citrus detection area is obtained based on the original citrus detection image data, and the actual detection area of ​​the citrus detection image is obtained according to the structural distribution information.

[0024] To accurately obtain the structured distribution information of the citrus detection area, it is necessary to clarify the actual detection area of ​​the citrus detection image. This embodiment introduces an image segmentation and extraction algorithm, using a deep learning-based segmentation network to perform pixel-level classification operations on the original citrus detection image. This segmentation network has been trained with a large amount of citrus image data and has strong feature learning capabilities. It can preliminarily segment the original citrus detection image into different areas such as citrus fruit, branches and leaves, and background based on the texture, color, shape and other features of different areas in the detection image. During the segmentation process, the above network can continuously optimize parameters and automatically identify the actual monitoring area in the image, laying the foundation for subsequent analysis.

[0025] After segmentation, the system conducts a structural analysis of the actual monitored area of ​​the citrus fruit. Using a series of image processing algorithms, it extracts information such as the fruit's outline area, perimeter, circumscribed ellipse axis length, number of pixels, center coordinates, percentage of occluded area, arrangement direction angle, and average fruit brightness. These structural features describe the structural distribution of the citrus fruit detection area from multiple dimensions, enabling in-depth analysis of key indicators such as citrus growth status, fruit density, and obstacle conditions, facilitating the subsequent development of more scientific and rational fruit harvesting and implementation plans.

[0026] Next, a detection image evaluation model was established by combining the target detection area and the actual detection area.

[0027] In order to comprehensively and accurately evaluate the effectiveness and accuracy of the citrus detection image, this embodiment introduces structural similarity as an evaluation indicator for the detection image. This indicator fully considers the global features and local details of the citrus detection image, and can ensure the reliability of the graphic evaluation results.

[0028] In the specific implementation process, in order to realize the normalization processing of the central matrix information and ensure the consistency and accuracy of the image evaluation results, a normalization factor is introduced in the embodiment. The selection of the normalization factor needs to be closely combined with the statistical characteristics of the citrus image in the citrus target detection area and the actual detection area, including but not limited to the average grayscale, regional area, etc. The normalization factor can be adjusted and optimized according to actual conditions.

[0029] At the same time, mathematical calculations are required to calculate the target detection area and the actual detection area of ​​citrus fruit to obtain the corresponding center matrix and normalization factor for each area. The center matrix is ​​an important indicator that describes the spatial distribution characteristics of the detection area and can accurately reflect the spatial distribution center of the citrus targets within the detection area. The normalization factor can eliminate the evaluation bias caused by differences in scale, brightness, and other factors between different areas, making the evaluation results between different detection areas more comparable and feasible.

[0030] In order to further enhance the flexibility and adaptability of the detection image evaluation model, the balance parameter of the citrus detection image is also introduced. , set its value range to ,This balance parameter can be used to dynamically adjust the weights of the target area and the actual ,detection area in the evaluation process, thereby flexibly allocating the importance of the two in the ,evaluation process according to actual needs.

[0031] The balance parameters of the citrus detection image need to satisfy the following mathematical relationship: in, represents the balance parameter of the citrus detection image, Indicates the area of ​​the actual citrus detection area, Represents the area of ​​the citrus target detection area.

[0032] In order to further quantitatively evaluate the structural similarity of citrus detection images, a detection image evaluation model of this embodiment is constructed by combining the central matrix and normalization factor corresponding to the above-mentioned target detection area and the actual detection area. This model integrates the information of the central matrix and normalization factor through logical relationships, thereby realizing a scientific evaluation of the structural similarity of citrus detection images.

[0033] The above detection image evaluation model satisfies the following relationship: in, Indicates the structural similarity between the target detection area and the actual detection area in the citrus detection image, represents the balance parameter of the citrus detection image, represents the center alignment matrix of the target detection area in the citrus detection image, express The corresponding normalization factor, represents the center alignment matrix of the actual detection area in the citrus detection image, express The corresponding normalization factor.

[0034] Then, the original citrus detection image data is evaluated and analyzed using the above-mentioned detection image evaluation model to obtain the structural similarity of the original citrus detection image.

[0035] Using this detection image evaluation model, we conducted a comprehensive and in-depth evaluation and analysis of the original citrus detection image data. During the evaluation process, the model comprehensively analyzed the structural differences and similarities between the target detection area and the actual detection area based on multi-dimensional information such as the center matrix, normalization factor, and balance parameter, ultimately quantifying the degree of structural similarity between the original citrus detection images.

[0036] Finally, the original citrus detection image data is screened and optimized according to the degree of structural similarity to obtain citrus target detection image information.

[0037] In order to obtain high-quality citrus target detection image information, the citrus original detection image data is systematically screened and optimized based on the structural similarity calculated above. In the embodiment, the structural similarity threshold of the citrus original detection image is pre-set. ,This threshold is determined based on a large amount of experimental data and ,actual detection needs, and is mainly used to judge whether the citrus ,detection image meets the target requirement benchmark.

[0038] Structural similarity between citrus target detection area and actual detection area Similarity threshold with preset structure , screen and analyze the initial citrus detection images, and calculate the structural similarity of each initial citrus detection image With threshold A comparison is performed to determine whether the image meets the target requirements, and then a set of citrus target detection images is obtained.

[0039] The above screening analysis process must follow the following mathematical relationship: in, represents the citrus detection image screening model, Indicates the structural similarity between the target detection area and the actual detection area in the citrus detection image, represents the structural similarity reference value of the citrus detection image, Indicates that the citrus detection image is qualified. Indicates that the citrus detection image fails.

[0040] In this example, a detection image evaluation model was constructed by combining the center matrix and normalization factor corresponding to the target detection area and the actual detection area. This logically integrated information from the center matrix and normalization factor, enabling a scientific assessment of the structural similarity of citrus detection images and providing technical support for quantitative evaluation of image quality. Comparing the structural similarity of the initial citrus detection images with a threshold accurately screened for qualified detection images, resulting in a high-quality set of citrus target detection images, providing a data foundation for subsequent citrus recognition and obstacle detection.

[0041] Furthermore, the method for acquiring citrus target detection images in this embodiment is only an optional condition of the present invention. In other embodiments, the method for acquiring citrus target detection images can be optimized and replaced according to the collection conditions of citrus detection images and the monitoring requirements of citrus information, which can meet diverse image monitoring needs and enhance the practicality and feasibility of citrus picking and obstacle detection methods.

[0042] S2. Determine the analysis feature parameters of the citrus detection image based on the requirements of citrus picking and obstacle detection, and construct a multi-level analysis system for the citrus detection image based on the analysis feature parameters. The specific implementation steps and contents are as follows: In order to complete the citrus picking and obstacle detection tasks accurately and efficiently, it is necessary to determine the key feature parameters of the citrus detection image according to the actual needs of citrus picking and obstacle detection, and on this basis build a multi-level analysis system for citrus detection images with a clear structure and distinct levels.

[0043] First, the analytical feature parameters for the citrus detection image are determined based on the requirements for citrus picking and obstacle detection. A thorough analysis of the various requirements during citrus picking and obstacle detection is conducted, selecting key feature parameters that accurately reflect the citrus growth status, fruit quality, and obstacle conditions. In this embodiment, a comprehensive analysis of multiple factors was conducted to determine a series of representative analytical feature parameters. These parameters primarily include: citrus geometric features, citrus texture features, citrus color features, spatial relationship features, citrus occlusion level, and citrus head and tail morphological features.

[0044] Citrus geometric features: covering roundness, aspect ratio, and convex hull ratio. These geometric features can intuitively reflect the shape information of citrus fruits and are important for distinguishing normal fruits (usually with a high degree of roundness) from obscured or stuck fruits (with a low degree of roundness or an abnormal aspect ratio).

[0045] Citrus texture features: mainly include local binary patterns (LBP) and contrast, entropy and other indicators in the gray-level co-occurrence matrix. Texture features can describe the texture complexity of the surface of citrus fruit and help to identify fruits obscured by leaves (higher texture complexity) and normal unobstructed fruits (relatively lower texture complexity).

[0046] Citrus color features: This involves HSV channel hue distribution and color histogram similarity. Color is an important basis for judging the maturity of citrus fruit. By analyzing color features, we can accurately distinguish between ripe fruit (with a high proportion of red or orange) and unripe or obscured fruit (with a high proportion of green or mixed colors).

[0047] Spatial relationship features: These include the Euclidean distance between fruits and the number of adjacent fruits. These features can reflect the spatial distribution of fruits and are important for detecting overlapping and adhering fruits (where the number of adjacent fruits is greater than 1 and the distance is less than a set threshold).

[0048] Citrus shading: This is measured by quantifying the percentage of shading (the ratio of the shading area to the fruit area) and determining the type of shading (e.g., leaves, branches, or fruit). Shading can be further categorized as slight shading (less than 20% shading) and leaf shading (greater than 20% shading).

[0049] The morphological characteristics of the head and tail of citrus mainly include the brightness difference between the head and tail (the grayscale difference between the top and bottom) and the curvature radius of the head and tail. The morphological characteristics of the head and tail can assist in judging the maturity of the fruit, among which the brightness difference between the head and tail is positively correlated with the maturity of the fruit.

[0050] Then, a multi-level analysis system for citrus detection images was constructed.

[0051] Based on the aforementioned citrus geometric, texture, color, spatial relationship, occlusion, and head-tail morphology features, a multi-level, systematic citrus inspection image analysis system was constructed. This system is divided into six layers: geometric, texture, color, spatial, occlusion, and head-tail morphology. Each layer performs in-depth analysis of specific feature categories. For details on the multi-level analysis system, see Table 1.

[0052] Table 1 Information table of multi-level analysis system for citrus detection images The multi-level analysis system for citrus fruit detection constructed in this embodiment rationally divides different feature analysis layers, with each layer analyzing specific feature analysis parameters, resulting in a clear structure and distinct levels. This systematic analysis system facilitates subsequent optimization and expansion of detection methods, improving the accuracy of citrus fruit recognition and obstacle detection.

[0053] Furthermore, in this embodiment, the construction mode of the multi-level analysis system for citrus detection images is only an optional condition of the present invention. In one or some other embodiments, the construction mode of the multi-level analysis system for citrus detection images can be optimized according to the requirements of citrus picking and obstacle detection and the characteristic attributes of citrus detection images, which can improve the reliability and accuracy of the multi-level analysis system and enhance the scalability and practical application effect of the citrus picking and obstacle detection method.

[0054] S3. Setting computational prediction models for different feature analysis layers in the multi-level analysis system for citrus detection images based on the analysis feature parameters. Then, combining the multi-level analysis system for citrus detection images and the computational prediction models, analyzing the citrus target detection image information to obtain a multi-level analysis result for the citrus detection image. The specific implementation steps and contents are as follows: In an optional embodiment, the citrus target detection image information is analyzed based on the geometric feature analysis layer and the computational prediction model to obtain key geometric feature parameters of the citrus detection image, such as circularity, aspect ratio, and convex hull rate.

[0055] The geometric feature analysis layer plays a role in the multi-level analysis system of citrus detection images. In order to achieve accurate analysis of the geometric features of citrus, this embodiment sets up a circularity analysis model, an aspect ratio analysis model and a convex hull analysis model.

[0056] Circularity analysis model: Roundness is an important indicator for measuring the degree to which the shape of citrus fruits is close to a circle. The mathematical expression of the roundness analysis model of this embodiment is as follows: in, Indicates the roundness of the citrus fruit. represents pi, represents the outline area of ​​the citrus, Represents the outline perimeter of a citrus fruit.

[0057] The outline area of ​​citrus fruit (total number of pixels).

[0058] The perimeter of a citrus fruit (the number of pixels along the edge of the outline).

[0059] The range of circularity is The closer the roundness value is to 1, the closer the citrus fruit is to a circle; conversely, the smaller the roundness value, the greater the deviation from a circle. A nearly round citrus fruit has a roundness very close to 1; however, when a citrus fruit is squeezed, obscured, or has an irregular growth shape, its roundness value will be significantly reduced.

[0060] Aspect ratio analysis model: The aspect ratio is mainly used to describe whether the shape of a citrus fruit is closer to round, flat, or elongated. The calculation formula of the aspect ratio analysis model in this embodiment is as follows: in, Indicates the length and width of citrus. The length of the minor axis of the smallest circumscribed ellipse (or rectangle) representing the outline of the citrus fruit. The length of the major axis of the minimum circumscribed ellipse (or rectangle) representing the outline of the citrus fruit.

[0061] when When , it means that the lengths of the main axis and the secondary axis of the citrus fruit are equal, and the fruit shape is completely round; when When it is smaller, it indicates that the fruit shape is flatter or more elongated, and can be oval, rectangular, etc.

[0062] In the actual calculation process, the minimum circumscribed rectangle can be used to obtain the lengths of the major and minor axes. During the analysis, the aspect ratio of normally grown citrus fruits is approximately equal to 1; if the fruit is blocked by branches or has deformed growth, its aspect ratio will be greater than 1.5. In other words, a citrus fruit that is severely blocked by branches may become slender in shape, resulting in a significant increase in the aspect ratio.

[0063] Convex hull analysis model: The convex hull ratio can reflect the degree of convexity of the citrus fruit outline and whether there are any concavities or irregularities. The convex hull ratio analysis model used in this embodiment needs to satisfy the following relationship: in, represents the convex hull rate of citrus, represents the actual coverage area of ​​citrus, The area of ​​the minimum convex polygon that circumscribes the outline of the orange.

[0064] The total number of pixels in the area inside the object's outline (i.e., the area actually covered by the object).

[0065] The area of ​​the minimum convex polygon surrounding the object's outline (calculated using OpenCV's cv2.convexHull method) Based on the above expression, we can first use the cv2.convexHull function to obtain the minimum circumscribed convex polygon of the citrus outline, and then use the contourArea function to calculate the contour area (that is, the actual coverage area) ) and the area of ​​the minimum circumscribed convex polygon .

[0066] The convexity of a normally grown citrus fruit is approximately 1; if the fruit surface is concave, overlapping, or blocked, the convexity will be less than 0.9. When , it means that the citrus fruit is a convex polygon, which can be a perfect circle, rectangle, etc.; when When the citrus fruit is concave or irregular in shape, it may be due to the concave surface of the citrus fruit caused by growth or external factors, multiple fruits overlapping or being blocked by leaves, etc.

[0067] Through the above three analysis models, the geometric feature parameters of the citrus detection image, such as roundness, aspect ratio and convex hull rate, can be comprehensively and accurately obtained, providing strong data support for subsequent citrus picking and obstacle detection.

[0068] In an optional embodiment, in order to analyze the key information contained in the citrus target detection image, the citrus target detection image information is analyzed based on the texture feature analysis layer and the computational prediction model to obtain the texture features, image contrast and image entropy of the citrus detection image.

[0069] In the multi-level analysis system of citrus detection images, the texture feature analysis layer is the key link in extracting the surface texture information of citrus. In this embodiment, a texture feature analysis model, an image contrast analysis model and an image entropy analysis model are set up to achieve effective extraction and analysis of image texture-related features.

[0070] Texture feature analysis model: Among them, Local Binary Pattern (LBP) is an operator widely used in image classification, target detection and feature extraction. It can effectively describe the local texture features of an image. By comparing the grayscale values ​​of the central pixel and the neighboring pixels, a binary code is generated to reflect the distribution of the local texture. The relevant implementation steps are as follows; The first step is to grayscale the image. The citrus monitoring image is converted to a grayscale image. In a color image, each pixel contains values ​​for three channels: red, green, and blue. However, after conversion to a grayscale image, each pixel has only one grayscale value, ranging primarily from 0 to 255. This step simplifies the image data and lays the foundation for subsequent texture analysis.

[0071] The second step is to divide the image into neighborhoods. The citrus monitoring image is divided into multiple The neighborhood is set to 3×3 in the embodiment, and each neighborhood includes a central pixel and surrounding neighboring pixels. This division method helps to analyze the texture features of the image from a local perspective.

[0072] The third step is to compare pixel grayscale values. Using the central pixel value of each neighborhood in the citrus monitoring image as the pixel threshold, the grayscale values ​​of other pixels in the neighborhood are compared with the grayscale value of the central pixel. This comparison process is a key step in generating LBP values, which can capture the changes in pixel grayscale values ​​within the local area.

[0073] The fourth step is to generate binary code and calculate LBP value. In the embodiment, binary code is generated according to the comparison result of grayscale value between neighboring pixels and central pixel. The specific calculation formula is as follows: in, Indicates the center pixel The LBP value, Indicates the total number of neighborhood pixels minus 1, represents the comparison function, Indicates the neighborhood The gray value of a pixel, represents the gray value of the center pixel, Indicates the index code of the neighborhood pixels; Center pixel The LBP value reflects the local texture features.

[0074] Center pixel The LBP value of represents the total number of neighborhood pixels minus 1. Right now −1, neighborhood; Represents a comparison function, if , returns 1; otherwise returns 0; Indicates the index code of the neighborhood pixel, the index of the neighborhood pixel ) The fifth step is to introduce the uniform model to optimize the LBP value calculation formula.

[0075] In this embodiment, in order to further optimize the calculation process of LBP value and improve the efficiency and accuracy of texture feature extraction of citrus detection image, a uniform model is introduced to calculate formula Optimize.

[0076] Under the uniform model, the expression of the LBP value of the central pixel is as follows: Furthermore, the optimized texture feature analysis model needs to satisfy the following relationship: in, Represents the center pixel of the citrus detection image The uniform LBP value of Indicates the total number of neighborhood pixels minus 1, represents the comparison function, Represents the neighborhood pixels of the citrus detection image The gray value of Represents the neighborhood pixels of the citrus detection image The gray value of Represents the index encoding of the neighborhood pixels of the citrus detection image, b represents the number of transitions in binary code. Indicates other situations.

[0077] In this embodiment, the uniform model is introduced to obtain the final texture feature analysis model, which can reduce the number of LBP patterns while retaining key texture information, further improving the analysis efficiency of the texture features of the citrus detection image.

[0078] Image contrast analysis model: During the image analysis process of citrus detection, image contrast is a key indicator for measuring image clarity and the depth of texture grooves. It can intuitively reflect the difference in brightness and darkness between citrus fruits and their surroundings or different areas on the fruit surface. This embodiment quantifies the contrast of citrus images based on the contrast calculation method in the grayscale co-occurrence matrix.

[0079] The gray-level co-occurrence matrix counts the frequency of occurrence of gray-level values ​​of pixel pairs with specific spatial position relationships in the image. The contrast calculation expression of the citrus image satisfies the following relationship: in, represents the contrast of the citrus detection image, Represents the gray value combination of pixel pairs, Represents the grayscale value of the first pixel in the citrus detection image, represents the grayscale value of the second pixel in the citrus detection image, Represents the grayscale value in the citrus detection image and Frequency of simultaneous occurrence.

[0080] The contrast of citrus images can reflect the clarity of the detected image and the depth of the texture grooves.

[0081] Gray-level co-occurrence matrix The probability value of the position, representing the gray value Frequency of simultaneous occurrence.

[0082] In the citrus detection image, different pixel pair grayscale value combinations reflect the grayscale changes at different positions in the image. When there are obvious light and dark contrast areas in the image, the pixel pair grayscale value differences are large. The value of will increase, resulting in an increase in the contrast value. In an alternative embodiment, when there is a large brightness difference between the citrus fruit surface and the background, or when there are obvious texture grooves on the fruit surface, the calculated contrast value will be relatively high. Therefore, the contrast of citrus images can effectively reflect the image clarity and the depth of the texture grooves, providing important feature information for citrus fruit recognition and obstacle detection.

[0083] Image entropy analysis model: Image entropy is one of the important parameters to measure the complexity and randomness of image texture. It can reflect the richness of texture information in citrus images. In citrus detection, image entropy helps to distinguish normal fruits from fruits with abnormal conditions such as occlusion and defects.

[0084] The calculation of image entropy in this embodiment also needs to be based on the gray-level co-occurrence matrix, and its specific mathematical expression is as follows: in, represents the entropy of the citrus detection image, Represents the gray value combination of pixel pairs, Represents the grayscale value of the first pixel in the citrus detection image, represents the grayscale value of the second pixel in the citrus detection image, Represents the grayscale value in the citrus detection image and Frequency of simultaneous occurrence.

[0085] When there is a branch occlusion area in the citrus detection image, the texture of the area is often more complex and irregular, and the gray value distribution of the pixel pair is more random, resulting in the probability value of each position in the gray level co-occurrence matrix. The calculated entropy value is usually higher when the distribution of pixel values ​​is relatively dispersed. Conversely, the surface texture of normally grown citrus fruit is relatively regular and uniform, and the grayscale value distribution of pixel pairs is relatively concentrated, resulting in a lower calculated entropy value. Therefore, the image entropy analysis model can provide a quantitative basis for anomaly detection in citrus images, helping to more accurately identify the actual growth status and obstacle conditions of citrus fruit.

[0086] The above-mentioned image contrast analysis model and image entropy analysis model can evaluate and analyze the features of citrus detection images from different angles, providing technical support for the automated detection of citrus fruits and obstacle detection.

[0087] In an optional embodiment, the citrus target detection image information is analyzed according to the color feature analysis layer and the computational prediction model to obtain the channel hue distribution and histogram similarity index information of the citrus detection image.

[0088] In the multi-level analysis system for citrus detection images, the channel hue distribution analysis model and the histogram similarity analysis model are embedded in the color feature analysis layer.

[0089] The channel hue distribution analysis model mainly includes the following steps: First, perform color space conversion. Convert the citrus image to be analyzed from RGB color space to HSV color space. This conversion process is based on the mathematical principles of color space conversion and can better meet the needs of subsequent color feature analysis.

[0090] Then perform histogram statistics. Perform histogram statistics on the H channel in the converted HSV image, and the H channel can be divided into 12 intervals (bins). The range of each interval is set to ; By counting the number of pixels in each interval, the histogram distribution of the H channel can be obtained.

[0091] Finally, the proportion is calculated and determined. The proportion of the number of pixels in each interval to the total number of pixels is calculated. For normal fruits, the red or orange area (in the HSV color space, the H value range is ) should account for a larger proportion of the overall image ; Unripe fruits show a higher proportion of green (H value range is ), if there is a leaf-blocked area, the green ratio will exceed .

[0092] Histogram similarity analysis model: After converting the citrus detection image from RGB color space to HSV color space, the color histogram similarity of the citrus detection image is calculated using the following formula.

[0093] in, represents the color histogram similarity of the citrus detection image, Represents the grayscale (or color interval) index of the histogram, Represents the grayscale level of a citrus image in the citrus detection image The normalized histogram frequency at , Indicates the grayscale level of another citrus image in the citrus detection image Normalized histogram frequency at .

[0094] In the embodiment, the color histogram similarity of the citrus image is set to , the larger the value, the higher the similarity.

[0095] In actual application, the fruit area is selected and compared with the HSV histogram of the standard color template (such as the image of ripe citrus) for similarity. A similarity threshold is set during the analysis process. In the embodiment, when the similarity is greater than 0.8, the fruit is judged to be normal fruit; when the similarity is less than 0.6, the fruit is judged to be immature fruit or there is occlusion by leaves.

[0096] In an optional embodiment, the spatial feature analysis layer and the computational prediction model are used to analyze the citrus target detection image information, so as to obtain the key spatial feature index, namely, the Euclidean distance between citrus fruits in the citrus detection image.

[0097] A Euclidean distance analysis model is set up in the spatial feature analysis layer. The spatial relationship features involved in this model mainly refer to the Euclidean distance between fruits and the number of adjacent fruits.

[0098] The calculation formula of the Euclidean distance between fruits satisfies the following relationship: in, Represents the distance between the center positions of the citrus detection images, Indicates the center position of a citrus in the detection image, Indicates the center position of another citrus in the detection image.

[0099] In the embodiment, the target detection algorithm is first used to process the citrus target detection image to obtain the center coordinates of each citrus fruit in the image; then, the distance between the center positions of any two citrus fruits is calculated according to the above-mentioned Euclidean distance formula. At the same time, a threshold for determining adjacent fruits is set. If the Euclidean distance between two citrus fruits is greater than 0. If the radius is less than 1.5 times the average radius of the citrus fruit, the two fruits are judged to be adjacent fruits.

[0100] In an optional embodiment, the citrus target detection image information is analyzed through an occlusion degree analysis layer and a computational prediction model to obtain indicators related to the occlusion area of ​​the citrus fruits in the citrus detection image.

[0101] In the occlusion analysis layer, an occlusion area analysis model is set up to quantify the occlusion of citrus fruits. The core evaluation indicator is the occlusion area ratio, which is calculated as follows: in, Indicates the occlusion area ratio of citrus in the citrus detection image, represents the area of ​​the occluded region in the citrus detection image, Represents the area of ​​citrus fruit in the citrus detection image.

[0102] In the embodiment, a segmentation algorithm is used to process the citrus target detection image to segment the fruit area and the occlusion area (such as the area covered by leaves and branches), which can clarify the distribution of pixels of different categories in the image and provide basic data for subsequent calculations.

[0103] After the segmentation is completed, the area ratio of the occlusion object and the fruit is calculated. This area ratio is a common indicator to measure the degree of overlap between two areas. By calculating the ratio of the intersection and union of the occlusion area and the fruit area, the coverage of the fruit by the occlusion object can be further evaluated, thereby assisting in determining the occlusion area.

[0104] In an optional embodiment, the head-tail morphological feature analysis layer and the computational prediction model are combined to analyze the citrus target detection image information, thereby obtaining two key morphological feature indicators of the head-tail brightness difference and the head-tail curvature radius of the citrus detection image.

[0105] A head-tail brightness difference analysis model and a head-tail curvature radius analysis model are set in the head-tail morphological feature analysis layer of the multi-level analysis system for citrus detection images.

[0106] Head-tail brightness difference analysis model (top and bottom grayscale difference): To eliminate the influence of ambient light intensity on brightness analysis results and ensure the comparability of brightness difference analysis results under different lighting conditions, the average grayscale values ​​of different areas of the citrus detection image are first normalized. The above steps can make the citrus detection image data under different lighting conditions in a unified scale range, laying the foundation for subsequent image analysis.

[0107] Weight matching is performed based on the grayscale value information of different regions. The rationality of weight distribution directly affects the accuracy of the head-tail brightness difference calculation, and can more realistically reflect the importance of different regions of the fruit in the overall image.

[0108] At the same time, the width difference analysis parameter value is introduced to further ensure the validity of the denominator of the head-tail brightness difference analysis model. The width difference analysis parameter value can avoid the abnormal situation where the denominator is zero during the calculation process, ensuring the stability of the head-tail brightness difference analysis model.

[0109] The above head-tail brightness difference analysis model satisfies the following relationship: in, represents the brightness difference between the head and tail of the citrus detection image, express The weight of represents the average grayscale value of the top area of ​​the citrus detection image, express The weight of represents the average grayscale value of the bottom area of ​​the citrus detection image, represents the minimum parameter; The above minimum value parameters can avoid the denominator being zero.

[0110] The brightness difference between the head and tail of citrus reflects the grayscale difference between the top and bottom of the fruit.

[0111] express The weight of express The weights of , and they satisfy the following relations: in, express The weight of Indicates the number of valid pixels in the top area after excluding occlusion. Indicates the number of valid pixels in the entire area after excluding occlusions.

[0112] in, express The weight of Indicates the number of valid pixels in the bottom area after excluding occlusion. Indicates the number of valid pixels in the entire area after excluding occlusions.

[0113] Divide the fruit into two equal parts along its main axis and calculate the grayscale mean of the upper and lower parts. This method can accurately obtain the average grayscale value of the top and bottom areas of the fruit, providing data support for the subsequent head-to-tail brightness difference calculation.

[0114] At the same time, a brightness difference judgment standard is set. For normal fruits, the brightness difference can be greater than 10 (and the top is brighter); for deformed fruits, the brightness difference can be less than 5. This standard can effectively distinguish normal fruits from deformed fruits, providing an important basis for citrus type identification and obstacle detection.

[0115] Analysis model of the head and tail curvature radius: In the head-tail morphological feature analysis layer of the multi-level analysis system for citrus inspection images, the head-tail curvature radius analysis model is primarily used to quantify the degree of curvature at the head and tail of citrus fruits. This model calculates the curvature radius of the head and tail by performing a polynomial fitting on the fruit contour, providing an important basis for accurate analysis of fruit morphology. The specific implementation steps are as follows: The first step is to extract and sort the contour points. The contour points of the citrus fruit are extracted from the citrus inspection image and sorted in a clockwise or counterclockwise order to ensure the consistency and order of the contour point data, laying the foundation for subsequent endpoint positioning and curvature calculation.

[0116] The second step involves endpoint position optimization. Based on geometric feature information and contour point information, the endpoint positions of the citrus fruit are optimized to ensure that the head and tail endpoints match their actual positions. This optimization method utilizes geometric feature positioning: the head is located as the highest point of the contour (where the y coordinate is maximum) or the point farthest from the center of mass; the tail is located as the lowest point of the contour (where the y coordinate is minimum) or the point closest to the center of mass. This optimization method allows for more accurate determination of the head and tail endpoints of the fruit, improving the accuracy of subsequent curvature radius calculations.

[0117] The third step is to calculate the curvature radius. Since direct calculation of the derivative will lead to numerical instability due to discretization error, the embodiment adopts the central difference method to perform multi-order derivative derivation analysis. Several points (such as multiple points before and after) are selected near the endpoint information and the head and tail endpoints, and the average value of the curvature radius is calculated. The average value of the curvature radius is calculated for multi-order derivative derivation analysis. , the specific calculation process is as follows: First-order derivative: Second-order derivative: The formula for calculating the radius of curvature is as follows: in, represents the curvature radius of the head and tail of the citrus in the citrus detection image, Represents the slope of the contour curve at a certain point in the citrus detection image, Represents the curvature of the contour curve at a certain point in the citrus detection image.

[0118] Finally, the head and tail curvature radius analysis model satisfies the following relationship: in, represents the head and tail curvature radius of the citrus detection image, Indicates the citrus detection image The coordinates of the next contour point, Indicates the citrus detection image The coordinates of the previous contour point, Represents the coordinates of the current citrus contour point in the citrus detection image.

[0119] The head curvature radius calculated by the head-tail curvature radius analysis model and the tail curvature radius , and further adopt the following determination method and steps to detect deformed fruits to distinguish normal fruits from deformed fruits.

[0120] The difference in curvature radius between the head and tail of a normal fruit is set to , that is, the following relationship is satisfied: The difference in curvature radius between the head and tail of deformed fruits is set as , that is, the following relationship is satisfied: in, represents the curvature radius of the head of the citrus fruit, represents the curvature radius of the tail of the citrus fruit, Indicates that the smaller value of the head and tail curvature radii is taken.

[0121] The thresholds 0.2 and 0.5 are empirical values. When the difference between the curvature radius of the head and tail exceeds the smaller value, The fruit shape may be abnormal. The difference in curvature radius between the head and tail of normal fruit is usually small. , while the difference in deformed fruits may be greater ,The head and tail curvature radius units of citrus can be adjusted and optimized according to the image calibration ,condition.

[0122] Furthermore, in this embodiment, the setting method and model establishment method of the multi-level analysis system for citrus detection images are only an optional condition of this embodiment. In one or some other embodiments, the setting method and model establishment method of the multi-level analysis system for citrus detection images can be optimized according to the analysis objectives of the citrus detection images and the actual conditions of citrus picking and obstacle detection. The analysis level and model parameters can be adjusted according to the actual picking environment to improve the detection accuracy of citrus and obstacles, and ensure that the picking robots or equipment can operate efficiently and safely.

[0123] S4. Analyze the citrus fruit type and obstacle conditions based on the multi-level analysis results and citrus target detection image information, and formulate different citrus picking plans based on the citrus fruit type and obstacle conditions. The specific implementation steps and contents are as follows: In this embodiment, a fruit type and obstacle recognition mechanism is also constructed. This mechanism combines a multi-level classification strategy with feature fusion rules to facilitate accurate identification of citrus fruits and obstacles.

[0124] The relevant content of the multi-level classification strategy is as follows: First-level classification: Citrus fruits are divided into two categories: normal fruits and obscured / adhered fruits. Normal fruits have complete outlines and clear features, while obscured / adhered fruits have incomplete outlines due to external factors or are connected to other objects.

[0125] Secondary classification: further subdivision for obscured / adhered fruits.

[0126] Occlusion: This category is further categorized by the cause of occlusion, including leaf occlusion, branch occlusion, and fruit overlap. Leaf occlusion occurs when citrus fruits are partially or completely covered by leaves; branch occlusion occurs when the fruit is blocked by branches; and fruit overlap occurs when multiple fruits overlap, making their outlines difficult to distinguish.

[0127] Adhesion: Based on the degree of adhesion and the number of objects involved, it can be divided into single-fruit adhesion and multi-fruit adhesion. Single-fruit adhesion is when the fruit is slightly adhered to surrounding non-fruit objects; multi-fruit adhesion is when multiple fruits are tightly connected, forming a larger adhesion.

[0128] The relevant content of feature fusion rules is as follows: Citrus geometric features and citrus texture features are integrated. When citrus fruit exhibits low roundness and high texture complexity, it can be judged as being occluded by leaves. Low roundness indicates an irregular fruit outline, while high texture complexity suggests that the fruit surface is covered by textures such as leaves.

[0129] Citrus color features and spatial relationship features are integrated. If the green ratio is high and the number of adjacent fruits is greater than 1, it can be judged as immature fruit adhesion. A high green ratio indicates that the fruit is not fully ripe, and a large number of adjacent fruits indicates that there is adhesion between the fruits.

[0130] Citrus head and tail morphological characteristics are integrated with citrus geometric characteristics. When the head and tail curvature radius differ significantly and the length-to-width ratio is abnormal, the fruit can be identified as deformed and requires separate labeling; large differences in the head and tail curvature radius and abnormal length-to-width ratio reflect irregular fruit morphology.

[0131] Finally, the categories can be classified into normal fruit, slightly obscured fruit, leaf obscured fruit, branch obscured fruit, and overlapping and sticky fruit.

[0132] Then, the fruit type and obstacle recognition mechanism, multi-level analysis results, and citrus target detection image information are combined to comprehensively identify and judge the citrus fruit type and obstacle situation. The specific implementation content is as follows: Based on the above implementation content, it can be seen that the geometric feature analysis layer obtains the roundness, aspect ratio and convex hull rate of the citrus detection image, among which the roundness reflects the roundness of the fruit contour, the aspect ratio reflects the shape characteristics of the fruit, and the convex hull rate is used to measure the convexity of the fruit contour.

[0133] The texture feature analysis layer extracts the texture features, image contrast and image entropy of the citrus detection image. The texture features describe the texture pattern of the fruit surface, the image contrast reflects the degree of difference between the light and dark areas in the image, and the image entropy is used to measure the information complexity of the image.

[0134] The color feature analysis layer calculates the channel hue distribution and histogram similarity of the citrus detection image. The above channel hue distribution can be used to analyze the distribution of fruit color, and the histogram similarity is used to compare the color similarity between different fruits or regions.

[0135] The spatial feature analysis layer obtains the Euclidean distance of the citrus detection image. The Euclidean distance is mainly used to measure the spatial distance relationship between fruits or obstacles.

[0136] The occlusion degree analysis layer calculates the occlusion area of ​​the citrus detection image, which directly reflects the degree of occlusion of the fruit.

[0137] The head-tail morphological feature analysis layer obtains the head-tail brightness difference and head-tail curvature radius of the citrus detection image. The head-tail brightness difference can be used to analyze the light and dark difference between the head and tail parts of the fruit, and the head-tail curvature radius reflects the degree of curvature of the head and tail ends of the fruit.

[0138] In an optional embodiment, the citrus fruit type and obstacle are determined as follows: First, the first-level classification judgment is performed: based on geometric features such as roundness, aspect ratio, and convex hull rate, the citrus fruit is judged to be normal fruit or occluded / adhesive fruit. If the roundness is high, the aspect ratio is close to 1, and the convex hull rate is normal, it is preliminarily judged as a normal fruit; if the roundness is low, the aspect ratio is abnormal, and the convex hull rate is low, it is preliminarily judged as an occluded / adhesive fruit.

[0139] Then the secondary classification judgment is performed: for fruits that are initially judged to be blocked / adhered, further analysis is performed based on texture features, color features, spatial features, etc.

[0140] Next is the occlusion type judgment: if the circularity is low and the texture complexity is high, the occlusion area of ​​the occlusion degree analysis layer is combined to determine whether it is leaf occlusion; if the occlusion area accounts for a large proportion, the Euclidean distance of the spatial feature analysis layer is combined to determine whether it is branch occlusion; if the number of adjacent fruits is large, the histogram similarity of the color feature analysis layer is combined to determine whether the fruits are overlapping.

[0141] Then the adhesion type is judged: according to the number of adjacent fruits and the degree of adhesion, it is judged whether it is a single fruit adhesion or multiple fruit adhesion. If the number of adjacent fruits is 1 and the degree of adhesion is light, it is judged as a single fruit adhesion; if the number of adjacent fruits is greater than 1 and the adhesion is tight, it is judged as multiple fruit adhesion.

[0142] Finally, a comprehensive judgment is made and the results are output. This multi-level analysis, combining geometric, texture, color, spatial, occlusion, and head-tail morphology, can improve the accuracy of the judgment. If the geometric features indicate an occluded / adhered fruit, and the texture and color features also support this, the fruit type and obstacle situation are further confirmed.

[0143] The final citrus picking judgment and obstacle detection results can be obtained. In the embodiment, citrus fruits can be divided into normal fruit, slightly obstructed fruit, leaf obstructed fruit, branch obstructed fruit, overlapping and sticking fruit and other categories according to the comprehensive judgment results, and the corresponding obstacle conditions can be output.

[0144] Based on the identification and judgment results of citrus fruit type and obstacle conditions, a targeted citrus picking plan is formulated.

[0145] The categories that can be picked include normal fruits and slightly obscured fruits. The above fruits are relatively easy to pick, have little impact on picking tools and operations, and can be picked directly.

[0146] Obstacle categories include leaf obstructions, branch obstructions, and overlapping and sticky fruits. The above situations will increase the difficulty of picking and require additional processing by picking equipment or manual labor, such as adjusting the picking angle, removing obstructions, or separating stuck fruits.

[0147] In an optional embodiment, the specific contents of different citrus picking plans are as follows: Picking plan for pickable categories.

[0148] Choosing tools for normal fruit picking. Because normal fruit has complete contours and clear features, making it easy to grasp and pick, a robotic arm picker with high-precision gripping capabilities can be used. These pickers are equipped with soft and flexible gripping components, such as silicone grippers, to prevent scratches or crushing damage to the fruit surface during the gripping process.

[0149] The picking parameters need to be set according to the size and weight range of normal citrus fruits, and the appropriate robotic arm grasping force and picking speed should be set. For normal fruits with a diameter of 6-8 cm, the grasping force can be set to 2-3 Newtons, and the picking speed can be controlled at 0.5-1 meter per second to ensure that the citrus fruits can be picked smoothly and safely.

[0150] The picking path planning uses the robot vision system to obtain the real-time location information of normal citrus fruits in the orchard, plan the optimal picking path, and give priority to picking normal fruits that are close to the picking equipment and easy to reach, thereby reducing the moving distance and time during the picking process and improving citrus picking efficiency.

[0151] For slightly obstructed fruits, the degree and location of the obstruction can be assessed using image analysis technology. If the obstructed portion is small and does not affect the main features and gripping points of the fruit, the fruit can be picked directly. If the obstructed portion is located in the key gripping area of ​​the fruit, such as near the fruit stem, appropriate treatment measures need to be taken.

[0152] Auxiliary picking measures can also be set up. When picking slightly obscured fruits, auxiliary tools can be used, such as a small pneumatic leaf blowing device or a light toggle rod. The pneumatic leaf blowing device can blow away the small amount of leaves blocking the surface of the fruit by blowing out air; the light toggle rod can gently move the branches or leaves to expose the key grasping area of ​​the fruit, making it easier for the picking equipment to grasp it.

[0153] Then, the picking strategy is adjusted. Since even slight obstructions can affect the picking equipment's grasping accuracy, the picking speed needs to be appropriately reduced during the picking process. Fruit positioning and grasping confirmation steps should be added. After grasping the fruit, sensors can be used to detect the grasping force and the fruit's positional stability to ensure that the fruit is firmly grasped before picking.

[0154] Obstacle category handling solution.

[0155] Removal of obstructions such as leaves. For severe leaf obstruction, a combination of mechanical removal and intelligent avoidance can be used. Mechanical removal involves installing a rotating cutting blade or pruning device to partially trim leaves obstructing the fruit before picking. Intelligent avoidance utilizes the robot's vision and sensor systems to monitor the position and movement of leaves in real time, automatically adjusting the angle and position of the picking equipment during the picking process to avoid interference from leaves.

[0156] Optimize the picking sequence. Fruit obscured by leaves is often scattered throughout an orchard. To improve picking efficiency, prioritize picking fruit that is not obscured by leaves or is less obscured by leaves, then focus on areas with significant leaf obscuration. Furthermore, plan picking routes based on the growth direction and distribution of leaves to reduce the number of round trips the picking equipment makes within the orchard.

[0157] When encountering branch obstruction, different approaches are needed depending on the thickness and hardness of the branch. Thinner branches can be quickly cut with electric shears or saws; thicker branches require initial handling with powerful tools like hydraulic shears, followed by fine cutting. When cutting branches, care should be taken to avoid damaging surrounding fruit and branches.

[0158] Adaptability of picking equipment: Because tree branches can alter the spatial position and posture of fruit, picking equipment needs to be more adaptable and flexible. Robotic arms can add multiple degrees of freedom, enabling more complex motion trajectories to navigate branches and accurately grasp fruit. Furthermore, the grasping components of the picking equipment require specialized designs, such as retractable or bendable structures, to accommodate fruit grasping at different angles and positions.

[0159] For overlapping and sticky fruits, adhesion separation technology can be introduced. For overlapping and sticky citrus fruits, a vibration device can be used to generate or apply moderate mechanical tension, causing relative displacement between the sticking citrus fruits, thereby effectively separating the overlapping and sticky fruits.

[0160] Picking order and strategy: When picking overlapping fruit, first determine the fruit's maturity and degree of adhesion. Prioritize those that are more mature and loosely attached. For tightly attached fruit, adopt a step-by-step picking strategy, picking one fruit first and then dealing with the remaining stuck fruit. Also, pay close attention to fruit damage during the picking process and adjust the picking force and method promptly.

[0161] This harvesting solution contributes to the goal of intelligent agriculture. By identifying and classifying citrus fruits and obstacles, the harvesting robot can precisely pick the fruit based on its actual condition, avoiding over-picking or missed picking. It can also rationally handle obstacles, helping to protect the orchard's ecological environment, promote the sustainable development of the citrus industry, and improve citrus fruit yield and quality.

[0162] See Figure 2In an optional embodiment, to efficiently implement the citrus picking and obstacle detection method provided by the present invention, the present invention further provides a citrus picking and obstacle detection system, wherein an input device, a processor, an output device, and a memory are interconnected in the system, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is configured to call the program instructions to execute the specific steps of the embodiments of the citrus picking and obstacle detection method provided by the present invention. The citrus picking and obstacle detection system of the present invention has a complete structure, is objective and stable, and can efficiently implement the citrus picking and obstacle detection method of the present invention, thereby improving the overall applicability and practical application capabilities of the present invention.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and description of the present invention.

Claims

1. A citrus picking and obstacle detection method, characterized in that: The steps include: Obtaining original citrus detection image data, establishing a detection image evaluation model to evaluate and analyze the original citrus detection image data, and obtaining citrus target detection image information; Determine analysis feature parameters of citrus detection images according to citrus picking and obstacle detection requirements, and construct a multi-level analysis system for citrus detection images based on the analysis feature parameters; Setting computational prediction models of different feature analysis layers in a multi-level analysis system for citrus detection images according to the analysis feature parameters, analyzing the citrus target detection image information in combination with the multi-level analysis system for citrus detection images and obtaining a multi-level analysis result of the citrus detection image; Based on the multi-level analysis results and the citrus target detection image information, the citrus fruit type and obstacle conditions are analyzed, and picking plans for different citrus fruits are formulated according to the citrus fruit type and the obstacle conditions.

2. The citrus picking and obstacle detection method according to claim 1, characterized in that: The acquiring of original citrus detection image data, establishing a detection image evaluation model to evaluate and analyze the original citrus detection image data, and obtaining citrus target detection image information includes: Obtaining structural distribution information of a citrus detection area based on the original citrus detection image data; Obtaining an actual detection area of ​​the citrus detection image according to the structural distribution information; Establishing a detection image evaluation model based on the target detection area and the actual detection area; The citrus original detection image data is evaluated and analyzed by the detection image evaluation model to obtain the structural similarity of the citrus original detection image; The original citrus detection image data is screened and optimized according to the structural similarity to obtain citrus target detection image information.

3. The citrus picking and obstacle detection method according to claim 2, characterized in that: The establishing of the detection image evaluation model by combining the target detection area and the actual detection area includes: Obtaining a center matrix and a normalization factor corresponding to the target detection area based on the target detection area; Obtaining a center matrix and a normalization factor corresponding to the actual detection area according to the actual detection area; The detection image evaluation model is established by combining the center matrix and the normalization factor; The detection image evaluation model satisfies the following relationship: in, Indicates the structural similarity between the target detection area and the actual detection area in the citrus detection image, represents the balance parameter of the citrus detection image, represents the center alignment matrix of the target detection area in the citrus detection image, express The corresponding normalization factor, represents the center alignment matrix of the actual detection area in the citrus detection image, express The corresponding normalization factor.

4. The citrus picking and obstacle detection method according to claim 1, characterized in that: The determining of analysis feature parameters of the citrus detection image according to the requirements of citrus picking and obstacle detection, and constructing a multi-level analysis system for the citrus detection image based on the analysis feature parameters include: Determine analysis feature parameters of the citrus detection image according to citrus picking and obstacle detection requirements, wherein the analysis feature parameters include citrus geometric features, citrus texture features, citrus color features, spatial relationship features, citrus occlusion degree, and citrus head and tail morphological features; A multi-level analysis system for citrus detection images is constructed based on the citrus geometric features, the citrus texture features, the citrus color features, the spatial relationship features, the citrus occlusion degree and the citrus head and tail morphological features. The multi-level analysis system for citrus detection images includes a geometric feature analysis layer, a texture feature analysis layer, a color feature analysis layer, a spatial feature analysis layer, an occlusion degree analysis layer and a head and tail morphological feature analysis layer.

5. The citrus picking and obstacle detection method according to claim 4, characterized in that: The calculation prediction models of different feature analysis layers are set in the multi-level analysis system of the citrus detection image according to the analysis feature parameters, the citrus target detection image information is analyzed in combination with the multi-level analysis system of the citrus detection image and the calculation prediction model, and the multi-level analysis results of the citrus detection image are obtained, including: Analyzing the citrus target detection image information according to the geometric feature analysis layer and the computational prediction model to obtain the circularity, aspect ratio, and convex hull ratio of the citrus detection image; Analyzing the citrus target detection image information based on the texture feature analysis layer and the computational prediction model to obtain texture features, image contrast, and image entropy of the citrus detection image; Analyzing the citrus target detection image information according to the color feature analysis layer and the computational prediction model to obtain the channel hue distribution and histogram similarity of the citrus detection image; Analyzing the citrus target detection image information using the spatial feature analysis layer and the computational prediction model to obtain the Euclidean distance of the citrus detection image; Analyzing the citrus target detection image information through the occlusion degree analysis layer and the calculation prediction model to obtain the occlusion area of ​​the citrus detection image; The citrus target detection image information is analyzed in combination with the head-tail morphological feature analysis layer and the computational prediction model to obtain the head-tail brightness difference and the head-tail curvature radius of the citrus detection image.

6. The citrus picking and obstacle detection method according to claim 1, characterized in that: The calculation prediction model of setting different feature analysis layers in the multi-level analysis system of citrus detection images according to the analysis feature parameters includes: In the geometric feature analysis layer of the multi-level analysis system for citrus detection images, a circularity analysis model, an aspect ratio analysis model, and a convex hull ratio analysis model are set; Setting a texture feature analysis model, an image contrast analysis model and an image entropy analysis model in the texture feature analysis layer of the multi-level analysis system for citrus detection images; The texture feature analysis model satisfies the following relationship: in, Represents the center pixel of the citrus detection image The uniform LBP value of Indicates the total number of neighborhood pixels minus 1, represents the comparison function, Represents the neighborhood pixels of the citrus detection image The gray value of Represents the neighborhood pixels of the citrus detection image The gray value of Represents the index encoding of the neighborhood pixels of the citrus detection image, b represents the number of transitions in binary code. Indicates other situations.

7. The citrus picking and obstacle detection method according to claim 1, characterized in that: The calculation prediction model of setting different feature analysis layers in the multi-level analysis system of citrus detection images according to the analysis feature parameters includes: A channel hue distribution analysis model and a histogram similarity analysis model are set in the color feature analysis layer of the multi-level analysis system for citrus detection images; Setting up a Euclidean distance analysis model in the spatial feature analysis layer of the multi-level analysis system for citrus detection images; An occlusion area analysis model is set in the occlusion degree analysis layer of the multi-level analysis system for citrus detection images.

8. The citrus picking and obstacle detection method according to claim 1, characterized in that: The calculation prediction model of setting different feature analysis layers in the multi-level analysis system of citrus detection images according to the analysis feature parameters includes: In the head-tail morphological feature analysis layer of the multi-level analysis system for citrus detection images, a head-tail brightness difference analysis model and a head-tail curvature radius analysis model are set; The head-tail brightness difference analysis model satisfies the following relationship: in, represents the brightness difference between the head and tail of the citrus detection image, express The weight of represents the average grayscale value of the top area of ​​the citrus detection image, express The weight of represents the average grayscale value of the bottom area of ​​the citrus detection image, represents the minimum parameter; The head and tail curvature radius analysis model satisfies the following relationship: in, represents the head and tail curvature radius of the citrus detection image, Indicates the citrus detection image The coordinates of the next contour point, Indicates the citrus detection image The coordinates of the previous contour point, Represents the coordinates of the current citrus contour point in the citrus detection image.

9. The citrus picking and obstacle detection method according to claim 1, characterized in that: Analyzing citrus fruit types and obstacle conditions based on the multi-level analysis results and the citrus target detection image information, and formulating picking plans for different citrus fruits according to the citrus fruit types and the obstacle conditions includes: Build a mechanism for identifying fruit types and obstacles; Identify and judge the citrus fruit type and obstacle situation by combining the fruit type and obstacle recognition mechanism, the multi-level analysis results, and the citrus target detection image information; Based on the identification and judgment results of citrus fruit type and obstacle conditions, picking plans for different citrus fruits are formulated.

10. A citrus picking and obstacle detection system, characterized in that: The system includes a processor, an input device, an output device and a memory, which are interconnected. The memory is used to store a computer program, and the computer program includes program instructions. The processor is configured to call the program instructions to execute the citrus picking and obstacle detection method according to any one of claims 1 to 9.

Images