Multi-spectrum-based intelligent robot inspection path planning method and system

Abstract

The invention belongs to the technical field of machine intelligence, and particularly relates to an intelligent robot inspection path planning method and system based on multispectrum, and the method comprises the steps: calculating an abnormal score according to a reflectivity vector and a vegetation index of a pixel point, forming an instantaneous abnormal point set of an inspection period through abnormal pixel points with the abnormal score greater than an abnormal threshold value, clustering to obtain a plurality of regions of interest, calculating an abnormal area change rate and an abnormal cumulative frequency of the regions of interest according to an overlapped region of the instantaneous abnormal point set and the regions of interest of the current inspection period and a plurality of previous historical inspection periods, and performing weighted summation to obtain a comprehensive inspection priority; and constructing a traveling salesman problem with priority constraint according to the regions of interest in the current inspection period and the comprehensive inspection priority thereof, and solving through an ant colony algorithm to obtain a shortest path traversing all the regions of interest as an optimal detailed inspection path of the current inspection period. The method ensures that the high-risk area is preferentially accessed.

Description

Technical Field

[0001] This invention relates to the field of machine intelligence technology. More specifically, this invention relates to a method and system for planning inspection paths for intelligent robots based on multispectral imaging. Background Technology

[0002] As a high-value economic crop, cranberries require extremely high precision in their cultivation and management. Due to the high water content, thin skin, and susceptibility to rot of cranberry fruits, as well as their special growing environment, which requires acidic peat soil and regular flooding management, the cultivation process is highly sensitive to pest and disease monitoring and environmental stress early warning.

[0003] Traditional manual inspection methods are not only inefficient, but also highly subjective and difficult to quantify, especially in their inability to identify early diseases (such as flower rot and fruit rot). As the scale of cranberry cultivation continues to expand, this inefficient and subjective inspection method can no longer meet the urgent needs of modern large-scale and standardized cultivation.

[0004] In modern precision agriculture management, inspection robots equipped with multispectral cameras are used to inspect large-scale cranberry planting areas. Multispectral imaging technology collects reflectance data of crops in blue, green, red, and near-infrared bands, which can calculate key indicators such as vegetation index. This effectively identifies crop growth monitoring, disease and pest early warning, and stress diagnosis. It can detect physiological changes in cranberry plants before symptoms appear, providing a valuable time window for timely intervention.

[0005] However, the core contradiction faced by existing technologies lies in the irreconcilable conflict between comprehensive inspection and efficient discovery: on the one hand, to ensure that no potential problem areas are missed, inspection robots need to execute full-coverage paths, but this results in a lot of time being wasted in healthy areas; on the other hand, to improve the efficiency of anomaly discovery, the focus should be on high-risk areas, but there is a lack of real-time judgment mechanisms, resulting in significant time lag. Summary of the Invention

[0006] To address the technical problem of the irreconcilable conflict between comprehensive inspection and efficient discovery during the inspection process of intelligent robots, this invention provides solutions in the following aspects.

[0007] In a first aspect, the present invention provides a multispectral-based intelligent robot inspection path planning method, comprising: at the beginning of an inspection cycle, the inspection robot collects multispectral images of the entire planting area according to a globally covered path, obtains the reflectance vector of each pixel, and calculates the vegetation index of each pixel; based on the reflectance vector and vegetation index of each pixel, calculates the anomaly score of each pixel, and forms an instantaneous anomaly point set for the inspection cycle by grouping anomaly pixels with anomaly scores greater than anomaly thresholds; clustering the instantaneous anomaly point set to obtain multiple regions of interest for the inspection cycle; and calculating the regions of interest based on the differences between the overlapping areas of the instantaneous anomaly point sets of the current inspection cycle and the regions of interest of multiple previous historical inspection cycles. The abnormal area change rate is used to calculate the cumulative frequency of anomalies in the region of interest (ROI) based on the area of ​​the overlapping region between the instantaneous anomaly set and the region of interest from multiple historical inspection cycles prior to the current inspection cycle. The ROI's comprehensive inspection priority is obtained by weighted summation of the ROI's abnormal area change rate and cumulative frequency. All ROIs in the current inspection cycle are sorted according to their comprehensive inspection priority from highest to lowest, and several high-risk regions are selected. Based on all high-risk regions and their comprehensive inspection priorities, a traveling salesman problem with priority constraints is constructed and solved using the ant colony algorithm to obtain the shortest path traversing all high-risk regions, which serves as the optimal detailed inspection path for the current inspection cycle.

[0008] This invention introduces a time dimension, comprehensively considering the rate of change of abnormal area and the frequency of abnormal accumulation in regions of interest derived from historical data analysis. This allows for the calculation of the comprehensive inspection priority for each region of interest. Based on this comprehensive inspection priority, a traveling salesman problem with priority constraints is constructed. This ensures that the planned path not only pursues efficiency in terms of distance but also guarantees that the robot prioritizes and thoroughly examines critical high-risk areas that are rapidly deteriorating or have been present for a long time. This significantly improves the intelligence, targeting, and timeliness of agricultural intervention during inspections, avoiding economic losses caused by delays in optimal treatment.

[0009] Preferably, the method for setting the time interval of the inspection cycle includes: the growth cycle of cranberries has a clear seasonal pattern: December to February is the dormant period, March to May is the budding period, June to July is the flowering period, August to October is the fruiting period, and November is the ripening period; the time interval between two inspection cycles is different in different growth stages of cranberries: during the dormant period, the time interval is set to 21 days; during the budding period, the time interval is set to 7 days; during the flowering period, the time interval is set to 3 days; during the fruiting period, the time interval is set to 5 days; and during the ripening period, the time interval is set to 2 days.

[0010] Based on the physiological characteristics and risk sensitivity of cranberries at different growth stages, this invention dynamically and differentiates the time interval of the inspection cycle, which can ensure that problems are detected and dealt with in a timely manner during critical growth periods, while avoiding unnecessary frequent inspections during non-critical periods, effectively reducing the robot's energy consumption and equipment wear and tear.

[0011] Preferably, the acquisition of multispectral images of the entire planting area, obtaining the reflectance vector of each pixel, and calculating the vegetation index of each pixel includes: the multispectral camera providing four key bands, including: blue light band, green light band, red light band, and near-infrared band; obtaining the reflectance of each pixel in the multispectral image in the blue light band, green light band, red light band, and near-infrared band. , , and And form the reflectance vector of the pixel. Based on the reflectivity of pixels in the red light band and reflectivity in the near-infrared band Calculate the vegetation index of each pixel. .

[0012] Preferably, the step of calculating the anomaly score of a pixel based on its reflectance vector and vegetation index includes: converting the reflectance vector of the pixel... Vegetation index of pixels The multispectral feature vectors of all pixels in the multispectral images collected during the inspection cycle are used to form the multispectral data for the inspection cycle. The isolated forest benchmark model in the isolated forest anomaly detection algorithm is trained using the multispectral data from the three historical inspection cycles prior to the current inspection cycle. The input is the multispectral feature vector of the pixel, and the output is the anomaly score of the pixel. The anomaly score of each pixel in the multispectral data of the current inspection cycle is calculated using the obtained isolated forest benchmark model.

[0013] Preferably, the step of calculating the rate of change of the abnormal area of ​​the region of interest based on the difference between the overlapping areas of the instantaneous abnormal point set of the current inspection cycle and multiple previous historical inspection cycles and the region of interest includes: In the formula, Indicates the first The rate of change of the abnormal area of ​​each region of interest; For a short period of time; This represents the set of instantaneous anomalies in the current inspection cycle; Indicates the first inspection cycle adjacent to the current inspection cycle. A set of instantaneous anomalies for each historical inspection cycle; Indicates the first A set consisting of all pixels in a region of interest; This indicates finding the intersection; This indicates the number of pixels in the set; This indicates the time interval between two inspection cycles.

[0014] This invention quantifies the dynamic evolution trend of abnormal areas in the short term by calculating the rate of change of abnormal area. It can clearly identify areas that are rapidly expanding or gradually shrinking, providing a key basis for priority assessment of urgency, making inspection decisions more forward-looking, and enabling priority response to rapidly developing and most threatening problems.

[0015] Preferably, the step of calculating the anomaly accumulation frequency of the region of interest based on the area of ​​the overlapping region between the instantaneous anomaly point set of multiple historical inspection cycles prior to the current inspection cycle and the region of interest includes: In the formula, Indicates the first The cumulative frequency of anomalies in each region of interest; For long duration, the following is required ; Indicates the first inspection cycle adjacent to the current inspection cycle. A set of instantaneous anomalies for each historical inspection cycle; Indicates the first A set consisting of all pixels in a region of interest; This indicates finding the intersection; This indicates the number of pixels in the set; It is an indicator function; its value is 1 when the condition inside the parentheses is true, and 0 otherwise. This is the area threshold.

[0016] This invention effectively identifies areas where problems have repeatedly occurred in history by calculating the cumulative frequency of anomalies. It can reveal chronic stress caused by fundamental and long-term factors such as soil problems and irrigation system defects, making up for the shortcomings of focusing only on instantaneous states. This allows path planning to take into account deep-seated problems that do not change rapidly but have existed for a long time, thus improving the comprehensiveness and fundamental nature of problem diagnosis.

[0017] Preferably, the step of obtaining the comprehensive inspection priority of the region of interest by weighted summing of the abnormal area change rate and abnormal accumulation frequency includes: In the formula, Indicates the first Comprehensive inspection priority for each area of ​​interest; Indicates the first The rate of change of the abnormal area of ​​each region of interest; Indicates the first The cumulative frequency of anomalies in each region of interest; Represents the normalization function; and It is a weighting coefficient, and , .

[0018] This invention constructs a comprehensive inspection priority by weighted summation of the rate of change of abnormal area and the frequency of abnormal accumulation. It can integrate the dynamic change trend of a region with its historical persistence, and provide a comprehensive and quantitative risk assessment result for each region of interest, making the subsequent path planning decision more scientific, comprehensive and reliable.

[0019] Preferably, the weighting coefficient and The setup methods include: dynamically adjusting the weight coefficients based on the growth stage. and During dormancy, Set it to 0.3, Set to 0.7; during the germination stage, Set to 0.6, Set to 0.4; during the flowering period, Set to 0.8, Set to 0.2; during the results period, Set it to 0.5, Set to 0.5; during the maturity stage, Set to 0.7, Set it to 0.3.

[0020] Preferably, the step of obtaining a shortest path traversing all high-risk areas using the ant colony algorithm includes: treating all high-risk areas as nodes, and the probability of an ant choosing the next node is determined by the pheromone concentration between two nodes and heuristic information. Specifically, when solving the priority-constrained traveling salesman problem using the ant colony algorithm, the heuristic information between two nodes is equal to the comprehensive inspection priority of the next node. Distance to two nodes reciprocal The product of; the path is selected using the roulette wheel selection method based on probability.

[0021] This invention improves heuristic information by integrating inspection priorities, so that when ants choose the next node, they are attracted by both high priority and proximity, ensuring that high-risk areas are visited first.

[0022] Secondly, the present invention provides a multispectral-based intelligent robot inspection path planning system, including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the above-mentioned multispectral-based intelligent robot inspection path planning method is implemented.

[0023] By adopting the above technical solution, the above-mentioned multispectral-based intelligent robot inspection path planning method is generated into a computer program and stored in a memory for loading and execution by a processor. This allows for the creation of a terminal device based on the memory and processor, facilitating its use.

[0024] The beneficial effects of this invention are as follows: Compared to existing technologies where robot inspection path planning typically focuses solely on the shortest physical path, this invention introduces a time dimension. It comprehensively considers the rate of change of abnormal areas and the frequency of abnormal accumulation in regions of interest, derived from historical data analysis. This allows for the calculation of the overall inspection priority for each region of interest. Based on this overall inspection priority, a traveling salesman problem with priority constraints is constructed. This ensures that the planned path not only prioritizes efficiency in terms of distance but also ensures that the robot prioritizes and thoroughly inspects critical high-risk areas that are rapidly deteriorating or have been present for a long time. Consequently, this significantly improves the intelligence, targeting, and timeliness of agricultural intervention during inspections, avoiding economic losses caused by delays in optimal treatment. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the multispectral-based intelligent robot inspection path planning method of the present invention; Figure 2 This is a flowchart illustrating step S3. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0028] This invention discloses a multispectral-based intelligent robot inspection path planning method, referring to... Figure 1 This includes steps S1 and S4: S1: At the beginning of the inspection cycle, the inspection robot collects multispectral images of the entire planting area according to the global coverage path, obtains the reflectance vector of the pixel, and calculates the vegetation index of the pixel.

[0029] It should be noted that multispectral data is the foundation of anomaly detection. Therefore, at the beginning of each inspection cycle, multispectral images of the entire planting area are collected globally.

[0030] Specifically, the growth cycle of cranberries exhibits a distinct seasonal pattern: December to February is the dormancy period, March to May is the budding period, June to July is the flowering period, August to October is the fruiting period, and November is the ripening period. The time interval between two inspection cycles varies depending on the different growth stages of the cranberry plant. Specifically: during dormancy, the plant's metabolic activity is at its lowest, and the above-ground parts cease growth; therefore, the time interval is set at 21 days. During budding, buds swell, new shoots begin to grow, and the plant is extremely sensitive to environmental changes; therefore, the time interval is set at 7 days. During flowering, bud formation, flowering, and pollination directly affect the fruit set rate; therefore, the time interval is set at 3 days. During fruiting, fruit development and enlargement are critical, and sugar accumulation is at its peak; therefore, the time interval is set at 5 days. During ripening, the fruit matures, the sugar-acid ratio changes, and harvesting is in preparation; therefore, the time interval is set at 2 days.

[0031] Furthermore, at the beginning of each inspection cycle, the inspection robot travels in the cranberry planting area according to a global coverage path and uses its onboard multispectral camera to collect multispectral images of the entire planting area. The multispectral camera provides four key bands, including: blue light band (corresponding to 450–520nm band), green light band (corresponding to 520–600nm band), red light band (corresponding to 630–690nm band), and near-infrared band (corresponding to 760–900nm band).

[0032] The global coverage path is a preset bow-shaped pattern or parallel line pattern. In order to improve the basic scanning efficiency, the inspection robot travels at a relatively fast speed, usually set to 0.8 m / s. The multispectral camera acquires multispectral images at a frequency of one frame every 10 seconds. The resolution of the multispectral camera is the standard resolution mode, i.e., 5 million pixels.

[0033] For each newly acquired multispectral image: First, georegistration is performed using GPS and IMU (Inertial Measurement Unit) data, and precise spatial alignment is achieved with historical multispectral images stored in the database to ensure accurate correspondence of pixels at the same geographical location across all multispectral images. Then, the calibrated multispectral images undergo preprocessing, including radiometric calibration and atmospheric correction, to eliminate the influence of environmental factors such as illumination variations on the multispectral images. Finally, the reflectance of each pixel in the multispectral image is obtained in the blue, green, red, and near-infrared bands. , , and And form the reflectance vector of the pixel. .

[0034] Additionally, based on the reflectivity of the pixel in the red light band and reflectivity in the near-infrared band Calculate the vegetation index of each pixel. The obtained vegetation index The value range of is [-1, 1]. Specifically, if... , .

[0035] S2: Calculate the anomaly score of the pixel based on the reflectance vector and vegetation index. Pixels with anomaly scores greater than the anomaly threshold are grouped into a set of instantaneous anomalies for the inspection cycle. Cluster the instantaneous anomaly sets to obtain multiple regions of interest for the inspection cycle.

[0036] It should be noted that by calculating and clustering anomaly scores, high-dimensional spectral anomalies are transformed into actionable spatial targets, providing precise targets for path planning.

[0037] Specifically, the reflectance vector of the pixel Vegetation index of pixels The multispectral feature vectors of all pixels in the multispectral images collected during the inspection cycle are used to form the multispectral data of the inspection cycle.

[0038] Furthermore, multispectral data from three historical inspection cycles prior to the current inspection cycle are loaded from the historical database to train the isolated forest benchmark model in the isolated forest anomaly detection algorithm. The input of the isolated forest benchmark model is the multispectral feature vector of the pixel, and the output is the anomaly score of the pixel. The isolated forest benchmark model integrates multi-band data and vegetation indices, which can improve the robustness of anomaly detection.

[0039] Finally, using the obtained isolated forest baseline model, the anomaly score of each pixel in the multispectral data of the current inspection cycle is calculated; an anomaly threshold is set, and pixels with anomaly scores greater than the threshold are recorded as anomalous pixels, forming the instantaneous anomaly set for the current inspection cycle. .

[0040] The outlier scores of the pixels calculated by the isolated forest benchmark model are between 0 and 1. The set outlier threshold is used to filter out outlier pixels with higher outlier scores from all pixels. Therefore, the outlier threshold is set to 0.7.

[0041] It should be noted that the Isolation Forest anomaly detection algorithm is used to determine the anomaly score of data points. The higher the anomaly score, the more anomalous the data point. The Isolation Forest anomaly detection algorithm is a well-known technology and will not be described in detail here.

[0042] It should be further noted that, in order to avoid the planned robot inspection path being fragmented by noise points, the discrete abnormal pixels output by the isolated forest anomaly detection algorithm need to be organized into spatially coherent regions of interest.

[0043] Therefore, the DBSCAN clustering algorithm is used to analyze the instantaneous anomaly set of the current inspection cycle. The abnormal pixels in the data are clustered, and each cluster is used as a region of interest for the current inspection cycle.

[0044] The parameters in the DBSCAN clustering algorithm, namely the neighborhood radius eps and the minimum number of samples min_samples, need to be adapted to the planting characteristics of cranberries. Therefore, the parameter settings in the DBSCAN clustering algorithm are as follows: (1) Based on experience, the row spacing of cranberry plants is usually 0.5 meters to 1 meter. In order to ensure that pixels in the same diseased area are clustered, the neighborhood radius eps of the DBSCAN clustering algorithm is set according to the median value of the row spacing, which is 0.75 meters. The empirical value of the row spacing of 0.75 meters is converted into the neighborhood radius eps in the image space through the ground resolution. Then the neighborhood radius eps is equal to , For ground resolution; (2) Based on experience, the area of ​​a single cranberry plant is 0.5 square meters, meaning the empirical value of the area of ​​a single cranberry plant is 0.5. To avoid noise interference, the minimum number of points (min_samples) for the DBSCAN clustering algorithm is set based on the empirical value of 0.5 for the area of ​​a single plant. This empirical value of 0.5 is then converted into the minimum number of points (min_samples) in the image space using ground resolution. Therefore, the minimum number of points (min_samples) is equal to... , This is the ground resolution.

[0045] Ground resolution is equal to the ratio of the actual distance in geographic space to the number of pixels in image space.

[0046] S3: Calculate the rate of change of the abnormal area of ​​the region of interest based on the difference between the overlapping areas of the instantaneous abnormal point sets of the current inspection cycle and the regions of interest in multiple previous historical inspection cycles. Calculate the cumulative frequency of abnormalities in the region of interest based on the area of ​​the overlapping areas of the instantaneous abnormal point sets of the current inspection cycle and the regions of interest in multiple previous historical inspection cycles. Calculate the weighted sum of the rate of change of the abnormal area and the cumulative frequency of abnormalities in the region of interest to obtain the comprehensive inspection priority of the region of interest.

[0047] Refer to the flowchart for step S3 Figure 2 This includes steps S301 to S303, specifically: S301. Calculate the rate of change of the abnormal area of ​​the region of interest based on the difference between the instantaneous abnormal point set of the current inspection cycle and the overlapping area of ​​the region of interest in the previous multiple historical inspection cycles.

[0048] It should be noted that a rapidly expanding area usually indicates that the disease or stress is intensifying, and its risk level and the urgency of inspection are higher. Therefore, by retrieving multiple historical inspection cycles from the historical database and identifying the set of pixels identified as anomalous in the region of interest in each historical inspection cycle, and subtracting the set of instantaneous anomalous points in the region of interest in the current inspection cycle, the rate of change of the anomalous area of ​​the region of interest is constructed. This indicator can capture the expansion or contraction trend of the region of interest in the recent time window.

[0049] Specifically, for each region of interest in the current inspection cycle: based on the difference between the overlapping areas of the instantaneous anomaly point sets in the current inspection cycle and multiple previous historical inspection cycles and the regions of interest, the rate of change of the anomaly area of ​​the region of interest is calculated. The specific calculation formula is as follows: ; In the formula, Indicates the first The rate of change of the abnormal area of ​​each region of interest; For short-term durations, it is used to obtain the rate of change of anomalous areas that characterizes short-term dynamic changes; therefore, Set to 3; This represents the set of instantaneous anomalies in the current inspection cycle; Indicates the first inspection cycle adjacent to the current inspection cycle. A set of instantaneous anomalies for each historical inspection cycle; Indicates the first A set consisting of all pixels in a region of interest; This indicates finding the intersection; This indicates the number of pixels in the set; This indicates the time interval between two inspection cycles.

[0050] in, This represents the set of instantaneous anomalies during the current inspection cycle. With the A set consisting of all pixels in a region of interest The intersection represents the points that fall within the current inspection cycle. The total area covered by abnormal pixels within the corresponding geographic range; Indicates the first inspection cycle adjacent to the current inspection cycle. Set of instantaneous anomalies in a historical inspection cycle With the A set consisting of all pixels in a region of interest The intersection of these points represents the first point adjacent to the current inspection cycle. During each historical inspection cycle, it fell into The total area covered by abnormal pixels within the corresponding geographical range.

[0051] The dynamic trend of the region of interest is characterized by calculating the net change in the area of ​​the abnormal region per unit time: when When the value is positive and large, it indicates that the region of interest is expanding rapidly, and the abnormal situation in the corresponding scene may be deteriorating rapidly; when... When the value is negative, it indicates that the region of interest is shrinking, and the anomaly may be improving or under control; when... When the value is close to zero, it indicates that the region of interest is in a stable state.

[0052] S302. Calculate the cumulative frequency of anomalies in the region of interest based on the area of ​​the overlapping region between the instantaneous anomaly point set of multiple historical inspection cycles prior to the current inspection cycle and the region of interest.

[0053] It should be noted that a newly emerging anomalous area may be merely an isolated phenomenon, while a recurring anomalous area indicates a deeper and more persistent source of risk. Therefore, after analyzing recent changes in the region of interest, it is necessary to further consider its persistence, i.e., whether the region of interest is an area with persistent and intractable problems. Thus, by statistically analyzing the percentage of times a specific geographical location corresponding to a region of interest was identified as significantly abnormal within multiple historical inspection cycles retrieved from historical databases, an anomalous cumulative frequency of the region of interest is constructed. This indicator aims to identify regions of interest that have historically experienced recurring problems. These regions typically represent chronic stress caused by fundamental factors such as soil issues and irrigation deficiencies, and also require close attention.

[0054] Specifically, for each region of interest in the current inspection cycle: the cumulative frequency of anomalies in the region of interest is calculated based on the area of ​​the overlapping region between the instantaneous anomaly point set of multiple historical inspection cycles prior to the current inspection cycle and the region of interest. The specific calculation formula is as follows: ; In the formula, Indicates the first The cumulative frequency of anomalies in each region of interest; The duration is long-term, used to obtain the cumulative frequency of anomalies characterizing long-term historical patterns; therefore, it requires... ,Will Set to 7; Indicates the first inspection cycle adjacent to the current inspection cycle. A set of instantaneous anomalies for each historical inspection cycle; Indicates the first A set consisting of all pixels in a region of interest; This indicates finding the intersection; This indicates the number of pixels in the set. Used to determine the first In each historical inspection cycle Does the area within the range exceed the area threshold? Abnormal areas; It is an indicator function; its value is 1 when the condition inside the parentheses is true, and 0 otherwise. This is the area threshold used to filter out tiny, negligible noise.

[0055] Based on experience, the empirical value for the area of ​​a single cranberry plant is equal to 0.5. Converting this empirical value into an area threshold in image space using ground resolution, the area threshold is then equal to... , This is the ground resolution.

[0056] Among them, the frequency of abnormal accumulation in the region of interest The value range is [0,1]: abnormal cumulative frequency A value close to 1 indicates that the region of interest is a hotspot where problems repeatedly occur, corresponding to persistent and recurring issues in the scenario; abnormal accumulation frequency A value close to 0 indicates that the region of interest has historically had few or no problems.

[0057] S303. The abnormal area change rate and abnormal cumulative frequency of the region of interest are weighted and summed to obtain the comprehensive inspection priority of the region of interest.

[0058] It should be noted that the dynamic evolution features obtained from steps S301 and S302, namely the rate of change of abnormal area and the persistence features, namely the frequency of abnormal accumulation, are combined to generate a comprehensive inspection priority for each region of interest clustered from abnormal pixels, thereby guiding path planning during subsequent inspections.

[0059] Specifically, for each region of interest in the current inspection cycle: the comprehensive inspection priority of the region of interest is obtained by weighted summation of the rate of change of abnormal area and the frequency of abnormal accumulation. The specific calculation formula is as follows: ; In the formula, Indicates the first Comprehensive inspection priority for each area of ​​interest; Indicates the first The rate of change of the abnormal area of ​​each region of interest; Indicates the first The cumulative frequency of anomalies in each region of interest; The normalization function, including but not limited to the minimum-maximum (Min-Max) normalization function, maps the rate of change of abnormal area to the interval [0,1] and unifies the units with the abnormal cumulative frequency so that it can be weighted and summed with the abnormal cumulative frequency; and It is a weighting coefficient, and , This is used to balance the weight of urgency and importance in the final decision.

[0060] For weighting coefficients and The adjustment can be made dynamically based on the growth stage: during periods when cranberry diseases are prevalent, such as the flowering period, the temperature can be appropriately increased. Prioritize treating rapidly spreading problem areas (such as flower rot and fruit rot), and appropriately reduce [the dosage] during the stable growth period of cranberries, such as the dormancy period. Prioritize areas with long-standing chronic problems (such as soil compaction and uneven irrigation), therefore, during the dormant period, Set it to 0.3, Set to 0.7, during the germination stage, Set to 0.6, Set to 0.4, during the flowering period, Set to 0.8, Set to 0.2, during the results period, Set it to 0.5, Set to 0.5, during the maturity period, Set to 0.7, Set it to 0.3.

[0061] Among them, the rate of change of abnormal area, which represents short-term dynamic changes, is obtained by weighted summation. and characterizing long-term historical patterns Merge into a unified priority score : A region of interest with a higher value indicates that it is a rapidly deteriorating problem, a long-standing and persistent problem, or both. Therefore, it should be given the highest priority during subsequent inspections.

[0062] S4: Sort all regions of interest in the current inspection cycle according to the comprehensive inspection priority from largest to smallest, and select multiple high-risk regions; based on all high-risk regions and their comprehensive inspection priorities, construct a traveling salesman problem with priority constraints, solve it using the ant colony algorithm, and obtain the shortest path to traverse all high-risk regions, which serves as the optimal detailed inspection path for the current inspection cycle.

[0063] It should be noted that integrating comprehensive inspection priorities into path planning enables intelligent resource allocation.

[0064] Specifically, according to the comprehensive inspection priority from highest to lowest, all areas of interest in the current inspection cycle are sorted, and the sorted areas are then ranked... Each area of ​​interest, designated as a high-risk area, requires prior... The sum of the number of pixels in each region of interest is less than or equal to 20% of the total number of pixels in the multispectral image of the entire planting area, while the former The sum of the number of pixels in each region of interest is greater than 20% of the total number of pixels in the multispectral image of the entire planting area.

[0065] For high-risk areas in the current inspection cycle: the position of the centroid of each high-risk area is used as its location information; all high-risk areas are treated as nodes, and a traveling salesman problem with priority constraints is constructed by combining the comprehensive inspection priority of high-risk areas: the goal is to start from the location of the inspection robot, traverse all high-risk areas, and obtain the shortest total path while prioritizing the high-risk areas with high comprehensive inspection priority.

[0066] Furthermore, the ant colony algorithm is used to solve the constructed Traveling Salesman Problem with priority constraints, obtaining a shortest path traversing all high-risk areas. This shortest path serves as the optimal detailed inspection path for the current inspection cycle. During this process, a roulette wheel selection method is used based on probability. The probability of an ant choosing the next node is determined by both the pheromone concentration between two nodes and the heuristic information: when solving the conventional Traveling Salesman Problem using the ant colony algorithm, the heuristic information between two nodes is equal to the distance between the two nodes. reciprocal When solving the Traveling Salesman Problem with priority constraints using the ant colony algorithm, the heuristic information between two nodes is equal to the overall inspection priority of the next node. Distance to two nodes reciprocal The product of.

[0067] In calculating the probability of selecting a path, the overall inspection priority is combined with distance. High-risk areas with higher overall inspection priority and closer distance will have a greater probability of being selected. The optimal detailed inspection path planned in this way can realize intelligent inspection with priority to high-risk areas.

[0068] When the inspection robot follows the optimal detailed inspection path, its travel speed is faster in non-high-risk areas, typically set to 0.8 m / s, and slower in high-risk areas, typically set to 0.3 m / s. The multispectral camera acquires one frame every 3 seconds, and its resolution is switched to high-resolution mode, i.e., 12 megapixels. Based on the reflectivity of pixels in the red light band in the acquired high-resolution multispectral images... and reflectivity in the near-infrared band Calculate the vegetation index of each pixel. .

[0069] The average vegetation index of all pixels in the high-risk area is used as the vegetation index of the high-risk area, which is then used to monitor the growth status of cranberries in the corresponding geographic space of the high-risk area. (1) During the dormant period, if the vegetation index of a high-risk area is less than 0.12, it indicates a serious risk of frost damage. If the vegetation index of a high-risk area is greater than 0.30, it indicates abnormal budding, which may lead to a decrease in cold resistance due to a warm winter.

[0070] (2) During the germination period, if the difference in vegetation index between two adjacent inspection cycles in a high-risk area is less than 0.02, it indicates that there is a lack of nutrients or root problems.

[0071] (3) During the flowering period, if the vegetation index of a high-risk area is less than 0.45, it indicates that the flower organs are underdeveloped and there is a risk of low fruit set rate. If the vegetation index of a high-risk area is greater than 0.65, it indicates that there is an excess of nutrients, which may lead to flower drop.

[0072] (4) During the fruiting period, if the vegetation index of a high-risk area is less than 0.48, it indicates that there is a problem of nutrient deficiency or early fruit rot. If the vegetation index of a high-risk area is greater than 0.72, it indicates that there is excessive nitrogen and poor fruit coloring.

[0073] (5) If the vegetation index of a high-risk area is greater than 0.55 during the maturity period, it indicates that the maturity is delayed and may affect the harvesting plan.

[0074] This invention also discloses a multispectral-based intelligent robot inspection path planning system, including a processor and a memory. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, the multispectral-based intelligent robot inspection path planning method according to this invention is implemented.

[0075] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.

Claims

1. A method for intelligent robot inspection path planning based on multispectral imaging, characterized in that, include: At the beginning of the inspection cycle, the inspection robot collects multispectral images of the entire planting area according to a global coverage path, obtains the reflectance vector of each pixel, and calculates the vegetation index of each pixel. Based on the reflectance vector and vegetation index of each pixel, the anomaly score of each pixel is calculated. Pixels with anomaly scores greater than the anomaly threshold are grouped into an instantaneous anomaly set for the inspection cycle. Clustering of instantaneous anomalies yields multiple regions of interest for the inspection cycle; Based on the differences between the overlapping areas of the instantaneous anomaly point sets and the region of interest in the current inspection cycle and several previous historical inspection cycles, the rate of change of the anomaly area of ​​the region of interest is calculated. Based on the area size of the overlapping areas of the instantaneous anomaly point sets and the region of interest in the several previous historical inspection cycles, the cumulative frequency of anomalies in the region of interest is calculated. The comprehensive inspection priority of the region of interest is obtained by weighted summation of the rate of change of the anomaly area and the cumulative frequency of anomalies. Sort all regions of interest in the current inspection cycle according to the comprehensive inspection priority from highest to lowest, and select several high-risk regions. Based on all high-risk regions and their comprehensive inspection priorities, construct a traveling salesman problem with priority constraints, solve it using the ant colony algorithm, and obtain the shortest path to traverse all high-risk regions, which serves as the optimal detailed inspection path for the current inspection cycle.

2. The method for planning inspection paths for intelligent robots based on multispectral imaging according to claim 1, characterized in that, The method for setting the time interval of the inspection cycle includes: Cranberries have a distinct seasonal growth cycle: December to February is the dormant period, March to May is the budding period, June to July is the flowering period, August to October is the fruiting period, and November is the ripening period. The time interval between two inspection cycles varies depending on the different growth stages of cranberries: during dormancy, the interval is set to 21 days; during budding, the interval is set to 7 days; during flowering, the interval is set to 3 days; during fruiting, the interval is set to 5 days; and during ripening, the interval is set to 2 days.

3. The method for planning inspection paths for intelligent robots based on multispectral imaging according to claim 1, characterized in that, The process of acquiring multispectral images of the entire planting area, obtaining the reflectance vectors of pixels, and calculating the vegetation index of pixels includes: The multispectral camera provides four key bands: blue, green, red, and near-infrared. It obtains the reflectance of each pixel in the multispectral image across these bands. , , and And form the reflectance vector of the pixel. ; Based on the reflectivity of pixels in the red light band and reflectivity in the near-infrared band Calculate the vegetation index of each pixel. .

4. The method for intelligent robot inspection path planning based on multispectral imaging according to claim 1, characterized in that, The step of calculating the anomaly score of a pixel based on its reflectance vector and vegetation index includes: The reflectance vector of the pixel Vegetation index of pixels The multispectral feature vectors of all pixels in the multispectral images collected during the inspection cycle are used to form the multispectral data of the inspection cycle. Using multispectral data from three historical inspection cycles prior to the current inspection cycle, the isolated forest baseline model in the isolated forest anomaly detection algorithm is trained. The input is the multispectral feature vector of each pixel, and the output is the anomaly score of each pixel. Using the obtained isolated forest baseline model, the anomaly score of each pixel in the multispectral data of the current inspection cycle is calculated.

5. The method for intelligent robot inspection path planning based on multispectral imaging according to claim 1, characterized in that, The step of calculating the rate of change of the abnormal area of ​​the region of interest based on the difference between the overlapping areas of the instantaneous abnormal point set of the current inspection cycle and multiple previous historical inspection cycles and the region of interest includes: ； In the formula, Indicates the first The rate of change of the abnormal area of ​​each region of interest; For a short period of time; This represents the set of instantaneous anomalies in the current inspection cycle; Indicates the first inspection cycle adjacent to the current inspection cycle. A set of instantaneous anomalies for each historical inspection cycle; Indicates the first A set consisting of all pixels in a region of interest; This indicates finding the intersection; This indicates the number of pixels in the set; This indicates the time interval between two inspection cycles.

6. The method for intelligent robot inspection path planning based on multispectral imaging according to claim 1, characterized in that, The step of calculating the anomaly accumulation frequency of the region of interest based on the area of ​​the overlapping region between the instantaneous anomaly point set of multiple historical inspection cycles prior to the current inspection cycle and the region of interest includes: ； In the formula, Indicates the first The cumulative frequency of anomalies in each region of interest; For long duration, the following is required ; Indicates the first inspection cycle adjacent to the current inspection cycle. A set of instantaneous anomalies for each historical inspection cycle; Indicates the first A set consisting of all pixels in a region of interest; This indicates finding the intersection; This indicates the number of pixels in the set; It is an indicator function; its value is 1 when the condition inside the parentheses is true, and 0 otherwise. This is the area threshold.

7. The method for intelligent robot inspection path planning based on multispectral imaging according to claim 1, characterized in that, The weighted summation of the abnormal area change rate and abnormal accumulation frequency of the region of interest to obtain the comprehensive inspection priority of the region of interest includes: ； In the formula, Indicates the first Comprehensive inspection priority for each area of ​​interest; Indicates the first The rate of change of the abnormal area of ​​each region of interest; Indicates the first The cumulative frequency of anomalies in each region of interest; Represents the normalization function; and It is a weighting coefficient, and , .

8. The method for planning inspection paths for intelligent robots based on multispectral imaging according to claim 2 or 7, characterized in that, The weighting coefficient and The setup methods include: The weighting coefficients are dynamically adjusted based on the growth stage. and : During dormancy, Set it to 0.3, Set to 0.7; In the budding stage, Set to 0.6, Set to 0.4; During the flowering period, Set to 0.8, Set to 0.2; During the results period, Set it to 0.5, Set to 0.5; During the maturity period, Set to 0.7, Set it to 0.

3.

9. The method for intelligent robot inspection path planning based on multispectral imaging according to claim 1, characterized in that, The method of solving using the ant colony algorithm to obtain a shortest path traversing all high-risk areas includes: Treating all high-risk areas as nodes, the probability of an ant choosing the next node is determined by the pheromone concentration between two nodes and heuristic information. Specifically, when solving the constructed Traveling Salesman Problem with priority constraints using the ant colony algorithm, the heuristic information between two nodes is equal to the overall inspection priority of the next node. Distance to two nodes reciprocal The product of; the path is selected using the roulette wheel selection method based on probability.

10. A multispectral-based intelligent robot inspection path planning system, characterized in that, include: A processor and a memory, wherein the memory stores computer program instructions that, when executed by the processor, implement the multispectral-based intelligent robot inspection path planning method according to any one of claims 1-9.

Images