Multi-spectral based intelligent robot inspection path planning method and system

Abstract

The application belongs to the technical field of machine intelligence, and particularly relates to a multi-spectrum-based intelligent robot inspection path planning method and system, which comprises the following steps: calculating an abnormal score according to a reflectivity vector and a vegetation index of a pixel point, forming an instantaneous abnormal point set of an inspection cycle by abnormal pixel points with an abnormal score greater than an abnormal threshold, clustering to obtain a plurality of interest regions, calculating an abnormal area change rate and an abnormal cumulative frequency of the interest regions according to the overlapping regions of the instantaneous abnormal point set and the interest regions in the current inspection cycle and a plurality of historical inspection cycles, obtaining a comprehensive inspection priority by weighted summation, constructing a traveling salesman problem with a priority constraint according to the interest regions in the current inspection cycle and the comprehensive inspection priority thereof, solving the problem by an ant colony algorithm, and obtaining a shortest path traversing all the interest regions as an optimal detailed inspection path of the current inspection cycle. The application ensures that high-risk regions are visited preferentially.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine intelligence. More particularly, the present application relates to a multi-spectrum-based intelligent robot inspection path planning method and system. BACKGROUND

[0002] As a high-value economic crop, the planting management of cranberry requires a high degree of refinement. Due to the high water content, thin skin, and perishable nature of cranberry fruits, as well as the special growth environment requiring acidic peat soil and regular flooding management, the planting process is extremely sensitive to disease and pest monitoring and environmental stress early warning.

[0003] Traditional manual inspection methods are not only inefficient, but also subjective and difficult to quantify. In particular, the ability to identify early-stage diseases such as flower rot and fruit rot is severely lacking. As the scale of cranberry cultivation continues to expand, this inefficient and subjective inspection method cannot meet the urgent needs of modern large-scale and standardized cultivation.

[0004] In modern precision agriculture management, inspection robots equipped with multi-spectral cameras are used to inspect large areas of cranberry cultivation. Multi-spectral imaging technology collects reflectance data of crops at blue, green, red, and near-infrared wavelengths, calculates key indicators such as vegetation index, and effectively identifies crop growth monitoring, disease and pest warning, and stress diagnosis. It can detect physiological changes in cranberry plants before symptoms appear, providing valuable time windows for timely intervention.

[0005] However, the core contradiction in existing technology is the irreconcilability between comprehensive inspection and efficient discovery. On the one hand, to ensure that no potential problem areas are missed, the inspection robot needs to perform a full coverage path, which results in a lot of time wasted in healthy areas. On the other hand, to improve the efficiency of abnormal discovery, focus should be on high-risk areas, but the lack of real-time judgment mechanism leads to significant time lag. SUMMARY

[0006] To solve the technical problem of the irreconcilability between comprehensive inspection and efficient discovery in the process of intelligent robot inspection, the present application provides solutions in the following aspects.

[0007] In a first aspect, the present application provides a multispectral-based intelligent robot inspection path planning method, comprising: in the early stage of the inspection cycle, the inspection robot collects multispectral images of the entire planting area according to a global coverage path, obtains a reflectance vector of a pixel point, and calculates a vegetation index of the pixel point; according to the reflectance vector and the vegetation index of the pixel point, the abnormal score of the pixel point is calculated, and the abnormal pixel points with an abnormal score greater than an abnormal threshold are combined to form a set of instantaneous abnormal points in the inspection cycle; the set of instantaneous abnormal points is clustered to obtain a plurality of interest areas in the inspection cycle; according to the difference between the overlapping areas of the set of instantaneous abnormal points and the interest areas in the current inspection cycle and a plurality of historical inspection cycles before the current inspection cycle, the abnormal area change rate of the interest area is calculated, and according to the area size of the overlapping areas of the set of instantaneous abnormal points and the interest areas in a plurality of historical inspection cycles before the current inspection cycle, the abnormal cumulative frequency of the interest area is calculated; the abnormal area change rate and the abnormal cumulative frequency of the interest area are weighted and summed to obtain a comprehensive inspection priority of the interest area; all interest areas in the current inspection cycle are sorted in descending order of the comprehensive inspection priority, and a plurality of high-risk areas are selected; according to all high-risk areas and their comprehensive inspection priorities, a traveling salesman problem with priority constraints is constructed, and a shortest path traversing all high-risk areas is obtained by solving the problem through an ant colony algorithm, serving as an optimal detailed inspection path in the current inspection cycle.

[0008] The present application comprehensively considers the abnormal area change rate and the abnormal cumulative frequency of the interest area analyzed from the historical data by introducing the time dimension, thereby calculating the comprehensive inspection priority of each interest area, constructing the traveling salesman problem with priority constraints based on the comprehensive inspection priority, so that the planned path not only pursues high efficiency in distance, but also ensures that the robot preferentially inspects the key high-risk areas that rapidly deteriorate or exist for a long time, thereby significantly improving the intelligence, pertinence and timeliness of agricultural intervention of the inspection, and avoiding economic losses caused by delaying the best treatment opportunity.

[0009] Preferably, the setting method of the time interval of the inspection cycle comprises that the growth cycle of the cranberry has obvious seasonal regularity: the dormant period is from December to February, the germination period is from March to May, the flowering period is from June to July, the fruiting period is from August to October, and the mature period is November; in different growth periods of the cranberry, the time interval between two inspection cycles is different: the time interval is set to 21 days in the dormant period, the time interval is set to 7 days in the germination period, the time interval is set to 3 days in the flowering period, the time interval is set to 5 days in the fruiting period, and the time interval is set to 2 days in the mature period.

[0010] Based on the physiological characteristics and risk sensitivity of cranberries at different growth stages, this invention dynamically and differentiates the time interval of the inspection cycle, which can ensure that problems are detected and dealt with in a timely manner during critical growth periods, while avoiding unnecessary frequent inspections during non-critical periods, effectively reducing the robot's energy consumption and equipment wear and tear.

[0011] Preferably, the acquisition of multispectral images of the entire planting area, obtaining the reflectance vector of each pixel, and calculating the vegetation index of each pixel includes: the multispectral camera providing four key bands, including: blue light band, green light band, red light band, and near-infrared band; obtaining the reflectance of each pixel in the multispectral image in the blue light band, green light band, red light band, and near-infrared band. , , and And form the reflectance vector of the pixel. Based on the reflectivity of pixels in the red light band and reflectivity in the near-infrared band Calculate the vegetation index of each pixel. .

[0012] Preferably, the step of calculating the anomaly score of a pixel based on its reflectance vector and vegetation index includes: converting the reflectance vector of the pixel... Vegetation index of pixels The multispectral feature vectors of all pixels in the multispectral images collected during the inspection cycle are used to form the multispectral data for the inspection cycle. The isolated forest benchmark model in the isolated forest anomaly detection algorithm is trained using the multispectral data from the three historical inspection cycles prior to the current inspection cycle. The input is the multispectral feature vector of the pixel, and the output is the anomaly score of the pixel. The anomaly score of each pixel in the multispectral data of the current inspection cycle is calculated using the obtained isolated forest benchmark model.

[0013] Preferably, the step of calculating the rate of change of the abnormal area of ​​the region of interest based on the difference between the overlapping areas of the instantaneous abnormal point set of the current inspection cycle and multiple previous historical inspection cycles and the region of interest includes: In the formula, Indicates the first The rate of change of the abnormal area of ​​each region of interest; For a short period of time; This represents the set of instantaneous anomalies in the current inspection cycle; Indicates the first inspection cycle adjacent to the current inspection cycle. A set of instantaneous anomalies for each historical inspection cycle; Indicates the first A set consisting of all pixels in a region of interest; This indicates finding the intersection; This indicates the number of pixels in the set; This indicates the time interval between two inspection cycles.

[0014] This invention quantifies the dynamic evolution trend of abnormal areas in the short term by calculating the rate of change of abnormal area. It can clearly identify areas that are rapidly expanding or gradually shrinking, providing a key basis for priority assessment of urgency, making inspection decisions more forward-looking, and enabling priority response to rapidly developing and most threatening problems.

[0015] Preferably, the step of calculating the anomaly accumulation frequency of the region of interest based on the area of ​​the overlapping region between the instantaneous anomaly point set of multiple historical inspection cycles prior to the current inspection cycle and the region of interest includes: In the formula, Indicates the first The cumulative frequency of anomalies in each region of interest; For long duration, the following is required ; Indicates the number of inspection cycles adjacent to the current one. A set of instantaneous anomalies for each historical inspection cycle; Indicates the first A set consisting of all pixels in a region of interest; This indicates finding the intersection; This indicates the number of pixels in the set; It is an indicator function; its value is 1 when the condition inside the parentheses is true, and 0 otherwise. This is the area threshold.

[0016] This invention effectively identifies areas where problems have repeatedly occurred in history by calculating the cumulative frequency of anomalies. It can reveal chronic stress caused by fundamental and long-term factors such as soil problems and irrigation system defects, making up for the shortcomings of focusing only on instantaneous states. This allows path planning to take into account deep-seated problems that do not change rapidly but have existed for a long time, thus improving the comprehensiveness and fundamental nature of problem diagnosis.

[0017] Preferably, the step of obtaining the comprehensive inspection priority of the region of interest by weighted summing of the abnormal area change rate and abnormal accumulation frequency includes: In the formula, Indicates the first Comprehensive inspection priority for each area of ​​interest; Indicates the first The rate of change of the abnormal area of ​​each region of interest; Indicates the first The cumulative frequency of anomalies in each region of interest; Represents the normalization function; and are weight coefficients, and , .

[0018] The application can integrate the dynamic change trend and the historical stubbornness of the region by weighting and summing the abnormal area change rate and the abnormal cumulative frequency, provide a comprehensive and quantitative risk assessment result for each interest region, and make the subsequent path planning decision basis more scientific, comprehensive and reliable.

[0019] Preferably, the weight coefficients and are set by a method comprising: dynamically adjusting the weight coefficients and based on the growth stage: in the dormant period, setting to 0.3 and setting to 0.7; in the germination period, setting to 0.6 and setting to 0.4; in the flowering period, setting to 0.8 and setting to 0.2; in the fruiting period, setting to 0.5 and setting to 0.5; and in the maturation period, setting to 0.7 and setting to 0.3.

[0020] Preferably, the shortest path traversing all high-risk regions is obtained by solving the traveling salesman problem with priority constraint by the ant colony algorithm, comprising: taking all high-risk regions as nodes, and the probability of the ant selecting the next node being determined by the pheromone concentration and the heuristic information between two nodes, wherein the heuristic information between two nodes is equal to the product of the reciprocal of the distance between the two nodes and the comprehensive inspection priority of the latter node ; and the path is selected by using the roulette wheel selection method according to the probability.

[0021] The application improves the heuristic information by the comprehensive inspection priority, so that the ant is attracted by both the high priority and the short distance when selecting the next node, and the high-risk region is ensured to be visited preferentially.

[0022] In the second aspect, the application provides a multi-spectrum-based intelligent robot inspection path planning system, comprising a processor and a memory, and the memory stores computer program instructions, which realize the multi-spectrum-based intelligent robot inspection path planning method when executed by the processor.​

[0023] By adopting the technical scheme, the multi-spectrum-based intelligent robot inspection path planning method is generated into a computer program and stored in a memory to be loaded and executed by a processor, so that a terminal device is manufactured according to the memory and the processor, and use is facilitated.

[0024] The present application has the advantages that:

[0025] Compared with the method of only taking the shortest physical path as the target in the prior art, the present application introduces the time dimension, comprehensively considers the abnormal area change rate and abnormal cumulative frequency of the interest region analyzed from the historical data, calculates the comprehensive inspection priority of each interest region, constructs the traveling salesman problem with priority constraint based on the comprehensive inspection priority, makes the planned path not only pursue the efficiency in distance, but also ensure that the robot preferentially checks the key high-risk region which rapidly deteriorates or long exists, thereby significantly improving the intelligence, pertinence and timeliness of agricultural intervention of the inspection, and avoiding the economic loss caused by delaying the best processing opportunity. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 is a flowchart schematically showing the multi-spectrum-based intelligent robot inspection path planning method in the present application;

[0027] Figure 2 is a flowchart schematically showing step S3. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0029] The specific embodiments of the present application will be described in detail below with reference to the drawings.

[0030] The embodiments of the present application disclose a multi-spectrum-based intelligent robot inspection path planning method, referring to Figure 1 , including steps S1, S4:

[0031] S1: In the early stage of the inspection cycle, the inspection robot collects multi-spectrum images of the entire planting area according to the global coverage path, obtains the reflectance vector of the pixel points, and calculates the vegetation index of the pixel points.

[0032] It should be noted that multispectral data is the foundation of anomaly detection. Therefore, at the beginning of each inspection cycle, multispectral images of the entire planting area are collected globally.

[0033] Specifically, the growth cycle of cranberries exhibits a distinct seasonal pattern: December to February is the dormancy period, March to May is the budding period, June to July is the flowering period, August to October is the fruiting period, and November is the ripening period. The time interval between two inspection cycles varies depending on the different growth stages of the cranberry plant. Specifically: during dormancy, the plant's metabolic activity is at its lowest, and the above-ground parts cease growth; therefore, the time interval is set at 21 days. During budding, buds swell, new shoots begin to grow, and the plant is extremely sensitive to environmental changes; therefore, the time interval is set at 7 days. During flowering, bud formation, flowering, and pollination directly affect the fruit set rate; therefore, the time interval is set at 3 days. During fruiting, fruit development and enlargement are critical, and sugar accumulation is at its peak; therefore, the time interval is set at 5 days. During ripening, the fruit matures, the sugar-acid ratio changes, and harvesting is in preparation; therefore, the time interval is set at 2 days.

[0034] Furthermore, at the beginning of each inspection cycle, the inspection robot travels in the cranberry planting area according to a global coverage path and uses its onboard multispectral camera to collect multispectral images of the entire planting area. The multispectral camera provides four key bands, including: blue light band (corresponding to 450–520nm band), green light band (corresponding to 520–600nm band), red light band (corresponding to 630–690nm band), and near-infrared band (corresponding to 760–900nm band).

[0035] The global coverage path is a preset bow-shaped pattern or parallel line pattern. In order to improve the basic scanning efficiency, the inspection robot travels at a relatively fast speed, usually set to 0.8 m / s. The multispectral camera acquires multispectral images at a frequency of one frame every 10 seconds. The resolution of the multispectral camera is the standard resolution mode, i.e., 5 million pixels.

[0036] For each newly acquired multispectral image: First, georegistration is performed using GPS and IMU (Inertial Measurement Unit) data, and precise spatial alignment is achieved with historical multispectral images stored in the database to ensure accurate correspondence of pixels at the same geographical location across all multispectral images. Then, the calibrated multispectral images undergo preprocessing, including radiometric calibration and atmospheric correction, to eliminate the influence of environmental factors such as illumination variations on the multispectral images. Finally, the reflectance of each pixel in the multispectral image is obtained in the blue, green, red, and near-infrared bands. , , and And form the reflectance vector of the pixel. .

[0037] In addition, according to the reflectivity of the pixel point in the red light band and the reflectivity of the pixel point in the near-infrared band , the vegetation index of the pixel point is calculated , and the obtained vegetation index has a value range of [-1, 1], and specifically, if , .

[0038] S2: According to the reflectivity vector of the pixel point and the vegetation index, the abnormal score of the pixel point is calculated, and the abnormal pixel points with an abnormal score greater than an abnormal threshold value are combined to form a transient abnormal point set of the inspection period; the transient abnormal point set is clustered to obtain a plurality of interest regions of the inspection period.

[0039] It should be noted that through the abnormal score calculation and clustering, the high-dimensional spectral anomaly is converted into an operable spatial target, providing accurate targets for path planning.

[0040] Specifically, the reflectivity vector of the pixel point and the vegetation index of the pixel point are combined to form a multispectral feature vector of the pixel point; and the multispectral feature vectors of all pixel points in the multispectral image collected in the inspection period are combined to form multispectral data of the inspection period.

[0041] Further, from the historical database, the multispectral data of the three historical inspection periods before the current inspection period is loaded for training an isolated forest benchmark model in the isolated forest anomaly detection algorithm, the input of the isolated forest benchmark model being the multispectral feature vector of the pixel point and the output being the abnormal score of the pixel point; the isolated forest benchmark model integrates multispectral data and vegetation index, and can improve the robustness of anomaly detection.

[0042] Finally, through the obtained isolated forest benchmark model, the abnormal score of each pixel point in the multispectral data of the current inspection period is calculated; an abnormal threshold value is set, and the pixel points with an abnormal score greater than the abnormal threshold value are recorded as abnormal pixel points to form a transient abnormal point set of the current inspection period .

[0043] Among them, the abnormal score of the pixel point calculated by the isolated forest benchmark model is between 0 and 1, and the abnormal threshold value is used to screen out abnormal pixel points with a larger abnormal score from all pixel points, so the abnormal threshold value is set to 0.7.

[0044] It should be noted that the isolated forest anomaly detection algorithm is used to determine the abnormal score of the data point, and the greater the abnormal score, the more abnormal the data point; the isolated forest anomaly detection algorithm is a known technology, and will not be described here.

[0045] Further need to be explained is that, in order to avoid the planned robot inspection path being fragmented by noise points, it is necessary to organize the discrete abnormal pixel points output by the Isolation Forest anomaly detection algorithm into spatially coherent interest regions.

[0046] Therefore, by the DBSCAN clustering algorithm, the abnormal pixel points in the instantaneous abnormal point set of the current inspection period are clustered, and each class cluster obtained is taken as an interest region of the current inspection period.

[0047] Among them, the parameters in the DBSCAN clustering algorithm, i.e. the neighborhood radius eps and the minimum number of points min_samples, need to be adapted to the planting characteristics of cranberries, therefore, the setting method of the parameters in the DBSCAN clustering algorithm is:

[0048] (1) According to experience, the row spacing of cranberry plants is usually 0.5 to 1 meters, in order to ensure that the pixel points of the same disease area are clustered, when setting the neighborhood radius eps of the DBSCAN clustering algorithm, the middle value of the plant row spacing, i.e. 0.75 meters, is set. The empirical value of 0.75 of the plant row spacing is converted into the neighborhood radius eps in the image space through the ground resolution, then the neighborhood radius eps is equal to , is the ground resolution;

[0049] (2) According to experience, the area of a single cranberry plant is 0.5 square meters, i.e. the empirical value of the area of a single cranberry plant is equal to 0.5; in order to avoid the influence of noise, when setting the minimum number of points min_samples of the DBSCAN clustering algorithm, the empirical value of 0.5 of the single plant area is set. The empirical value of 0.5 of the single plant area is converted into the minimum number of points min_samples in the image space through the ground resolution, then the minimum number of points min_samples is equal to , is the ground resolution.

[0050] Among them, the ground resolution is equal to the ratio of the actual distance in the geographic space to the number of pixel points in the image space.

[0051] S3: According to the difference between the overlapping areas of the instantaneous abnormal point set and the interest region of the current inspection period and the multiple historical inspection periods before the current inspection period, calculate the abnormal area change rate of the interest region, according to the area size of the overlapping areas of the instantaneous abnormal point set and the interest region of the multiple historical inspection periods before the current inspection period, calculate the abnormal cumulative frequency of the interest region; the weighted sum of the abnormal area change rate and the abnormal cumulative frequency of the interest region is obtained. The comprehensive inspection priority of the interest region.

[0052] The flow chart of step S3 is referred to​Figure 2 This includes steps S301 to S303, specifically:

[0053] S301. Calculate the rate of change of the abnormal area of ​​the region of interest based on the difference between the instantaneous abnormal point set of the current inspection cycle and the overlapping area of ​​the region of interest in the previous multiple historical inspection cycles.

[0054] It should be noted that a rapidly expanding area usually indicates that the disease or stress is intensifying, and its risk level and the urgency of inspection are higher. Therefore, by retrieving multiple historical inspection cycles from the historical database and identifying the set of pixels identified as anomalous in the region of interest in each historical inspection cycle, and subtracting it from the set of instantaneous anomalous pixels in the region of interest in the current inspection cycle, the rate of change of the anomalous area of ​​the region of interest can be constructed. This indicator can capture the expansion or contraction trend of the region of interest in the recent time window.

[0055] Specifically, for each region of interest in the current inspection cycle: based on the difference between the overlapping areas of the instantaneous anomaly point sets in the current inspection cycle and multiple previous historical inspection cycles and the regions of interest, the rate of change of the anomaly area of ​​the region of interest is calculated. The specific calculation formula is as follows:

[0056] ;

[0057] In the formula, Indicates the first The rate of change of the abnormal area of ​​each region of interest; For short-term durations, it is used to obtain the rate of change of anomalous areas that characterizes short-term dynamic changes; therefore, Set to 3; This represents the set of instantaneous anomalies in the current inspection cycle; Indicates the first inspection cycle adjacent to the current inspection cycle. A set of instantaneous anomalies for each historical inspection cycle; Indicates the first A set consisting of all pixels in a region of interest; This indicates finding the intersection; This indicates the number of pixels in the set; This indicates the time interval between two inspection cycles.

[0058] in, This represents the set of instantaneous anomalies during the current inspection cycle. With the A set consisting of all pixels in a region of interest The intersection represents the points that fall within the current inspection cycle. The total area covered by abnormal pixels within the corresponding geographical range; Indicates the first inspection cycle adjacent to the current inspection cycle. Set of instantaneous anomalies in a historical inspection cycle With the A set consisting of all pixels in a region of interest The intersection of these points represents the first point adjacent to the current inspection cycle. During each historical inspection cycle, it fell into The total area covered by abnormal pixels within the corresponding geographical range.

[0059] The dynamic trend of the region of interest is characterized by calculating the net change in the area of ​​the abnormal region per unit time: when When the value is positive and large, it indicates that the region of interest is expanding rapidly, and the abnormal situation in the corresponding scene may be deteriorating rapidly; when... When the value is negative, it indicates that the region of interest is shrinking, and the anomaly may be improving or under control; when... When the value is close to zero, it indicates that the region of interest is in a stable state.

[0060] S302. Calculate the cumulative frequency of anomalies in the region of interest based on the area of ​​the overlapping region between the instantaneous anomaly point set of multiple historical inspection cycles prior to the current inspection cycle and the region of interest.

[0061] It should be noted that a newly emerging anomalous area may be merely an isolated phenomenon, while a recurring anomalous area indicates a deeper and more persistent source of risk. Therefore, after analyzing recent changes in the region of interest, it is necessary to further consider its persistence, i.e., whether the region of interest is an area with persistent and intractable problems. Thus, by statistically analyzing the percentage of times a specific geographical location corresponding to a region of interest was identified as significantly abnormal within multiple historical inspection cycles retrieved from historical databases, an anomalous cumulative frequency of the region of interest is constructed. This indicator aims to identify regions of interest that have historically experienced recurring problems. These regions typically represent chronic stress caused by fundamental factors such as soil issues and irrigation deficiencies, and also require close attention.

[0062] Specifically, for each region of interest in the current inspection cycle: the cumulative frequency of anomalies in the region of interest is calculated based on the area of ​​the overlapping region between the instantaneous anomaly point set of multiple historical inspection cycles prior to the current inspection cycle and the region of interest. The specific calculation formula is as follows:

[0063] ;

[0064] In the formula, Indicates the first The cumulative frequency of anomalies in each region of interest; The duration is long-term, used to obtain the cumulative frequency of anomalies characterizing long-term historical patterns; therefore, it is required that... ,Will is set to 7; represents the set of instantaneous abnormal points in the historical inspection period adjacent to the current inspection period; represents the set of all pixel points in the i-th interest region; represents the intersection; represents the number of pixel points in the set, is used to determine whether there is an abnormal region with an area greater than the area threshold in the i-th historical inspection period, is the indicator function, whose value is 1 when the condition in the parentheses is true, otherwise 0; is the area threshold value, used to filter out small and negligible noise. According to experience, the empirical value of the area of a single cranberry plant is equal to 0.5. By converting the empirical value of the area of a single cranberry plant into an area threshold value in image space through the ground resolution, the area threshold value is equal to

[0065] , is the ground resolution. wherein the value range of the abnormal accumulation frequency of the interest region is [0, 1]: the abnormal accumulation frequency

[0066] close to 1 indicates that the interest region is a hot spot area with repeated problems, which corresponds to the stubbornness and persistence of the problem in the scene; the abnormal accumulation frequency close to 0 indicates that the interest region rarely or never has problems in history. S303, the abnormal area change rate and the abnormal accumulation frequency of the interest region are weighted and summed to obtain the comprehensive inspection priority of the interest region.

[0067] It should be noted that the dynamic evolution feature obtained by the comprehensive steps S301 and S302, i.e., the abnormal area change rate and the persistence feature, i.e., the abnormal accumulation frequency, generate a comprehensive inspection priority for each interest region generated by clustering of abnormal pixel points, thereby guiding the path planning in subsequent inspection.

[0068] Specifically, for each interest region of the current inspection period: the abnormal area change rate and the abnormal accumulation frequency of the interest region are weighted and summed to obtain the comprehensive inspection priority of the interest region, and the specific calculation formula is as follows:

[0069]

[0070] ;

[0071] In the formula, represents the i-th interest region​​​​​​ a comprehensive inspection priority of the interest region; an abnormal area change rate representing the an abnormal cumulative frequency representing the a normalization function, including but not limited to a Min-Max normalization function, mapping the abnormal area change rate to the interval [0, 1] and unifying the dimension of the abnormal cumulative frequency, so as to be weighted summed with the abnormal cumulative frequency; and are weight coefficients, and , for balancing the proportion of urgency and importance in the final decision.

[0072] For the weight coefficients and , they can be dynamically adjusted based on the growth stage: in the high-incidence period of cranberry diseases, such as the flowering period, the value of is appropriately increased to prioritize the processing of problem areas that spread rapidly (such as flower rot and fruit rot), and in the stable growth period of cranberry, such as the dormancy period, the value of is appropriately reduced to prioritize the processing of areas with chronic problems that exist for a long time (such as soil compaction and uneven irrigation), so that in the dormancy period, is set to 0.3, and is set to 0.7, in the germination period, is set to 0.6, and is set to 0.4, in the flowering period, is set to 0.8, and is set to 0.2, in the fruiting period, is set to 0.5, and is set to 0.5, in the maturation period, is set to 0.7, and is set to 0.3.

[0073] wherein the abnormal area change rate representing short-term dynamic changes and representing long-term historical patterns are fused into a unified priority score by weighted summation: The higher the value of the interest region, the more likely it is a rapidly deteriorating problem point or a long-standing stubborn problem point, or both, so it should be given the highest priority in subsequent inspections.

[0074] ​​S4: According to the order of the comprehensive inspection priority from large to small, sort all the interest areas in the current inspection period, and select a plurality of high-risk areas; according to all the high-risk areas and their comprehensive inspection priorities, construct a traveling salesman problem with priority constraints, solve it by using an ant colony algorithm, and obtain a shortest path traversing all the high-risk areas as the optimal detailed inspection path of the current inspection period.

[0075] It should be noted that the comprehensive inspection priority is integrated into the path planning to realize intelligent resource allocation.

[0076] Specifically, according to the order of the comprehensive inspection priority from large to small, sort all the interest areas in the current inspection period, and select a plurality of high-risk areas; according to all the high-risk areas and their comprehensive inspection priorities, construct a traveling salesman problem with priority constraints, solve it by using an ant colony algorithm, and obtain a shortest path traversing all the high-risk areas as the optimal detailed inspection path of the current inspection period. The first interest area is a high-risk area, and the number of pixel points in the first interest area is less than or equal to 20% of the number of pixel points in the multispectral image of the entire planting area. The number of pixel points in the first interest area is greater than 20% of the number of pixel points in the multispectral image of the entire planting area.

[0077] For the high-risk areas in the current inspection period: the position of the centroid of each high-risk area is taken as its position information; all the high-risk areas are taken as nodes, and a traveling salesman problem with priority constraints is constructed by combining the comprehensive inspection priorities of the high-risk areas: the goal is to start from the position of the inspection robot, traverse all the high-risk areas, and have the shortest total path and preferentially access high-risk areas with high comprehensive inspection priorities.

[0078] Further, the traveling salesman problem with priority constraints constructed by using the ant colony algorithm is solved to obtain a shortest path traversing all the high-risk areas as the optimal detailed inspection path of the current inspection period. In this process, the path is selected by using a roulette wheel selection method according to a probability; and the probability of an ant selecting a next node is determined by the pheromone concentration and heuristic information between two nodes: when a conventional traveling salesman problem is solved by using the ant colony algorithm, the heuristic information between two nodes is equal to the reciprocal of the distance between the two nodes; when the traveling salesman problem with priority constraints is constructed and solved by using the ant colony algorithm, the heuristic information between two nodes is equal to the product of the comprehensive inspection priority of the latter node and the reciprocal of the distance between the two nodes.

[0079] ​​​​​​Wherein, in the calculation of the probability of selecting the path, the inspection priority and the distance are combined, the high-risk area with higher inspection priority and closer distance will obtain greater selection probability, and the optimal detailed inspection path planned in this way can realize intelligent inspection with high-risk area priority.

[0080] Wherein, when the inspection robot inspects according to the optimal detailed inspection path: in the non-high-risk area, the travel speed is faster, usually set to 0.8 m / s, and in the high-risk area, the travel speed is slower, usually set to 0.3 m / s; the frequency of the multispectral camera collecting multispectral images is one frame every 3 seconds, and the resolution of the multispectral camera is switched to the high-resolution mode, i.e. 12 million pixels; according to the reflectivity of the pixel point in the red light waveband and the reflectivity of the pixel point in the near-infrared waveband in the high-resolution multispectral image collected, the vegetation index of the pixel point is calculated .

[0081] The average value of the vegetation index of all pixel points in the high-risk area is taken as the vegetation index of the high-risk area, which is used for growth state monitoring of the cranberry in the geographical space corresponding to the high-risk area:

[0082] (1) In the dormant period, if the vegetation index of the high-risk area is less than 0.12, it means that there is a serious frost damage risk, and if the vegetation index of the high-risk area is greater than 0.30, it means that abnormal sprouting has occurred, which may lead to a decrease in cold resistance due to a warm winter.

[0083] (2) In the germination period, if the difference between the vegetation indexes of the high-risk area in two adjacent inspection cycles is less than 0.02, it means that there is a problem of nutrient deficiency or root system.

[0084] (3) In the flowering period, if the vegetation index of the high-risk area is less than 0.45, it means that the flower organ development is poor, and there is a risk of low fruit setting rate, and if the vegetation index of the high-risk area is greater than 0.65, it means that there is nutrient excess, which may lead to flower drop.

[0085] (4) In the fruiting period, if the vegetation index of the high-risk area is less than 0.48, it means that there is a problem of nutrient deficiency or early fruit rot disease, and if the vegetation index of the high-risk area is greater than 0.72, it means that there is a problem of poor fruit color due to excessive nitrogen.

[0086] (5) In the mature period, if the vegetation index of the high-risk area is greater than 0.55, it means that the maturation is delayed, which may affect the harvesting plan.

[0087] The embodiment of the present application also discloses a multi-spectrum-based intelligent robot inspection path planning system, comprising a processor and a memory, and the memory stores computer program instructions.

[0088] The system also comprises other components such as a communication bus and a communication interface which are well known to those skilled in the art, and the setting and functions thereof are known in the art, thus no further description is given herein.

Claims

1. A method for intelligent robot inspection path planning based on multispectral imaging, characterized in that, include: At the beginning of the inspection cycle, the inspection robot collects multispectral images of the entire planting area according to a global coverage path, obtains the reflectance vector of each pixel, and calculates the vegetation index of each pixel. Based on the reflectance vector and vegetation index of each pixel, the anomaly score of each pixel is calculated. Pixels with anomaly scores greater than the anomaly threshold are grouped into an instantaneous anomaly set for the inspection cycle. Clustering of instantaneous anomalies yields multiple regions of interest for the inspection cycle; Based on the differences between the overlapping areas of the instantaneous anomaly point sets and the region of interest in the current inspection cycle and several previous historical inspection cycles, the rate of change of the anomaly area of ​​the region of interest is calculated. Based on the area size of the overlapping areas of the instantaneous anomaly point sets and the region of interest in the several previous historical inspection cycles, the cumulative frequency of anomalies in the region of interest is calculated. The comprehensive inspection priority of the region of interest is obtained by weighted summation of the rate of change of the anomaly area and the cumulative frequency of anomalies. Sort all regions of interest in the current inspection cycle according to the comprehensive inspection priority from highest to lowest, and select several high-risk regions. Based on all high-risk regions and their comprehensive inspection priorities, construct a traveling salesman problem with priority constraints, solve it using the ant colony algorithm, and obtain the shortest path to traverse all high-risk regions, which serves as the optimal detailed inspection path for the current inspection cycle.

2. The method for planning inspection paths for intelligent robots based on multispectral imaging according to claim 1, characterized in that, The method for setting the time interval of the inspection cycle includes: Cranberries have a distinct seasonal growth cycle: December to February is the dormant period, March to May is the budding period, June to July is the flowering period, August to October is the fruiting period, and November is the ripening period. The time interval between two inspection cycles varies depending on the different growth stages of cranberries: during dormancy, the interval is set to 21 days; during budding, the interval is set to 7 days; during flowering, the interval is set to 3 days; during fruiting, the interval is set to 5 days; and during ripening, the interval is set to 2 days.

3. The method for planning inspection paths for intelligent robots based on multispectral imaging according to claim 1, characterized in that, The process of acquiring multispectral images of the entire planting area, obtaining the reflectance vectors of pixels, and calculating the vegetation index of pixels includes: The multispectral camera provides four key bands: blue, green, red, and near-infrared. It obtains the reflectance of each pixel in the multispectral image across these bands. , , and And form the reflectance vector of the pixel. ; Based on the reflectivity of pixels in the red light band and reflectivity in the near-infrared band Calculate the vegetation index of each pixel. .

4. The method for intelligent robot inspection path planning based on multispectral imaging according to claim 1, characterized in that, The step of calculating the anomaly score of a pixel based on its reflectance vector and vegetation index includes: The reflectance vector of the pixel Vegetation index of pixels The multispectral feature vectors of all pixels in the multispectral images collected during the inspection cycle are used to form the multispectral data of the inspection cycle. Using multispectral data from three historical inspection cycles prior to the current inspection cycle, the isolated forest baseline model in the isolated forest anomaly detection algorithm is trained. The input is the multispectral feature vector of each pixel, and the output is the anomaly score of each pixel. Using the obtained isolated forest baseline model, the anomaly score of each pixel in the multispectral data of the current inspection cycle is calculated.

5. The method for intelligent robot inspection path planning based on multispectral imaging according to claim 1, characterized in that, The step of calculating the rate of change of the abnormal area of ​​the region of interest based on the difference between the overlapping areas of the instantaneous abnormal point set of the current inspection cycle and multiple previous historical inspection cycles and the region of interest includes: ； In the formula, Indicates the first The rate of change of the abnormal area of ​​each region of interest; For a short period of time; This represents the set of instantaneous anomalies in the current inspection cycle; Indicates the first inspection cycle adjacent to the current inspection cycle. A set of instantaneous anomalies for each historical inspection cycle; Indicates the first A set consisting of all pixels in a region of interest; This indicates finding the intersection; This indicates the number of pixels in the set; This indicates the time interval between two inspection cycles.

6. The method for intelligent robot inspection path planning based on multispectral imaging according to claim 1, characterized in that, The step of calculating the anomaly accumulation frequency of the region of interest based on the area of ​​the overlapping region between the instantaneous anomaly point set of multiple historical inspection cycles prior to the current inspection cycle and the region of interest includes: ； In the formula, Indicates the first The cumulative frequency of anomalies in each region of interest; For long duration, the following is required ; Indicates the first inspection cycle adjacent to the current inspection cycle. A set of instantaneous anomalies for each historical inspection cycle; Indicates the first A set consisting of all pixels in a region of interest; This indicates finding the intersection; This indicates the number of pixels in the set; It is an indicator function; its value is 1 when the condition inside the parentheses is true, and 0 otherwise. This is the area threshold.

7. The method for intelligent robot inspection path planning based on multispectral imaging according to claim 1, characterized in that, The weighted summation of the abnormal area change rate and abnormal accumulation frequency of the region of interest to obtain the comprehensive inspection priority of the region of interest includes: ； In the formula, Indicates the first Comprehensive inspection priority for each area of ​​interest; Indicates the first The rate of change of the abnormal area of ​​each region of interest; Indicates the first The cumulative frequency of anomalies in each region of interest; Represents the normalization function; and It is a weighting coefficient, and , .

8. The method for planning inspection paths for intelligent robots based on multispectral imaging according to claim 7, characterized in that, The weighting coefficient and The setup methods include: The weighting coefficients are dynamically adjusted based on the growth stage. and : During dormancy, Set it to 0.3, Set to 0.7; In the budding stage, Set to 0.6, Set to 0.4; During the flowering period, Set to 0.8, Set to 0.2; During the results period, Set it to 0.5, Set to 0.5; During the maturity period, Set to 0.7, Set it to 0.

3.

9. The method for intelligent robot inspection path planning based on multispectral imaging according to claim 1, characterized in that, The method of solving using the ant colony algorithm to obtain a shortest path traversing all high-risk areas includes: Treating all high-risk areas as nodes, the probability of an ant choosing the next node is determined by the pheromone concentration between two nodes and heuristic information. Specifically, when solving the constructed Traveling Salesman Problem with priority constraints using the ant colony algorithm, the heuristic information between two nodes is equal to the overall inspection priority of the next node. Distance to two nodes reciprocal The product of; the path is selected using the roulette wheel selection method based on probability.

10. A multispectral-based intelligent robot inspection path planning system, characterized in that, include: A processor and a memory, wherein the memory stores computer program instructions that, when executed by the processor, implement the multispectral-based intelligent robot inspection path planning method according to any one of claims 1-9.

Images