Large-scale crop maturity detection method suitable for edge equipment

Abstract

The invention relates to a large-scale crop maturity detection method suitable for edge equipment, and the method comprises the following steps: S1, obtaining a crop image, and outputting a corrected real threshold value; s2, on the basis of the corrected real threshold value, calculating the proportion of pixels, meeting the real threshold value, of all position id parts in the crop image to the total pixels of the image, and judging the crop state of the position id; and S3, after judging that the crop at the position id is immature, delaying for a period of time, continuing to execute S1 and S2 in the period of time, if the crop at the position id in the period of time is not changed into mature, outputting a crop immature signal, and otherwise, outputting a mature signal. Compared with the prior art, the method has the advantages of reducing misjudgment of crop maturity caused by weather and illumination changes, improving judgment accuracy and the like.

Description

Technical Field

[0001] The present invention relates to the fields of smart agriculture and machine vision, and in particular to a large-scale crop maturity detection method suitable for edge devices. Background Art

[0002] With the rapid development of artificial intelligence technology, its technical applications have been widely used in industry production. "Smart agriculture" is an important development direction in the agricultural field in recent years. It is an agricultural form formed by the deep integration of modern information technology and agricultural production and management. It realizes intelligent management of the entire agricultural production process by integrating modern information technologies such as the Internet of Things, big data, and cloud computing. The core of smart agriculture lies in dataization, which provides a scientific basis for agricultural production through real-time monitoring and data analysis of various indicators of the farmland environment. It uses various sensors, satellite remote sensing and other equipment to collect a large amount of data, and then analyzes and processes the data through artificial intelligence and big data technology to generate suggestions and decisions that are instructive for agricultural production.

[0003] In the context of smart agriculture, the application of agricultural Internet of Things technology is promoting the transformation of traditional agriculture to modernization. Agricultural Internet of Things technology uses intelligent sensing technology, information transmission technology and intelligent processing technology to monitor and remotely control all aspects of agricultural activities in real time, promote the intelligent informatization of agricultural production, business management and strategic decision-making, and realize the efficiency, intensification, scale and standardization of agricultural production. Especially in the monitoring of forest cash crops, by deploying high-precision sensors and cameras, various production environment data of crops can be monitored in real time. Farmers can use mobile phones or computers to accurately control water, fertilizers and medicines in the planting process, effectively reducing production costs and reducing soil erosion. The application of this technology not only improves the efficiency and quality of agricultural production, but also provides strong support for the sustainable development of agriculture.

[0004] The areas involved in the "forest economy" or other large-scale agricultural and forestry environments may be in mountainous and hilly areas, where there is no mobile network signal and no power facility coverage. The current mainstream algorithm solution for maturity detection is individual detection based on deep learning detection algorithms such as YOLO and SSD. It has high requirements for hardware equipment and consumes a lot of energy. The detection algorithm for edge devices needs to meet the hardware requirements of the device and be able to detect large areas in long-term standby conditions. It is summarized that the current algorithm or technical architecture has the following bottlenecks in implementation:

[0005] Current detection technology is targeted at the individual crop level. Large-scale or large-area detection requires the deployment of more imaging equipment to complete, and due to changes in weather and light, it is easy to misjudge the maturity of crops. Summary of the invention

[0006] The purpose of the present invention is to reduce the misjudgment of crop maturity caused by changes in weather and light and to improve the accuracy of judgment, thereby providing a large-scale crop maturity detection method suitable for edge devices.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] A large-scale crop maturity detection method suitable for edge devices, the method comprising the following steps:

[0009] S1, obtaining a crop image, wherein a position ID is set on the image, each image corresponds to a moment, deduplication is performed on the image and the ID number of the image is recorded, the deduplication image is input into a color threshold detector, and the color threshold detector outputs a corrected real threshold;

[0010] S2. Based on the corrected real threshold, calculate the ratio of pixels at each position id in the crop image that meet the real threshold to the total pixels of the image. If the ratio is greater than the ratio threshold, the position id meets the crop maturity condition at this moment, and set the mark X of the position id at this moment. t =1, otherwise set the mark X of the position id at that moment t = 0, repeat the above steps multiple times to obtain the labels of multiple position IDs corresponding to multiple moments, and determine whether there are at least m enter The moment meets the crop maturity condition, m enter is the number threshold, if yes, the crop state of the position id is determined to be mature, otherwise the crop state of the position id is determined to be immature;

[0011] S3, after determining that the crop at the position id is mature, delay for a period of time, and continue to execute S1 and S2 during the period of time. If the crop state at the position id does not change to immature during the period of time, output a crop maturity signal, otherwise output an immature signal;

[0012] After determining that the crop at the position id is immature, a delay is made, and S1 and S2 are continued during the time period. If the crop state at the position id does not change to mature during the time period, an immature crop signal is output, otherwise a mature crop signal is output;

[0013] In the above steps, the ID number and time are aligned regularly.

[0014] Furthermore, the crop maturity condition is:

[0015]

[0016] Among them, X t Indicates the mark, m enter Indicates the number of times threshold.

[0017] Furthermore, the corrected true threshold is:

[0018] thresh=f′(Le)=[[H l , S l , V l ],[H h , S h , V h ]]

[0019] Among them, f′ represents the mapping relationship, Le represents the ambient light, H l , H h Indicates the maximum and minimum values ​​of the H channel, S 1 , S h Indicates the maximum and minimum values ​​of the S channel, V l 、V h Indicates the maximum value of V channel and the minimum value of V channel;

[0020] The true threshold is satisfied when, for a crop image, the H component, the S component, and the V component of its pixel are all within the corresponding channel maximum and minimum values, then the true threshold is satisfied.

[0021] Furthermore, the specific steps for deduplication of images are as follows:

[0022] If the moving range is greater than the range threshold, then for the two crop images, find the matching feature point pairs (P 1i , P 2i ) where P 1i is the matching feature point of the first crop image, P 2i are the matching feature points of the second crop image;

[0023] Compute the homography matrix;

[0024] Based on the homography matrix, the bounding box of the first crop image is transformed into the coordinate system of the second crop image, and a mapping B1′ of the shooting range bounding box of the first crop image in the coordinate system of the second crop image is obtained, that is, the part of the second crop image that contains the shooting content of the first crop image;

[0025] Calculate the overlapping area based on the mapping B1′, remove the overlapping area for the second crop image, and correct the camera shooting frequency according to the overlapping area;

[0026] After removing the overlapping area, calculate the angle occupied by the farmland that was not photographed between the two images, determine the missing part of the picture based on the angle occupied by the farmland, cut out a picture at a certain angle in the left and right adjacent images of the missing part of the picture, calculate the ratio of pixels of mature crops in the cut-out picture to the total pixels of the picture, take the average as the approximate ratio of pixels of mature crops in the missing part to the total pixels of the picture, and the approximate value is used as a part of the ratio of pixels that meet the real threshold at each position id to the total pixels of the picture.

[0027] Furthermore, the response matrix is:

[0028]

[0029] Among them, i represents the matching feature point number, x 2y2 1 represents the homogeneous coordinates of the feature points in the second crop image, x 1 y 1 1 represents the homogeneous coordinates of the feature points in the first crop image.

[0030] Furthermore, the mapping B1′ is:

[0031] B1′(x, y)={H·[x 1 y 1 1] T for(x 1 ,y 1 )∈B1}

[0032] Among them, B1 represents the bounding box of the first crop image in its own coordinate system.

[0033] Furthermore, the overlapping area is:

[0034] R=B1′∩B2

[0035] Among them, R represents the overlapping area, and B2 represents the bounding box of the second crop image in its own coordinate system.

[0036] Furthermore, the corrected camera shooting frequency is:

[0037] f = argmin(size(R)|size(R)>0)

[0038] Wherein, f represents the corrected camera shooting frequency, and size(R) represents the size of the overlapping area R.

[0039] Furthermore, the angle of the unphotographed farmland between the two images is:

[0040] α gap =ωt interval -HFOV

[0041] Among them, αgap is the angle of the unphotographed farmland between the two images, ω is the camera rotation angular velocity, t interval is the shooting interval, and HFOV is the horizontal field of view.

[0042] Furthermore, the approximate value of the ratio of the pixels of the missing part of the mature crops to the total pixels of the picture is:

[0043] p gap =mean(p β_0left , p β_0right )

[0044] Among them, p gap is an approximate value, p β_0left It indicates the ratio of the pixels of the mature crops in the cut left adjacent image to the total pixels of the image, p β_0right It shows the ratio of the pixels of the mature crops in the cut-out right adjacent image to the total pixels of the image.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] The present invention corrects the threshold value based on the illumination and designs the following judgment rules: if the area that meets the color judgment threshold value is greater than the set value for a certain number of days or a certain period of time, the area is judged to be mature and can be picked. At the same time, in order to eliminate the threshold judgment caused by special weather, we adopt the method of delayed notification and reconfirmation at different times to ensure that no misjudgment is made. Once the scale maturity condition is met and the delayed confirmation condition is met, the scale maturity state is set to Tru e , after reaching the mature state, the method continues to run. If the threshold value is not reached for a period of time, the mature state will be revoked, indicating that the previous round of judgment is wrong. In addition, based on the assumption that the crop maturity is approximately continuously changing with respect to the geographical location, the present invention calculates the screen proportion of the mature crops cut out of the left and right pictures respectively, and takes the average as the approximate value of the screen proportion of the missing mature crops. The beneficial effect is that compared with directly using the overall average value, the crop maturity of the missing part of the picture can be calculated more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic diagram of the structure of a system corresponding to the method of the present invention;

[0048] Figure 2 A flowchart of deduplication of the present invention;

[0049] Figure 3 The figure is a flow chart of the color correction method of the present invention. DETAILED DESCRIPTION

[0050] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0051] The system currently used for large-scale agricultural and forestry detection adopts a sensor-client-server architecture, which collects data through sensors, transmits it to the server after partial data processing locally, performs large-scale recognition processing, and then sends the recognition results to the client. The system is generally divided into three layers:

[0052] 1. Information perception layer: Generally, it is a sensor device used to collect information such as status and pictures in a specific small area. Generally, the device is small and has a single function. It can only collect specific information and has no processing capabilities.

[0053] 2. Data aggregation layer: It is used to manage terminal devices such as sensors in a certain large area, collect their information, have certain information processing capabilities, and can perform preliminary processing or packaging of information. It can report sensor anomalies and issue monitoring status alarm information, etc.

[0054] 3. Information processing layer: It is the terminal for collecting information flow. After all the collected information is gathered in the information processing layer, it will be processed and analyzed at a higher level to achieve intelligent judgment and decision-making. Generally speaking, the center of the information processing layer is the server-level equipment. After the processing is completed, the judgment information will be returned to the terminal equipment of the data aggregation layer for processing, or processed through manual intervention.

[0055] For example, some orchard environmental monitoring systems use sensors arranged in lines to certain areas to monitor the status of small areas at fixed points. After processing by integrated sensors or distributed controllers, the data is transmitted to the upper-level IoT gateway through the ZigBee network, and then sent to the monitoring host or monitoring server through the 4G / 5G communication network through the gateway protocol. Another agricultural and forestry climate monitoring system uses microclimate sensor nodes to monitor fixed small areas, sends them to the data aggregation platform through the ZigBee network, and after data packaging, sends them to the user / back-end data processing platform through the Internet / GPRS network.

[0056] In the existing agricultural Internet of Things technology implementation, it is difficult to transmit information in the harsh environment of agricultural production. Harsh environmental conditions, such as high temperature, low temperature, large humidity changes, complex soil conditions, etc., often cause traditional communication networks to face problems such as unstable signals, data loss, and transmission delays in agricultural applications. The existing technology mainly uses low-power wide area network technology (LPWAN), long-range radio technology (LoRa) and other anti-interference communication technologies and satellite communication and edge computing combined methods, intelligent routing and self-organizing network technology, adaptive communication protocols and other multi-measure solutions to alleviate the pain points of harsh environments.

[0057] For situations where communication may not be possible under more extreme conditions, edge computing and distributed processing of local data can be used to move data processing functions to the device side or local nodes, rather than relying entirely on remote servers. For extreme cases where real-time communication is only temporarily unavailable, offline mode and periodic synchronization can also be used. Edge devices switch to offline mode in harsh conditions to independently collect and store data. After collecting data for a period of time, these devices will periodically synchronize the locally stored data with the central system.

[0058] In order to achieve better disaster recovery capabilities, agricultural IoT devices can usually also be set up with a backup independent power supply to cope with scenarios where they cannot rely on external power supply. Through low-power module design, the device can achieve long-term independent operation without an external power supply, and can be supplemented with power through wireless charging or other means when necessary.

[0059] In some extreme environments, it may not be possible to perform complex data analysis tasks. In this case, the device can use simplified information processing algorithms to perform preliminary analysis of key data and take necessary basic measures. For example, the sensor can directly determine whether the temperature and humidity exceed the preset thresholds. If so, it will automatically trigger an alarm or take some simple control measures without relying on complex cloud computing. This simplified processing solution ensures that the device can still operate effectively and provide necessary functions even under extreme conditions.

[0060] It is concluded that the current algorithms or technical architectures have the following bottlenecks in implementation:

[0061] 1. The current detection technology is targeted at the individual crop level. Large-scale or large-area detection requires the deployment of more imaging devices to complete.

[0062] 2. For the design of the Internet of Things system, the current mainstream design considers a three-layer working mechanism, in which information must be guaranteed to be effectively transmitted. However, in remote forest or mountainous areas, the communication quality cannot be guaranteed, and existing methods are basically not applicable.

[0063] 3. In the case where communication and energy supply cannot be guaranteed, the method is required to complete simple information processing and give judgment results independently on the edge device, and the method is required to be efficient and low in energy consumption. Traditional methods cannot meet this requirement.

[0064] In view of the shortcomings of existing solutions, the present invention designs a new crop detection method for the hardware of outdoor environment in forest areas. Through a small amount of visible light images and infrared sensors of the terminal, large-scale crop maturity detection is completed, and an agricultural emergency alarm function is provided.

[0065] The present invention proposes a large-scale crop maturity detection method suitable for edge devices. The system structure diagram corresponding to the method is as follows: Figure 1 The method comprises the following steps:

[0066] S1, obtaining a crop image, wherein a position ID is set on the image, each image corresponds to a moment, deduplication is performed on the image and the ID number of the image is recorded, the deduplication image is input into a color threshold detector, and the color threshold detector outputs a corrected real threshold;

[0067] S2. Based on the corrected real threshold, calculate the ratio of pixels at each position id in the crop image that meet the real threshold to the total pixels of the image. If the ratio is greater than the ratio threshold, the position id meets the crop maturity condition at this moment, and set the mark X of the position id at this moment. t =1, otherwise set the mark X of the position id at that moment t = 0, repeat the above steps multiple times to obtain the labels of multiple position IDs corresponding to multiple moments, and determine whether there are at least m enter The moment meets the crop maturity condition, m enter is the number threshold, if yes, the crop state of the position id is determined to be mature, otherwise the crop state of the position id is determined to be immature;

[0068] S3, after determining that the crop at the position id is mature, delay for a period of time, and continue to execute S1 and S2 during the period of time. If the crop state at the position id does not change to immature during the period of time, output a crop maturity signal, otherwise output an immature signal;

[0069] After determining that the crop at the position id is immature, a delay is made, and S1 and S2 are continued during the time period. If the crop state at the position id does not change to mature during the time period, an immature crop signal is output, otherwise a mature crop signal is output;

[0070] In the above steps, the ID number and time are aligned regularly.

[0071] The system consists of a detection and compensation module, a timing statistics module and a splicing module, among which the innovations are mainly in the compensation module and the timing statistics module. The overall idea is that the equipment is stationary or reciprocating in a certain area, and periodically takes pictures. The pictures taken are judged by the maturity area of ​​a single picture, and the splicing module performs regional alignment. By performing "window sliding" recognition on the aligned pictures, the planting monitoring of a larger monitoring area can be carried out under displacement monitoring. At the same time, in order to further improve the accuracy of recognition and prevent misjudgment due to weather and light reasons, a timing confirmation module is designed to prevent accidental recognition errors by designing the conditions for establishment under the time period.

[0072] Figure 1 In the process, the displacement sensor of the central controller reports the position. When the position moves to the set value, a wide-angle camera is used to take a picture to monitor the largest possible position. After being sent to the image fusion module, the image overlap analysis is performed to remove duplicates. The processed image is then sent to the color threshold detector for detection. At the same time, the image fusion module provides the corresponding ID number for the time series statistics to correspond to the detection status of the color threshold at different times. Finally, the results are reported to the central controller regularly. Considering that the devices supported by this system may be reciprocating motion, the central controller will align the ID and position for the time series statistics.

[0073] 1. Color threshold detection module

[0074] In order to reduce the computing requirements, the maturity detection and recognition method we adopted is completed by using basic image processing technology and judging by color value. The general idea is that for most crops, the color after maturity is obviously different from that during the growth period. For maturity monitoring, it is not necessary to use individual level and counting. In large-scale planting, for large areas, it is only necessary to constrain the color and count the blocks that meet the constrained color values ​​to monitor large-scale planting. Since this constraint condition will change due to different lighting conditions in the environment, we obtain real-time ambient lighting information by placing a color correction plate, and establish a mapping relationship from ambient light to the color threshold of mature crops, so as to dynamically adjust the color threshold so that the algorithm can detect mature crops more accurately in all weather and time.

[0075] In the color threshold detection module, threshold intelligent adaptive calibration is performed. For the judgment standard of crop maturity, modeling is performed in the HSV color space. Assuming that under standard lighting conditions, the threshold judgment method is:

[0076] thresh 0 =[[H l , S l , V l ],[H h , S h , V h]]

[0077] Assume that the ambient light at a certain moment is L in real life e =[H e , S e , V e ];

[0078] At this time, the true color threshold is affected by the ambient light, that is:

[0079] thresh=f(thresh 0 , L)

[0080] Due to thresh 0 It is only related to the crop and can be regarded as a constant, so the original formula can be simplified to:

[0081] thresh = f′(Le)

[0082] The specific steps of threshold judgment are:

[0083] The ambient light information is obtained through the color correction plate, and the real threshold is obtained through the mapping relationship f′

[0084] thresh=[[H l , S l , V l ],[H h , S h , V h ]]

[0085] For the H channel, when the H component of a pixel meets the conditions:

[0086] H l ≤H real ≤H h

[0087] It means that it falls within the threshold; the same is true for the S and V channels. When all three channels meet the threshold requirements, it is determined that the pixel falls within the threshold. The flowchart of the color correction method is as follows Figure 3 shown.

[0088] 2. Time series statistics module

[0089] In some extreme cases, there is no guarantee that the calibrated color can accurately identify mature crops. For example, on a cloudy day, due to the light, the compensated color threshold is met and the color area area is met, but this situation may only last for a period of time or only cause misjudgment in certain weather and light conditions. In response to this. In response to this, we designed the judgment rule that if the area that meets the color judgment threshold is greater than the set value on certain days or during certain time periods, the area is judged to be mature and can be picked. At the same time, in order to eliminate the threshold judgment caused by special weather, we adopt the method of delayed notification and reconfirmation at different times to ensure that no misjudgment is made. Once the scale maturity conditions are met and the delayed confirmation conditions are met, the scale maturity status is set to Tru e , and send a signal to the network at the same time. After the mature state, the algorithm continues to run and relaxes the judgment threshold. If the monitoring threshold fails to reach the threshold for a period of time, the mature state will be revoked and a signal will be sent again, indicating that the previous round of judgment was wrong.

[0090] The specific requirements for satisfying time series statistics are:

[0091] Over a period of time, the image with the calibrated color threshold is continuously tested. When a continuous period of time or multiple discontinuous periods of time meet the threshold image area, the color threshold ratio of the image is reported. The central controller will count the total color threshold according to the location ID. When the sum or average of the threshold area of ​​all location IDs exceeds the set value, the maturity of the area will be reported as suitable for picking.

[0092] For a certain edge device, it continuously takes crop images and performs color threshold detection at certain time intervals during operation; the time of each operation is simply recorded as a moment. Assume that it performs a detection at time t, and obtains the ratio of pixels that meet the color threshold to the total pixels of the screen pt; if the ratio is greater than or equal to the set area ratio threshold, that is, p t ≥p thresh , then it is considered that the location id meets the crop maturity condition at this moment, recorded as X t =1, otherwise, recorded as X t =0.

[0093] Set the threshold m enter , a total of n detections were made within a period of time, if at least m enter The crop maturity conditions are met at the moment, namely:

[0094]

[0095] It is considered that the location ID meets the crop maturity condition and sends a status update signal to the network.

[0096] Anti-satisfaction timing statistics:

[0097] When the continuous time threshold of a single sheet (single location ID) is less than the threshold (this threshold is different from the satisfaction threshold and is less than the satisfaction threshold), the controller is not satisfied with the area and the average of its current satisfaction threshold. When the total / average threshold calculated by the central controller is less than the cancellation setting threshold, the central controller sends a cancellation message.

[0098] Similar to satisfying time series statistics, count the number of moments that meet the crop maturity conditions within a period of time and set the anti-satisfaction threshold m exit , if the number of moments that meet the crop maturity condition within a period of n detections is less than m exit ,Right now:

[0099]

[0100] It is considered that the location ID no longer meets the crop maturity conditions, and a status update signal is sent to the network.

[0101] 3. Image fusion module

[0102] Considering that the design of the equipment using this method may be movable within a certain range, if the setting is non-fixed, periodic collection of a large area is required. After processing the collected data at different locations, the overall maturity rate needs to be considered before making a judgment. The design method is as follows:

[0103] Case 1: The moving range is small, and the captured images must have overlapping parts

[0104] In this case, it is necessary to identify the overlapping parts and then calculate the actual area to be taken into consideration through deduplication method. Assuming that the device is performing reciprocating motion and the algorithm cannot obtain the starting and ending points of the reciprocating motion, the specific steps of the deduplication algorithm are:

[0105] (1) Find the matching feature point pairs (P 1i , P 2i ):

[0106] FeatureMap(I 1 , I 2 )={(P 1i , P 2i )}

[0107] (2) Calculate the homography matrix to describe the projection transformation between two planar images:

[0108]

[0109] Among them, the calculation of the homography matrix is:

[0110]

[0111] (3) Image I 1 The bounding box of:

[0112] (X 1,min ,y 1,min ), (X 1,max ,y 1,min ), (X 1,max ,y 1,max ), (X 1,min ,y 1,max )

[0113] Transform to I 2 Coordinate system:

[0114] B1′(x,y)={H·[x1y11] T jor(x1,y1)∈B1}

[0115] (4) Calculate the overlapping area:

[0116] R=B1′∩B2

[0117] (5) In image I 2 Remove the overlapping areas.

[0118] Case 2: The moving range is large, and the captured images have overlapping parts with a small probability.

[0119] In view of this situation, it is necessary to increase the shooting frequency of the moving camera as much as possible to reduce the missed shooting of the area, which is determined by the central controller.

[0120] According to the size (R) of the overlapping area R calculated in 1, the camera shooting frequency is corrected to find the frequency f that tends to satisfy the requirement that the overlapping area is as small as possible but no sampling is missed:

[0121] f = argmin(size(R)|size(R)>0)

[0122] At the same time, the overlapping judgment method in situation 1 is applied to perform repeated judgment.

[0123] When ensuring that there are no actual duplications, a direct statistical method is used, and we increase the amount of mature perception of non-overlapping edges, as follows:

[0124] (1) The angle of the unphotographed farmland between two images is calculated by using the camera rotation angular velocity, camera field of view angle, and image shooting interval time:

[0125] α gap =ωt interval -HFOV

[0126] where α gap is the interval angle, ω is the camera rotation angular velocity, t interval is the shooting interval, and HFOV is the horizontal field of view.

[0127] (2) Use interpolation to cut out the β values ​​in the left and right adjacent images of the missing part of the picture. 0 The image ratio of the left and right images with the mature crops cut out is calculated for each (pre-set) angle, and the average is taken as the approximate ratio of the missing mature crops:

[0128] p gap =mean(P β_0left , p β_0right )

[0129] Where p is the proportion of mature crops in the picture.

[0130] The deduplication flowchart is as follows Figure 2 shown.

[0131] In terms of data collection, the present invention uses a single wide-angle camera to capture the crop growth situation in a large mask area at high altitude with a follower device. A single device can monitor a large area. A single device can cover an area of ​​up to 500 square meters.

[0132] Aiming at the requirements of high-efficiency, low-energy and mature detection, the designed algorithm is based on the traditional visual algorithm and completes the adaptive adjustment of the color value detection of the picture. The amount of calculation required by this algorithm is 2-3 orders of magnitude lower than that of deep learning detection algorithms such as YOLO. At the same time, the infrared sensing device attached to the device is combined to calibrate different environments, seasons and meteorological conditions, making the algorithm judgment more accurate.

[0133] The algorithm is aimed at maturity detection at a large area rather than at an individual level, and can improve efficiency for understory planting, which requires less manpower and requires large-scale management.

[0134] The algorithm has added abnormal value prompts for pests and diseases, as well as detection prompts for fires, mountain torrents, etc., which are more practical for monitoring under-forest planting.

[0135] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A large-scale crop maturity detection method suitable for edge devices, characterized in that: The method comprises the following steps: S1, obtaining a crop image, wherein a position ID is set on the image, each image corresponds to a moment, deduplication is performed on the image and the ID number of the image is recorded, the deduplication image is input into a color threshold detector, and the color threshold detector outputs a corrected real threshold; S2. Based on the corrected real threshold, calculate the ratio of pixels at each position id in the crop image that meet the real threshold to the total pixels of the image. If the ratio is greater than the ratio threshold, the position id meets the crop maturity condition at this moment, and set the mark X of the position id at this moment. t =1, otherwise set the mark X of the position id at that moment t = 0, repeat the above steps multiple times to obtain the labels of multiple position IDs corresponding to multiple moments, and determine whether there are at least m enter The moment meets the crop maturity condition, m enter is the number threshold, if yes, the crop state of the position id is determined to be mature, otherwise the crop state of the position id is determined to be immature; S3, after determining that the crop at the position id is mature, delay for a period of time, and continue to execute S1 and S2 during the period of time. If the crop state at the position id does not change to immature during the period of time, output a crop maturity signal, otherwise output an immature signal; After determining that the crop at the position id is immature, a delay is made, and S1 and S2 are continued during the time period. If the crop state at the position id does not change to mature during the time period, an immature crop signal is output, otherwise a mature crop signal is output; In the above steps, the ID number and time are aligned regularly.

2. A large-scale crop maturity detection method suitable for edge devices according to claim 1, characterized in that: The conditions for satisfying crop maturity are: Among them, X t Indicates the mark, m enter Indicates the number of times threshold.

3. The large-scale crop maturity detection method applicable to edge devices according to claim 1, characterized in that: The corrected true threshold is: thresh=f′(Le)=[[H l ,S l ,V l ],[H h ,S h ,V h ]] Among them, f′ represents the mapping relationship, Le represents the ambient light, H l , H h Indicates the maximum and minimum values ​​of the H channel, S l , S h Indicates the maximum and minimum values ​​of the S channel, V l 、V h Indicates the maximum value of V channel and the minimum value of V channel; The true threshold is satisfied when, for a crop image, the H component, the S component, and the V component of its pixel are all within the corresponding channel maximum and minimum values, then the true threshold is satisfied.

4. The large-scale crop maturity detection method applicable to edge devices according to claim 1, characterized in that: The specific steps to deduplicate an image are as follows: If the moving range is greater than the range threshold, then for the two crop images, find the matching feature point pairs (P 1i ,P 2i ) where P 1i is the matching feature point of the first crop image, P 2i are the matching feature points of the second crop image; Compute the homography matrix; Based on the homography matrix, the bounding box of the first crop image is transformed into the coordinate system of the second crop image, and a mapping B1′ of the shooting range bounding box of the first crop image in the coordinate system of the second crop image is obtained, that is, the part of the second crop image that contains the shooting content of the first crop image; Calculate the overlapping area based on the mapping B1′, remove the overlapping area for the second crop image, and correct the camera shooting frequency according to the overlapping area; After removing the overlapping area, calculate the angle occupied by the farmland that was not photographed between the two images, determine the missing part of the picture based on the angle occupied by the farmland, cut out a picture at a certain angle in the left and right adjacent images of the missing part of the picture, calculate the ratio of pixels of mature crops in the cut-out picture to the total pixels of the picture, take the average as the approximate ratio of pixels of mature crops in the missing part to the total pixels of the picture, and the approximate value is used as a part of the ratio of pixels that meet the real threshold at each position id to the total pixels of the picture.

5. A large-scale crop maturity detection method suitable for edge devices according to claim 4, characterized in that: The homography matrix is: Among them, i represents the matching feature point number, x2y21 represents the homogeneous coordinates of the feature point in the second crop image, and x1y11 represents the homogeneous coordinates of the feature point in the first crop image.

6. A large-scale crop maturity detection method suitable for edge devices according to claim 5, characterized in that: The mapping B1′ is: B1′(x,y)={H·[x1y11] T for(x1,y1)∈B1} Among them, B1 represents the bounding box of the first crop image in its own coordinate system.

7. A large-scale crop maturity detection method suitable for edge devices according to claim 6, characterized in that: The overlapping area is: R=B1′∩B2 Among them, R represents the overlapping area, and B2 represents the bounding box of the second crop image in its own coordinate system.

8. The large-scale crop maturity detection method applicable to edge devices according to claim 4, characterized in that: The corrected camera shooting frequency is: f = argmin(size(R)|size(R)>0) Wherein, f represents the corrected camera shooting frequency, and size(R) represents the size of the overlapping area R.

9. A large-scale crop maturity detection method suitable for edge devices according to claim 8, characterized in that: The angle of unphotographed farmland between the two images is: a gap =ωt interval -HFOV Among them, α gap is the angle of the unphotographed farmland between the two images, ω is the camera rotation angular velocity, t interval is the shooting interval, and HFOV is the horizontal field of view.

10. A large-scale crop maturity detection method applicable to edge devices according to claim 9, characterized in that: The approximate value of the ratio of the pixels of the missing part of the mature crop to the total pixels of the picture is: p gap =mean(p β_0left ,p β_0right ) Among them, p gap is an approximate value, p β_0left It indicates the ratio of the pixels of the mature crops in the cut left adjacent image to the total pixels of the image, p β_0right It shows the ratio of the pixels of the mature crops in the cut-out right adjacent image to the total pixels of the image.

Images