Cherry development monitoring method based on image recognition

Abstract

The invention provides a cherry development monitoring method based on image recognition, and relates to the technical field of agricultural intelligent monitoring. According to the method, a high-definition camera is installed at the position 2-3 m above a cherry plant, RGB images are collected regularly, and Gaussian filtering, white balance adjustment, illumination compensation and color space conversion preprocessing operation are sequentially carried out. An improved YOLOv5s model is adopted to position and segment cherry fruits in the image, and four development stages of the fruits are identified in combination with a ResNet-18 classification network. And after the identification result is subjected to visualization processing, the identification result is uploaded to an agricultural management platform through a 4G communication module, so that remote monitoring and data management are realized. The method can automatically adapt to different illumination conditions, has good identification precision and generalization ability, is suitable for fruit development monitoring of various fruit trees, and helps to improve the intelligent level of orchard management.

Description

Technical Field

[0001] The present invention relates to the technical field of agricultural intelligent monitoring, and in particular to a cherry development monitoring method based on image recognition. Background Art

[0002] With the development of intelligent modern agriculture, automated crop growth monitoring technology has become a research hotspot. Cherries are a highly economically valuable fruit, and their fruit development significantly impacts harvest timing, quality control, and resource allocation. Traditional cherry fruit development monitoring relies primarily on manual field observation, with regular field inspections recording fruit color changes and morphological characteristics to determine maturity. However, this method is inefficient, highly subjective, and difficult to scale, making it unable to meet the demands of refined management in modern agriculture.

[0003] In recent years, intelligent agricultural monitoring systems based on image recognition have been increasingly applied to analyzing the growth status of fruits and vegetables. Previous studies have attempted to use RGB cameras or drones to capture images of fruit trees and combine them with traditional image processing algorithms or machine learning models for fruit recognition and classification. For example, some literature reports methods based on color feature extraction and rule matching to determine the color ratio of fruit and thus infer its degree of ripeness.

[0004] However, the existing technology still has the following significant defects: most systems use ground mobile devices or handheld cameras for image acquisition, which has limited shooting angles and makes it difficult to cover the entire cherry tree crown area, resulting in unstable image quality and affecting subsequent recognition accuracy.

[0005] Existing methods mostly use simple grayscale or histogram equalization processing, which lacks an effective compensation mechanism for changes in natural lighting. The image quality deteriorates significantly on rainy days or in low light conditions, making it difficult to accurately extract fruit features.

[0006] Traditional image recognition methods rely on manually designed features, have weak recognition capabilities for small target fruits such as young cherries, and are prone to false detection and missed detection under complex background interference.

[0007] Existing technologies often use shallow classification models or rule-based judgments based on color thresholds. These methods are unable to effectively distinguish the gradual transition from green to red to crimson in cherry fruit, resulting in ambiguous developmental stage delineation and inaccurate classification results. Most systems lack timed data acquisition, remote transmission, and visualization capabilities, requiring manual intervention in the image processing process, making continuous monitoring and real-time data feedback difficult. Therefore, we address this issue by proposing a cherry development monitoring method based on image recognition. Summary of the Invention

[0008] The purpose of the present invention is to address the problems raised by the existing background technology. In order to achieve the above-mentioned invention purpose, the present invention provides the following technical solutions: a cherry development monitoring method based on image recognition, comprising the following steps:

[0009] Step 1: Image acquisition: Using a high-definition camera installed 2 to 3 meters above the cherry plants, RGB images with a resolution of 1920 × 1080 pixels were collected between 9:00 and 11:00 a.m. daily.

[0010] Step 2: The collected images are subjected to Gaussian filtering for denoising, white balance adjustment, illumination compensation based on Retinex theory, and conversion from RGB color space to HSV color space.

[0011] Step 3: Detect the cherry fruit in the image based on the YOLOv5s model, and implement binary segmentation of the fruit area by adding a Mask branch;

[0012] Step 4: Use a convolutional neural network built with the ResNet-18 architecture to classify the segmented fruit images. The input image size is 224×224 pixels, and the output results correspond to the four developmental stage labels of cherry fruits.

[0013] Step 5: Superimpose the classification results on the original image to export a visual image, and upload the classification label data to the agricultural management platform through the 4G communication module;

[0014] Step 6: Repeat the above process every day to continuously monitor the development and changes of cherry fruits from the 10th day after flowering to the maturity stage.

[0015] As a preferred technical solution of the present invention, the image acquisition device has high resolution and wide dynamic range characteristics, and is equipped with an adjustable fill light device, which is connected to the light sensor. When the light intensity monitored by the light sensor is lower than 5000 lux, the fill light device turns on and supplements the light intensity to 8000-10000 lux; when the light intensity is higher than 12000 lux, the fill light device turns off.

[0016] As a preferred technical solution of the present invention, the collected image is sequentially subjected to Gaussian filtering denoising and white balance adjustment processing steps: a 3×3 median filtering window is used to perform median filtering denoising on the original image; the denoised image is enhanced using a histogram equalization method; the cherries are separated from the background using a threshold segmentation algorithm based on color features. In the RGB color space, the threshold range is set to R∈[120,255], G∈[50,180], B∈[30,120], and a circular structuring element with a radius of 3 is used to perform a morphological opening operation on the segmented image to remove noise and small connected areas.

[0017] As a preferred technical solution of the present invention, CSPDarknet53 is used as the backbone network in the YOLOv5s model, PANet is used as the feature pyramid structure, the detection confidence threshold is set to 0.5, and the non-maximum suppression NMS threshold is set to 0.4.

[0018] As a preferred technical solution of the present invention, the cross entropy loss function is used in the ResNet-18 model training process, the initial value of the learning rate is 0.001, it decays once every 30 training cycles, the momentum is set to 0.9, and the weight decay coefficient is 0.0005.

[0019] As a preferred technical solution of the present invention, the four developmental stage labels are: green in the young fruit stage, mixed red and green in the color change stage, mainly red in the early maturity stage, and dark red in the maturity stage, and each label corresponds to an independent Softmax classification output.

[0020] As a preferred technical solution of the present invention, the image acquisition device is started at a set time every day according to a set schedule. If the ambient light intensity is lower than 500 lux, the fill light is automatically enabled for auxiliary lighting.

[0021] As a preferred technical solution of the present invention, during the model training stage, the original image dataset is enhanced, including random rotation of ±15°, horizontal flipping, brightness perturbation of ±20%, and contrast perturbation of ±0.2 times.

[0022] As a preferred technical solution of the present invention, the 4G communication module adopts the MQTT protocol for data transmission, and the uploaded content includes fruit position coordinates, development stage labels, image timestamps and device ID information.

[0023] As a preferred technical solution of the present invention, the method is applicable to the development monitoring of fruits of other fruit trees. During specific implementation, the training data set of the target detection model is replaced, and the deployment height and angle of the image acquisition equipment are adjusted to adapt to different tree species structures.

[0024] Beneficial effects of the present invention: The present invention collects images at regular intervals by setting a high-definition camera at a height of 2 to 3 meters above the cherry plants. It can obtain high-quality fruit images without interfering with the growth of the plants, thereby improving the degree of automation and stability of image acquisition. Gaussian filtering, white balance adjustment, illumination compensation and color space conversion steps are used in the image preprocessing process to effectively eliminate the impact of natural illumination changes on image quality and enhance the contrast and feature expression of the fruit area. Based on the target detection model YOLOv5s, the Mask branch is introduced to achieve pixel-level segmentation of cherry fruits, thereby improving the recognition accuracy and positioning accuracy of small target fruits. The classification model is constructed using the ResNet-18 architecture, combined with the cross-entropy loss function and the transfer learning strategy, and a high classification accuracy can still be obtained on limited sample data, which is suitable for actual deployment needs in agricultural scenarios. The fruit classification results correspond to four clear developmental stage labels, including the young fruit stage, the color change stage, the early maturity stage and the mature stage, which helps to accurately judge the maturity state of the fruit and provide data support for harvesting decisions.

[0025] The system of the present invention has a visualization function, which superimposes the recognition results on the original image to export a visual image, allowing users to intuitively understand the development and distribution of fruits in the orchard. The recognition results are uploaded to the agricultural management platform using a 4G communication module, enabling remote monitoring and centralized data management, thereby improving the level of informatization of agricultural production. The image acquisition equipment is equipped with a fill light and a timed shooting mechanism, which can operate stably under different lighting conditions and adapt to various field environments. During the model training process, multiple data enhancement methods are introduced, including rotation, flipping, brightness perturbation, and contrast perturbation, which improves the generalization ability and robustness of the model in complex backgrounds.

[0026] The method of the present invention has a strong structural versatility and can quickly adapt to the fruit development monitoring tasks of various fruit trees such as strawberries, blueberries, and apples by replacing the training data set and adjusting the image acquisition parameters. It has good scalability and application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic flow chart of the cherry development monitoring method provided by the present invention;

[0028] Figure 2 A schematic diagram of image acquisition parameters provided by the present invention;

[0029] Figure 3 A schematic diagram of the image preprocessing process provided by the present invention;

[0030] Figure 4 Schematic diagram of model parameters provided by the present invention DETAILED DESCRIPTION

[0031] To make the purpose, technical solutions and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them.

[0032] Therefore, the following detailed description of the embodiments of the present invention is not intended to limit the scope of the claimed invention, but merely represents some embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features and technical solutions in the embodiments can be combined with each other. It should be noted that similar numbers and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0033] Example 1: A cherry development monitoring method based on image recognition, comprising the following steps:

[0034] Step 1: Image acquisition: Using a high-definition camera installed 2 to 3 meters above the cherry plants, RGB images with a resolution of 1920 × 1080 pixels were collected between 9:00 and 11:00 a.m. daily.

[0035] Step 2: The collected images are subjected to Gaussian filtering for denoising, white balance adjustment, illumination compensation based on Retinex theory, and conversion from RGB color space to HSV color space.

[0036] Step 3: Detect the cherry fruit in the image based on the YOLOv5s model, and implement binary segmentation of the fruit area by adding a Mask branch;

[0037] Step 4: Use a convolutional neural network built with the ResNet-18 architecture to classify the segmented fruit images. The input image size is 224×224 pixels, and the output results correspond to the four developmental stage labels of cherry fruits.

[0038] Step 5: Superimpose the classification results on the original image to export a visual image, and upload the classification label data to the agricultural management platform through the 4G communication module;

[0039] Step 6: Repeat the above process every day to continuously monitor the development and changes of cherry fruits from the 10th day after flowering to the maturity stage.

[0040] The image acquisition device has high resolution and wide dynamic range characteristics, and is equipped with an adjustable fill light device, which is connected to the light sensor. When the light intensity detected by the light sensor is lower than 5000 lux, the fill light device turns on and supplements the light intensity to 8000-10000 lux; when the light intensity is higher than 12000 lux, the fill light device turns off.

[0041] The collected images were subjected to Gaussian filtering denoising and white balance adjustment processing steps in sequence: a 3×3 median filtering window was used to perform median filtering denoising on the original image; the denoised image was enhanced using the histogram equalization method; the cherries were separated from the background using a threshold segmentation algorithm based on color features. In the RGB color space, the threshold range was set to R∈[120,255], G∈[50,180], B∈[30,120], and a circular structuring element with a radius of 3 was used to perform morphological opening operation on the segmented image to remove noise and small connected areas.

[0042] The YOLOv5s model uses CSPDarknet53 as the backbone network, PANet as the feature pyramid structure, the detection confidence threshold is set to 0.5, and the non-maximum suppression (NMS) threshold is set to 0.4.

[0043] The cross entropy loss function is used in the ResNet-18 model training process. The initial value of the learning rate is 0.001, which decays every 30 training cycles. The momentum is set to 0.9 and the weight decay coefficient is 0.0005.

[0044] The four developmental stage labels are: green in the young fruit stage, mixed red and green in the color change stage, mainly red in the early maturity stage, and dark red in the maturity stage. Each label corresponds to an independent Softmax classification output.

[0045] The image acquisition device starts at a set time every day according to the set schedule. If the ambient light intensity is lower than 500 lux, the fill light is automatically enabled for auxiliary lighting.

[0046] During the model training phase, the original image dataset was enhanced by random rotation of ±15°, horizontal flipping, brightness perturbation of ±20%, and contrast perturbation of ±0.2 times.

[0047] The 4G communication module uses the MQTT protocol for data transmission, and the uploaded content includes fruit location coordinates, development stage labels, image timestamps, and device ID information.

[0048] This method is applicable to the development monitoring of fruits of other fruit trees. In specific implementation, the training dataset of the target detection model is replaced, and the deployment height and angle of the image acquisition equipment are adjusted to adapt to the structure of different tree species.

[0049] The present invention provides a cherry development monitoring method based on image recognition. Its core lies in combining image acquisition and processing with deep learning model recognition to achieve automated identification and continuous monitoring of cherry fruit development stages. The specific working principle is as follows:

[0050] Image acquisition: A high-definition camera is installed 2 to 3 meters above the cherry plants, with the camera angle set at a 60° angle to the ground, ensuring coverage within a 3-meter radius of the fruit area. The camera regularly captures RGB images with a resolution of 1920 × 1080 pixels between 9:00 AM and 11:00 AM daily. When the ambient light intensity falls below 500 lux, a fill light is automatically activated to assist imaging.

[0051] Image preprocessing stage: The collected original images were processed in sequence through Gaussian filtering with a kernel size of 5×5 and σ=1.5, grayscale world white balance algorithm adjustment, illumination compensation based on single-scale Retinex theory, and conversion from RGB color space to HSV color space to improve image quality and enhance the contrast of the fruit area.

[0052] During the fruit localization and segmentation phase, the image was processed using the YOLOv5s object detection model. The backbone network employed the CSPDarknet53 architecture, while the feature fusion architecture employed the PANet architecture. A mask branch was added to the existing YOLOv5 architecture to achieve binary pixel-level segmentation of the cherry fruit. The detection confidence threshold was set to 0.5, and the non-maximum suppression (NMS) threshold was set to 0.4.

[0053] Developmental stage classification: Each segmented fruit image was cropped and resized to 224×224 pixels. It was then fed into a convolutional neural network based on the ResNet-18 architecture for classification and recognition. The model was trained using a cross-entropy loss function with an initial learning rate of 0.001, decayed every 30 training cycles, a momentum of 0.9, and a weight decay coefficient of 0.0005. The final output corresponded to four developmental stage labels: green for young fruit, a mixed red and green color at veraison, predominantly red at early maturity, and deep red at maturity.

[0054] Results visualization and transmission: The classification results are superimposed on the original image to create a visual image. The image also records the fruit location coordinates, developmental stage labels, image timestamps, and device ID information. This information is then uploaded to an agricultural management platform via a 4G communication module using the MQTT protocol for remote viewing and analysis.

[0055] System Update and Adaptation Mechanism: Data augmentation techniques are introduced during model training, including random rotation of ±15°, horizontal flipping, brightness perturbation of ±20%, and contrast perturbation of ±0.2x, to improve model generalization. The system supports changing the training dataset for the object detection model and adjusting the deployment height and angle of the image acquisition device to meet the development monitoring needs of strawberries, blueberries, apples, and other fruit trees.

[0056] The Python code for the cherry development monitoring method based on image recognition is as follows:

[0057] import cv2

[0058] import torch

[0059] from torchvision import transforms,models

[0060] import numpy as np

[0061] from datetime import datetime

[0062] import requests#Simulate 4G communication module to upload data

[0063] #Step 1: Image Acquisition

[0064] def image_capture:

[0065] #Simulate the acquisition of an RGB image with a resolution of 1920×1080 pixels

[0066] #Here we use local image reading instead of actual camera acquisition

[0067] image=cv2.imread('cherry_image.jpg')

[0068] image=cv2.cvtColor(image,cv2.COLOR_BGR2RGB)

[0069] return image

[0070] #Step 2: Image Preprocessing

[0071] def preprocess_image(image):

[0072] #Gaussian filter denoising

[0073] blurred=cv2.GaussianBlur(image,(5,5),0)

[0074] #White balance adjustment (simple grayscale world algorithm example)

[0075] def white_balance(img):

[0076] result=cv2.cvtColor(img,cv2.COLOR_RGB2LAB)

[0077] avg_a=np.average(result[1

[0078] avg_b=np.average(result[2

[0079] result[1]=result[1]-((avg_a-128)*(result[0] / 255.0)*1.1)

[0080] result[2]=result[2]-((avg_b-128)*(result[0] / 255.0)*1.1)

[0081] result=cv2.cvtColor(result,cv2.COLOR_LAB2RGB)

[0082] return result

[0083] balanced=white_balance(blurred)

[0084] #Lighting compensation based on Retinex theory (simple single-scale Retinex example)

[0085] def single_scale_retinex(img,sigma):

[0086] retinex=np.log10(img+1)-np.log10(cv2.GaussianBlur(img,(0,0),sigma)+1)

[0087] return retinex

[0088] enhanced=single_scale_retinex(balanced,30)

[0089] enhanced=np.uint8(np.clip(enhanced*255,0,255))

[0090] #Conversion from RGB color space to HSV color space

[0091] hsv_image=cv2.cvtColor(enhanced,cv2.COLOR_RGB2HSV)

[0092] return hsv_image

[0093] #Step 3: Cherry fruit detection and segmentation based on YOLOv5s model

[0094] def detect_and_segment(image):

[0095] #Load the pre-trained YOLOv5s model

[0096] model=torch.hub.load('ultralytics / yolov5','yolov5s',pretrained=True)

[0097] results=model(image)

[0098] #Simulate adding Mask branch to realize binary segmentation of fruit area

[0099] masks

[0100] for detection in results.xyxy[0]:

[0101] x1,y1,x2,y2=map(int,detection[:4

[0102] mask=np.zeros(image.shape[:2],dtype=np.uint8)

[0103] mask[y1:y2,x1:x2]=255

[0104] masks.append(mask)

[0105] return masks

[0106] #Step 4: Use ResNet-18 architecture to classify fruit images

[0107] def classify_fruits(masks,image):

[0108] # Load the pre-trained ResNet-18 model

[0109] model = models.resnet18(pretrained = False)

[0110] num_ftrs = model.fc.in_features model.fc = torch.nn.Linear(num_ftrs, 4) # 4 labels for developmental stages model.load_state_dict(torch.load('resnet18_cherry.pth')) model.eval()

[0111] transform = transforms.Compose

[0112] transforms.ToPILImage(),

[0113] transforms.Resize((224, 224)),

[0114] transforms.ToTensor(),

[0115] transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]) for mask in masks:

[0116] masked_image = cv2.bitwise_and(image, image, mask = mask)

[0117] input_tensor = transform(masked_image)

[0118] input_batch = input_tensor.unsqueeze(0)

[0119] with torch.no_grad():

[0120] output = model(input_batch)

[0121] _, predicted = torch.max(output.data, 1)

[0122] labels.append(predicted.item)

[0123] return labels

[0124] #Step 5: Visualization and data upload def visualize_and_upload(image,labels,masks):

[0125] #Superimpose the classification results on the original image to export the visual image for i,mask in enumerate(masks):

[0126] color={0:(0,255,0),1:(255,0,0),2:(0,0,255),3:(255,255,0)}[labels[i]]

[0127] image[mask>0]=color

[0128] cv2.imwrite('visualized_image.jpg',cv2.cvtColor(image,cv2.COLOR_RGB2BGR))

[0129] #Simulate uploading the classified label data to the agricultural management platform through the 4G communication module

[0130] url='http: / / agriculture_management_platform.com / upload'

[0131] data={'labels':labels}

[0132] response=requests.post(url,json=data)

[0133] print('Data uploaded:',response.status_code)

[0134] #Step 6: Repeat the process daily

[0135] def daily_monitoring:

[0136] now=datetime.now

[0137] if 9<=now.hour<=11:

[0138] image=image_capture

[0139] preprocessed_image=preprocess_image(image)

[0140] masks=detect_and_segment(preprocessed_image)

[0141] labels=classify_fruits(masks,preprocessed_image)

[0142] visualize_and_upload(image,labels,masks)

[0143] else:

[0144] print('Not in the image capture time range.')

[0145] #Main program

[0146] if__name__=="__main__":

[0147] daily_monitoring

[0148] Code Explanation:

[0149] Image acquisition: Use cv2.imread to read local images to simulate images captured by a high-definition camera.

[0150] Image preprocessing: Gaussian filtering denoising, white balance adjustment, illumination compensation based on Retinex theory, and color space conversion are performed in sequence.

[0151] Cherry fruit detection and segmentation: Use torch.hub.load to load the pre-trained YOLOv5s model for detection and simulate adding a mask branch to achieve binary segmentation.

[0152] Fruit image classification: Load the pre-trained ResNet-18 model to classify the segmented fruit images.

[0153] Visualization and data upload: The classification results are superimposed on the original image to generate a visualization image, and the requests library is used to simulate the 4G communication module to upload the classification label data to the agricultural management platform.

[0154] Repeat daily: Check whether the current time is between 9:00 and 11:00. If it is within this time period, the monitoring process will be executed; otherwise, it will prompt that it is not within the collection time range.

[0155] Example 2: A method for monitoring cherry development based on image recognition, implementation environment preparation:

[0156] HD camera: Choose a high-definition camera with high resolution (supports 1920×1080 pixels) and wide dynamic range, and install it at a height of 2 to 3 meters above the cherry plants to ensure that the cherry fruits can be clearly photographed.

[0157] Light sensor and fill light: Install a light sensor near the camera and connect an adjustable fill light. When the light sensor detects light intensity below 5,000 lux, the fill light turns on, increasing the light intensity to 8,000-10,000 lux. When the light intensity exceeds 12,000 lux, the fill light turns off.

[0158] 4G communication module: used to upload classified label data to the agricultural management platform.

[0159] Server: Used to run image processing and model inference programs, with sufficient computing resources and storage capabilities.

[0160] Operating system: Select Ubuntu Linux operating system.

[0161] Deep Learning Framework: Install the PyTorch deep learning framework for building and training ResNet-18 and YOLOv5s models.

[0162] Image processing library: Install the OpenCV library for image acquisition, preprocessing, and visualization operations.

[0163] A large number of cherry fruit images at different developmental stages were collected, covering four stages: green in the young fruit stage, mixed red and green in the color change stage, mainly red in the early stage of maturity, and dark red in the mature stage.

[0164] The images are annotated. For the YOLOv5s model, the bounding boxes of the cherry fruits are annotated; for the ResNet-18 model, the corresponding developmental stage labels are annotated for each image.

[0165] YOLOv5s model training

[0166] Configure the YOLOv5s model, use CSPDarknet53 as the backbone network, and PANet as the feature pyramid structure.

[0167] Set the detection confidence threshold to 0.5 and the non-maximum suppression (NMS) threshold to 0.4.

[0168] Use the labeled image data to train the YOLOv5s model and continuously adjust the model parameters until a good detection effect is achieved.

[0169] ResNet-18 model training

[0170] Construct a convolutional neural network with the ResNet-18 architecture, with an input image size of 224×224 pixels.

[0171] The cross entropy loss function is used, the initial value of the learning rate is set to 0.001, it decays every 30 training cycles, the momentum is set to 0.9, and the weight decay coefficient is 0.0005.

[0172] The ResNet-18 model is trained using labeled image data, and the model parameters are optimized through multiple iterations to improve the classification accuracy.

[0173] Every morning between 9:00 and 11:00, a high-definition camera captures RGB images with a resolution of 1920 × 1080 pixels. A light sensor monitors light intensity in real time and automatically switches the fill light on and off based on the lighting conditions, ensuring optimal lighting conditions for image acquisition.

[0174] Gaussian filtering denoising: Use OpenCV's cv2.GaussianBlur function to perform Gaussian filtering on the collected image to remove noise in the image.

[0175] Median filtering denoising: A 3×3 median filtering window is used to perform median filtering denoising on the original image, which is implemented using the cv2.medianBlur function.

[0176] Histogram equalization enhancement: Use the histogram equalization method to enhance the denoised image and improve the contrast of the image. This is achieved using the cv2.equalizeHist function.

[0177] Illumination compensation based on Retinex theory: Use the single-scale Retinex algorithm to perform illumination compensation on the image and adjust the brightness and contrast of the image.

[0178] Color space conversion: Use cv2.cvtColor function to convert the image from RGB color space to HSV color space.

[0179] Threshold segmentation: Use the color feature-based threshold segmentation algorithm to separate the cherries from the background. In the RGB color space, set the threshold range to R∈[120,255], G∈[50,180], B∈[30,120].

[0180] Morphological opening: Perform morphological opening on the segmented image using a circular structuring element with a radius of 3 to remove noise and small connected regions. This is implemented using the cv2.morphologyEx function.

[0181] Load the trained YOLOv5s model and input the preprocessed image into the model for cherry fruit detection.

[0182] By adding the Mask branch, the binary segmentation of the fruit area is realized, and the mask image of each cherry fruit is obtained.

[0183] Fruit image classification: Extract each cherry fruit image from the mask image and resize it to 224×224 pixels. Load the trained ResNet-18 model and input the processed fruit images into the model for classification. This yields a developmental stage label for each fruit, with each label corresponding to a separate Softmax classification output.

[0184] The classification results are superimposed on the original image, and the developmental stage of each cherry fruit is marked with different colors to produce a visual image.

[0185] The classification label data is uploaded to the agricultural management platform through the 4G communication module, allowing growers to understand the development of cherry fruits in real time.

[0186] The above process was repeated daily, starting from the 10th day after cherry flowering, and the cherry fruits were continuously monitored until maturity, and the developmental changes of the cherry fruits were recorded.

[0187] Example 3: Development monitoring in a small-scale cherry orchard

[0188] Implementation Background:

[0189] A small family cherry orchard, covering approximately one mu (approximately one acre), planted approximately 50 cherry trees. The orchard owner wanted real-time information on cherry fruit development to better manage farm activities, such as fertilizing, watering, and determining harvest times.

[0190] Implementation steps:

[0191] Equipment installation and image acquisition:

[0192] A high-definition camera is installed 2 meters above the cherry plants. The camera has high resolution and wide dynamic range characteristics, and is equipped with an adjustable fill light device connected to a light sensor.

[0193] Starting from the tenth day after cherry blossom bloom, RGB images with a resolution of 1920×1080 pixels were collected daily between 9:00 AM and 11:00 AM. When the light intensity detected by the light sensor fell below 5,000 lux, the supplemental lighting device activated and increased the light intensity to 8,000–10,000 lux. When the light intensity exceeded 12,000 lux, the supplemental lighting device was turned off.

[0194] Image preprocessing:

[0195] The collected images are processed in sequence: a 3×3 median filter window is used to perform median filtering denoising on the original image; the denoised image is enhanced using the histogram equalization method; the cherries are segmented from the background using a threshold segmentation algorithm based on color features. In the RGB color space, the threshold range is set to R∈[120,255], G∈[50,180], B∈[30,120], and a circular structuring element with a radius of 3 is used to perform morphological opening on the segmented image to remove noise and small connected areas; white balance adjustment, illumination compensation based on Retinex theory, and conversion from RGB color space to HSV color space are performed.

[0196] Fruit detection and segmentation:

[0197] Cherry fruits in the image are detected based on the YOLOv5s model. The YOLOv5s model uses CSPDarknet53 as the backbone network and PANet as the feature pyramid structure. The detection confidence threshold is set to 0.5, the non-maximum suppression (NMS) threshold is set to 0.4, and the mask branch is added to achieve binary segmentation of the fruit area.

[0198] Fruit classification:

[0199] The convolutional neural network built using the ResNet-18 architecture is used to classify the segmented fruit images. The input image size is 224×224 pixels, and the output results correspond to the four developmental stage labels of cherry fruits.

[0200] Result output and upload

[0201] The classification results are superimposed on the original image to export a visual image, which allows orchard owners to intuitively view the development of cherry fruits.

[0202] At the same time, the classification label data is uploaded to the agricultural management platform through the 4G communication module. The owner can log in to the platform through a mobile phone or computer to check the development data of the cherry fruit at any time.

[0203] Continuous monitoring:

[0204] The above process was repeated daily to continuously monitor the developmental changes of cherry fruits from the 10th day after flowering to maturity.

[0205] Implementation effect:

[0206] Through visual images and data from the agricultural management platform, orchard owners can clearly understand the developmental stages of cherry fruits and adjust farming management measures in a timely manner, thereby improving cherry yield and quality. For example, by applying fertilizer appropriately according to the developmental stage, the cherry fruits become fuller and more vibrant in color.

[0207] Example 4: Growth monitoring of a large cherry planting base

[0208] Implementation Background:

[0209] A large cherry planting base, covering 100 mu (approximately 1,000 acres) and housing approximately 5,000 cherry trees, required efficient and accurate growth monitoring of the large cherry orchard to enable scientific and intelligent management.

[0210] Implementation steps:

[0211] Equipment layout and image acquisition:

[0212] High-definition cameras are strategically placed throughout the cherry orchard, with one installed at regular intervals, three meters above the cherry plants, to ensure coverage of the entire planting area. The cameras also feature high resolution, wide dynamic range, and are equipped with adjustable fill lights connected to light sensors.

[0213] Starting from the tenth day after cherry blossoms bloomed, each camera simultaneously captured RGB images with a resolution of 1920×1080 pixels between 9:00 and 11:00 a.m. A light sensor monitored light intensity in real time. When light intensity fell below 5,000 lux, the fill light device activated and increased the light intensity to 8,000-10,000 lux. When light intensity exceeded 12,000 lux, the fill light device deactivated.

[0214] Image preprocessing and transmission:

[0215] The images captured by each camera are pre-processed locally, including 3×3 median filter window denoising, histogram equalization enhancement, color feature-based threshold segmentation, morphological opening operation, white balance adjustment, illumination compensation, and color space conversion operations.

[0216] The pre-processed images are transmitted to the data center at the base via a wired network.

[0217] Fruit detection and segmentation:

[0218] In the data center, high-performance servers are used to detect cherry fruits in images based on the YOLOv5s model. The YOLOv5s model uses CSPDarknet53 as the backbone network and PANet as the feature pyramid structure. The detection confidence threshold is set to 0.5, the non-maximum suppression (NMS) threshold is set to 0.4, and a mask branch is added to achieve binary segmentation of the fruit area.

[0219] Fruit classification:

[0220] The convolutional neural network built using the ResNet-18 architecture is used to classify the segmented fruit images. The input image size is 224×224 pixels, and the output results correspond to the four developmental stage labels of cherry fruits.

[0221] Result analysis and management:

[0222] The classification results are superimposed on the original image to export a visual image, which is then displayed on a large screen in the data center, allowing managers to intuitively understand the development of cherry fruits in each area.

[0223] The classified label data is then uploaded to the agricultural management platform via a 4G communication module. Managers can log in via mobile phone or computer and precisely manage cherry orchards in different areas. For example, they can rationally arrange picking personnel and equipment according to the growth stage to improve picking efficiency.

[0224] Continuous monitoring:

[0225] The above process was repeated daily to continuously monitor the developmental changes of cherry fruits from the 10th day after flowering to maturity.

[0226] Implementation effect:

[0227] Large-scale cherry plantations have achieved efficient management of their vast cherry orchards through this monitoring system. It can promptly identify differences in cherry fruit development in different areas, enabling targeted agricultural operations and improving overall production efficiency and economic benefits. For example, irrigation can be rationally arranged according to fruit development stages, conserving water while ensuring the growth needs of the cherries.

[0228] Test example:

[0229] Test example: Accuracy verification test of cherry development monitoring method based on image recognition

[0230] Experimental background:

[0231] This experiment was conducted to verify the accuracy and reliability of a cherry development monitoring method based on image recognition. By comparing it with traditional manual observation methods, the effectiveness of this monitoring method in practical applications was evaluated, providing a scientific basis for its promotion and application.

[0232] Test materials and equipment:

[0233] Experimental materials: A cherry orchard with an area of ​​about 5 mu was selected. The cherry trees planted in the orchard were of uniform varieties, about 5 years old, and in good growth condition.

[0234] Test equipment:

[0235] High-definition camera: installed 2-3 meters above the cherry plants, with high resolution and wide dynamic range characteristics, equipped with an adjustable fill light device connected to the light sensor.

[0236] 4G communication module: used to upload classified label data to the agricultural management platform.

[0237] Computer: used to run image preprocessing, fruit detection, and classification algorithm programs.

[0238] Test steps:

[0239] Image acquisition and manual observation are carried out simultaneously:

[0240] Starting from the tenth day after cherry blossoms bloomed, high-definition cameras were used to capture RGB images with a resolution of 1920×1080 pixels between 9:00 and 11:00 a.m. daily. Professional agricultural technicians also conducted manual observations of 100 randomly selected cherry trees within the orchard, recording the developmental stage of the cherry fruit on each tree.

[0241] The light sensor monitors the light intensity in real time. When the light intensity is lower than 5000 lux, the fill light device turns on and replenishes the light intensity to 8000-10000 lux; when the light intensity is higher than 12000 lux, the fill light device turns off.

[0242] Image preprocessing:

[0243] The collected images are processed in sequence: a 3×3 median filter window is used to perform median filtering denoising on the original image; the denoised image is enhanced using the histogram equalization method; the cherries are segmented from the background using a threshold segmentation algorithm based on color features. In the RGB color space, the threshold range is set to R∈[120,255], G∈[50,180], B∈[30,120], and a circular structuring element with a radius of 3 is used to perform morphological opening on the segmented image to remove noise and small connected areas; white balance adjustment, illumination compensation based on Retinex theory, and conversion from RGB color space to HSV color space are performed.

[0244] Fruit detection and segmentation:

[0245] Cherry fruits in the image are detected based on the YOLOv5s model. The YOLOv5s model uses CSPDarknet53 as the backbone network and PANet as the feature pyramid structure. The detection confidence threshold is set to 0.5, the non-maximum suppression (NMS) threshold is set to 0.4, and the mask branch is added to achieve binary segmentation of the fruit area.

[0246] Fruit classification:

[0247] The convolutional neural network built using the ResNet-18 architecture is used to classify the segmented fruit images. The input image size is 224×224 pixels, and the output results correspond to the four developmental stage labels of cherry fruits.

[0248] Results comparison and analysis:

[0249] The cherry fruit development stage labels obtained by image recognition were compared one by one with the results of manual observation records, and the consistency between the two was statistically analyzed.

[0250] Calculate the accuracy, recall, and F1 value indicators of the image recognition method to evaluate its performance. The calculation formula is as follows:

[0251] Accuracy = (number of correctly classified samples / total number of samples) × 100%

[0252] Recall rate = (number of correctly classified positive samples / actual number of positive samples) × 100%

[0253] F1 value = 2 × (precision × recall) / (precision + recall)

[0254] Continuous monitoring and data logging

[0255] Repeat the above process every day to continuously monitor the development and changes of cherry fruits from the 10th day after flowering to maturity, and record the daily comparison results and performance indicators.

[0256] Test results and conclusions:

[0257] Test results:

[0258] During the entire monitoring period, the image recognition method was highly consistent with manual observation results, with an average accuracy of over 90%, a recall rate of over 85%, and an F1 value of around 0.88.

[0259] As the cherry fruit develops, the image recognition accuracy decreases slightly at certain special stages (such as the period when the color changes rapidly when approaching maturity), but overall it remains at a high level.

[0260] in conclusion

[0261] The cherry development monitoring method based on image recognition has high accuracy and reliability, and can meet the monitoring needs in actual production.

[0262] This method can be used as an effective auxiliary means to help agricultural producers understand the development of cherry fruits in a timely and accurate manner, thereby improving the efficiency and scientific nature of agricultural management. The above embodiments are only used to illustrate the present invention and are not intended to limit the technical solutions described herein. Although this specification has described the present invention in detail with reference to the above embodiments, the present invention is not limited to the above specific implementation methods. Therefore, any modification or replacement of the present invention; and all technical solutions and improvements that do not depart from the spirit and scope of the invention are included in the scope of the claims of the present invention.

Claims

1. A cherry development monitoring method based on image recognition, characterized in that: The following steps are involved: Step 1: Image acquisition: Using a high-definition camera installed 2 to 3 meters above the cherry plants, RGB images with a resolution of 1920 × 1080 pixels were collected between 9:00 and 11:00 a.m. daily. Step 2: The collected images are subjected to Gaussian filtering for denoising, white balance adjustment, illumination compensation based on Retinex theory, and conversion from RGB color space to HSV color space. Step 3: Detect the cherry fruit in the image based on the YOLOv5s model, and implement binary segmentation of the fruit area by adding a Mask branch; Step 4: Use a convolutional neural network built with the ResNet-18 architecture to classify the segmented fruit images. The input image size is 224×224 pixels, and the output results correspond to the four developmental stage labels of cherry fruits. Step 5: Superimpose the classification results on the original image to export a visual image, and upload the classification label data to the agricultural management platform through the 4G communication module; Step 6: Repeat the above process every day to continuously monitor the development and changes of cherry fruits from the 10th day after flowering to the maturity stage.

2. A cherry development monitoring method based on image recognition according to claim 1, characterized in that, The image acquisition device has high resolution and wide dynamic range characteristics, and is equipped with an adjustable fill light device, which is connected to the light sensor. When the light intensity monitored by the light sensor is lower than 5000 lux, the fill light device turns on and supplements the light intensity to 8000-10000 lux; when the light intensity is higher than 12000 lux, the fill light device turns off.

3. The cherry development monitoring method based on image recognition according to claim 1, characterized in that: The collected images were subjected to Gaussian filtering denoising and white balance adjustment processing steps in sequence: a 3×3 median filtering window was used to perform median filtering denoising on the original image; the denoised image was enhanced using the histogram equalization method; the cherries were separated from the background using a threshold segmentation algorithm based on color features. In the RGB color space, the threshold range was set to R∈[120,255], G∈[50,180], B∈[30,120], and a circular structuring element with a radius of 3 was used to perform morphological opening operation on the segmented image to remove noise and small connected areas.

4. The cherry development monitoring method based on image recognition according to claim 1, characterized in that: The YOLOv5s model uses CSPDarknet53 as the backbone network, PANet as the feature pyramid structure, the detection confidence threshold is set to 0.5, and the non-maximum suppression (NMS) threshold is set to 0.

4.

5. The cherry development monitoring method based on image recognition according to claim 1, characterized in that: The cross entropy loss function is used in the ResNet-18 model training process. The initial value of the learning rate is 0.001, which decays every 30 training cycles. The momentum is set to 0.9 and the weight decay coefficient is 0.0005.

6. The cherry development monitoring method based on image recognition according to claim 1, characterized in that: The four developmental stage labels are: green in the young fruit stage, mixed red and green in the color change stage, mainly red in the early maturity stage, and dark red in the maturity stage. Each label corresponds to an independent Softmax classification output.

7. The cherry development monitoring method based on image recognition according to claim 1, characterized in that: The image acquisition device starts at a set time every day according to the set schedule. If the ambient light intensity is lower than 500 lux, the fill light is automatically enabled for auxiliary lighting.

8. The cherry development monitoring method based on image recognition according to claim 1, characterized in that: During the model training phase, the original image dataset was enhanced by random rotation of ±15°, horizontal flipping, brightness perturbation of ±20%, and contrast perturbation of ±0.2 times.

9. The cherry development monitoring method based on image recognition according to claim 1, characterized in that: The 4G communication module uses the MQTT protocol for data transmission, and the uploaded content includes fruit location coordinates, development stage labels, image timestamps, and device ID information.

10. The cherry development monitoring method based on image recognition according to any one of claims 1 to 9, characterized in that: The method is applicable to the development monitoring of fruits of other fruit trees. During implementation, the training data set of the target detection model is replaced, and the deployment height and angle of the image acquisition equipment are adjusted to adapt to different tree species structures.

Images