Watermelon maturity identification method and device and medium

Abstract

The invention belongs to the technical field of image processing, and discloses a watermelon maturity identification method and device and a medium, and the method comprises the steps: collecting watermelon image data which comprises watermelon images of different varieties, different maturity and different shooting angles; performing preprocessing of size adjustment, cutting and denoising on the image data to generate standardized image data; training the pre-processed image, extracting features of the watermelon, and establishing a watermelon identification model which can accurately extract feature information of watermelon variety and maturity based on input image data and output corresponding classification labels; and performing efficient classification and maturity evaluation in response to an input watermelon image according to the recognition model. According to the method, the maturity of the watermelon can be quickly and accurately judged from the image, and reliable technical support is provided for agricultural production and market circulation.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically, to a method, device and medium for identifying the maturity of a watermelon. Background Art

[0002] In the process of picking watermelons, workers usually rely on their experience to classify and judge the maturity of watermelons, which not only consumes a lot of manpower and material resources, but also because similar maturity is difficult to distinguish accurately, the classification process is often affected by strong subjective factors, resulting in too coarse classification. Therefore, how to conveniently, efficiently, objectively and accurately identify the maturity of watermelons has become a key technical problem that needs to be solved urgently.

[0003] In the prior art, for example, publication number CN117007552A discloses a method, device, system, electronic device and storage medium for detecting the maturity of watermelon, which relates to the field of agricultural technology. The method comprises: performing data preprocessing on the spectral data of the pulp of the watermelon to be detected based on the target data preprocessing method to obtain the original spectral data; performing feature screening on the original spectral data based on the target algorithm to determine the target characteristic band; after determining the spectral data with the target characteristic band in the original spectral data as the target spectral data, the target spectral data is input into the watermelon maturity detection model to obtain the maturity detection result of the watermelon to be detected output by the watermelon maturity detection model. The watermelon maturity detection method, device, system, electronic device and storage medium provided by the technology can detect the maturity of watermelon more accurately and efficiently, can realize batch non-destructive detection of watermelon maturity, and can better meet the needs of sorting watermelons in large watermelon planting bases.

[0004] In the above-mentioned public technology, the maturity of watermelon is recognized through spectrum, but it is impossible to recognize the maturity of watermelon based on image. The acquisition method of image is generally more convenient, low-cost and easy to operate than the acquisition of spectrum. For example, watermelon farmers can directly obtain watermelon images by taking pictures with their mobile phones. Therefore, it is particularly necessary to develop a technology for accurately identifying maturity based on watermelon images. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides a watermelon maturity recognition method, device and medium, which can quickly and accurately judge the maturity of watermelons from images, and provide assistance to farmers and pickers by judging whether the watermelons are ripe.

[0006] According to a first aspect of the present invention, a method for identifying watermelon maturity is provided, the method comprising: The image data is resized, cropped, and denoised to obtain a preprocessed image, specifically including: The input image is uniformly resized to the set pixel size, maintaining the aspect ratio and filling the missing parts; Normalize the adjusted image and scale the pixel values ​​to the [0,1] interval; The normalized image is converted into a color space from RGB to HSV space to better extract color features.

[0007] Performing data enhancement processing on the image after color space conversion; wherein the data enhancement processing includes random horizontal flipping, random rotation, random adjustment of brightness, contrast and saturation and / or Mosaic data enhancement; the angle range of the random rotation is [-15°, +15°], the range of the random adjustment of brightness, contrast and saturation is [0.8, 1.2], and the Mosaic data enhancement is to stitch multiple pictures into one; Denoising and enhancing the image after data enhancement processing; wherein the denoising and enhancement include using bilateral filtering to suppress noise, maintaining edge information, using adaptive histogram equalization to enhance image contrast, and applying median filtering to remove salt and pepper noise; Region extraction is performed on the denoised and enhanced image to obtain a preprocessed image; wherein the region extraction includes generating candidate regions using a selective search algorithm, performing non-maximum suppression on the candidate regions, and retaining target regions with confidence levels higher than a set threshold.

[0008] Furthermore, the recognition model includes: A backbone network, wherein the backbone network uses an SPPF module to enhance feature extraction capability and applies a C3 module to perform feature fusion; A feature pyramid network, wherein the feature pyramid network adopts an adaptive feature fusion module to achieve feature fusion; A detection head, wherein the detection head is connected to a classification branch, a regression branch, and a maturity branch; wherein the classification branch is used to output the probability of a watermelon variety category, the regression branch is used to predict the target bounding box coordinates, and the maturity branch is used to predict the maturity level of the watermelon; A loss function module, wherein the loss function module is used for training the recognition model.

[0009] Furthermore, the fusion process of the adaptive feature fusion module is expressed as:

[0010] in, represents the fusion feature, Represents different levels of features. is the learned attention weight, is the feature level number, is the number of feature levels; Furthermore, the loss function module is configured with an overall loss function; the overall loss function is composed of a weighted combination of variety classification loss, position regression loss and maturity loss, and is expressed as:

[0011] in, For the overall loss, , and are the weight coefficients for classification loss, regression loss and maturity loss respectively; , and They are classification loss, regression loss and maturity loss respectively; The classification loss is expressed as:

[0012] in, is the sample size, is the sample serial number, For the The true labels of samples, For the The predicted probability of a sample.

[0013] The regression loss is expressed as:

[0014] in, is the Euclidean distance, and are the predicted box and the true box respectively. is the minimum diagonal length of the outer frame, is the aspect ratio consistency parameter, is a positive trading parameter; The maturity loss is expressed as:

[0015] in, is the number of maturity levels.

[0016] Furthermore, the features of the watermelon contained in the preprocessed image are extracted, including: Extracting visual features; wherein the visual features include color features, texture features and shape features, the color features are obtained by extracting hue, saturation and brightness features of the HSV space, the texture features are obtained by extracting texture information using the LBP (local binary pattern) operator; the shape features are obtained by extracting geometric features of the perimeter, area and roundness of the target; Extract deep features; Extract semantic features: Use pre-trained models to extract high-level semantic features; construct feature vectors; apply feature selection algorithms to select the most discriminative features as semantic features.

[0017] Furthermore, deep features are extracted, including: Extract multi-level convolutional features through CSPDarknet; And use the attention mechanism to highlight important feature areas to obtain deep features. The calculation process is:

[0018] in, is the query matrix; is the key matrix; is the value matrix; is the key vector dimension; Further, outputting an identification label of the watermelon in the preprocessed image based on the feature of the watermelon includes: Feature fusion processing: weighted fusion of visual features, depth features and semantic features; adaptive weight allocation of different features using attention mechanism; and feature aggregation module to integrate multimodal information and obtain fused features; Classification prediction: classify varieties based on fusion features; use the Softmax function to calculate the probability of each category; output the variety category with the highest confidence; Maturity assessment: Establish a maturity level assessment indicator system;

[0019] in, Score the color feature; Score the texture features; Score shape features; , and are the weights of color features, texture features, and shape features respectively; The maturity is comprehensively assessed based on color and texture characteristics; Output maturity level; wherein the maturity level includes underripe, ripe and / or overripe.

[0020] According to a second aspect of the present invention, a device for identifying the maturity of a watermelon is provided, the device comprising: A data acquisition module is configured to acquire image data, wherein the image data includes images of watermelons of different varieties, different maturity levels, and different angles; An image preprocessing module is configured to perform size adjustment, cropping, and denoising operations on the image data to obtain a preprocessed image; A model training module is configured to train the preprocessed image, extract the features of the watermelon, and establish a recognition model, wherein the recognition model responds to the input preprocessed image and extracts the features of the watermelon contained in the preprocessed image, and outputs an identification label of the watermelon in the preprocessed image based on the features of the watermelon, wherein the identification label includes a watermelon variety label and a watermelon maturity label; The recognition and classification module is configured to perform recognition and classification in response to the input watermelon image according to the recognition model, and output the variety and / or maturity of the watermelon.

[0021] According to a third aspect of the present invention, there is provided a readable storage medium storing one or more programs, wherein the one or more programs can be executed by one or more processors to implement the method as described above.

[0022] The present invention has at least the following beneficial effects: The present invention is based on watermelon images. First, a series of preprocessing operations are performed to obtain a target area with high confidence as a preprocessing image to facilitate subsequent feature extraction. Then, during feature extraction, the attention mechanism is integrated and features at different levels, such as visual features, depth features, and semantic features, are integrated to obtain fused features. The overall loss is designed to measure the predicted loss to further ensure the accuracy of recognition. Therefore, the present invention can accurately identify the maturity of watermelons based on watermelon images, and has a wide range of application scenarios, such as assisting watermelon farmers in picking watermelons and helping consumers in selecting watermelons. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flow chart of a method for identifying the maturity of watermelon according to an embodiment of the present invention is shown.

[0024] Figure 2 A schematic diagram of the three-dimensional structure of a watermelon detection vehicle according to an embodiment of the present invention is shown.

[0025] Figure 3 A partial schematic diagram of a watermelon detection vehicle according to an embodiment of the present invention is shown.

[0026] Figure 4 A diagram showing the composition of detection components in a watermelon detection vehicle according to an embodiment of the present invention is shown.

[0027] Figure 5 A schematic diagram showing the connection relationship of an environmental monitoring module in a watermelon detection vehicle according to an embodiment of the present invention is shown.

[0028] Figure 6 A schematic diagram showing the connection relationship of a path planning module in a watermelon detection vehicle according to an embodiment of the present invention is shown.

[0029] Figure 7 A structural diagram of a recognition model according to an embodiment of the present invention is shown.

[0030] Figure 8 A network structure diagram of watermelon maturity detection according to an embodiment of the present invention is shown.

[0031] Fig. 9 A performance comparison chart of different classification methods according to an embodiment of the present invention on the watermelon maturity recognition task is shown.

[0032] Fig.10 A structural diagram of a watermelon maturity identification device according to an embodiment of the present invention is shown.

[0033] Fig.11 An example diagram of original sampling of watermelon according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0034] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings and specific embodiments, but are not intended to limit the present invention. For the various steps described herein, if there is no necessity for a causal relationship between each other, the order in which they are described as examples herein should not be regarded as a limitation, and those skilled in the art should know that they can be adjusted in order, as long as the logic between them is not destroyed, resulting in the inability to implement the entire process.

[0035] Figure 1 Flow chart of a method for identifying the maturity of a watermelon provided by an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps: S1. Acquire image data, where the image data includes watermelon images of different varieties, different maturity levels, and different angles.

[0036] In this embodiment, the image data can be images actually collected on site, or existing images can be selected. The purpose of the image data is to train the recognition model constructed in the subsequent steps, so as to complete the real-time watermelon maturity recognition through the trained recognition model, thereby ensuring that the watermelon can be picked at the best time.

[0037] For example, in this embodiment, the input watermelon image is a real-time image of the watermelon's actual growth state. Here, a watermelon detection vehicle capable of automatically collecting watermelon field images in real time is provided. Figure 2 and Figure 3As shown, the watermelon detection trolley includes a chassis 10, a walking part 20, a suspension part 30 and a detection part 40. Among them, the chassis 10 includes an upper support plate 11, a lower support plate 12 and an intermediate pillar 13, and the upper support plate 11 is connected to the lower support plate 12 through the intermediate pillar 13. The walking part 20 includes a moving wheel group 21. The moving wheel group 21 includes a first walking wheel 211, a second walking wheel 212, a central transmission sleeve 213 and a driving assembly 214, the first walking wheel 211 is connected to the second walking wheel 212 through the central transmission sleeve 213, and the driving assembly 214 is connected to the central transmission sleeve 213 to drive the first walking wheel 211 and the second walking wheel 212 to move through the central transmission sleeve 213. The suspension component 30 includes a spring suspension 31 and a nylon swing arm 32; wherein one end of the spring suspension 31 is connected to the upper support plate 11, and the other end of the spring suspension 31 is connected to the nylon swing arm 32, the outer end of the nylon swing arm 32 is hinged to the central transmission sleeve 213, and the inside of the nylon swing arm 32 is connected to the lower support plate 12. The detection component 40 includes a temperature and humidity sensor 41, a radar 42, and a visual component 43; wherein the temperature and humidity sensor is arranged on one side of the lower support plate 12, and the radar 42 and the visual component 43 are both arranged on the upper end of the upper support plate 11.

[0038] In this embodiment, the watermelon detection vehicle can adapt to the terrain and move forward steadily by setting the suspension component 30. When encountering an uneven road surface, the spring suspension 31 will be deformed and compressed, and in some cases it can quickly return to a normal state, adapt to different terrains, ensure the stability of the chassis 10, provide a stable environment for data collection of the detection component 40, and improve measurement accuracy.

[0039] It should be noted that the moving wheel set 21 included in the walking component 20 mentioned in this embodiment can be set to one or more, such as Figure 1As shown, it is set to two. When two moving wheel sets 21 are provided, their structures can be the same or different. If four-wheel drive is required, the structures of the two moving wheel sets 21 are the same. If rear-wheel drive or front-wheel drive is required, only the moving wheel set 21 arranged at the back or the front will be connected to the corresponding drive assembly 214. Exemplarily, the present embodiment adopts the front-wheel steering mode of front-wheel drive, and the drive assembly 214 may include a motor, a steering gear, a battery and a control panel, etc. The drive assembly 214 is installed on the lower support plate 12, that is, located in the space between the upper support plate 11 and the lower support plate 12. The battery supplies power to the motor, the steering gear and the control panel, and the trolley is equipped with a positioning module (such as GPS, Beidou positioning unit, etc.). In the case of existing positioning, the control panel can control the motor to provide travel force based on the set route, and control the offline adjustment direction so that the trolley can travel according to the set route. During the driving process, the suspension component 30 provides a stable detection environment, and the detection component realizes data collection, such as temperature and humidity parameters, image data, image data such as watermelon images, watermelon images, and watermelon planting environment images.

[0040] like Figure 2 As shown, the visual component 43 includes a fixed seat 431, a telescopic rod outer sleeve 432, a telescopic rod inner sleeve 433 and a camera 434. The telescopic rod outer sleeve 432 is set on the fixed seat 431, the telescopic rod inner sleeve 433 is movably connected in the telescopic rod outer sleeve 432, and the camera 434 is installed at the end of the telescopic rod inner sleeve 433.

[0041] In this embodiment, the fixing seat 431 serves to assemble the telescopic rod outer sleeve 432, and the telescopic rod inner sleeve 433 is movably connected to the telescopic rod outer sleeve 432, so that the camera 434 can adjust its shooting angle when the telescopic rod inner sleeve 433 moves up and down, and the data can be collected by adjusting the camera height and the camera collection range, and the collected data can be analyzed. For example, the maturity of crops can be monitored in real time to ensure that they are picked at the best time.

[0042] like Figure 4 As shown, the detection component 40 also includes an infrared sensor 44 and a soil sensor 45 , and the infrared sensor 44 and the soil sensor 45 are arranged on the lower support plate 12 .

[0043] In this embodiment, based on the existing image data, temperature and humidity data, the pH value and nutrient content of the soil can also be detected and analyzed. Intelligent collection device using sensor fusion. The car integrates multiple sensors, such as cameras, infrared sensors, soil moisture sensors, etc., and performs intelligent data analysis by fusing multi-source data to provide more comprehensive and accurate planting management suggestions.

[0044] Therefore, the watermelon detection car can combine the Internet of Things technology with agricultural machinery and equipment through the layout of the detection component 40 to realize the functions of automatic navigation, automated operation, data collection and analysis of agricultural machinery. For example, equipment such as intelligent planters and automated harvesters can improve the planting efficiency and production quality of crops. Big data technology can also be used to analyze and mine the massive data generated in agricultural production to provide a scientific basis for agricultural decision-making. Through data analysis, weather changes, market demand, outbreaks of pests and diseases, etc. can be predicted to help farmers formulate reasonable planting plans and sales strategies.

[0045] like Figure 5 As shown, the watermelon detection cart also includes an environmental monitoring module 50, which is signal-connected to the infrared sensor 44 and the soil sensor 45, and is used to determine the pH value and nutrient content of the soil based on the data fed by the infrared sensor 44 and the soil sensor 45, and adjust the fertilization plan based on the pH value and nutrient content of the soil.

[0046] Specifically, the pH value and nutrient thresholds of the soil are set. If the current soil pH value does not reach the set threshold, the fertilizers related to the soil pH value in the fertilization plan are adjusted. If the current soil pH value exceeds the set threshold, the use of fertilizers related to increasing the pH value in the fertilization plan is reduced. There are many types of nutrients. For example, taking nutrient A as an example, its set content threshold is a~b. If the current measured content of nutrient A is less than or equal to a, the use of fertilizers that can increase the nutrient A in the fertilization plan is increased. Conversely, if the current measured content of nutrient A is greater than or equal to b, the use of fertilizers that can increase the nutrient A in the fertilization plan is reduced.

[0047] Therefore, this embodiment monitors environmental factors such as soil, water quality, and air quality through sensor networks and remote sensing technology, timely discovers and solves environmental problems in agricultural production, and protects the health of farmland ecological environment. Soil detection and nutritional analysis: The car is equipped with soil sensors that can detect and analyze the pH value and nutritional components of the soil to help farmers adjust fertilization plans.

[0048] In some embodiments, Figure 6 As shown, the watermelon detection car also includes a path planning module 60, a signal output end of the path planning module 60 is connected to a signal input end of the driving component 214, and a signal output end of the radar 42 and the camera 434 is connected to a signal input end of the path planning module 60. The path planning module 60 is used to adaptively plan the path of the car according to the radar data and image data fed by the radar 42 and the camera 434 using a particle swarm algorithm, and control the driving component based on the adaptively planned path so that the car moves along the adaptively planned path.

[0049] It should be noted that the particle swarm algorithm is an existing algorithm, which can be configured in the path planning module 60 for implementation. The implementation process of the algorithm is not specifically described in this embodiment. This embodiment combines radar and vision so that when the car encounters obstacles during driving, it can avoid them at any time, reduce the risk of rollover, and ensure the effective data collection.

[0050] Through the setting of the path planning module 60, this embodiment enables the vehicle to have an adaptive operation scheduling system, which can perform intelligent operation task scheduling and path planning according to the actual situation of the farm and environmental changes, thereby improving operation efficiency and adaptability.

[0051] S2. Standardize, enhance, denoise and extract feature regions of the image data to obtain a preprocessed image.

[0052] In some embodiments, step S2 specifically includes: S21, image standardization processing, including: S211, uniformly resize the input image to 512x512 pixels, keep the aspect ratio, and fill the insufficient part; S212, normalize the image and scale the pixel values ​​to the interval [0,1]; S213, performing color space conversion from RGB to HSV space to better extract color features.

[0053] S22, data enhancement processing, including: S221, random horizontal flip, probability is 50%; S222, random rotation, angle range is [-15°, +15°]; S223, randomly adjust brightness, contrast and saturation in the range of [0.8, 1.2]; S224, Mosaic data enhancement, splicing 4 pictures into one to enhance training diversity.

[0054] S23, image denoising and enhancement, including: S231, using bilateral filtering to suppress noise while maintaining edge information; S232, using adaptive histogram equalization (CLAHE) to enhance image contrast; S233, applying median filtering to remove salt and pepper noise and further improve image quality.

[0055] S24, feature region extraction, including: S241, generating candidate regions using a selective search algorithm; S242, performing non-maximum suppression (NMS) on the candidate region, setting the threshold to 0.5, removing low confidence regions, and improving the accuracy of the candidate region; S3. Train the preprocessed images, extract the visual features, depth features and semantic features of watermelon, and establish a recognition model. The recognition model responds to the input preprocessed images and outputs watermelon variety and maturity labels.

[0056] In some embodiments, Figure 7 The figure shows the structure of the recognition model, which includes a backbone network, a feature pyramid network, a detection head, and a loss function module.

[0057] In this embodiment, the backbone network uses CSPDarknet to extract multi-level convolutional features; the SPPF (SpatialPyramid Pooling Fast) module is used to enhance the feature extraction capability; and the C3 module is applied to perform feature fusion to improve computational efficiency.

[0058] Feature Pyramid Network (FPN): Adopts adaptive feature fusion module (PANet) to achieve multi-scale feature fusion;

[0059] in Represents different levels of features. is the learned attention weight, is the number of feature levels.

[0060] The feature pyramid network is used to construct multi-scale feature maps, namely P3 (80×80), P4 (40×40) and P5 (20×20); jump connections are used to achieve the fusion of features at different levels.

[0061] The multi-task detection head (Detection Head) adopts a decoupled detection head design; it is connected separately: Classification branch: output the probability of watermelon variety category; Regression branch: predicts the target bounding box coordinates; Maturity branch: predict the maturity level of watermelon.

[0062] The loss function module is configured with an overall loss function; the overall loss function is composed of a weighted combination of variety classification loss, position regression loss, and maturity loss, and is expressed as:

[0063] in, For the overall loss, , and are the weight coefficients for classification loss, regression loss and maturity loss respectively; , and They are classification loss, regression loss and maturity loss respectively; The classification loss is expressed as:

[0064] in, is the sample size, is the sample serial number, For the The true labels of samples, For the The predicted probability of a sample.

[0065] The regression loss is expressed as:

[0066] in, is the Euclidean distance, and are the predicted box and the true box respectively. is the minimum diagonal length of the outer frame, is the aspect ratio consistency parameter, is a positive trading parameter; The maturity loss is expressed as:

[0067] in, is the number of maturity levels.

[0068] In some embodiments, Figure 8 As shown in the figure, it is the structure diagram of the watermelon maturity detection network. The watermelon maturity detection network is a part of the recognition model, including the backbone network, the feature pyramid network and the multi-task detection head. The function of the backbone network is to extract the features of the watermelon contained in the preprocessed image; and the function of the feature pyramid network is to fuse the features extracted by the backbone network and feed them to the multi-task detection head for category prediction.

[0069] Specifically, the steps of extracting the features of the watermelon contained in the preprocessed image based on the watermelon maturity detection network specifically include: S301, visual feature extraction: Color features: extract hue, saturation, and brightness features of the HSV space; Texture features: Use LBP (local binary pattern) operator to extract texture information; Shape features: Extract geometric features such as the circumference, area, and roundness of the target.

[0070] S302, deep feature extraction: Extract multi-level convolutional features through CSPDarknet; Use attention mechanism to highlight important feature areas;

[0071] in is the query matrix; is the key matrix; is the value matrix; is the key vector dimension; Multi-scale feature fusion is achieved through feature pyramid network.

[0072] S303, semantic feature extraction: Extract high-level semantic features using pre-trained models; Construct a feature vector containing appearance, shape, and texture information; Apply feature selection algorithms to select the most discriminative features.

[0073] In some embodiments, outputting an identification label of a watermelon in a preprocessed image based on the feature of the watermelon comprises the following steps: S310, feature fusion processing: weighted fusion of visual features, depth features and semantic features; use attention mechanism to adaptively assign weights to different features; use feature aggregation module to integrate multimodal information.

[0074] S320, classification prediction: classify varieties based on fusion features; use the Softmax function to calculate the probability of each category; take the value with the highest confidence as the variety category result.

[0075] S330, Maturity Assessment: Establish a maturity rating system based on color, texture and shape;

[0076] in, Score the color feature; Score the texture features; Score shape features; , and are the weights of color, texture and shape features respectively.

[0077] The maturity is comprehensively assessed based on color, texture and other characteristics; Output the maturity level of watermelon (e.g. unripe, moderately ripe, overripe).

[0078] S340, label generation and optimization: Integrate the variety and maturity results output by the multi-task detection head; apply post-processing algorithms to optimize predictions and generate standardized labels.

[0079] S4. According to the recognition model, recognition and classification are performed in response to the input watermelon image, and the variety and / or maturity of the watermelon are output.

[0080] In this embodiment, the recognition model can be a model obtained after training with a watermelon image data set. The process of the recognition model recognizing and classifying watermelon images has been described in detail in the embodiment in step S3 above, so it will not be repeated here.

[0081] This embodiment has made a systematic plan for the collection and processing of watermelon image data. First, in the watermelon image collection, 16 varieties including Luhongmi 520, Sumeng 301-106, Sweet No. 6, Lucai Mi 131, Sumeng No. 10, Xiaotailan, Meixin No. 2, Sumeng No. 12, Sumeng 206-003, Baoyu No. 1, Youmeng w-103, Sumeng No. 6, Lanhui, Xiuli No. 3, SP100 Huangtianyu, and Sumeng 311-306 were selected, and each variety was grouped according to three maturity levels: unripe, ripe, and overripe. Each maturity level was photographed from five different angles: front, left side, right side, top, and bottom, and numbered next to them (example figure as shown in the figure). Fig.11 ), an average of no less than 300 images were acquired for each variety and at each angle, thus covering the appearance details of each maturity stage.

[0082] In the data processing and augmentation stage, the present invention unifies all images to a resolution of 512×512 pixels, and adopts standardized naming and labeling methods (such as "SuMeng_Unripe_Front_01.jpg") for classification and retrieval. At the same time, in order to improve the robustness and generalization ability of the model, the present invention adopts a variety of data enhancement methods such as rotation, flipping, cropping, illumination change and color perturbation, so that the total amount of data increases by about 1.5-2 times on the original basis, and effectively reduces the model overfitting caused by insufficient sample distribution. Through the above-mentioned multi-variety, multi-maturity, multi-angle acquisition methods, as well as data enhancement methods, a data set of more than 12,000 original images was finally formed. After this systematic acquisition and processing, the constructed watermelon image data set can meet the needs of subsequent recognition, classification and quality assessment in terms of scale and diversity, and provides high-quality and traceable data support for related intelligent algorithms.

[0083] Different classification methods show obvious differences in the task of watermelon maturity recognition. Fig. 9As shown in the figure, the accuracy of traditional machine learning methods such as Random Forest, K-nearest neighbor (KNN) and support vector machine (SVM) are all around 45%, specifically 44.76%, 45.65% and 45.60%, which are significantly lower than the level that deep learning models can achieve. In contrast, the recognition accuracy of the three convolutional neural network methods, CNN (shallow convolutional network), ResNet (residual network) and Ours (the method proposed in this embodiment), are 89.47%, 87.34% and 93.03% respectively, which can more effectively capture the complex features of watermelon images. Among them, the network model of this patent method performs best, with an accuracy rate of over 90%. This shows that deep learning methods have higher generalization ability and recognition accuracy when identifying watermelon images of different maturity levels.

[0084] The embodiment of the present invention also provides a device for identifying the maturity of watermelon. Fig.10 As shown, the watermelon maturity identification device comprises: The data acquisition module 101 is configured to acquire image data, wherein the image data includes images of watermelons of different varieties, different maturity levels, and different angles; The image preprocessing module 102 is configured to perform size adjustment, cropping, and denoising operations on the image data to obtain a preprocessed image; The model training module 103 is configured to train the preprocessed image, extract the features of the watermelon, and establish a recognition model, wherein the recognition model responds to the input preprocessed image and extracts the features of the watermelon contained in the preprocessed image, and outputs the recognition label of the watermelon in the preprocessed image based on the features of the watermelon, wherein the recognition label includes a watermelon variety label and a watermelon maturity label; The recognition and classification module 104 is configured to perform recognition and classification in response to the input watermelon image according to the recognition model, and output the variety and / or maturity of the watermelon.

[0085] In some embodiments, the image preprocessing module is further configured to: The input image is uniformly resized to the set pixel size, maintaining the aspect ratio and filling the missing parts; Normalize the adjusted image and scale the pixel values ​​to the [0,1] interval; The normalized image is converted into a color space from RGB to HSV space to better extract color features.

[0086] Performing data enhancement processing on the image after color space conversion; wherein the data enhancement processing includes random horizontal flipping, random rotation, random adjustment of brightness, contrast and saturation and / or Mosaic data enhancement; the angle range of the random rotation is [-15°, +15°], the range of the random adjustment of brightness, contrast and saturation is [0.8, 1.2], and the Mosaic data enhancement is to stitch multiple pictures into one; Denoising and enhancing the image after data enhancement processing; wherein the denoising and enhancement include using bilateral filtering to suppress noise, maintaining edge information, using adaptive histogram equalization to enhance image contrast, and applying median filtering to remove salt and pepper noise; Region extraction is performed on the denoised and enhanced image to obtain a preprocessed image; wherein the region extraction includes generating candidate regions using a selective search algorithm, performing non-maximum suppression on the candidate regions, and retaining target regions with confidence levels higher than a set threshold.

[0087] In some embodiments, the recognition model includes: A backbone network, wherein the backbone network uses an SPPF module to enhance feature extraction capability and applies a C3 module to perform feature fusion; A feature pyramid network, wherein the feature pyramid network adopts an adaptive feature fusion module to achieve feature fusion; A detection head, wherein the detection head is connected to a classification branch, a regression branch, and a maturity branch; wherein the classification branch is used to output the probability of a watermelon variety category, the regression branch is used to predict the target bounding box coordinates, and the maturity branch is used to predict the maturity level of the watermelon; A loss function module, wherein the loss function module is used for training the recognition model.

[0088] In some embodiments, the fusion process of the adaptive feature fusion module is expressed as:

[0089] in, represents the fusion feature, Represents different levels of features. is the learned attention weight, is the feature level number, is the number of feature levels; In some embodiments, the loss function module is configured with an overall loss function; the overall loss function is composed of a weighted combination of a variety classification loss, a position regression loss, and a maturity loss, and is expressed as:

[0090] in, For the overall loss, , and are the weight coefficients for classification loss, regression loss and maturity loss respectively; , and They are classification loss, regression loss and maturity loss respectively; The classification loss is expressed as:

[0091] in, is the sample size, is the sample serial number, For the The true labels of samples, For the The predicted probability of a sample.

[0092] The regression loss is expressed as:

[0093] in, is the Euclidean distance, and are the predicted box and the true box respectively. is the minimum diagonal length of the outer frame, is the aspect ratio consistency parameter, is a positive trading parameter; The maturity loss is expressed as:

[0094] in, is the number of maturity levels.

[0095] In some embodiments, the model training module is further configured to: Extract visual features; wherein the visual features include color features, texture features and shape features, the color features are obtained by extracting the hue, saturation and brightness features of the HSV space, the texture features are obtained by extracting texture information using the LBP (local binary pattern) operator; the shape features are obtained by extracting the geometric features of the perimeter, area and roundness of the target Extract deep features; Extract semantic features: Use pre-trained models to extract high-level semantic features; construct feature vectors; apply feature selection algorithms to select the most discriminative features as semantic features.

[0096] In some embodiments, the model training module is further configured to: Extract multi-level convolutional features through CSPDarknet; And use the attention mechanism to highlight important feature areas to obtain deep features. The calculation process is:

[0097] in, is the query matrix; is the key matrix; is the value matrix; is the key vector dimension; In some embodiments, the model training module is further configured to: Feature fusion processing: weighted fusion of visual features, depth features and semantic features; adaptive weight allocation of different features using attention mechanism; and feature aggregation module to integrate multimodal information and obtain fused features; Classification prediction: classify varieties based on fusion features; use the Softmax function to calculate the probability of each category; output the variety category with the highest confidence; Maturity assessment: Establish a maturity level assessment indicator system;

[0098] in, Score the color feature; Score the texture features; Score shape features; , and are the weights of color features, texture features, and shape features respectively; The maturity is comprehensively assessed based on color and texture characteristics; Output maturity level; wherein the maturity level includes underripe, ripe and / or overripe.

[0099] It should be noted that the various device structures described in this embodiment belong to the same technical concept as the previously described method, and achieve the same technical effect through the same principle, which will not be repeated here.

[0100] An embodiment of the present invention further provides a readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the method described in any of the above embodiments.

[0101] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present invention with equivalent elements, modifications, omissions, combinations (e.g., various embodiments intersecting schemes), adaptations or changes. The elements in the claims are to be interpreted broadly based on the language adopted in the claims, and are not limited to the examples described in this specification or during the prosecution of this application, which examples are to be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, and the true scope and spirit are indicated by the following claims and the full scope of their equivalents.

[0102] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more of them) may be used in combination with each other. For example, a person of ordinary skill in the art may use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features may be grouped together to simplify the present invention. This should not be interpreted as an intention that a feature of an invention that is not claimed for protection is necessary for any claim. On the contrary, the subject matter of the present invention may be less than all the features of an embodiment of a particular invention. Thus, the following claims are incorporated herein as examples or embodiments in the specific embodiments, wherein each claim is independently a separate embodiment, and it is considered that these embodiments may be combined with each other in various combinations or arrangements. The scope of the present invention should be determined with reference to the appended claims and the full scope of equivalent forms granted by these claims.

Claims

1. A method for identifying watermelon maturity, characterized in that: The method comprises: Acquire image data, wherein the image data includes images of watermelons of different varieties, different maturity levels, and different angles; Performing size adjustment, cropping, and denoising operations on the image data to obtain a preprocessed image; Training the preprocessed image, extracting the features of the watermelon, and establishing a recognition model, wherein the recognition model responds to the input preprocessed image and extracts the features of the watermelon contained in the preprocessed image, and outputs an identification label of the watermelon in the preprocessed image based on the features of the watermelon, wherein the identification label includes a watermelon variety label and a watermelon maturity label; According to the recognition model, recognition and classification are performed in response to the input watermelon image, and the variety and / or maturity of the watermelon are output.

2. The method according to claim 1, characterized in that The image data is resized, cropped, and denoised to obtain a preprocessed image, specifically including: The input image is uniformly resized to the set pixel size, maintaining the aspect ratio and filling the missing parts; Normalize the adjusted image and scale the pixel values ​​to the [0,1] interval; Perform color space conversion on the normalized image, from RGB to HSV space to better extract color features; Performing data enhancement processing on the image after color space conversion; wherein the data enhancement processing includes random horizontal flipping, random rotation, random adjustment of brightness, contrast and saturation and / or Mosaic data enhancement; the angle range of the random rotation is [-15°, +15°], the range of the random adjustment of brightness, contrast and saturation is [0.8, 1.2], and the Mosaic data enhancement is to stitch multiple pictures into one; Denoising and enhancing the image after data enhancement processing; wherein the denoising and enhancement include using bilateral filtering to suppress noise, maintaining edge information, using adaptive histogram equalization to enhance image contrast, and applying median filtering to remove salt and pepper noise; Region extraction is performed on the denoised and enhanced image to obtain a preprocessed image; wherein the region extraction includes generating candidate regions using a selective search algorithm, performing non-maximum suppression on the candidate regions, and retaining target regions with confidence levels higher than a set threshold.

3. The method according to claim 1, characterized in that: The recognition model includes: A backbone network, wherein the backbone network uses an SPPF module to enhance feature extraction capability and applies a C3 module to perform feature fusion; A feature pyramid network, wherein the feature pyramid network adopts an adaptive feature fusion module to achieve feature fusion; A detection head, wherein the detection head is connected to a classification branch, a regression branch, and a maturity branch; wherein the classification branch is used to output the probability of a watermelon variety category, the regression branch is used to predict the target bounding box coordinates, and the maturity branch is used to predict the maturity level of the watermelon; A loss function module, wherein the loss function module is used for training the recognition model.

4. The method according to claim 3, characterized in that The fusion process of the adaptive feature fusion module is expressed as: ； in, represents the fusion feature, Represents different levels of features. is the learned attention weight, is the feature level number, is the number of feature levels.

5. The method according to claim 3, characterized in that: The loss function module is configured with an overall loss function; the overall loss function is composed of a weighted combination of variety classification loss, position regression loss, and maturity loss, and is expressed as: ； in, For the overall loss, , and are the weight coefficients for classification loss, regression loss and maturity loss respectively; , and They are classification loss, regression loss and maturity loss respectively; The classification loss is expressed as: ； in, is the sample size, is the sample serial number, For the The true labels of samples, For the The predicted probability of samples; The regression loss is expressed as: ； in, is the Euclidean distance, and are the predicted box and the true box respectively. is the minimum diagonal length of the outer frame, is the aspect ratio consistency parameter, is a positive trading parameter; The maturity loss is expressed as: ； in, is the number of maturity levels.

6. The method according to claim 3, characterized in that Extract the features of the watermelon contained in the preprocessed image, including: Extract visual features; wherein the visual features include color features, texture features and shape features, the color features are obtained by extracting the hue, saturation and brightness features of the HSV space, the texture features are obtained by extracting texture information using the LBP (local binary pattern) operator; the shape features are obtained by extracting the geometric features of the perimeter, area and roundness of the target Extract deep features; Extract semantic features: Use pre-trained models to extract high-level semantic features; construct feature vectors; apply feature selection algorithms to select the most discriminative features as semantic features.

7. The method according to claim 6, characterized in that Extract deep features, including: Extract multi-level convolutional features through CSPDarknet; And use the attention mechanism to highlight important feature areas to obtain deep features. The calculation process is: ； in, is the query matrix; is the key matrix; is the value matrix; is the key vector dimension.

8. The method according to claim 1, characterized in that Outputting an identification label of a watermelon in a preprocessed image based on the feature of the watermelon includes: Feature fusion processing: weighted fusion of visual features, depth features and semantic features; adaptive weight allocation of different features using attention mechanism; and feature aggregation module to integrate multimodal information and obtain fused features; Classification prediction: classify varieties based on fusion features; use the Softmax function to calculate the probability of each category; output the variety category with the highest confidence; Maturity assessment: Establish a maturity level assessment indicator system; ； in, Score the color feature; Score the texture features; Score shape features; , and are the weights of color features, texture features, and shape features respectively; The maturity is comprehensively assessed based on color and texture characteristics; Output maturity level; wherein the maturity level includes underripe, ripe and / or overripe.

9. A watermelon maturity identification device, characterized in that: The device comprises: A data acquisition module is configured to acquire image data, wherein the image data includes images of watermelons of different varieties, different maturity levels, and different angles; An image preprocessing module is configured to perform size adjustment, cropping, and denoising operations on the image data to obtain a preprocessed image; A model training module is configured to train the preprocessed image, extract the features of the watermelon, and establish a recognition model, wherein the recognition model responds to the input preprocessed image and extracts the features of the watermelon contained in the preprocessed image, and outputs an identification label of the watermelon in the preprocessed image based on the features of the watermelon, wherein the identification label includes a watermelon variety label and a watermelon maturity label; The recognition and classification module is configured to perform recognition and classification in response to the input watermelon image according to the recognition model, and output the variety and / or maturity of the watermelon.

10. A non-transitory computer-readable storage medium storing instructions, characterized in that: When the instructions are executed by a processor, the method according to any one of claims 1 to 8 is performed.

Images