Apple fruit image recognition method under conditions of multi-fruit overlapping and existence of complex sheltering objects

Abstract

The invention provides an apple fruit image recognition method, system and device under the conditions of multi-fruit overlapping and existence of complex sheltering objects and a storage medium, and the method comprises the steps: carrying out the image collection of an apple tree region, and forming at least two apple sub-regions; preprocessing the collected image, wherein the preprocessing comprises graying, filtering and edge detection; performing feature extraction on each apple sub-region based on an image feature extraction algorithm; based on the feature information and a machine learning algorithm, performing classification identification on each apple sub-region to obtain an identification result of the apple fruits; and combining the identification results of all the sub-regions through an image fusion technology to obtain an identification result of the whole apple tree region. According to the invention, an efficient and accurate apple fruit image recognition method can be realized, and the agricultural automation and intelligence level can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and more specifically, to an apple fruit image recognition method, system, medium and computing device in the presence of multiple overlapping fruits and complex obstructions. Background Art

[0002] In existing image recognition technologies, especially in the process of identifying apple fruits in the agricultural field, problems such as multiple fruit overlap and complex occlusions are often encountered, which poses a challenge to the accurate identification of fruits. Traditional image recognition methods often rely on simple image processing technologies, which are difficult to cope with complex practical scenes. In technical principles, these methods may only perform basic image acquisition and preprocessing, lack in-depth feature extraction and intelligent classification and recognition algorithms, resulting in low recognition accuracy in practical applications. In the process of implementing the embodiments of the present invention, the inventors found that there are at least the following problems or defects in the prior art: the inability to effectively handle the problems of multiple fruit overlap and occlusion, resulting in insufficient recognition accuracy; the lack of in-depth analysis and utilization of image features limits the performance improvement of the recognition algorithm. Summary of the invention

[0003] In a first aspect of the present invention, a method for recognizing apple fruit images in the presence of multiple overlapping fruits and complex obstructions is provided, comprising:

[0004] Collecting images of the apple tree area to form at least two apple sub-areas;

[0005] Preprocess the acquired images, including grayscale, filtering, and edge detection;

[0006] Extract features from each apple sub-region based on the image feature extraction algorithm;

[0007] Classify and identify each apple sub-region based on feature information and machine learning algorithms to obtain the recognition result of apple fruit;

[0008] The recognition results of all sub-areas are combined through image fusion technology to obtain the recognition result of the entire apple tree area.

[0009] Furthermore, the collected images are preprocessed, specifically including:

[0010] Use the grayscale formula

[0011] I gray (i,j)=0.299·I original (i,j) R +0.587·I original (i,j) G +0.114·I original (i,j)B

[0012] For each pixel (i, j) the original RGB image I original To convert, I original (i,j) R ,I original (i,j) G ,I original (i,j) B Respectively represent the red, green, and blue channel values ​​of the pixel.

[0013] Use the filter formula

[0014]

[0015] Filter the grayscale image, where F(m,n) is the coefficient of the filter kernel and k is the filter kernel size.

[0016] The Canny edge detection algorithm is used to calculate the edge strength and direction, and finally the binary edge image I is obtained. edges (i,j), where I edges (i, j) is the pixel value of the edge image at point (i, j), and the value is 0 or 255 to represent non-edge and edge.

[0017] Furthermore, feature extraction is performed on each apple sub-region based on an image feature extraction algorithm, specifically including:

[0018] Using feature extraction formula

[0019] Features=Fe(I edges )={HoG,Color,Texture}

[0020] Extract the image after edge detection I edges features, where Fe is the feature extraction function, HoG represents the histogram of oriented gradients, Color represents the color features, and Texture represents the texture features.

[0021] Furthermore, each apple sub-region is classified and identified based on feature information and machine learning algorithms, including:

[0022] Use feature vector Features = [f1,f2,...,f M ], where f1,f2,...,f M Represents the M features extracted.

[0023] Input the feature vector into the classifier and use the classification formula

[0024] C=ML(Features)=arg Features)

[0025] Classification is performed, where ML is the machine learning classification function, C is the set of possible categories, and P(c|Features) is the posterior probability of category c under given feature conditions.

[0026] Furthermore, the recognition results of all sub-areas are combined through image fusion technology, including:

[0027] Use I final =∪C i The classification results of all sub-regions C i To perform fusion, ∪ is a set union operation.

[0028] Furthermore, a deep learning algorithm is used to identify apple fruit images, specifically including:

[0029] Define the network structure of the deep learning model, including the input layer, multiple hidden layers, and the output layer;

[0030] Using loss function

[0031]

[0032] Calculate the prediction result P i and the true label Y i The mean square error loss between , where N is the number of samples;

[0033] The network weights W and bias b are updated through the back-propagation algorithm to optimize the model performance.

[0034] Furthermore, the deep learning model is characterized by the following formula:

[0035] O (l) =f (l) (W (l) ·O (l-1) +b (l) )

[0036] Among them, O (l) is the output of the lth layer, W (l) is the weight matrix from layer l-1 to layer l, b (l) is the bias term of the lth layer, f (l) is the activation function of the lth layer.

[0037] In a second aspect of the present invention, there is provided an apple fruit image recognition system in the case of multiple fruits overlapping and complex obstructions, comprising:

[0038] An image acquisition module, used for acquiring images of the apple tree area;

[0039] A preprocessing module is used to grayscale, filter and detect edges of the acquired images;

[0040] A feature extraction module, used for extracting features from each apple sub-region based on an image feature extraction algorithm;

[0041] A classification and recognition module, used to classify and recognize each apple sub-region based on feature information and machine learning algorithms;

[0042] The image fusion module is used to merge the recognition results of all sub-areas to obtain the recognition result of the entire apple tree area.

[0043] In a third aspect of the present invention, an electronic device is provided, comprising: at least one processor, a memory and an input-output unit; wherein the memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute any one of the methods described in the first aspect.

[0044] In a fourth aspect of the present invention, a computer-readable storage medium is provided, which includes instructions, and when the instructions are executed on a computer, the computer executes any one of the methods in the first aspect.

[0045] The above-mentioned embodiments of the present invention have at least the following beneficial effects: by adopting advanced image processing technology, the present invention can effectively perform preprocessing operations such as graying, filtering and edge detection on the collected images, thereby improving the image quality and laying a solid foundation for subsequent feature extraction and recognition. In addition, by applying feature extraction algorithms, key information such as directional gradient histograms, color features and texture features can be accurately extracted from the preprocessed images, and these features provide an important basis for the classification and recognition of apple fruits. Furthermore, by using machine learning algorithms to perform intelligent classification and recognition of feature information, the accuracy and efficiency of recognition can be significantly improved. Finally, by effectively integrating the recognition results of all sub-areas through image fusion technology, accurate recognition results of the entire apple tree area can be obtained, thereby providing strong technical support for agricultural automation and intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, in which:

[0047] Figure 1 A schematic diagram of a flow chart of an apple fruit image recognition method in the presence of multiple overlapping fruits and complex obstructions provided by an embodiment of the present invention;

[0048] Figure 2A schematic diagram of the structure of an apple fruit image recognition system in the case of multiple fruits overlapping and complex obstructions provided by an embodiment of the present invention;

[0049] Figure 3 The schematic diagram schematically shows the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.

[0051] Those skilled in the art know that the embodiments of the present invention can be implemented as a system, device, apparatus, method or computer program product. Therefore, the present invention can be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0052] It should be noted that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction and does not have any limiting meaning.

[0053] Reference below Figure 1 , Figure 1 The figure is a flow chart of a method for recognizing apple fruit images in the presence of multiple overlapping fruits and complex occlusions provided by an embodiment of the present invention. Figure 1 As shown, a method 100 for recognizing apple fruit images in the presence of multiple overlapping fruits and complex occlusions includes:

[0054] Step 101, collecting images of an apple tree region to form at least two apple sub-regions;

[0055] Step 102, preprocessing the collected image, including grayscale, filtering and edge detection;

[0056] Step 103, extracting features from each apple sub-region based on an image feature extraction algorithm;

[0057] Step 104, classifying and identifying each apple sub-region based on the feature information and the machine learning algorithm to obtain an identification result of the apple fruit;

[0058] Step 105, combining the recognition results of all sub-areas by image fusion technology to obtain the recognition result of the entire apple tree area.

[0059] It should be noted that the implementation of the present invention first includes image acquisition of the apple tree area, which is usually performed using a high-resolution camera at different angles and lighting conditions. The preprocessing step involves converting the acquired color image into a grayscale image using a specific grayscale formula, such as using coefficients of 0.299, 0.587 and 0.114 corresponding to the weighted sum of the red, green and blue channels, respectively. The filtering step uses an adjustable filter kernel to smooth the image and reduce noise. Edge detection uses the Canny algorithm to determine edge strength and direction by setting an appropriate threshold.

[0060] Preferably, image acquisition can be further optimized by using a camera with autofocus and exposure control to ensure clear images under different lighting conditions. In the preprocessing stage, in addition to grayscale, adaptive thresholding can also be considered to enhance the contrast of the image. The size of the filter kernel can be adjusted according to the specific situation of the image, such as using a larger filter kernel when the image is blurry. In edge detection, the edge information can be captured more accurately by dynamically adjusting the high and low thresholds of the Canny algorithm. In addition, in the feature extraction step, in addition to the directional gradient histogram, color features, and texture features, shape features and size features can also be considered to be added to further improve the accuracy of recognition.

[0061] In some embodiments, the collected image is preprocessed, specifically including:

[0062] Use the grayscale formula

[0063] I gray (i,j)=0.299·I original (i,j) R +0.587·I original (i,j) G +0.114·I original (i,j) B

[0064] For each pixel (i, j) the original RGB image I original To convert, I original (i,j) R ,I original (i,j) G ,I original (i,j) B Respectively represent the red, green, and blue channel values ​​of the pixel.

[0065] Use the filter formula

[0066]

[0067] Filter the grayscale image, where F(m,n) is the coefficient of the filter kernel and k is the filter kernel size.

[0068] The Canny edge detection algorithm is used to calculate the edge strength and direction, and finally the binary edge image I is obtained. edges (i,j), where I edges (i, j) is the pixel value of the edge image at point (i, j), and the value is 0 or 255 to represent non-edge and edge.

[0069] It should be noted that image preprocessing includes three main steps: grayscale, filtering and edge detection. Grayscale is to convert the RGB image into a grayscale image using the formula

[0070] I gray (x,y)=0.299·I red (x,y)+0.587·I green (x,y)+0.114·I blue (x,y)

[0071] Among them I gray ,I red ,I green and I blue Represents the grayscale value and RGB channel value of the pixel respectively. Filtering uses the convolution kernel to process the grayscale image to eliminate noise. The coefficient h(i,j) and size of the convolution kernel can be selected according to the specific content of the image. Edge detection uses the Canny algorithm to determine the edge in the image by calculating the amplitude and direction of the gradient, and finally generates a binary edge image.

[0072] Preferably, the grayscale process can adopt more advanced methods, such as adaptive grayscale, to dynamically adjust the weight coefficients of each channel according to the image content. In the filtering stage, the size of the convolution kernel can be adjusted according to the resolution and noise level of the image to achieve the best denoising effect. For the Canny edge detection algorithm, the sensitivity of the edge can be controlled by adjusting the high threshold and the low threshold to adapt to scenes of different complexity. In addition, after edge detection, morphological operations such as dilation or erosion can be used to further optimize the continuity and integrity of the edge.

[0073] In some embodiments, feature extraction is performed on each apple sub-region based on an image feature extraction algorithm, specifically including:

[0074] Using feature extraction formula

[0075] Features=Fe(I edges )={HoG,Color,Texture}

[0076] Extract the image after edge detection Iedges features, where Fe is the feature extraction function, HoG represents the histogram of oriented gradients, Color represents the color features, and Texture represents the texture features.

[0077] It should be noted that the embodiment of the present invention elaborates in detail the process of extracting features from each apple sub-region based on an image feature extraction algorithm, which is a key step in the recognition system and directly affects the accuracy of subsequent classification and recognition.

[0078] Specifically, feature extraction includes the extraction of oriented gradient histogram (HOG), color features, and texture features. The oriented gradient histogram is a method for describing the local shape features of an image by calculating the gradient direction and magnitude of each pixel in the image, and then quantizing this information into different directional histograms. Color features can be based on statistical information in color space (such as RGB, HSV), such as the mean and variance of the color. Texture features describe the roughness or repetitive patterns of the image area and can be extracted through methods such as the gray level co-occurrence matrix (GLCM). Feature extraction formula

[0079] Features=Fe(I edges )={HoG,Color,Texture}

[0080] In the above example, Fe represents the feature extraction function, I edges is the image after edge detection.

[0081] Preferably, the feature extraction process can be further refined or alternatives can be provided. For example, for the histogram of oriented gradients, different numbers of quantization directions and bin sizes of the histogram can be selected to accommodate different image features. The extraction of color features can be extended to include shape descriptors of color correlation or color distribution. In addition to the grayscale co-occurrence matrix, the extraction of texture features can also consider the use of methods such as local binary patterns (LBP) or Gabor filters. In addition, multi-scale analysis can be introduced in the feature extraction process, that is, the feature extraction steps are repeated at different image resolutions to capture feature information at different scales. Feature selection methods can also be used to reduce feature dimensions, improve classifier performance and reduce computational complexity.

[0082] In some embodiments, each apple sub-region is classified and identified based on feature information and a machine learning algorithm, specifically including:

[0083] Use feature vector Features = [f1,f2,...,f M ], where f1,f2,...,f M Represents the M features extracted.

[0084] Input the feature vector into the classifier and use the classification formula

[0085] C=ML(Features)=arg Features

[0086] Classification is performed, where ML is the machine learning classification function, C is the set of possible categories, and P(c|Features) is the posterior probability of category c under given feature conditions.

[0087] It should be noted that the embodiments of the present invention describe in detail how to classify and identify each apple sub-region based on feature information and machine learning algorithms. This step is the core link for achieving high-precision apple fruit recognition.

[0088] Specifically, the classification and recognition process first involves using the feature vector Features = [f1, f2, ..., f M ], where f1,f2,...,f M Represents the M features extracted. These feature vectors are then input into the classifier. The design of the classifier can be based on a variety of machine learning algorithms, such as support vector machine (SVM), random forest, neural network, etc. The classification formula C = ML (Features) = arg Features is used to determine the most likely category C under given feature conditions, where P (c | Features) is the posterior probability, which represents the probability of category c under the condition of the feature vector Features.

[0089] Preferably, the classification and recognition process can be further refined. For example, the selection of feature vectors can be optimized by feature importance evaluation to exclude redundant or irrelevant features, thereby improving the accuracy and efficiency of classification. The choice of classifier can be determined according to the specific application scenario and the characteristics of the data set, and it may be necessary to evaluate the performance of different algorithms through methods such as cross-validation. In addition, the calculation of the posterior probability can be implemented using a Bayesian method or through a softmax function in a neural network. In practical applications, it is also possible to consider using ensemble learning methods such as random forests or gradient boosted decision trees (GBDT) to improve the robustness of classification. For deep learning algorithms, model performance can be optimized by adjusting hyperparameters such as network structure, learning rate, and batch size.

[0090] In some embodiments, the recognition results of all sub-regions are merged by image fusion technology, specifically including:

[0091] Use I final =∪C i The classification results of all sub-regions C i Fusion is performed, where ∪ is a set union operation.

[0092] It should be noted that image fusion technology involves integrating the classification results of each sub-region. This process can be achieved through a set merge operation, that is, the recognition results of all sub-regions are merged through a logical or operation to form a unified recognition result. In this process, it may be necessary to define a fusion rule to determine how to derive the final recognition result from the results of multiple sub-regions. For example, if most sub-regions recognize an object as an apple, the final result will also recognize the object as an apple.

[0093] Preferably, the image fusion process can be further refined or alternatives can be provided. For example, the fusion rule can be based on a voting mechanism, in which the recognition result of each sub-region is equivalent to one vote, and the final result is determined by the category with the most votes. In addition, the concept of confidence can be introduced, that is, the recognition result of each sub-region can be accompanied by a confidence value, indicating the level of confidence in the result. When fusing, results with high confidence can be given priority. It is also possible to consider using a weighted fusion method, in which the recognition results of different sub-regions are given different weights according to their importance or reliability. In addition, if there are spatial relationships between sub-regions, these relationships can also be used to optimize the fusion results, for example, by considering the results of adjacent sub-regions to solve potential boundary problems.

[0094] In some embodiments, the apple fruit image is recognized using a deep learning algorithm, specifically including:

[0095] Define the network structure of the deep learning model, including the input layer, multiple hidden layers, and the output layer;

[0096] Using loss function

[0097]

[0098] Calculate the prediction result P i and the true label Y i The mean square error loss between , where N is the number of samples;

[0099] The network weights W and bias b are updated through the back-propagation algorithm to optimize the model performance.

[0100] It should be noted that the embodiments of the present invention describe in detail the process of using a deep learning algorithm to identify apple fruit images. As an advanced artificial intelligence technology, deep learning can improve the accuracy of recognition by learning the features of a large amount of data.

[0101] Specifically, the construction of a deep learning model includes defining the network structure, which usually involves an input layer, multiple hidden layers, and an output layer. The input layer receives the preprocessed image data, while the hidden layer transforms and extracts features from the data through weight matrices and bias terms. The output layer gives the final classification result based on the output of the hidden layer. The choice of loss function is crucial to the training process. Mean square error loss is a commonly used loss function that measures the difference between the predicted result and the true label. Through the back-propagation algorithm, the weights and biases of the network can be updated according to the results of the loss function to optimize the model performance.

[0102] Preferably, the construction and training process of the deep learning model can be further refined. For example, different types of activation functions, such as ReLU or Sigmoid, can be selected to increase the nonlinear expression ability of the model. The design of the network structure can include convolutional layers, pooling layers, and fully connected layers to adapt to the characteristics of image data. In addition to the mean square error, the loss function can also consider cross entropy loss or other custom loss functions to better adapt to specific classification tasks. During the training process, the training effect of the model can be optimized by adjusting hyperparameters such as learning rate, batch size, and number of iterations. In addition, regularization techniques, such as L1 or L2 regularization, can be used to prevent the model from overfitting.

[0103] More specifically, each layer of the deep learning model can be further defined. For example, the weight matrix can be set by initialization methods such as Glorot initialization or He initialization to promote the convergence of the network. The bias term is usually initialized to zero or a small constant value. The choice of activation function can be adjusted according to the needs of the hidden layer, such as using ReLU activation function after the convolutional layer and softmax activation function in the output layer for multi-class classification. The implementation of the backpropagation algorithm can adopt various optimization techniques, such as Momentum or Adam optimizer, to speed up the training process and improve the generalization ability of the model. In addition, the robustness of the model to different image transformations can be increased by introducing data augmentation techniques such as rotation, scaling or cropping.

[0104] In some embodiments, the deep learning model is characterized by the following formula:

[0105] O (l) =f (l) (W (l) ·O (l-1) +b (l) )

[0106] Among them, O (l) is the output of the lth layer, W (l) is the weight matrix from layer l-1 to layer l, b (l) is the bias term of the lth layer, f(l) is the activation function of the lth layer.

[0107] It should be noted that the representation of the deep learning model uses a recursive formula

[0108] O (l) =σ(W (l) ·O (l-1) +b (l) )

[0109] Among them, (l) represents the output of layer l, W (l) is the weight matrix from layer l-1 to layer l, b (l) is the bias term of the lth layer, and σ is the activation function. This formula shows how the output of each layer depends on the output of the previous layer and the parameters of the current layer. The activation function σ is usually used to introduce nonlinearity, allowing the model to learn and simulate more complex function mappings.

[0110] Preferably, the implementation of the deep learning model can be further refined. For example, the weight matrix W (l) It can be set by randomly initializing or pre-training the model parameters to help the model converge quickly. (l) It is usually initialized to zero or a small random value to break the symmetry. The choice of activation function σ can be determined according to different layers and task requirements. Common activation functions include ReLU, Sigmoid or Tanh. In addition, the number of layers of the model, the number of neurons in each layer, and the connection mode are all parameters that need to be considered in model design.

[0111] More specifically, the implementation of deep learning models can be further optimized by: using batch normalization to speed up the training process and improve model stability; using residual connections to solve the gradient vanishing problem in deep network training; introducing attention mechanisms to enhance the model's ability to identify key areas in the image; and using different optimization algorithms, such as Adam or RMSprop, to adjust the model's learning rate to make it more adaptable to the characteristics of the training data. In addition, regularization techniques, such as Dropout or L1 / L2 regularization, can be used to prevent model overfitting and improve the model's generalization ability.

[0112] The above-mentioned embodiments of the present invention have the following beneficial effects: the apple fruit image recognition method of the present invention can achieve high-accuracy image recognition in the case of multiple fruit overlaps and complex occlusions through a series of innovative technical steps. This method first ensures the quality of image data through image acquisition and preprocessing, laying a solid foundation for subsequent feature extraction and classification recognition. The application of feature extraction algorithms can extract key visual information from images, including directional gradient histograms, color features, and texture features, which can provide rich input data for machine learning algorithms. The introduction of machine learning algorithms enables the system to accurately classify and identify apple sub-regions based on feature information. Finally, through image fusion technology, the recognition results of all sub-regions are merged, which can further improve the accuracy and reliability of the entire apple tree area recognition.

[0113] In addition, the present invention also specifically introduces a deep learning algorithm, which can further enhance the intelligence level and processing power of the recognition system by constructing a complex neural network model. The network structure design of the deep learning model includes a multi-layer hidden layer, which enables the model to automatically learn deeper feature representations. The calculation of the loss function and the application of the back propagation algorithm can provide optimization directions for the weights and biases of the model to ensure the effectiveness of model training. The application of this deep learning method can not only improve the accuracy of recognition, but also adapt to changing environments and conditions, making the recognition system more robust and flexible. Through the comprehensive application of these technologies, the image recognition method of the present invention can be widely used in the field of agricultural automation and intelligence, bringing significant efficiency improvements and cost savings to agricultural production.

[0114] like Figure 2 As shown, in some embodiments, an apple fruit image recognition system 200 in the case of multiple fruits overlapping and complex occlusions, the system 200 includes:

[0115] An image acquisition module 201 is used to acquire images of the apple tree area;

[0116] A preprocessing module 202 is used to grayscale, filter and detect edges of the collected images;

[0117] A feature extraction module 203 is used to extract features from each apple sub-region based on an image feature extraction algorithm;

[0118] A classification and identification module 204, used for classifying and identifying each apple sub-region based on feature information and a machine learning algorithm;

[0119] The image fusion module 205 is used to merge the recognition results of all sub-areas to obtain the recognition result of the entire apple tree area.

[0120] It is understandable that the modules recorded in the apple fruit image recognition system 200 under the condition of multiple overlapping fruits and complex obstructions are the same as those in the reference Figure 1 The steps in the method for recognizing apple fruit images under the condition of multiple overlapping fruits and complex occlusions described above correspond to each other. Therefore, the operations, features and beneficial effects described above for the method for recognizing apple fruit images under the condition of multiple overlapping fruits and complex occlusions are also applicable to the apple fruit image recognition system 200 under the condition of multiple overlapping fruits and complex occlusions and the modules contained therein, and will not be repeated here.

[0121] Reference below Figure 3 , which shows a schematic diagram of a structure 300 of an electronic device suitable for implementing some embodiments of the present invention. The electronic devices in some embodiments of the present invention may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0122] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0123] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0124] Furthermore, the storage medium of the embodiment of the present application stores program instructions that can implement all the above methods, wherein the program instructions can be stored in the above storage medium in the form of a software product, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.

[0125] The above descriptions are only some preferred embodiments of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present invention is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present invention to form a technical solution.

Claims

1. A method for recognizing apple fruit images in the presence of multiple overlapping fruits and complex obstructions, characterized in that: The method comprises: Collecting images of the apple tree area to form at least two apple sub-areas; Preprocess the acquired images, including grayscale, filtering, and edge detection; Extract features from each apple sub-region based on the image feature extraction algorithm; Classify and identify each apple sub-region based on feature information and machine learning algorithms to obtain the recognition result of apple fruit; The recognition results of all sub-areas are combined through image fusion technology to obtain the recognition result of the entire apple tree area.

2. The apple fruit image recognition method according to claim 1, characterized in that: Preprocess the collected images, including: Use the grayscale formula I gray (i,j)=0.299·I original (i,j) R +0.587·I original (i,j) G +0.114·I original (i,j) B For each pixel (i, j) the original RGB image I orighinal To convert, I original (i,j) R ,I original (i,j) G ,I original (i,j) B Respectively represent the red, green, and blue channel values ​​of the pixel. Use the filter formula Filter the grayscale image, where F(m,n) is the coefficient of the filter kernel and k is the filter kernel size. The Canny edge detection algorithm is used to calculate the edge strength and direction, and finally the binary edge image I is obtained. edges (i,j), where I edges (i, j) is the pixel value of the edge image at point (i, j), and the value is 0 or 255 to represent non-edge and edge.

3. The apple fruit image recognition method according to claim 1, characterized in that: Feature extraction is performed on each apple sub-region based on the image feature extraction algorithm, including: Using feature extraction formula Features=Fe(I edges )={HoG,Color,Texture} Extract the image after edge detection I edges features, where Fe is the feature extraction function, HoG represents the histogram of oriented gradients, Color represents the color features, and Texture represents the texture features.

4. The apple fruit image recognition method according to claim 1, characterized in that: Classify and identify each apple sub-region based on feature information and machine learning algorithms, including: Use feature vector Features = [f1,f2,...,f M ], where f1,f2,...,f M Represents the M features extracted; Input the feature vector into the classifier and use the classification formula C=ML(Features)=arg Features) Classification is performed, where ML is the machine learning classification function, C is the set of possible categories, and P(c|Features) is the posterior probability of category c under given feature conditions.

5. The apple fruit image recognition method according to claim 1, characterized in that: The recognition results of all sub-areas are combined through image fusion technology, including: Use I final =∪C i The classification results of all sub-regions C i Fusion is performed, where ∪ is a set union operation.

6. The apple fruit image recognition method according to claim 1, characterized in that: Use deep learning algorithms to identify apple fruit images, including: Define the network structure of the deep learning model, including the input layer, multiple hidden layers, and the output layer; Using loss function Calculate the prediction result P i and the true label Y i The mean square error loss between , where N is the number of samples; The network weights W and bias b are updated through the back-propagation algorithm to optimize the model performance.

7. The apple fruit image recognition method according to claim 6, characterized in that: The deep learning model is characterized by the following formula: O (l) =f (l) (W (l) O (l-1) +b (l) ) Among them, O (l) is the output of the lth layer, W (l) is the weight matrix from layer l-1 to layer l, b (l) is the bias term of the lth layer, f (l) is the activation function of the lth layer.

8. An apple fruit image recognition system in the presence of multiple overlapping fruits and complex obstructions, characterized in that: The system comprises: An image acquisition module, used for acquiring images of the apple tree area; A preprocessing module is used to grayscale, filter and detect edges of the acquired images; A feature extraction module, used for extracting features from each apple sub-region based on an image feature extraction algorithm; A classification and recognition module, used to classify and recognize each apple sub-region based on feature information and machine learning algorithms; The image fusion module is used to merge the recognition results of all sub-areas to obtain the recognition result of the entire apple tree area.

9. An electronic device, characterized in that: It comprises a processor and a memory coupled to the processor, wherein the memory stores program instructions executable by the processor; when the processor executes the program instructions stored in the memory, the apple fruit image recognition method as described in any one of claims 1 to 8 is implemented.

10. A storage medium, characterized in that: The storage medium stores program instructions, which, when executed by a processor, can implement the apple fruit image recognition method according to any one of claims 1 to 8.

Images