Close-up fruit image recognition method, device, storage medium and apparatus

Abstract

The application relates to the technical field of image processing, and discloses a close-range color fruit image recognition method, equipment, a storage medium and a device, the method comprising the following steps: acquiring an image to be recognized, recognizing the image to be recognized through an improved YOLOv5 network model, and obtaining a recognition result, wherein the improved YOLOv5 network model comprises a hybrid attention module, and the hybrid attention module is used for enhancing the image recognition rate; since the improved YOLOv5 network model adds the hybrid attention module to a backbone network, the feature information of the image can be enhanced, and the image recognition rate is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a near-sky fruit image recognition method, device, storage medium and apparatus. BACKGROUND

[0002] With the progress of science and technology and the high-quality development of China's agriculture, agriculture has been transformed from automation to artificial intelligence, and traditional manual picking methods have been unable to meet the picking needs of farmers for citrus fruits. Today, China is carrying out research on intelligent fruit picking robots to meet the actual production needs. Researching intelligent fruit picking robots can effectively save labor costs and promote the intelligent development of agriculture. Computer vision technology is a key technology for realizing the operational capabilities of intelligent fruit picking robots.

[0003] However, in actual applications, due to the similarity of near-sky fruit and natural background color, the recognition difficulty is high, and the recognition accuracy is low. For example, when green citrus is identified, due to the similarity with the color of the leaves, the recognition accuracy is low. SUMMARY

[0004] The main purpose of the present application is to provide a near-sky fruit image recognition method, device, storage medium and apparatus, which aims to solve the technical problems of the prior art, such as the similarity of near-sky fruit and natural background color, high recognition difficulty and low recognition accuracy.

[0005] To achieve the above purpose, the present application provides a near-sky fruit image recognition method, which comprises the following steps:

[0006] Obtaining an image to be identified;

[0007] Identifying the image to be identified by an improved YOLOv5 network model to obtain a recognition result, wherein the improved YOLOv5 network model comprises a hybrid attention module, which is used to enhance the image recognition rate.

[0008] Optionally, the hybrid attention module comprises a channel attention SE module and a collaborative attention CA module, the channel attention SE module is arranged after a first convolutional layer of the improved YOLOv5 network model, and the collaborative attention CA module is arranged before a third C3 module of the improved YOLOv5 network model, the channel attention SE module is used to determine the information between channels, and the collaborative attention CA module is used to determine the direction perception information and the position perception information.

[0009] Optionally, the step of identifying the image to be identified by the improved YOLOv5 network model to obtain a recognition result comprises:

[0010] The weight values of each channel of the image to be recognized are adjusted through the improved channel attention SE module of the improved YOLOv5 network model.

[0011] The direction perception feature map and the position perception feature map of the image to be recognized are determined through the improved collaborative attention CA module of the improved YOLOv5 network model.

[0012] The image to be recognized is identified according to the adjusted weight values, the direction perception feature map and the position perception feature map, and an identification result is obtained.

[0013] Optionally, the step of determining the direction perception feature map and the position perception feature map of the image to be recognized through the improved collaborative attention CA module of the improved YOLOv5 network model comprises:

[0014] The coordinate attention is decomposed into one-dimensional feature encoding through the improved collaborative attention CA module of the improved YOLOv5 network model.

[0015] The features are aggregated along the spatial dimension through the collaborative attention CA module based on the one-dimensional feature encoding, and a direction perception feature map and a position perception feature map are obtained.

[0016] Optionally, the improved YOLOv5 network model further comprises an improved model feature fusion connection structure, which is arranged before the C3 module and is used for performing a Concat feature fusion operation.

[0017] Optionally, before the step of obtaining the image to be recognized, the method further comprises:

[0018] An image sample is obtained, and the image sample is input into an initial YOLOv5 network model for training, and a Varifocal Loss loss function is obtained.

[0019] The initial YOLOv5 network model is adjusted based on the Varifocal Loss loss function, and an improved YOLOv5 network model is obtained.

[0020] Optionally, before the step of obtaining the image sample and inputting the image sample into the initial YOLOv5 network model for training to obtain the Varifocal Loss loss function, the method further comprises:

[0021] An initial image is obtained.

[0022] The initial image is data enhanced through a Mosaic data enhancement module of the improved YOLOv5 model, and an image sample is obtained.

[0023] In addition, to achieve the above object, the present application also provides a near-view color fruit image recognition device, which comprises a memory, a processor and a near-view color fruit image recognition program stored in the memory and executable on the processor, and the near-view color fruit image recognition program is configured to implement the near-view color fruit image recognition method as described above.

[0024] In addition, to achieve the above object, the present application also provides a storage medium, which stores a near-view color fruit image recognition program, and the near-view color fruit image recognition program implements the near-view color fruit image recognition method as described above when executed by a processor.

[0025] In addition, to achieve the above object, the present application also provides a near-view color fruit image recognition device, which comprises: an image acquisition module and an image recognition module.

[0026] The image acquisition module is configured to acquire a to-be-recognized image.

[0027] The image recognition module is configured to recognize the to-be-recognized image through an improved YOLOv5 network model to obtain a recognition result, and the improved YOLOv5 network model comprises a hybrid attention module configured to enhance the image recognition rate.

[0028] In the present application, the to-be-recognized image is acquired, and the to-be-recognized image is recognized through the improved YOLOv5 network model to obtain a recognition result, and the improved YOLOv5 network model comprises a hybrid attention module configured to enhance the image recognition rate. Since the improved YOLOv5 network model of the present application adds the hybrid attention module to the backbone network, the feature information of the image can be enhanced, and the image recognition rate can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 FIG. 1 is a structural schematic diagram of a near-view color fruit image recognition device of a hardware running environment related to an embodiment scheme of the present application;

[0030] Figure 2 FIG. 2 is a flowchart of a near-view color fruit image recognition method according to a first embodiment of the present application;

[0031] Figure 3 FIG. 3 is a schematic diagram of an existing YOLOv5 network model;

[0032] Figure 4 FIG. 4 is a schematic diagram of an improved YOLOv5 network model according to an embodiment of the near-view color fruit image recognition method of the present application;

[0033] Figure 5A flowchart of a second embodiment of the near-view color fruit image recognition method of the present application is shown in the figure.

[0034] Figure 6 A flowchart of a third embodiment of the near-view color fruit image recognition method of the present application is shown in the figure.

[0035] Figure 7 A structure block diagram of a first embodiment of the near-view color fruit image recognition device of the present application is shown in the figure.

[0036] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0037] It should be understood that the specific embodiments described herein are merely illustrative of the present application and do not limit the present application.

[0038] Reference Figure 1 , Figure 1 A near-view color fruit image recognition device structure schematic diagram of a hardware running environment involved in the embodiment scheme of the present application is shown in the figure.

[0039] As Figure 1 shown, the near-view color fruit image recognition device can include a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 can include a display screen (Display), and the optional user interface 1003 can also include a standard wired interface, a wireless interface, and the wired interface of the user interface 1003 can be a USB interface in the present application. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a wireless fidelity (Wi-Fi) interface). The memory 1005 can be a high-speed random access memory (RAM), and can also be a stable memory (NVM), such as a disk memory. The memory 1005 can also be an independent storage device from the aforementioned processor 1001.

[0040] Those skilled in the art can understand Figure 1 that the structure shown in the figure does not constitute a limitation on the near-view color fruit image recognition device, and can include more or fewer components than the figure, or combine certain components, or different component arrangements.

[0041] As Figure 1As shown, the memory 1005, which is identified as a computer storage medium, can include an operating system, a network communication module, a user interface module, and a close-range color fruit image recognition program.

[0042] In Figure 1 In the close-range color fruit image recognition device shown, the network interface 1004 is mainly used to connect a background server and communicate data with the background server; the user interface 1003 is mainly used to connect a user device; the close-range color fruit image recognition device calls the close-range color fruit image recognition program stored in the memory 1005 through the processor 1001, and executes the close-range color fruit image recognition method provided by the embodiment of the application.

[0043] Based on the above hardware structure, embodiments of the close-range color fruit image recognition method of the application are proposed.

[0044] Reference Figure 2 , Figure 2 The flowchart of the first embodiment of the close-range color fruit image recognition method of the application is shown, and the first embodiment of the close-range color fruit image recognition method of the application is proposed.

[0045] Step S10: obtaining an image to be recognized.

[0046] It should be understood that the execution subject of the embodiment can be a close-range color fruit image recognition device with data processing, network communication and program running functions, such as a computer or other electronic devices capable of achieving the same or similar functions, and the embodiment does not limit this.

[0047] It should be noted that the image to be recognized can be input by the user in advance, for example, the image to be recognized can be a green citrus image input by the user in advance.

[0048] It can be understood that, in order to improve the processing efficiency, the initial image can be preprocessed in the embodiment to obtain the image to be recognized. The preprocessing can be denoising and the like to improve the image quality.

[0049] Step S20: identifying the image to be recognized through an improved YOLOv5 network model to obtain an identification result, the improved YOLOv5 network model comprising a hybrid attention module, the hybrid attention module being used to enhance the image recognition rate.

[0050] It should be noted that the hybrid attention module can be a channel attention SE module and / or a collaborative attention CA module, and the embodiment does not limit this.

[0051] The identification result can include the positioning and classification of each part in the image to be recognized, for example, when the image to be recognized is a green citrus image, the identification result is the position of the green citrus.

[0052] For the sake of easy understanding, reference Figure 3 is made to the accompanying drawings, but the present solution is not limited thereto. Figure 3 is a schematic diagram of the existing YOLOv5 network model, in which the input end of the YOLOv5 network has a Mosaic data enhancement strategy to ensure the balance of the picture samples. The backbone network of the YOLOv5 model mainly includes a Focus module and a C3 module. The Focus module performs slicing operation on the picture before entering the backbone network, stacks the four adjacent positions of the image, splits the high-resolution feature map into multiple low-resolution feature maps, realizes downsampling while reducing the amount of calculation, and improves the network speed. The C3 module is improved on the basis of the BottleneckCSP, and its bottleneck structure has one less convolution layer than the BottleneckCSP, so that the volume of the model becomes smaller, and the activation function in the convolution module after concatenation is also replaced by SiLU, and the detection performance is continuously enhanced. The structure of FPN combined with PAN is adopted in the Neck, and its main idea comes from the PANet network structure. The FPN layer conveys strong semantic features from top to bottom, and then combines with the feature pyramid from bottom to top, connects the low-level features and high-level features, and enhances the semantic information of the model. The CIOU_Loss is used as the bounding box loss function in the Prediction, and the network outputs three size feature maps for detection, which are used to detect target objects of different sizes, and finally the generated image is labeled by the predicted bounding box and the class.

[0053] For the sake of easy understanding, reference Figure 4 is made to the accompanying drawings, but the present solution is not limited thereto. Figure 4 is a schematic diagram of the improved YOLOv5 network model, in which a hybrid attention mechanism is added to the backbone network of the existing YOLOv5 network model, that is, the channel attention SE module and the collaborative attention CA module are combined and put into the backbone network, which can not only capture direction perception and position perception information, but also further strengthen the model's capture of channel information, so that the model can better extract and locate the feature information of the citrus picture and improve the citrus recognition rate in the natural environment.

[0054] In the first embodiment, an image to be recognized is acquired, and the improved YOLOv5 network model is used to recognize the image to be recognized to obtain a recognition result. The improved YOLOv5 network model includes a hybrid attention module, which is used to enhance the image recognition rate. Since the improved YOLOv5 network model of the present embodiment adds a hybrid attention module to the backbone network, the feature information of the image can be enhanced, and the image recognition rate can be improved.

[0055] Reference is made to Figure 5 , Figure 5This is a flowchart illustrating the second embodiment of the close-up fruit image recognition method of the present invention, based on the above. Figure 2 The first embodiment shown presents a second embodiment of the close-up fruit image recognition method of the present invention.

[0056] In the second embodiment, the hybrid attention module includes a channel attention (SE) module and a collaborative attention (CA) module. The channel attention (SE) module is positioned after the first convolutional layer of the improved YOLOv5 network model, and the collaborative attention (CA) module is positioned before the third C3 module of the improved YOLOv5 network model. The channel attention (SE) module is used to determine inter-channel information, and the collaborative attention (CA) module is used to determine orientation awareness information and position awareness information.

[0057] For ease of understanding, please refer to Figure 4 This explanation does not limit the scope of this solution. Figure 4 The diagram shows the improved YOLOv5 network model. In the diagram, a hybrid attention mechanism is added to the backbone of the existing YOLOv5 network model. Specifically, SE attention is embedded in the second layer of the backbone network, and CA attention is embedded in the eleventh layer. The two attention mechanisms are used together to not only capture orientation and position information, but also to further enhance the model's capture of channel information. This allows the model to better extract and locate the feature information of citrus images and improve the citrus recognition rate in natural environments.

[0058] In the second embodiment, the improved YOLOv5 network model further includes an improved model feature fusion connection structure, which is set before the C3 module and is used to perform Concat feature fusion operation.

[0059] It should be understood that convolutional neural networks are specifically designed to process image data, a type of network-like data structure. Through convolution operations, they can better extract feature information from images. The C3 module of the YOLOv5 model has one less convolutional layer than the Bottleneck CSP module, preventing gradient explosion and network degradation during convolution. The C3 module involves multiple convolution operations, and the YOLOv5 model uses a large number of C3 structures, while the model itself also operates on the feature map through other convolution operations. Because convolution operations involve a large number of parameters, and citrus images are similar in color to the background, and when there are many citrus fruits, their volume is small, the feature extraction process is prone to losing feature information of citrus objects after a large number of convolutions. Therefore, to overcome the above defects, the improved YOLOv5 network model in this embodiment also includes an improved model feature fusion connection structure, which is placed before the C3 module for performing concat feature fusion operations.

[0060] For the sake of easy understanding, reference Figure 4 is made to the Figure 4 For improving the YOLOv5 network model, the YOLOv5 model adopts the Concat method to fuse the upper layer feature branch and the lower layer feature branch, in order to reduce the convolution times of the lower layer feature branch and the parameter amount of the Concat feature fusion, and improve the performance of the network model in identifying citrus, the lower layer feature branch is improved before the C3 module, and then the Concat feature fusion is performed with another upper layer feature branch, so as to reduce the parameter amount in the Concat connection operation.

[0061] In the second embodiment, the step S20 comprises:

[0062] Step S201: adjusting the weight value of each channel of the to-be-identified image through the channel attention SE module of the improved YOLOv5 network model.

[0063] It should be understood that the citrus picture is similar to the background color of the natural environment, and its channel feature information is limited. In the case of a large number of citrus, the pixel value of the image is also limited, so the inter-channel feature information needs to be increased for identification.

[0064] It can be understood that the channel attention SE module pays attention to the information between the channels, and can optimize the learning of the feature information of a specific class in a deep network. First, the channel attention SE module performs a compression (Squeeze) operation on the given input X. The compression operation of the Cth channel can be represented by the following formula:

[0065]

[0066] In the formula, Zc is the output related to the cth channel, and the global average pooling operation is performed on the feature map after the convolution operation to obtain the global compression feature quantity of the current feature map.

[0067] Secondly, after the excitation (Excitation) operation, the following formula can be used to represent:

[0068]

[0069] In the formula, σ represents a sigmoid activation function, is the result after the conversion function is generated.

[0070] The result after the conversion function is generated can be represented by the following formula:

[0071]

[0072] In the formula, T1 and T2 are two linear transformations to capture the importance of each channel, and finally the features between channels are weighted through a scaling operation, that is, the weight value between channels is multiplied by the features of the original feature map to obtain the adjusted weight value.

[0073] Step S202: determining the direction perception feature map and the position perception feature map of the to-be-identified image through the cooperative attention CA module of the improved YOLOv5 network model.

[0074] It should be understood that in actual application, attention needs to be paid to the position perception information during identification, which helps the model to more accurately locate and identify the citrus object.

[0075] It can be understood that the cooperative attention CA module not only pays attention to the information between channels, but also pays attention to the direction perception and position perception information. The cooperative attention CA module decomposes the coordinate attention into two one-dimensional feature encodings, respectively aggregates the features along two spatial directions, one direction captures long-range dependencies, and the other direction preserves precise position information. Finally, the direction perception and position perception feature maps are formed respectively and applied complementarily to the input feature map.

[0076] Step S203: identifying the to-be-identified image according to the adjusted weight value, the direction perception feature map and the position perception feature map to obtain an identification result.

[0077] It should be understood that after obtaining the adjusted weight value, the direction perception feature map and the position perception feature map, subsequent processing can be performed according to the adjusted weight value, the direction perception feature map and the position perception feature map to identify the to-be-identified image and obtain an identification result.

[0078] Further, the step S204 comprises:

[0079] The cooperative attention CA module of the improved YOLOv5 network model decomposes the coordinate attention into one-dimensional feature encoding;

[0080] Based on the one-dimensional feature encoding, the cooperative attention CA module aggregates the features along the spatial dimension to obtain the direction perception feature map and the position perception feature map.

[0081] Specifically, in order to realize the embedding of coordinate information, the cooperative attention CA module first encodes each channel of the horizontal X direction and the vertical Y direction through global average pooling to obtain two feature maps. Then, the two feature maps are connected by Concat, and then a shared 1x1 convolution transformation function F1 is used for operation:

[0082] f=δ(F1([z h ,z w ]))

[0083] wherein, δ represents a nonlinear activation function, [·, ·] represents a connection operation along the spatial dimension, f R C / r×(H+W) denote the intermediate feature maps that encode spatial information in horizontal and vertical directions, and r represents a scaling coefficient. Then, f is split into two independent tensors f h C / r×H and f w C / r×W To make the channel numbers of the tensors the same, two 1x1 convolutional transformations Fh and Fw are used to transform fh and fw, respectively, and the following formula is obtained:

[0084] g h = σ(F h (f h ))

[0085] g w = σ(F w (f w ))

[0086] σ represents a sigmoid activation function, and here, a scaling coefficient r is also used to reduce the channel number of f. Then, the expanded gh and gw are used as the weights of attention, respectively. Finally, the expression of the output of the attention module can be as follows:

[0087]

[0088] In the second embodiment, the weight values of the channels of the image to be recognized are adjusted by improving the channel attention SE module of the improved YOLOv5 network model, the initial image is data enhanced by the improved Mosaic data enhancement module of the YOLOv5 model, and the image sample is obtained. The image to be recognized is recognized according to the adjusted weight value, the direction perception feature map and the position perception feature map, and a recognition result is obtained. Since the channel attention SE module is also used to pay attention to the information between the channels, and the collaborative attention CA module is used to pay attention to the direction perception and position perception information, the accuracy of the recognition result can be further improved.

[0089] Referring to Figure 6 , Figure 6 is a flowchart of the third embodiment of the present application for near-range color fruit image recognition method, based on the first embodiment shown in the above Figure 2 The third embodiment of the present application for near-range color fruit image recognition method is proposed based on the first embodiment shown in the above.

[0090] In the third embodiment, before the step S10, the method further comprises:

[0091] ​​Step S01: Obtain an image sample, and input the image sample into an initial YOLOv5 network model for training to obtain a Varifocal Loss loss function.

[0092] It should be understood that in actual application, green citrus has different growth forms and is easily affected by factors such as background objects such as leaves and branches and self-overlapping, resulting in low recognition accuracy. Therefore, in order to overcome the above defects, in the embodiment, the loss function of the YOLOv5 network model is improved when the YOLOv5 network model is trained, thereby enhancing the recognition accuracy and robustness in the case of citrus overlapping and shielding.

[0093] It should be noted that the initial YOLOv5 network model can be an improved YOLOv5 network model that has not been trained.

[0094] It can be understood that the main task of image recognition is positioning and classification, that is, positioning and labeling the target of the picture and giving the corresponding category. The loss function is to compare the predicted value of the network with the actual label information during the training of the neural network, and calculate the gap between the two. At this time, the network model is constantly updated in reverse iteration, and the parameters are constantly adjusted and updated to reduce the loss function, so that the final prediction of the network is more and more accurate. Therefore, the loss function of the target detection task can be divided into two categories: one is the classification loss function, and the other is the regression loss function.

[0095] The classification loss function of the existing YOLOv5 network model adopts the Focal Loss loss function, which is defined as follows:

[0096]

[0097] In the formula, y represents the true class, p represents the predicted probability of the foreground class, α represents the weight used to balance the positive and negative samples, (1-p) γ and p γ are used to adjust the weight of each sample, and the Focal Loss loss function is the same for positive and negative samples.

[0098] In order to improve the recognition rate of citrus in a natural scene, the Varifocal Loss loss function is introduced to replace the Focal Loss loss function of the original YOLOv5 network model. The Varifocal Loss loss function is an improved classification loss function based on the Focal Loss loss function, and is defined as follows:

[0099]

[0100] where p is the predicted classification score, q is the score of the target IoU, and a represents the weight used to balance the positive and negative samples. p γ represents the scaling coefficient. Unlike Focal Loss, the scaling coefficient q is added to weight the positive samples, and the weight of the loss function will be larger if the real IoU score of the positive sample is higher, so as to focus on the high-quality samples during training. Since the number of positive samples is small, in order to fully utilize the supervision information of the positive samples, the scaling coefficient p γ only adjusts the negative samples. In order to balance the overall number of positive and negative samples, the negative samples are weighted using a.

[0101] Step S02: adjusting the initial YOLOv5 network model based on the Varifocal Loss loss function to obtain an improved YOLOv5 network model.

[0102] It should be understood that adjusting the initial YOLOv5 network model based on the Varifocal Loss loss function can be continuously updating the initial YOLOv5 network model based on the Varifocal Loss loss function, continuously adjusting the parameters, reducing the loss function, and making the final prediction of the network more and more accurate.

[0103] In a third embodiment, an initial YOLOv5 network model is trained based on image samples, and a Varifocal Loss loss function is obtained. The initial YOLOv5 network model is adjusted based on the Varifocal Loss loss function to obtain an improved YOLOv5 network model. Since the loss function of the YOLOv5 network model is improved during training, the recognition accuracy and robustness in the case of citrus overlap and occlusion are enhanced.

[0104] Further, in order to further enhance the training of small target citrus or citrus occlusion, the step S01 further comprises:

[0105] obtaining an initial image;

[0106] performing data augmentation on the initial image through the Mosaic data augmentation module of the improved YOLOv5 model to obtain an image sample.

[0107] It should be understood that in order to improve the problem of insufficient number of citrus pictures in some classification scenarios, the initial citrus image is data-augmented through the Mosaic data augmentation module of the improved YOLOv5 model.

[0108] It should be noted that the initial image can be a citrus image input by a user.

[0109] It can be understood that the YOLOv5 model has a Mosaic data enhancement module, and for the case that the number of citrus pictures in each scene is not balanced, the Mosaic data enhancement module is used to improve the problem that the number of citrus pictures in some classification scenes is not enough, and further enhance the training under the condition of small target citrus or citrus occlusion. Four pictures are randomly cropped and spliced into one picture for training. When using Mosaic enhancement training, since the data of 4 pictures can be directly calculated, it is not necessary to increase the size of the minimum batch (Mini-batch), which reduces the use rate of GPU, and one GPU can achieve better results, so that the robustness of the network is better. Therefore, the image after using this method is beneficial to better fit the images in the training set during the training process, and the Mosaic data enhancement module is a training strategy that can improve the performance of the model with a small cost.

[0110] In a third embodiment, an initial image is obtained, and the initial image is data enhanced by a Mosaic data enhancement module of an improved YOLOv5 model to obtain a to-be-recognized image. Since the initial citrus image is also data enhanced by the Mosaic data enhancement module of the improved YOLOv5 model in this embodiment, the images in the training set can be better fitted during the training process.

[0111] In addition, an embodiment of the present application also provides a storage medium, wherein the storage medium stores a near-scene color fruit image recognition program, and the near-scene color fruit image recognition program is executed by a processor to realize the near-scene color fruit image recognition method as described above.

[0112] In addition, with reference to Figure 7 , an embodiment of the present application also provides a near-scene color fruit image recognition device, which comprises an image acquisition module 10 and an image recognition module 20.

[0113] The image acquisition module 10 is used to acquire a to-be-recognized image.

[0114] It should be noted that the to-be-recognized image can be input by a user in advance, for example, the to-be-recognized image can be a green citrus image input by a user in advance.

[0115] It can be understood that, in order to improve the processing efficiency, in this embodiment, the initial image can be preprocessed to obtain the to-be-recognized image. The preprocessing can be denoising and the like to improve the image quality.

[0116] The image recognition module 20 is configured to recognize the to-be-recognized image by using the improved YOLOv5 network model to obtain a recognition result, and the improved YOLOv5 network model comprises a mixed attention module, which is configured to enhance the image recognition rate.

[0117] It should be noted that the mixed attention module can be a channel attention SE module and / or a collaborative attention CA module, and the present embodiment is not limited in this regard.

[0118] The recognition result can include the location and category of each part in the to-be-recognized image, for example, when the to-be-recognized image is a green citrus image, the recognition result is the location of the green citrus.

[0119] For the sake of understanding, reference Figure 3 will be made to Figure 3 illustrate the present solution, but the present solution is not limited in this regard. The figure is a schematic diagram of the existing YOLOv5 network model, in which the input end of the YOLOv5 network has a Mosaic data enhancement strategy to ensure the balance of the picture samples. The backbone network of the YOLOv5 model mainly includes a Focus module and a C3 module. The Focus module performs slicing operation on the picture before entering the backbone network, stacks the four adjacent positions of the image, splits the high-resolution feature map into multiple low-resolution feature maps, realizes downsampling while reducing the amount of calculation, and improves the network speed. The C3 module is improved on the basis of the BottleneckCSP, and its bottleneck structure has one less convolution layer than the BottleneckCSP, so that the volume of the model becomes smaller, and the activation function in the convolution module after concatenation is replaced by SiLU, and the detection performance is continuously enhanced. The structure of FPN combined with PAN is adopted in the Neck, and the main idea comes from the PANet network structure. The FPN layer conveys strong semantic features from top to bottom, and then combines with the feature pyramid from bottom to top, connects the low-level features and high-level features, and enhances the semantic information of the model. The Prediction adopts CIOU_Loss as the boundary box loss function, and the network outputs three size feature maps for detection, which are used to detect target objects of different sizes, and finally the generated image is labeled by the predicted boundary box and the category.

[0120] For the sake of understanding, reference Figure 4 will be made to Figure 4In order to improve the rear YOLOv5 network model schematic diagram, in the existing YOLOv5 network model backbone network, a mixed attention mechanism is added, that is, a channel attention SE module and a collaborative attention CA module are combined and placed into the backbone network, which can not only capture direction perception and position perception information, but also further strengthen the model to capture channel information, so that the model can better extract and locate the feature information of the citrus picture, and improve the citrus recognition rate in the natural environment.

[0121] In the embodiment, the image to be recognized is acquired, the improved YOLOv5 network model is used to recognize the image to be recognized, and the recognition result is obtained.

[0122] Other embodiments or specific implementations of the near-view fruit image recognition device described in the application can refer to the above-mentioned method embodiments, which will not be described here.

[0123] It should be noted that in this paper, the term "includes", "contains" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or system. Without more limitations, the element defined by the statement "includes a" does not exclude the presence of another identical element in the process, method, article or system including the element.

[0124] The above-mentioned embodiment number of the application is only for description, not representing the advantages and disadvantages of the embodiments.

[0125] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the application can be embodied in the form of software product, which is stored in a storage medium (such as read only memory image (Read Only Memory image, ROM) / random access memory (Random Access Memory, RAM), magnetic disc, optical disc), including a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, air conditioner or network device, etc.) execute the method described in each embodiment of the application.

[0126] The above merely provides the preferred embodiments of the present application, and is not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made according to the content of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A close-up color fruit image recognition method, characterized by, The near-view fruit image recognition method comprises the following steps: acquiring an image to be recognized; recognizing the image to be recognized through an improved YOLOv5 network model to obtain a recognition result, wherein the improved YOLOv5 network model comprises a mixed attention module, which is used to enhance the image recognition rate; the mixed attention module comprises a channel attention SE module and a collaborative attention CA module, the channel attention SE module is arranged after a first convolutional layer of the improved YOLOv5 network model, and the collaborative attention CA module is arranged before a third C3 module of the improved YOLOv5 network model, the channel attention SE module is used to determine the information between channels, and the collaborative attention CA module is used to determine the direction perception information and the position perception information; the improved YOLOv5 network model further comprises an improved model feature fusion connection structure arranged before the C3 module and used for performing a Concat feature fusion operation.

2. The near-view fruit image recognition method according to claim 1, wherein The step of recognizing the image to be recognized through the improved YOLOv5 network model to obtain a recognition result comprises: adjusting the weight values of each channel of the image to be recognized through the channel attention SE module of the improved YOLOv5 network model; determining a direction perception feature map and a position perception feature map of the image to be recognized through the collaborative attention CA module of the improved YOLOv5 network model; recognizing the image to be recognized according to the adjusted weight values, the direction perception feature map and the position perception feature map to obtain a recognition result.

3. The near-view fruit image recognition method according to claim 2, wherein The step of determining the direction perception feature map and the position perception feature map of the image to be recognized through the collaborative attention CA module of the improved YOLOv5 network model comprises: decomposing the coordinate attention into one-dimensional feature encoding through the collaborative attention CA module of the improved YOLOv5 network model; aggregating features along the spatial dimension through the collaborative attention CA module based on the one-dimensional feature encoding to obtain a direction perception feature map and a position perception feature map.

4. The near view fruit image recognition method according to any one of claims 1 to 3, wherein, Before the step of acquiring an image to be recognized, the method further comprises the following steps: acquiring an image sample and inputting the image sample into an initial YOLOv5 network model for training to obtain a VarifocalLoss loss function; adjusting the initial YOLOv5 network model based on the VarifocalLoss loss function to obtain an improved YOLOv5 network model.

5. The near-view fruit image recognition method according to claim 4, wherein Before the step of acquiring an image sample and inputting the image sample into an initial YOLOv5 network model for training to obtain a VarifocalLoss loss function, the method further comprises the following steps: acquiring an initial image; performing data enhancement on the initial image through a Mosaic data enhancement module of the improved YOLOv5 model to obtain an image sample.

6. A close-up color fruit image recognition apparatus characterized by comprising: The near-view color fruit image recognition device includes a memory, a processor, and a near-view color fruit image recognition program stored on the memory and executable on the processor, and the near-view color fruit image recognition program, when executed by the processor, implements the near-view color fruit image recognition method according to any one of claims 1 to 5.

7. A storage medium, characterized by The storage medium stores a near-view color fruit image recognition program, and the near-view color fruit image recognition program, when executed by a processor, implements the near-view color fruit image recognition method according to any one of claims 1 to 5.

8. A close-up color fruit image recognition apparatus, characterized by comprising: The near-view color fruit image recognition device includes an image acquisition module and an image recognition module. The image acquisition module is configured to acquire a to-be-recognized image. The image recognition module is configured to recognize the to-be-recognized image by using an improved YOLOv5 network model to obtain a recognition result, and the improved YOLOv5 network model includes a hybrid attention module configured to enhance the image recognition rate. The hybrid attention module includes a channel attention SE module and a collaborative attention CA module, the channel attention SE module is arranged after a first convolutional layer of the improved YOLOv5 network model, the collaborative attention CA module is arranged before a third C3 module of the improved YOLOv5 network model, the channel attention SE module is configured to determine information between channels, and the collaborative attention CA module is configured to determine direction perception information and position perception information. The improved YOLOv5 network model further includes an improved model feature fusion connection structure arranged before the C3 module and configured to perform a Concat feature fusion operation.

Images