Litchi detection method and device based on YOLOv5, equipment, medium and product

Abstract

The application relates to the technical field of image detection, and provides a litchi detection method and device based on a lightweight YOLOv5, a computer device, a storage medium and a computer program product. The application can improve the accuracy of litchi detection on a to-be-detected litchi image. The method comprises the following steps: replacing Maxpool pooling in an SPP module in a lightweight YOLOv5 network structure with SoftPool pooling, establishing weighted feature fusion between a low-layer network structure and a high-layer network structure in the lightweight YOLOv5 network structure, replacing convolution in a CBL module in a backbone network in the lightweight YOLOv5 network structure with a Ghost module, obtaining a litchi detection model, obtaining a to-be-detected litchi image, and detecting the to-be-detected litchi image by using the litchi detection model to obtain a litchi detection result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image detection, in particular to a litchi detection method and device based on lightweight YOLOv5 with weighted feature fusion, a computer device, a storage medium and a computer program product. BACKGROUND

[0002] With the development of automatic picking technology, the picking efficiency of automatic picking is obviously higher than that of manual picking. The image detection technology of the picked object (such as litchi) is the premise of the development of automatic picking technology, so the image detection technology of the picked object (such as litchi) becomes an important detection technology.

[0003] The traditional technology usually detects the litchi image to be detected by using the SSD algorithm to obtain a litchi detection result, but the accuracy of the litchi detection of the litchi image to be detected by using the technology is low. SUMMARY

[0004] Therefore, it is necessary to provide a litchi detection method and device based on lightweight YOLOv5 with weighted feature fusion, a computer device, a computer readable storage medium and a computer program product in view of the above technical problems.

[0005] In a first aspect, the present application provides a litchi detection method based on lightweight YOLOv5 with weighted feature fusion. The method comprises:

[0006] replacing Maxpool pooling in the SPP module in the lightweight YOLOv5 network structure with SoftPool pooling, establishing weighted feature fusion between the low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure, and replacing the convolution in the CBL module in the backbone network in the lightweight YOLOv5 network structure with the Ghost module to obtain a litchi detection model;

[0007] obtaining a litchi image to be detected;

[0008] detecting the litchi image to be detected by using the litchi detection model to obtain a litchi detection result.

[0009] In one embodiment, the litchi detection model comprises an Input part, a Backbone part, a Neck part and a Prediction part; the Neck part of the litchi detection model comprises a feature map pyramid network and a pyramid self-attention network.

[0010] In one embodiment, the establishment of the weighted feature fusion between the low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure comprises:

[0011] The low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure are established to obtain weighted feature summation through a feature pyramid network and a pyramid self-attention network.

[0012] In one embodiment, the fusion factor of the weighted feature summation is obtained according to the ratio between the target quantity in the low-level network feature map and the target quantity in the high-level network feature map in the lightweight YOLOv5 network structure.

[0013] In one embodiment, the method further comprises:

[0014] The Adam optimization algorithm is used to train the litchi detection model in an end-to-end manner.

[0015] In one embodiment, the loss function of the litchi detection model comprises a bbox loss part, a classification loss part and a confidence loss part; the bbox loss part comprises a GIoU loss function; and the classification loss part and the confidence loss part comprise a binary cross-entropy loss function.

[0016] In a second aspect, the present application further provides a litchi detection device based on a lightweight YOLOv5 with weighted feature fusion. The device comprises:

[0017] The detection model obtaining module is configured to replace Maxpool pooling in the SPP module in the lightweight YOLOv5 network structure with SoftPool pooling, establish weighted feature fusion between the low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure, and replace convolution in the CBL module in the backbone network in the lightweight YOLOv5 network structure with a Ghost module to obtain a litchi detection model.

[0018] The image obtaining module is configured to obtain a litchi image to be detected.

[0019] The result obtaining module is configured to detect the litchi image to be detected by using the litchi detection model to obtain a litchi detection result.

[0020] In a third aspect, the present application further provides a computer device. The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0021] The Maxpool pooling in the SPP module in the lightweight YOLOv5 network structure is replaced by SoftPool pooling, the low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure are established to fuse weighted features, and the convolution in the CBL module in the backbone network in the lightweight YOLOv5 network structure is replaced by a Ghost module, to obtain a litchi detection model; a litchi image to be detected is acquired; and the litchi detection model is used to detect the litchi image to be detected to obtain a litchi detection result.

[0022] In a fourth aspect, the present application also provides a computer readable storage medium. The computer readable storage medium has a computer program stored thereon, and the computer program, when executed by a processor, implements the following steps:

[0023] The Maxpool pooling in the SPP module in the lightweight YOLOv5 network structure is replaced by SoftPool pooling, the low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure are established to fuse weighted features, and the convolution in the CBL module in the backbone network in the lightweight YOLOv5 network structure is replaced by a Ghost module, to obtain a litchi detection model; a litchi image to be detected is acquired; and the litchi detection model is used to detect the litchi image to be detected to obtain a litchi detection result.

[0024] In a fifth aspect, the present application also provides a computer program product. The computer program product comprises a computer program, and the computer program, when executed by a processor, implements the following steps:

[0025] The Maxpool pooling in the SPP module in the lightweight YOLOv5 network structure is replaced by SoftPool pooling, the low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure are established to fuse weighted features, and the convolution in the CBL module in the backbone network in the lightweight YOLOv5 network structure is replaced by a Ghost module, to obtain a litchi detection model; a litchi image to be detected is acquired; and the litchi detection model is used to detect the litchi image to be detected to obtain a litchi detection result.

[0026] The litchi detection method, device, computer equipment, storage medium and computer program product based on the weighted feature fusion lightweight YOLOv5 replace the Maxpool pooling in the SPP module in the lightweight YOLOv5 network structure with the SoftPool pooling, and establish weighted feature fusion between the low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure, and replace the convolution in the CBL module in the backbone network in the lightweight YOLOv5 network structure with the Ghost module, obtain a litchi detection model, obtain a litchi image to be detected, and detect the litchi image to be detected by using the litchi detection model to obtain a litchi detection result. The original Maxpool pooling in the SPP module in the lightweight YOLOv5 network structure is replaced with the SoftPool pooling, and at the same time, the low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure are weighted and fused, and at the same time, the original convolution in the CBL module in the backbone network in the lightweight YOLOv5 network structure is replaced with the Ghost module, so as to obtain the litchi detection model, then obtain the litchi image to be detected, input the litchi image to be detected into the litchi detection model, obtain the litchi detection result output by the litchi detection model, and improve the accuracy of litchi detection on the litchi image to be detected. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 A flowchart of a litchi detection method based on weighted feature fusion lightweight YOLOv5 in an embodiment is shown;

[0028] Figure 2 A flowchart of a litchi detection model in an embodiment is shown;

[0029] Figure 3 A construction diagram of an SPP_s module in an embodiment is shown;

[0030] Figure 4 A construction diagram of a CBL module in an embodiment is shown;

[0031] Figure 5 A construction diagram of a GBL module in an embodiment is shown;

[0032] Figure 6 A construction diagram of a Ghost module, a GhostBottleneck1 module and a GhostBottleneck2 module in an embodiment is shown;

[0033] Figure 7 A construction diagram of a Focus module in an embodiment is shown;

[0034] Figure 8 A construction diagram of a CSP1_X module in an embodiment is shown;

[0035] Figure 9 A construction schematic of a Res unit module in an embodiment;

[0036] Figure 10 A construction schematic of a CSP2_X module in an embodiment;

[0037] Figure 11 A structural block diagram of a litchi detection device based on a lightweight YOLOv5 with weighted feature fusion in an embodiment;

[0038] Figure 12 An internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0039] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0040] In an embodiment, as shown in Figure 1 , a litchi detection method based on a lightweight YOLOv5 with weighted feature fusion is provided, and the present embodiment takes the method applied to a terminal as an example, including the following steps:

[0041] In step S101, Maxpool pooling in the SPP module in the lightweight YOLOv5 network structure is replaced by SoftPool pooling, weighted feature fusion is established between the low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure, and the convolution in the CBL module in the backbone network in the lightweight YOLOv5 network structure is replaced by the Ghost module, and a litchi detection model is obtained.

[0042] In this step, as shown in Figure 2 , the lightweight YOLOv5 network structure can be a YOLOv5 algorithm, the low-level network structure can be the 4th layer feature map, the 6th layer feature map and the 9th layer feature map in the backbone network (Backbone part) in the lightweight YOLOv5 network structure, and the high-level network structure can be the 17th layer feature map, the 19th layer feature map and the 22nd layer feature map in the Neck part in the lightweight YOLOv5 network structure.

[0043] Specifically, the terminal replaces the original Maxpool pooling in the SPP module in the lightweight YOLOv5 network structure with SoftPool pooling, and simultaneously performs weighted feature fusion on the low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure, and simultaneously replaces the original convolution in the CBL module in the backbone network in the lightweight YOLOv5 network structure with a Ghost module, thereby constructing the litchi detection model.

[0044] As shown in Figure 2 and Figure 3 , the terminal replaces the original Maxpool pooling in the SPP module in the lightweight YOLOv5 network structure with SoftPool pooling to obtain SPP_s, where SPP_s represents Spatial Pyramid Pooling, and the parameters are (c_in, c_out, [kernel_size1, kernel_size2, kernel_size3]), where c_in represents the size of the input channel, c_out represents the size of the output channel, and kernel_size represents the size of the convolution kernel. For example, the SPP_s module represents that after the data input into the SPP_s module is processed by three SoftPool poolings, the data is merged (Concat) with the data input into the SPP_s module and then output, where the SoftPool pooling is to accumulate the activations in an exponential weighting manner, and each activation a i A weight w i is applied to the activation a i , which is calculated as the ratio of the natural exponential of the activation to the sum of the natural exponentials of all activations in the neighborhood R. For example, the weight w i is calculated as follows:

[0045]

[0046] The output value of the SoftPool operation is obtained by standard summing all weighted activations in the kernel neighborhood R, and the formula for the output value of the SoftPool operation is as follows:

[0047]

[0048] The terminal simultaneously performs weighted feature fusion on the 9th feature map and the 22nd feature map, the 6th feature map and the 19th feature map, and the 4th feature map and the 17th feature map in the lightweight YOLOv5 network structure; as shown in Figure 4 , Figure 5 and Figure 6 ​As shown, the terminal replaces the original convolution (Conv) in the CBL module in the backbone network of the lightweight YOLOv5 network structure with a Ghost module to obtain a GBL module, wherein the CBL module can be composed of a convolution (Conv), a BN (Batch Normalization, for example, a normalization network layer with the characteristics of accelerating training convergence) and a Leakyrelu (activation function), and the GBL module can be composed of a Ghost module, a BN and a Leakyrelu, as shown in Figure 2 and Figure 6 As shown, the parameters of the CBL module and the GBL module are (c_in, c_out, kernel_size, stride), wherein stride represents the step size, as shown in Figure 6 As shown in (a) Ghost module diagram in the above, the schematic diagram of the Ghost module is shown, for example, after the data (such as WxHxC format, wherein W represents width, H represents height, and C represents the size of the channel) input to the Ghost module is processed by convolution (Conv), data in W1xH1xC1 format is obtained, after the data in W1xH1xC1 format is processed by Identify (such as original features), the intrinsic feature map is obtained, at the same time, the data in W1xH1xC1 format is split into multiple sub-data such as φ1, φ2……φn, and then fused into a Ghost feature map, after the intrinsic feature map and the Ghost feature map are fused, the output data of the Ghost module is obtained, in addition, as shown in Figure 6 As shown in (b) GhostBottleneck1 diagram in the above, when the step size of the GBL module is 1, the GBL module can be a GhostBottleneck1 module, the data input to the GhostBottleneck1 module is first input to the Ghost module (Ghost module), after being processed by BN and ReLU (such as Batch Normalization and ReLU processing), it is input to the Ghost module (Ghost module) again, after being processed by BN, the output data of the GhostBottleneck1 module is obtained by superimposing (such as adding) the data input to the GhostBottleneck1 module, and as shown in Figure 6As shown in the (c) GhostBottleneck2 diagram, when the step of the GBL module is 2, the GBL module can be a GhostBottleneck2 module, the data input into the GhostBottleneck2 module is first input into a Ghost module, after being processed by BN and ReLU (such as Batch Normalization and ReLU processing), it is input into a DWConv (depthwise separable convolution) with a stride of 2, and after being processed by BN and ReLU (such as Batch Normalization and ReLU processing), it is input into a Ghost module, and after being processed by BN, it is superimposed (such as added) with the data input into the GhostBottleneck2 module to obtain the output data of the GhostBottleneck2 module; and the terminal thereby constructs a litchi detection model.

[0049] In some embodiments, as shown in Figure 2 The litchi detection model includes an Input part, a Backbone part, a Neck part, and a Prediction part, and the Neck part of the litchi detection model includes a feature pyramid network and a pyramid self-attention network. As shown in Figure 2 The Input part is used to perform processing such as enhancement, adaptive anchor frame calculation, and adaptive picture scaling on the litchi image to be detected, as shown in Figure 2 The 0-9 layer feature maps in the left box constitute the Backbone part (main network), and the Backbone part is used to perform slicing on the image output by the Input part through a Focus module, expand the dimensions of the sliced feature maps through Concat (such as merging operation), and then use 4 GBL modules, 4 CSP1_X (X can be 1 or 3) modules, and 1 SPP_s module (Spatial Pyramid Pooling, spatial pyramid pooling) to obtain a new group of feature maps, as shown in Figure 2The 10-23 layer feature maps in the middle box constitute the Neck part. The Neck part adopts the structure of Feature Pyramid Networks (FPN) and Pyramid Attention Networks (PAN). The FPN layer conveys strong semantic features from top to bottom, and the PAN conveys strong positioning features from bottom to top. Different detection layers are fused with features from different backbone layers, and features are cross-connected by referring to BiFPN. The Prediction part processes P1 (such as a 17 layer feature map), P2 (such as a 20 layer feature map) and P3 (such as a 23 layer feature map) three branch results in the Neck part through convolution to generate a bounding box and a predicted target class, such as Figure 2 The outputs of the 17, 20 and 23 layer feature maps are inputs of the Prediction function part (including Conv2d and sigmoid activation function). As shown in Figure 2 The parameters of the Focus module are (c_in, c_out, kernel_size, stride). The Focus module is constructed as shown in Figure 7 The data input into the Focus module is processed by a plurality of (such as 4) slice operations, then concatenated, input into the CBL module and output. As shown in Figure 2 and Figure 8 The parameters of the CSP1_X (X can be 1 or 3) module represent (c_in, c_out) x the number of times the module is stacked (such as Figure 2 The 9th layer feature map indicates that the size of the input channel is 1024, the size of the output channel is 1024, and x3 indicates that the module is stacked 3 times. The input data input into the CSP1_X module is sequentially processed by the CBL module, the Res unit module and the Conv, then concatenated, processed by BN and Leakyrelu, and output as output data. As shown in Figure 9 The input data input into the Res unit module is processed by two CBL modules, then stacked (such as added) and output. As shown in Figure 2 and Figure 10As shown, the parameters of the CSP2_X (X can be 1) module represent (c_in, c_out) x the number of times the module is stacked, when X is 1, the input data input to the CSP2_X module is sequentially subjected to 2 CBL modules, Conv processing, and the input data subjected to Conv processing is subjected to Concat, BN, Leakyrelu processing to output as output data.

[0050] In step S102, a litchi image to be detected is acquired.

[0051] Specifically, the terminal acquires a litchi image to be detected.

[0052] In step S103, a litchi detection model is used to detect the litchi image to be detected to obtain a litchi detection result.

[0053] Specifically, the terminal inputs the litchi image to be detected into the litchi detection model to obtain the litchi detection result output by the litchi detection model. For example, Figure 2 As shown, when the terminal inputs a certain litchi image to be detected into the litchi detection model, the litchi detection model detects the litchi image to be detected, wherein the scale_factor of UpSample is 2, that is, the output image is 2 times the input.

[0054] In the above litchi detection method based on the lightweight YOLOv5 network structure, the Maxpool pooling in the SPP module of the lightweight YOLOv5 network structure is replaced by the SoftPool pooling, the low-level network structure and the high-level network structure of the lightweight YOLOv5 network structure are established to perform weighted feature fusion, and the convolution in the CBL module of the backbone network of the lightweight YOLOv5 network structure is replaced by the Ghost module to obtain the litchi detection model, acquire the litchi image to be detected, and use the litchi detection model to detect the litchi image to be detected to obtain the litchi detection result. The original Maxpool pooling in the SPP module of the lightweight YOLOv5 network structure is replaced by the SoftPool pooling, and at the same time, the low-level network structure and the high-level network structure of the lightweight YOLOv5 network structure are weighted and fused, and at the same time, the original convolution in the CBL module of the backbone network of the lightweight YOLOv5 network structure is replaced by the Ghost module, thereby obtaining the litchi detection model, then acquiring the litchi image to be detected, inputting the litchi image to be detected into the litchi detection model, and obtaining the litchi detection result output by the litchi detection model, thereby improving the accuracy of litchi detection on the litchi image to be detected.

[0055] In one embodiment, the step S101 of establishing weighted feature fusion between the low-level network structure and the high-level network structure of the lightweight YOLOv5 network structure specifically includes:

[0056] The low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure are established to perform weighted feature summation through the feature pyramid network and the pyramid self-attention network.

[0057] Specifically, as shown in Figure 2 , the terminal establishes the weighted feature summation (W_Sum) of the low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure through the feature pyramid network (FPN) and the pyramid self-attention network (PAN). For example, as shown in Figure 2 , the terminal performs the weighted feature summation (W_Sum) of the 9th feature map and the 22nd feature map, the 6th feature map and the 19th feature map, and the 4th feature map and the 17th feature map in the lightweight YOLOv5 network structure through the feature pyramid network (FPN) and the pyramid self-attention network (PAN).

[0058] Further, in one embodiment, the fusion factor of the weighted feature summation can be obtained according to the ratio between the target quantity in the low-level network feature map and the high-level network feature map in the lightweight YOLOv5 network structure. The fusion factor is used to balance the contribution of different layers, so that the fused features after the weighted feature summation of the low-level network structure and the high-level network structure are suitable for small object detection. The fusion factor is a weighted coefficient, and the calculation method of the fusion factor a is as follows: first, calculate the IoU value of each anchor box, select the anchor box with the maximum IoU as the target in the image; second, based on the positive anchor box in each layer and the predefined number of anchor boxes, calculate the real target quantity in each layer; third, repeat the first step and the second step for each image in the data set to obtain statistical results, and then calculate the fusion factor a according to the formula:

[0059]

[0060] The fusion factor a is calculated, where, and respectively represent the target quantity of the P i+1 and P i layer in the feature fusion.

[0061] The technical scheme of the embodiment establishes the weighted feature summation of the low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure through the feature pyramid network and the pyramid self-attention network, and the fusion factor is obtained according to the ratio between the target quantity in the low-level network feature map and the high-level network feature map in the lightweight YOLOv5 network structure, so as to facilitate the litchi detection model to obtain more feature information and more accurate feature information, thereby facilitating subsequent improvement of the accuracy of litchi detection on the litchi image to be detected.

[0062] In one embodiment, the method described above can also train the litchi detection model by the following steps, specifically comprising: training the litchi detection model in an end-to-end training manner based on an Adam optimization algorithm.

[0063] In this embodiment, the end-to-end training manner can be a training manner of directly training a target algorithm model by inputting data into the network of the litchi detection model, and the training process does not need human intervention.

[0064] Specifically, the terminal trains the litchi detection model in an end-to-end training manner based on the Adam optimization algorithm method, wherein the image used for training can be an image with a size of 640*640, a batch-size of 8 and an epoch of 300.

[0065] Further, the loss function of the litchi detection model includes a bbox loss part, a classification loss part and a confidence loss part, the bbox loss part includes a GIoU loss function, and the classification loss part and the confidence loss part include a binary cross entropy loss function. Wherein, the loss function LOSS of the litchi detection model is composed of the bbox loss L box , the classification loss L cls and the confidence loss L obj The bbox loss L box adopts a GIoU (Generalized Intersection over Union) loss function, the classification loss L cls and the confidence loss L obj adopt a binary cross entropy (BCE) loss function.

[0066] The technical scheme of this embodiment trains the litchi detection model and uses the loss function of the litchi detection model, thereby facilitating to obtain a litchi detection model with higher detection accuracy, and thereby facilitating to improve the accuracy of litchi detection on the litchi image to be detected in the future.

[0067] It should be understood that although each step in the flowchart involved in each embodiment as described above is shown in sequence according to the direction of the arrow, these steps are not necessarily executed in sequence according to the direction of the arrow. Unless explicitly stated herein, there is no strict order limitation for the execution of these steps, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately or alternately executed with at least part of other steps or steps or stages in other steps.

[0068] Based on the same inventive concept, the embodiments of the present application also provide a weighted feature fusion-based lightweight YOLOv5 lychee detection device for implementing the above-mentioned weighted feature fusion-based lightweight YOLOv5 lychee detection method. The problem-solving implementation scheme provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more weighted feature fusion-based lightweight YOLOv5 lychee detection device embodiments provided below can be referred to the limitations of the weighted feature fusion-based lightweight YOLOv5 lychee detection method in the above, which will not be repeated here.

[0069] In one embodiment, as shown in FIG. 11, a weighted feature fusion-based lightweight YOLOv5 lychee detection device is provided, which can include: Figure 11

[0070] The detection model obtaining module 1101 is configured to replace Maxpool pooling in the SPP module in the lightweight YOLOv5 network structure with SoftPool pooling, establish weighted feature fusion between the low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure, and replace the convolution in the CBL module in the backbone network in the lightweight YOLOv5 network structure with the Ghost module to obtain a lychee detection model.

[0071] The image obtaining module 1102 is configured to obtain a lychee image to be detected.

[0072] The result obtaining module 1103 is configured to detect the lychee image to be detected by using the lychee detection model to obtain a lychee detection result.

[0073] ​In one embodiment, the litchi detection model comprises an Input part, a Backbone part, a Neck part and a Prediction part; the Neck part of the litchi detection model comprises a feature pyramid network and a pyramid self-attention network.

[0074] In one embodiment, the detection model obtaining module 1101 is further configured to establish weighted feature summation of the low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure through the feature pyramid network and the pyramid self-attention network.

[0075] In one embodiment, the fusion factor of the weighted feature summation is obtained according to the ratio between the target quantity in the low-level network feature map and the target quantity in the high-level network feature map in the lightweight YOLOv5 network structure.

[0076] In one embodiment, the device 1100 further comprises a model training module configured to train the litchi detection model in an end-to-end manner based on an Adam optimization algorithm.

[0077] In one embodiment, the loss function of the litchi detection model comprises a bbox loss part, a classification loss part and a confidence loss part; the bbox loss part comprises a GIoU loss function; the classification loss part and the confidence loss part comprise a binary cross-entropy loss function.

[0078] The above-mentioned various modules of the litchi detection device based on the lightweight YOLOv5 of the weighted feature fusion can be realized by software, hardware and their combinations in whole or in part. The above-mentioned various modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to call and execute the operations corresponding to the above-mentioned various modules by the processor.

[0079] In one embodiment, a computer device is provided, which can be a terminal, and the internal structure diagram thereof can be as shown in Figure 12As shown in the figure. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used for wired or wireless communication with external terminals. Wireless mode can be achieved through WIFI, mobile cellular network, NFC (Near Field Communication) or other technologies. The computer program is executed by the processor to implement a litchi detection method based on weighted feature fusion of lightweight YOLOv5. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad provided on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0080] Those skilled in the art can understand that, Figure 12 The skilled in the art can understand that,

[0081] In one embodiment, a computer device is provided, comprising a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the following steps:

[0082] Replace Maxpool pooling in the SPP module in the lightweight YOLOv5 network structure with SoftPool pooling, and establish weighted feature fusion between the low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure, and replace the convolution in the CBL module in the backbone network in the lightweight YOLOv5 network structure with the Ghost module, to obtain a litchi detection model;

[0083] Obtain a litchi image to be detected;

[0084] Detect the litchi image to be detected using the litchi detection model to obtain a litchi detection result.

[0085] In one embodiment, the litchi detection model includes an Input part, a Backbone part, a Neck part and a Prediction part; the Neck part of the litchi detection model includes a feature map pyramid network and a pyramid self-attention network.

[0086] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0087] The low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure are established to perform weighted feature summation through the feature pyramid network and the pyramid self-attention network.

[0088] In one embodiment, the fusion factor of the weighted feature summation is obtained according to the ratio between the target quantity in the low-level network feature map and the target quantity in the high-level network feature map in the lightweight YOLOv5 network structure.

[0089] In one embodiment, the processor, when executing the computer program, also implements the following steps:

[0090] The Adam optimization algorithm is used to train the litchi detection model in an end-to-end manner.

[0091] In one embodiment, the loss function of the litchi detection model includes a bbox loss part, a classification loss part and a confidence loss part; the bbox loss part includes a GIoU loss function; the classification loss part and the confidence loss part include a binary cross-entropy loss function.

[0092] In one embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0093] The Maxpool pooling in the SPP module in the lightweight YOLOv5 network structure is replaced by the SoftPool pooling, the low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure are established to perform weighted feature fusion, and the convolution in the CBL module in the backbone network in the lightweight YOLOv5 network structure is replaced by the Ghost module, thereby obtaining a litchi detection model.

[0094] A litchi image to be detected is obtained.

[0095] The litchi detection model is used to detect the litchi image to be detected to obtain a litchi detection result.

[0096] In one embodiment, the litchi detection model includes an Input part, a Backbone part, a Neck part and a Prediction part; the Neck part of the litchi detection model includes a feature pyramid network and a pyramid self-attention network.

[0097] In one embodiment, the computer program is executed by the processor to implement the following steps:

[0098] The low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure are established to obtain weighted feature summation through a feature pyramid network and a pyramid self-attention network.

[0099] In one embodiment, a fusion factor of the weighted feature summation is obtained according to a ratio between a target quantity in a low-level network feature map and a target quantity in a high-level network feature map in the lightweight YOLOv5 network structure.

[0100] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0101] The litchi detection model is trained in an end-to-end manner based on an Adam optimization algorithm.

[0102] In one embodiment, a loss function of the litchi detection model includes a bbox loss part, a classification loss part and a confidence loss part; the bbox loss part includes a GIoU loss function; and the classification loss part and the confidence loss part include a binary cross-entropy loss function.

[0103] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the following steps:

[0104] The Maxpool pooling in the SPP module in the lightweight YOLOv5 network structure is replaced by SoftPool pooling, the low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure are established to obtain weighted feature fusion, and the convolution in the CBL module in the backbone network in the lightweight YOLOv5 network structure is replaced by a Ghost module to obtain the litchi detection model.

[0105] An image of a litchi to be detected is obtained.

[0106] The litchi detection model is used to detect the image of the litchi to be detected to obtain a litchi detection result.

[0107] In one embodiment, the litchi detection model includes an Input part, a Backbone part, a Neck part and a Prediction part; and the Neck part of the litchi detection model includes a feature pyramid network and a pyramid self-attention network.

[0108] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0109] The low-level network structure and the high-level network structure in the lightweight YOLOv5 network structure are established to obtain weighted feature summation through a feature pyramid network and a pyramid self-attention network.

[0110] In one embodiment, the fusion factor of the weighted feature summation is obtained according to the ratio between the target quantity in the low-layer network feature map and the high-layer network feature map in the lightweight YOLOv5 network structure.

[0111] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0112] The litchi detection model is trained in an end-to-end manner based on an Adam optimization algorithm.

[0113] In one embodiment, the loss function of the litchi detection model includes a bbox loss part, a classification loss part and a confidence loss part; the bbox loss part includes a GIoU loss function; and the classification loss part and the confidence loss part include a binary cross-entropy loss function.

[0114] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties.

[0115] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0116] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0117] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A litchi detection method based on lightweight YOLOv5 of weighted feature fusion, characterized in that, The method comprises: replacing the original Maxpool pooling in the SPP module in the lightweight YOLOv5 network structure with a SoftPool pooling to obtain an SPP_s module, and performing weighted feature summation on the 9th feature map and the 22nd feature map in the lightweight YOLOv5 network structure through a feature pyramid network and a pyramid self-attention network, performing weighted feature summation on the 6th feature map and the 19th feature map, and performing weighted feature summation on the 4th feature map and the 17th feature map, and replacing the convolution in the CBL module in the backbone network in the lightweight YOLOv5 network structure with a Ghost module to obtain a litchi detection model; the SPP_s module is used to output the data input into the SPP_s module after three SoftPool pooling processes and merging of the data input into the SPP_s module; obtaining a litchi image to be detected; detecting the litchi image to be detected by using the litchi detection model to obtain a litchi detection result; The fusion factor of the weighted feature summation is obtained according to the ratio between the target quantity in the low-layer network feature map and the target quantity in the high-layer network feature map in the lightweight YOLOv5 network structure; the fusion factor is used to balance the contribution of different layers, so that the fused features after the weighted feature summation of the low-layer network structure and the high-layer network structure are adapted to the detection of small objects; the fusion factor is a weighted coefficient, and the calculation method of the fusion factor comprises: first, calculating the IoU value of each anchor box, selecting the anchor box with the maximum IoU as the target in the image; second, calculating the real target quantity in each layer based on the positive anchor box in each layer and a predetermined number of anchor boxes; third, repeating the first step and the second step for each image in the data set to obtain statistical results.

2. The method of claim 1, wherein, The litchi detection model comprises an Input part, a Backbone part, a Neck part and a Prediction part; the Neck part in the litchi detection model comprises the feature pyramid network and the pyramid self-attention network.

3. The method of claim 2, wherein, The Input part is used for enhancing the litchi image to be detected, calculating adaptive anchor boxes and adaptively scaling pictures.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: training the litchi detection model in an end-to-end manner based on an Adam optimization algorithm.

5. The method of claim 4, wherein, The loss function of the litchi detection model comprises a bbox loss part, a classification loss part and a confidence loss part; the bbox loss part comprises a GIoU loss function; the classification loss part and the confidence loss part comprise a binary cross-entropy loss function.

6. A litchi detection device based on lightweight YOLOv5 of weighted feature fusion, characterized in that, The device comprises: The detection model obtaining module is configured to replace the original Maxpool pooling in the SPP module in the lightweight YOLOv5 network structure with a SoftPool pooling to obtain an SPP_s module, and perform weighted feature summation on the 9th feature map and the 22nd feature map in the lightweight YOLOv5 network structure through a feature pyramid network and a pyramid self-attention network, perform weighted feature summation on the 6th feature map and the 19th feature map, and perform weighted feature summation on the 4th feature map and the 17th feature map, and replace the convolution in the CBL module in the backbone network in the lightweight YOLOv5 network structure with a Ghost module to obtain a lychee detection model; the SPP_s module is configured to merge and output data input into the SPP_s module after three SoftPool pooling processes; The image acquisition module is configured to acquire a lychee image to be detected; The result obtaining module is configured to detect the lychee image to be detected by using the lychee detection model to obtain a lychee detection result. The fusion factor of the weighted feature summation is obtained according to the ratio between the number of targets in the low-layer network feature map and the number of targets in the high-layer network feature map in the lightweight YOLOv5 network structure; the fusion factor is used to balance the contribution of different layers, so that the fused features after the weighted feature summation of the low-layer network structure and the high-layer network structure are adapted to the detection of small objects; the fusion factor is a weighted coefficient, and the calculation method of the fusion factor includes: first, calculating the IoU value of each anchor box, selecting the anchor box with the maximum IoU as the target in the image; second, calculating the number of real targets in each layer based on the positive anchor box in each layer and a predefined number of anchor boxes; and third, repeating the first step and the second step for each image in the data set to obtain statistical results.

7. The apparatus of claim 6, wherein, The device further includes a model training module configured to train the lychee detection model in an end-to-end training manner based on an Adam optimization algorithm.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 5.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 5.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 5. The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 5.

Images