Date online detection method and system based on deep regression neural network, storage medium and electronic equipment

Abstract

The invention discloses a jujube online detection method and system based on a deep regression neural network, a storage medium and electronic equipment, and the method comprises the following steps: collecting various jujube image data, and building a sample image data set; designing a jujube classification sorting model based on a deep regression neural network; deploying a neural network model; designing an automatic sorting system, and extending the automatic sorting system into an importance matrix; according to the method, samples are collected through a self-made image data collection device, manual and machine joint labeling and signing are utilized, a jujube data set with multiple varieties, cross-producing areas, unified standards and perfect indexes is constructed, sample expansion is carried out through technologies such as a cyclic generative adversarial network and an image enhancement strategy, and preprocessing is carried out; and a dynamic configurable multi-dataset database management system is designed to manage a dataset, so that data support is provided for deep regression neural network model training and later research on agricultural products such as jujubes, and the detection precision can be improved while the real-time detection speed is ensured.

Description

Technical Field

[0001] The present invention relates to the field of smart agricultural technology, and in particular to an online detection method, system, storage medium and electronic device for jujube based on a deep regression neural network. Background Art

[0002] At present, my country has a large-scale planting of agricultural products such as dates, plums, and cherry tomatoes, with rich cultivated varieties. In particular, the output of dates accounts for more than 99% of the world. In 2021, the cultivated area of ​​jujube in my country reached 3.31 million hm2.

[0003] The industrialized production of jujubes needs to rely on scientific and technological progress, implement regional layout, specialized production, integrated operation, enterprise management, and social services to promote the long-term development of the jujube industry. In addition, with the improvement of people's living standards, the quality requirements for jujubes are becoming more and more diversified. Therefore, the jujube industry needs to transform towards innovation and health to adapt to the diversification of consumption scenarios and add new vitality to the healthy development of the fruit industry. Therefore, the intelligentization, systematization, and efficiency of testing equipment are crucial to the quality improvement and scale expansion of jujubes, plums and other fruits. Summary of the invention

[0004] The purpose of the present invention is to provide an online detection method for jujubes based on a deep regression neural network, which can solve the technical problem of limited overall feature extraction capability of the model.

[0005] The technical solution adopted by the present invention is:

[0006] An online detection method for jujubes based on a deep regression neural network comprises the following steps:

[0007] Step 1, collect various jujube image data and establish a sample image data set;

[0008] Step 2, designing a jujube grading and sorting model based on deep regression neural network;

[0009] Step 3, deploy the neural network model;

[0010] Step 4: Design an automatic sorting system and extend it into an importance matrix.

[0011] Collect various jujube image data and establish a sample image data set, including:

[0012] Step 1.1, classifying the jujube dataset into normal, shrunken, crushed and scratched categories; Step 1.2, preprocessing the jujube dataset in step 1.1 by using upright and side-lying jujube conveying methods;

[0013] Step 1.3, using an image annotation tool (LabelImg) to add the jujube annotation information to the image data set in step 1.2;

[0014] Step 1.4: Auto Augment and CycleGAN are used to enhance and transfer the style of the training data in step 1.3 to increase the diversity of jujube data.

[0015] Step 1.5, using a cyclic generative adversarial network (CycleGAN) based on a Transformer model to pre-train a generator in a self-supervised manner to perform diverse style transfer on the samples in step 1.4 to establish a standard, unified, large-scale jujube dataset;

[0016] Step 1.6, using database management to collect and manage the jujube data set in step 1.5.

[0017] The design is based on a deep regression neural network-based grading and sorting model for agricultural products such as dates.

[0018] include:

[0019] Step 2.1, in the YOLOv5 target detection model, the thickness of the CSP module is expanded in the shallow feature extraction process, and the CA attention mechanism is introduced through feedback iteration to fully extract the detailed features of various jujube targets in the feature map of the step 1.6 dataset with gradually reduced size.

[0020]

[0021] The corresponding formula is;

[0022] Formula (1) and Formula (2) represent the E feature map of the cth channel with a height of h and a width of w, respectively, and i and j represent the row and column corresponding to the feature map, respectively; and It is the output of the feature map on channel c for h and w. The output value is not only related to the number of channels c, but also to the position coordinates i, j and the direction perception h, w:

[0023] f=δ(F1([z h ,z w ])) (3);

[0024] In the Coordinate feature mapping stage, the feature maps of the width and height of the global receptive field are concatenated together, and then sent to a convolution module with a shared convolution kernel of 1×1 to reduce its dimension to the original c / r. Then, the batch normalized feature map is sent to the Sigmoid activation function to obtain a feature map f of the shape of 1×(w+h)×c / r, as shown in formula (3);

[0025] Then, in the Coordinate attention generation stage, the feature map f is convolved along the spatial dimension with a convolution kernel of 1×1 according to the original height and width to obtain the feature map f with the same number of channels as the original one. h ,f w , where f h ∈R C / r×H ,f w ∈R C / r×W , and then use two 1×1 convolution transformation functions to transform f h ,f w Transform to obtain the attention feature weights g in the height and width directions h ,g w , the specific formulas are shown in (4) and (5), where σ is the sigmoid activation function;

[0026] g h =σ(F h (f h )) (4)

[0027] g w =σ(F w (f w )) (5)

[0028] Finally, by multiplying the original feature map again with weights, we get the final feature map with attention weights in the width and height directions, as shown in formula (6):

[0029]

[0030] Step 2.2, using the improved bidirectional feature pyramid network to perform feature fusion on the feature map in the data set of step 2.1, the corresponding formula is:

[0031]

[0032] in, Represents the feature map output from the top-up stage of the P4 layer, is the output feature of the 4th layer on the bottom-up path. Resize transforms the feature map in the spatial domain and channel domain, which includes double upsampling / downsampling and 1×1 convolution. i Represents the corresponding weight, ε is a minimum value to ensure the stability of the data;

[0033] Step 2.3, design the SAR-IoU loss function to optimize the feature map after feature fusion in step 2.2. The corresponding formula is as follows:

[0034]

[0035] Among them, ρ 2 (b,b gt ) represents the distance between the center point of the predicted box b and the real box b gt The Euclidean distance of the center point, c represents the diagonal distance between the predicted box and the real box, α is the weight coefficient, and v SAR is the consistency factor of the long and short sides, and the specific calculation method can be expressed as:

[0036] v SAR =σ-G(r) (10)

[0037]

[0038] Among them, σ 2 is the variance of the length-width ratio of the real boxes of all similar targets, μ is the mean of the real length-short side ratio of all similar targets, and r represents the ratio of the long side to the short side;

[0039] Step 2.4, using multi-scale target detection to train the network model data obtained after optimization in step 2.3.

[0040] The deployment of the neural network model includes:

[0041] Step 3.1, converting the network model weight file (.pt) trained in step 2.4 into a format (.onnx / .wts) supported by the embedded device;

[0042] Step 3.2, generating a deployment engine on Jetson Nano for the network model converted in step 3.1;

[0043] Step 3.3, use TensorRT to accelerate reasoning and deploy the model using the engine in step 3.2.

[0044] The automatic sorting system is designed and extended into an importance matrix, including:

[0045] Step 4.1, using an image acquisition preprocessing system based on a high-speed digital signal processing integrated circuit DSP to collect, segment, denoise, and equalize the images deployed in step 3.3, and upload the processing results to the main control system;

[0046] Step 4.2, designing a pneumatic sorting system so that after receiving the material separation command transmitted from the main control system in step 4.1, the optical coupling relay controls the high-speed solenoid valve to be turned on and off, and the high-pressure gas generated by the air pump blows the material to be processed out of the conveyor belt;

[0047] Step 4.3, using the dual-machine communication mode of the HTTP protocol to communicate with the Blynk IoT platform, accessing WIFI through the Esp8266 IoT module, and accessing the Bylnk server through WIFI. At the same time, the server and the mobile terminal also access the Bylnk server through the Internet, so as to realize real-time data communication interaction among the three, and then complete the mechanical operation functions such as feeding the jujubes in step 4.2 and evenly distributing them in the bins of the conveyor belt;

[0048] Step 4.4, design a mobile APP to monitor the mobile terminal in step 4.3;

[0049] Step 4.5, connecting the server to the sorting control core embedded device Jetson Nano of multiple sorting lines to perform online learning on the mobile APP terminal monitoring in step 4.4.

[0050] An online detection system for jujubes based on a deep regression neural network comprises a conveyor chain, a loading box, an image acquisition device, a material distribution plate, an electromagnetic valve, a fan device, an air pipeline, a controller and a material drop plate, wherein the loading box is above one end of the conveyor chain, and the material drop plate is arranged at the other end of the conveyor chain, and the image acquisition device and the material distribution plate are arranged between the two in sequence, the material distribution plate is provided with a fan, and the air outlet of the fan is connected to the air inlet of the material distribution plate through a pipeline, and a plurality of electromagnetic valves are arranged on the pipeline, the output end of the image acquisition device is connected to the input end of the controller, and the output end of the controller is respectively connected to the control input ends of the plurality of electromagnetic valves, and the controller is provided with an online detection method for jujubes based on a deep regression neural network as described in any one of claims 1 to 5.

[0051] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the device where the computer-readable storage medium is located executes the online detection method for jujubes based on a deep regression neural network.

[0052] An electronic device comprises: a memory and a processor, wherein the memory stores a program that can be run on the processor, and when the processor executes the program, the online detection method for jujubes based on a deep regression neural network is implemented.

[0053] The present invention has the following beneficial effects:

[0054] First, the present invention collects samples through a self-made image data acquisition device, and uses manual and machine joint labeling to construct a jujube dataset with multiple varieties, cross-origin, unified standards, and complete indicators. Cycle Generative Adversarial Network (CycleGAN) and image enhancement strategy (Auto Augment) and other technologies are used to expand samples for preprocessing, and a dynamically configurable multi-dataset database management system is designed to manage the data set, providing data support for deep regression neural network model training and later research on agricultural products such as jujube.

[0055] Second, a new target detection model for agricultural products such as jujube based on deep regression neural network is constructed by expanding and iterating the shallow CSP (Constraint Satisfaction Problem) module, introducing the position attention module, adopting the improved bidirectional feature pyramid network (Bidirectional Feature Pyramid Network), adding a small target detection layer, and designing a new SAR-IoU loss loss function (Smoothed Approximate Regularized IoU loss). The model is designed from multiple aspects such as feature extraction, feature fusion, target detection, model training, etc., which can improve the detection accuracy while ensuring real-time detection speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 is a flow chart of the present invention;

[0058] Figure 2 is an overall block diagram of the communication system of the present invention;

[0059] Figure 3 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] like Figure 1 , 2 As shown in 3, the present invention comprises the following steps:

[0062] Step 1, collect various jujube image data and establish a sample image data set;

[0063] Step 2, design a grading and sorting model for agricultural products such as dates based on a deep regression neural network;

[0064] Step 3, deploy the neural network model;

[0065] Step 4: Design an automatic sorting system and extend it into an importance matrix.

[0066] The following is a detailed explanation of each of the above steps:

[0067] refer to Figure 1 , showing the process of some embodiments of a method for studying an online detection system for jujubes based on a deep regression neural network according to the present invention. The method for studying an online detection system for jujubes based on a deep regression neural network comprises the following steps:

[0068] Step 1: Collect various jujube image data and establish a sample image data set, which includes:

[0069] Step 1.1, classify the jujube dataset into normal, shriveled, bruised, and scratched categories;

[0070] Step 1.2, preprocessing the jujube data set in step 1.1 by using a method of conveying jujubes in an upright manner and a side-lying manner;

[0071] Step 1.3, using an image annotation tool (LabelImg) to add the jujube annotation information to the image data set in step 1.2;

[0072] The use of an image annotation tool (LabelImg) to add the jujube annotation information to the image data set in step 1.2 includes the following steps:

[0073] Step 1.3.1, in the image annotation tool (LabelImg), open the menu of the image data in step 1.2 and select "Open Dir", and then select the folder of the image data in step 1.2.

[0074] Step 1.3.2, select one of the images in step 1.3.1, select or add the category "Dates" in the category list on the right, and use the annotation tool to select the area in the image that contains the dates. This is usually done by clicking and dragging in the image to create a rectangular box, making sure to select the entire area of ​​the dates.

[0075] Step 1.3.3, after creating the labeling box in step 1.3.2, click the "Save" button in the upper left corner to save the labeling information. The image labeling tool (LabelImg) will save the labeling information in the same directory as the image file and generate a labeling file with the same name but the suffix .xml.

[0076] Step 1.4, using the image automatic enhancement strategy (Auto Augment) and the cycle adversarial generative network (CycleGAN) to enhance and transfer the style of the training data in step 1.3, so as to increase the diversity of the jujube data;

[0077] The steps of using the image automatic enhancement strategy (Auto Augment) and the cycle adversarial generative network (CycleGAN) to enhance the training data in step 1.3 and transfer the style include:

[0078] Step 1.4.1, prepare the training data set in step 1.3.3 and an appropriate stylized image data set;

[0079] Step 1.4.2, using a Cycle Generative Adversarial Network (CycleGAN) model to stylize part of the training data set in step 1.4.1 into a target style;

[0080] Step 1.4.3, using the image automatic enhancement strategy (Auto Augment) to enhance the training data set and the generated stylized data set in step 1.4.2 to generate more diverse training samples.

[0081] Step 1.5, using a cyclic generative adversarial network (CycleGAN) based on a Transformer model to pre-train a generator in a self-supervised manner to perform diverse style transfer on the samples in step 1.4 to establish a standard, unified, large-scale jujube dataset;

[0082] Step 1.6, using database management to collect and manage the jujube data set in step 1.5.

[0083] Step 2: Design a grading and sorting model for agricultural products such as dates based on a deep regression neural network, including:

[0084] Step 2.1, in the YOLOv5 (You Only Look Once version 5) target detection model, the thickness of the CSP (Constraint Satisfaction Problem) module is expanded in the shallow feature extraction process, and the CA (Content-based Attention Mechanism) attention mechanism is introduced through feedback iteration to fully extract the feature map of the step 1.6 data set with gradually reduced size. The corresponding formula for the detailed features of various jujube targets is:

[0085]

[0086] Formula (1) and Formula (2) represent the feature map of the cth channel with a height of h and a width of w, respectively, and i and j represent the row and column corresponding to the feature map, respectively. and It is the output of the feature map on channel c for h and w. The output value is not only related to the number of channels c, but also to the position coordinates i, j and the direction perception h, w.

[0087] f=δ(F1([z h ,z w ])) (3)

[0088] In the Coordinate feature mapping stage, the feature maps of the width and height of the global receptive field are concatenated together, and then sent to a convolutional module with a shared convolution kernel of 1×1 to reduce its dimension to the original c / r. The batch normalized feature map is then sent to the Sigmoid activation function to obtain a feature map f of the shape of 1×(w+h)×c / r, as shown in formula (3).

[0089] Then, in the Coordinate attention generation stage, the feature map f is convolved along the spatial dimension with a convolution kernel of 1×1 according to the original height and width to obtain the feature map f with the same number of channels as the original one. h ,f w , where f h ∈R C / r×H ,f w ∈R C / r×W , and then use two 1×1 convolution transformation functions to transform f h ,f w Transform to obtain the attention feature weights g in the height and width directions h ,g w, the specific formulas are shown in (4) and (5), where σ is the sigmoid activation function.

[0090] g h =σ(F h (f h )) (4)

[0091] g w =σ(F w (f w )) (5)

[0092] Finally, the original feature map is re-weighted by multiplication to obtain the final feature map with attention weights in the width and height directions, as shown in formula (6).

[0093]

[0094] Step 2.2, using the improved bidirectional feature pyramid network (Bidirectional Feature Pyramid Network) to perform feature fusion on the feature map in the data set of step 2.1. The corresponding formula is:

[0095]

[0096]

[0097] in, Represents the feature map output from the top-up stage of the P4 layer, is the output feature of the 4th layer on the bottom-up path. Resize transforms the feature map in the spatial domain and channel domain, which includes double upsampling / downsampling and 1×1 convolution. i Represents the corresponding weight, and ε is a minimum value to ensure the stability of the data.

[0098] Step 2.3, design the SAR-IoU loss function (Smoothed Approximate Regularized IoUloss) to optimize the feature map after feature fusion in step 2.2. The corresponding formula is as follows:

[0099]

[0100] Among them, ρ 2 (b,b gt ) represents the distance between the center point of the predicted box b and the real box b gt The Euclidean distance of the center point, c represents the diagonal distance between the predicted box and the real box, α is the weight coefficient, and vSAR is the consistency factor of the long and short sides, and the specific calculation method can be expressed as:

[0101] v SAR =σ-G(r) (10)

[0102]

[0103] Among them, σ 2 is the variance of the length-width ratio of the real boxes of all similar targets, μ is the mean of the real length-short side ratio of all similar targets, and r represents the ratio of the long side (the largest width and height side) to the short side (the smallest width and height side).

[0104] Step 2.4, using multi-scale target detection to train the network model data obtained after optimization in step 2.3.

[0105] The training of the network model data obtained after the optimization in step 2.3 using multi-scale target detection includes the following steps:

[0106] Step 2.4.1, randomly cropping, rotating, mirror-flipping, etc. the network model data set obtained after the optimization in step 2.3 to increase its generalization ability;

[0107] Step 2.4.2, performing preprocessing operations such as normalization and resizing on the image generalized in step 2.4.1;

[0108] Step 2.4.3, adopt a multi-scale training strategy to train the image in step 2.4.2.

[0109] Step 3: Deploy the neural network model, including:

[0110] Step 3.1, convert the network model weight file (.pt) trained in step 2.4.3 into a format (.onnx / .wts) supported by the embedded device;

[0111] Step 3.2, generating a deployment engine on Jetson Nano (NVIDIA JetsonNano) for the network model converted in step 3.1;

[0112] Step 3.3, use TensorRT to accelerate reasoning and deploy the model using the engine in step 3.2.

[0113] Step 4: Design an automatic sorting system, including:

[0114] Step 4.1, using an image acquisition preprocessing system based on a high-speed digital signal processing integrated circuit DSP (Digital Signal Processor) to collect, segment, denoise, and balance the images deployed in step 3.3, and upload the processing results to the main control system;

[0115] Step 4.2, designing a pneumatic sorting system so that after receiving the material separation command transmitted from the main control system in step 4.1, the optical coupling relay controls the high-speed solenoid valve to be turned on and off, and the high-pressure gas generated by the air pump blows the material to be processed out of the conveyor belt;

[0116] Step 4.3, using the HTTP (Hyper Text Transfer Protocol) protocol dual-machine communication mode to communicate with the Blynk IoT platform, accessing WIFI (Wireless Fidelity) through the Esp8266 IoT module, WIFI accessing the Bylnk server, and the server and mobile terminal also accessing the Bylnk server through the Internet, to achieve real-time data communication interaction among the three, thereby completing the mechanical operation functions of step 4.2 such as feeding the jujubes and evenly distributing them in the bins of the conveyor belt;

[0117] Step 4.4, design a mobile APP (Application) to monitor the mobile terminal in step 4.3;

[0118] Step 4.5, connect the server to the sorting control core embedded device Jetson Nano (NVIDIA Jetson Nano) of multiple sorting lines to perform online learning on the mobile APP (Application) terminal monitoring of step 4.4.

[0119] The present invention collects samples through a self-made image data acquisition device, and uses manual and machine joint labeling to construct a jujube dataset with multiple varieties, cross-origin, unified standards and complete indicators. Cycle Generative Adversarial Network (CycleGAN) and image enhancement strategy (Auto Augment) and other technologies are used to expand samples for preprocessing, and a dynamically configurable multi-dataset database management system is designed to manage the data set, providing data support for deep regression neural network model training and later research on agricultural products such as jujube. Furthermore, the present invention constructs a new target detection model for agricultural products such as jujube based on deep regression neural network by expanding and iterating the shallow CSP (Constraint Satisfaction Problem) module, introducing the position attention module, adopting the improved bidirectional feature pyramid network (Bidirectional Feature Pyramid Network), adding a small target detection layer, and designing a new SAR-IoUloss loss function (Smoothed Approximate Regularized IoU loss). The model is designed from multiple aspects such as feature extraction, feature fusion, target detection, model training, etc., which can improve the detection accuracy while ensuring real-time detection speed.

[0120] The invention discloses an online detection system for jujubes based on a deep regression neural network, comprising a conveyor chain, a feeding box, an image acquisition device, a material distribution plate, an electromagnetic valve, a fan device, an air pipeline, a controller and a material drop plate. The feeding box is located above one end of the conveyor chain, and the material drop plate is arranged at the other end of the conveyor chain. An image acquisition device and a material distribution plate are arranged between the two in sequence. The material distribution plate is provided with a fan, and an air outlet of the fan is connected to an air inlet of the material distribution plate through a pipeline. A plurality of electromagnetic valves are arranged on the pipeline. The output end of the image acquisition device is connected to the input end of the controller, and the output end of the controller is respectively connected to the control input ends of a plurality of electromagnetic valves. The controller is provided with the online detection method for jujubes based on the deep regression neural network.

[0121] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the device where the computer-readable storage medium is located executes the online detection method for dates based on a deep regression neural network as described above. The computer program includes a computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory and other memories, etc.

[0122] An electronic device comprises: a memory and a processor, wherein the memory stores a program that can be run on the processor, and when the processor executes the program, the online detection method for jujubes based on a deep regression neural network as described above is implemented.

[0123] If the module / unit integrated in the electronic device described in this application is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware devices through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented.

[0124] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0125] Computer-readable instructions are stored in the computer-readable storage medium, and the computer-readable instructions are executed by a processor in an electronic device to implement the online detection method for jujubes based on a deep regression neural network as described in any of the above embodiments.

[0126] Compared with traditional technical methods, the present invention has higher accuracy, faster detection speed, better adaptability and automation level, and can significantly improve detection efficiency and quality and reduce costs.

[0127] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0128] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0129] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent application shall be subject to the attached claims.

[0130] It should be noted that the terms "including" and "having" and any variations thereof in the specification and claims of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0131] Note that the above are only preferred embodiments of the present invention and the principles of the application technology. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention is described in more detail through the above embodiments, the present invention is not limited to the specific embodiments described herein, and may include more other effective embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. An online detection method for jujube based on deep regression neural network, characterized in that: The following steps are involved: Step 1, collect various jujube image data and establish a sample image data set; Step 2, designing a jujube grading and sorting model based on deep regression neural network; Step 3, deploy the neural network model; Step 4: Design an automatic sorting system and extend it into an importance matrix.

2. According to claim 1, a method for studying an online detection system for jujubes based on a deep regression neural network is characterized in that: Collect various jujube image data and establish a sample image data set, including: Step 1.1, classify the jujube dataset into normal, shriveled, bruised, and scratched categories; Step 1.2, preprocessing the jujube data set in step 1.1 by using a method of conveying jujubes in an upright manner and a side-lying manner; Step 1.3, using an image annotation tool (LabelImg) to add the jujube annotation information to the image data set in step 1.2; Step 1.4: Auto Augment and CycleGAN are used to enhance and transfer the style of the training data in step 1.3 to increase the diversity of jujube data. Step 1.5, using a cyclic generative adversarial network (CycleGAN) based on a Transformer model to pre-train a generator in a self-supervised manner to perform diverse style transfer on the samples in step 1.4 to establish a standard, unified, large-scale jujube dataset; Step 1.6, using database management to collect and manage the jujube data set in step 1.

5.

3. According to claim 2, a method for studying an online detection system for jujubes based on a deep regression neural network is characterized in that: The design of the grading and sorting model for agricultural products such as dates based on a deep regression neural network includes: Step 2.1, in the YOLOv5 target detection model, the thickness of the CSP module is expanded in the shallow feature extraction process, and the CA attention mechanism is introduced through feedback iteration to fully extract the feature map of the step 1.6 data set to reduce the size step by step. The corresponding formula for the detailed features of various jujube targets is: Formula (1) and Formula (2) represent the E feature map of the cth channel with a height of h and a width of w, respectively, and i and j represent the row and column corresponding to the feature map, respectively; and It is the output of the feature map on channel c for h and w. The output value is not only related to the number of channels c, but also to the position coordinates i, j and the direction perception h, w: ； In the Coordinate feature mapping stage, the feature maps of the width and height of the global receptive field are concatenated together, and then sent to a convolution module with a shared convolution kernel of 1×1 to reduce its dimension to the original c / r. Then, the batch normalized feature map is sent to the Sigmoid activation function to obtain a feature map f of the shape of 1×(w+h)×c / r, as shown in formula (3); Then, in the Coordinate attention generation stage, the feature map f is convolved along the spatial dimension with a convolution kernel of 1×1 according to the original height and width to obtain the feature map f with the same number of channels as the original one. h ,f w , where f h ∈R C / r×H ,f w ∈R C / r×W , and then use two 1×1 convolution transformation functions to transform f h ,f w Transform to obtain the attention feature weights g in the height and width directions h ,g w , the specific formulas are shown in (4) and (5), where σ is the sigmoid activation function; g h =σ(F h (f h )) (4) g w =σ(F w (f w )) (5) Finally, by multiplying the original feature map again with weights, we get the final feature map with attention weights in the width and height directions, as shown in formula (6): Step 2.2, using the improved bidirectional feature pyramid network to perform feature fusion on the feature map in the data set of step 2.1, the corresponding formula is: in, Represents the feature map output from the top-up stage of the P4 layer, is the output feature of the 4th layer on the bottom-up path. Resize transforms the feature map in the spatial domain and channel domain, which includes double upsampling / downsampling and 1×1 convolution. i Represents the corresponding weight, ε is a minimum value to ensure the stability of the data; Step 2.3, design the SAR-IoU loss function to optimize the feature map after feature fusion in step 2.

2. The corresponding formula is as follows: Among them, ρ 2 (b,b gt ) represents the distance between the center point of the predicted box b and the real box b gt The Euclidean distance of the center point, c represents the diagonal distance between the predicted box and the real box, α is the weight coefficient, and v SAR is the consistency factor of the long and short sides, and the specific calculation method can be expressed as: v SAR =σ-G(r) (10) Among them, σ 2 is the variance of the length-width ratio of the real boxes of all similar targets, μ is the mean of the real length-short side ratio of all similar targets, and r represents the ratio of the long side to the short side; Step 2.4, using multi-scale target detection to train the network model data obtained after optimization in step 2.

3.

4. According to claim 3, a method for studying an online detection system for jujubes based on a deep regression neural network is characterized in that: The deployment of the neural network model includes: Step 3.1, converting the network model weight file (.pt) trained in step 2.4 into a format (.onnx / .wts) supported by the embedded device; Step 3.2, generating a deployment engine on Jetson Nano for the network model converted in step 3.1; Step 3.3, use TensorRT to accelerate reasoning and deploy the model using the engine in step 3.

2.

5. According to claim 4, a method for studying an online detection system for jujubes based on a deep regression neural network is characterized in that: The automatic sorting system is designed and extended into an importance matrix, including: Step 4.1, using an image acquisition preprocessing system based on a high-speed digital signal processing integrated circuit DSP to collect, segment, denoise, and equalize the images deployed in step 3.3, and upload the processing results to the main control system; Step 4.2, designing a pneumatic sorting system so that after receiving the material separation command transmitted from the main control system in step 4.1, the optical coupling relay controls the high-speed solenoid valve to be turned on and off, and the high-pressure gas generated by the air pump blows the material to be processed out of the conveyor belt; Step 4.3, using the dual-machine communication mode of the HTTP protocol to communicate with the Blynk IoT platform, accessing WIFI through the Esp8266 IoT module, and accessing the Bylnk server through WIFI. At the same time, the server and the mobile terminal also access the Bylnk server through the Internet, so as to realize real-time data communication interaction among the three, and then complete the mechanical operation functions such as feeding the jujubes in step 4.2 and evenly distributing them in the bins of the conveyor belt; Step 4.4, design a mobile APP to monitor the mobile terminal in step 4.3; Step 4.5, connecting the server to the sorting control core embedded device Jetson Nano of multiple sorting lines to perform online learning on the mobile APP terminal monitoring in step 4.

4.

6. The online detection system of jujube based on deep regression neural network is characterized by: It includes a conveyor chain, a loading box, an image acquisition device, a material distribution plate, an electromagnetic valve, a fan device, an air pipeline, a controller and a blanking plate. The loading box is above one end of the conveyor chain, and the blanking plate is arranged at the other end of the conveyor chain. The image acquisition device and the material distribution plate are arranged between the two in sequence. The material distribution plate is provided with a fan, and the air outlet of the fan is connected to the air inlet of the material distribution plate through a pipeline. A plurality of electromagnetic valves are arranged on the pipeline. The output end of the image acquisition device is connected to the input end of the controller, and the output end of the controller is respectively connected to the control input ends of the plurality of electromagnetic valves. The controller is provided with an online detection method for jujubes based on a deep regression neural network as described in any one of claims 1 to 5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the device where the computer-readable storage medium is located executes the online detection method for jujubes based on deep regression neural network as described in any one of claims 1 to 5.

8. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a program that can be run on the processor, and when the processor executes the program, the online detection method for jujubes based on a deep regression neural network as described in any one of claims 1-5 is implemented.

Images