Crop insect pest recognition method and device, electronic equipment and storage medium

Abstract

The invention discloses a crop pest recognition method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a to-be-identified video stream, and determining a to-be-identified image according to the to-be-identified video stream; determining correlation parameters between each pixel point in the to-be-identified image and other pixel points except the pixel point, and determining a first feature map corresponding to the to-be-identified image according to the correlation parameters; two-way feature fusion is carried out on the first feature maps of different scales to obtain a second feature map, the first feature maps of different scales correspond to different resolutions respectively, and the fusion sequence of the two-way feature fusion on the first feature maps of different scales comprises the sequence of the resolutions from high to low and the sequence of the resolutions from low to high; and determining a detection frame in the to-be-recognized image according to the second feature map. According to the invention, the technical problems of insufficient precision and relatively large resource consumption of an insect pest detection method in the prior art are solved.

Description

Technical Field

[0001] The present application relates to the field of computer vision application technology, and more specifically, to a crop pest identification method, device, electronic device and storage medium. Background Art

[0002] With the modernization and intelligence of agricultural production, real-time monitoring and prevention of crop pests have become increasingly important. Due to the diversity and complexity of crop pests, pest detection schemes in related technologies may lead to false detection or missed detection in actual applications, and there is a problem of insufficient detection accuracy. In addition, target detection models in related technologies usually require a large amount of computing resources and memory, which may burden mobile embedded systems during actual deployment.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a crop pest identification method, device, electronic device and storage medium to at least solve the technical problems of insufficient accuracy and high resource consumption of pest detection methods in related technologies.

[0005] According to one aspect of an embodiment of the present application, a method for identifying crop pests is provided, comprising: obtaining a video stream to be identified, and determining an image to be identified based on the video stream to be identified, wherein the video stream to be identified is a video stream containing images of crops; determining a correlation parameter between each pixel in the image to be identified and other pixels except the pixel, and determining a first feature map corresponding to the image to be identified based on the correlation parameter, wherein the correlation parameter is used to characterize the degree of correlation between different pixels in the image to be identified; performing bidirectional feature fusion on first feature maps of different scales to obtain a second feature map, wherein each first feature map of different scales corresponds to a different resolution, and the fusion order of bidirectional feature fusion on first feature maps of different scales includes: in order from high to low resolution, and in order from low to high resolution; determining a detection frame in the image to be identified based on the second feature map, wherein the detection frame is used to indicate an insect pest area in the image to be identified.

[0006] Optionally, determining the image to be identified based on the video stream to be identified includes: acquiring a first image frame and a second image frame in the video stream to be identified, wherein the second image frame is an image frame immediately following the first image frame in the video stream to be identified; determining a differential image based on the first image frame and the second image frame, wherein the pixel value of a pixel in the differential image is used to characterize the difference between the pixel in the first image frame and the second image frame; binarizing the differential image based on a preset pixel threshold, and fusing the binarized differential image with the second image frame to obtain the image to be identified.

[0007] Optionally, based on the correlation parameters, determining the first feature map corresponding to the image to be identified includes: using the backbone network in the target detection model to extract features of the image to be identified, and obtaining original features corresponding to pixels in the image to be identified, wherein the target detection model is used to identify crop pests; based on the correlation parameters between the pixel point and each other pixel point, determining the weighted weight corresponding to the pixel point, and based on the weighted weight, determining the enhanced features corresponding to the pixel point in the image to be identified, wherein the enhanced features are features that capture all image context information of the image to be identified; and fusing the original features and enhanced features corresponding to each pixel point in the image to be identified to obtain the first feature map.

[0008] Optionally, before using the backbone network in the target detection model to extract features from the image to be identified, the method also includes: determining the amount of data in the video stream to be identified and the amount of available computing resources in the system; determining a composite scaling factor based on the amount of data and the amount of available computing resources, wherein the composite scaling factor is used to reduce the amount of computation of the backbone network when performing feature extraction while ensuring computational efficiency; adjusting the network structure parameters of the backbone network based on the composite scaling factor, wherein the network structure parameters include: network depth, network width and resolution, wherein the network depth is used to characterize the number of network layers in the network structure, the network width is used to characterize the number of channels of each network layer, and the resolution is used to characterize the size of the image input to the network structure.

[0009] Optionally, performing bidirectional feature fusion on first feature maps of different scales to obtain a second feature map includes: performing feature fusion on first feature maps of different scales in order of resolution from high to low to obtain a third feature map; performing feature fusion on first feature maps of different scales in order of resolution from low to high to obtain a fourth feature map; respectively determining weight coefficients corresponding to the third feature map and the fourth feature map, and determining the second feature map based on the third feature map, the fourth feature map, and the corresponding weight coefficients.

[0010] Optionally, determining a detection frame in the image to be identified based on the second feature map includes: using a detection head in a target detection model to make predictions based on the second feature map to obtain a detection frame in the image to be identified, wherein the target detection model is trained based on a training data set, and the training data set includes: multiple training images marked with real detection frames; wherein the training process of the target detection model includes: using the target detection model to make predictions based on the training images to obtain a predicted detection frame; determining a target loss function value based on the predicted detection frame and the real detection frame in the training data; and adjusting model parameters of the target detection model based on the target loss function value to complete the training of the target detection model.

[0011] Optionally, determining the target loss function value based on the predicted detection frame and the real detection frame in the training data includes: determining the intersection-and-union ratio between the predicted detection frame and the real detection frame, wherein the intersection-and-union ratio is used to characterize the degree of overlap between the predicted detection frame and the real detection frame; determining the Euclidean distance between the center point of the predicted detection frame and the center point of the real detection frame; determining the angle difference value between the rotation angle of the predicted detection frame relative to the vertical direction and the rotation angle of the real detection frame relative to the vertical direction; determining first size data of the predicted detection frame and second size data of the real detection frame, and determining the size matching degree between the predicted detection frame and the real detection frame based on the first size data and the second size data; determining the target loss function value based on the intersection-and-union ratio, the Euclidean distance, the angle difference value, and the size matching degree.

[0012] According to another aspect of the embodiment of the present application, there is also provided a crop pest identification device, comprising: an image acquisition module, used to acquire a video stream to be identified, and determine an image to be identified based on the video stream to be identified, wherein the video stream to be identified is a video stream containing images of crops; a feature extraction module, used to determine a correlation parameter between each pixel point in the image to be identified and other pixel points except the pixel point, and determine a first feature map corresponding to the image to be identified based on the correlation parameter, wherein the correlation parameter is used to characterize the degree of correlation between different pixels in the image to be identified; a feature fusion module, used to perform bidirectional feature fusion on first feature maps of different scales to obtain a second feature map, wherein each first feature map of different scales corresponds to a different resolution, and the fusion order of bidirectional feature fusion on first feature maps of different scales includes: in order from high to low resolution, and in order from low to high resolution; a target detection module, used to determine a detection frame in the image to be identified based on the second feature map, wherein the detection frame is used to indicate the pest area in the image to be identified.

[0013] According to another aspect of the embodiments of the present application, there is also provided an electronic device, including: a memory and a processor, the processor being configured to run a program stored in the memory, wherein the crop pest identification method is executed when the program is run.

[0014] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, the non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the crop pest identification method by running the computer program.

[0015] According to another aspect of the embodiments of the present application, a computer program product is provided, including a computer program, which implements the steps of the crop pest identification method when the computer program is executed by a processor.

[0016] In an embodiment of the present application, a video stream to be identified is obtained, and an image to be identified is determined based on the video stream to be identified, wherein the video stream to be identified is a video stream containing images of crops; a correlation parameter between each pixel in the image to be identified and other pixels except the pixel is determined, and a first feature map corresponding to the image to be identified is determined based on the correlation parameter, wherein the correlation parameter is used to characterize the degree of correlation between different pixels in the image to be identified; bidirectional feature fusion is performed on the first feature maps of different scales to obtain a second feature map, wherein each first feature map of different scales corresponds to a different resolution, and the fusion order of bidirectional feature fusion of the first feature maps of different scales includes: in order from high to low resolution, and in order from low to high resolution; a detection frame is determined in the image to be identified based on the second feature map, wherein the detection frame is used to indicate the pest area in the image to be identified, and by optimizing the target detection algorithm and the model structure, the purpose of improving detection accuracy, realizing real-time detection, and reducing the consumption of computing and storage resources is achieved, thereby solving the technical problems of insufficient accuracy and large resource consumption of pest detection methods in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 It is a hardware structure block diagram of a computer terminal (or electronic device) for implementing a method for identifying crop pests according to an embodiment of the present application;

[0019] Figure 2 is a schematic diagram of a method flow for identifying crop pests provided in an embodiment of the present application;

[0020] Figure 3 It is a schematic diagram of a lightweight crop pest video recognition and detection method process provided according to an embodiment of the present application;

[0021] Figure 4 It is a structural schematic diagram of a crop pest identification device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] In order to facilitate those skilled in the art to better understand the embodiments of the present application, some technical terms or nouns involved in the embodiments of the present application are explained as follows:

[0025] Frame Difference Method: It is a video processing technique used to detect the motion of objects in a video. Its basic principle is to calculate the pixel differences between adjacent video frames to identify the areas that have changed between these frames. This method can effectively highlight dynamic targets, reduce background interference, and improve the accuracy and efficiency of detection.

[0026] Efficient Net: is an efficient convolutional neural network architecture that uses compound scaling to evenly adjust the depth, width, and resolution of the network, thereby significantly reducing the amount of computation and model size while maintaining efficient feature extraction capabilities. Efficient Net performs well on multiple visual tasks and is particularly suitable for resource-constrained devices.

[0027] Non-local neural network: is a deep learning mechanism for capturing global contextual information. It enhances the globality and contextual information of feature representation by calculating the correlation between all pixels in the image. This method can improve the quality of feature extraction, especially when dealing with complex scenes and high-order semantic information.

[0028] BiFPN (Bi-directional Feature Pyramid Network): is an improved version of the Feature Pyramid Network (FPN), which aims to improve the efficiency of multi-scale feature fusion. Compared with PANet (Path Aggregation Network) in related technologies, BiFPN can reduce computational overhead while maintaining feature expression capabilities through weighted feature fusion and simplified connection methods. BiFPN has higher computational efficiency and stronger feature utilization capabilities.

[0029] SIoU (Siamese Intersection over Union) loss function: is an improved bounding box regression loss function designed to improve the accuracy of small object detection. It optimizes the bounding box regression accuracy by considering the geometric center distance, angle and scale matching between the target box and the predicted box. Compared with the CIoU (CompleteIoU) loss function in related technologies, SIoU performs better in small and fine-grained object detection.

[0030] With the modernization and intelligence of agricultural production, real-time monitoring and prevention of crop pests have become increasingly important. Pests not only have a serious impact on crop yield and quality, but may also cause environmental problems. Therefore, the development of an efficient pest monitoring system is of great significance for improving agricultural production efficiency and ensuring food safety.

[0031] In related technologies, crop pest detection technology is usually based on computer vision technology and target detection algorithms. These technologies use deep learning models to identify and locate pests in images or videos. Common target detection algorithms include: YOLO (You Only Look Once) series algorithms, SSD (Single Shot Multi Box Detector) algorithm, FasterR-CNN algorithm, etc. These algorithms usually use convolutional neural networks (CNN) as the backbone network, and combine feature extraction and region proposal technology to achieve target detection and positioning, which usually have the following problems:

[0032] 1) Insufficient accuracy: Due to the diversity and complexity of crop pests, related technical solutions may face the problem of false detection or missed detection in practical applications. This is mainly because different types of pests may have similar appearance characteristics, and changes in the background environment may also affect the detection accuracy.

[0033] 2) Unable to meet real-time requirements: The detection of crop pests needs to be carried out in real-time video streams so that timely discovery and prevention measures can be taken. However, the relevant technical solutions have certain deficiencies in processing speed and real-time performance, and may not meet the needs of real-time monitoring, especially in resource-constrained environments.

[0034] 3) High resource consumption: The training and reasoning process of deep learning models usually requires a lot of computing resources and memory. The training process requires a large amount of labeled data and computing power, and in the actual deployment process, the reasoning of the model may also require high computing resources and memory consumption, which may cause a burden on mobile embedded systems.

[0035] In order to solve the above problems, relevant solutions are provided in the embodiments of the present application, and a lightweight target detection method based on YOLO improvement is proposed. By optimizing the feature extraction, target detection and post-processing steps of the model, and combining a lightweight network structure, it aims to solve the problems of insufficient accuracy, poor real-time performance and high resource consumption in crop pest detection of related technical solutions, which are described in detail below.

[0036] According to an embodiment of the present application, a method embodiment for identifying crop pests is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal (or electronic device) for implementing a method for identifying crop pests. Figure 1As shown, the computer terminal 10 (or electronic device) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0038] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or electronic device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0039] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the crop pest identification method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the above-mentioned crop pest identification method is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0040] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0041] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or electronic device).

[0042] Under the above operating environment, the present application embodiment provides a method for identifying crop pests. Figure 2 is a schematic diagram of a method flow for identifying crop pests according to an embodiment of the present application, such as Figure 2 As shown, the method comprises the following steps:

[0043] Step S202, obtaining a video stream to be identified, and determining an image to be identified based on the video stream to be identified, wherein the video stream to be identified is a video stream containing an image of crops;

[0044] Step S204, determining a correlation parameter between each pixel point in the image to be identified and other pixel points except the pixel point, and determining a first feature map corresponding to the image to be identified based on the correlation parameter, wherein the correlation parameter is used to characterize the degree of correlation between different pixel points in the image to be identified;

[0045] Step S206, performing bidirectional feature fusion on the first feature maps of different scales to obtain a second feature map, wherein each first feature map of different scales corresponds to a different resolution, and the fusion order of performing bidirectional feature fusion on the first feature maps of different scales includes: an order from high to low resolution, and an order from low to high resolution;

[0046] Step S208: determining a detection frame in the image to be identified based on the second feature map, wherein the detection frame is used to indicate an insect pest area in the image to be identified.

[0047] Through the above steps, by optimizing the target detection algorithm and model structure, the purpose of improving detection accuracy, realizing real-time detection, and reducing the consumption of computing and storage resources is achieved, thereby solving the technical problems of insufficient accuracy and high resource consumption of pest detection methods in related technologies.

[0048] The following is a further introduction to the crop pest identification method in steps S202 to S208 of the embodiment of the present application.

[0049] Figure 3 is a schematic diagram of a lightweight crop pest video recognition and detection method process provided in accordance with an embodiment of the present application, such as Figure 3 shown.

[0050] First, obtain the video stream to be identified. For example, the system can capture the real-time video stream from the smart camera through the OpenCV library. Then, the video stream data can be enhanced and optimized through the inter-frame difference method to obtain the image to be identified in each frame of the video stream. The specific steps are as follows.

[0051] In some embodiments of the present application, determining an image to be identified based on a video stream to be identified includes the following steps: acquiring a first image frame and a second image frame in the video stream to be identified, wherein the second image frame is an image frame immediately following the first image frame in the video stream to be identified; determining a differential image based on the first image frame and the second image frame, wherein the pixel value of a pixel in the differential image is used to characterize the difference between the pixel in the first image frame and the second image frame; binarizing the differential image based on a preset pixel threshold, and fusing the binarized differential image with the second image frame to obtain the image to be identified.

[0052] In order to enhance the detection capability of dynamic targets, the embodiment of the present application introduces the inter-frame difference method in the input preprocessing stage. The moving target will show significant pixel changes between adjacent frames, while the static background changes are small. By analyzing the differences between video frames, the moving target (such as insect pests) can be better highlighted and the interference of the static background on the model can be reduced. The specific inter-frame difference result process is as follows: First, extract continuous frames from the video, set the current frame (i.e., the second image frame) as I t and the previous frame (i.e., the first image frame) is I t-1 ; Then calculate the current frame I t and the previous frame I t-1 The difference between them is obtained by t , where each pixel value of the differential image represents the degree of difference between the current frame and the previous frame. The calculation formula of the differential image is as follows:

[0053] D t (i,j)=|I t (i,j)-I t-1 (i,j)|

[0054] Among them, D t (i, j) is the difference image value at position (i, j), I t (i, j) and It-1 (i, j) are the pixel values ​​at position (i, j) of the current frame (second image frame) and the previous frame (first image frame), respectively.

[0055] Afterwards, the difference image D t Apply thresholding (binarization) to facilitate further processing. By setting a threshold T (i.e., preset pixel threshold), the part of the difference image that is greater than the threshold is retained, and the other parts are set to 0. The image B after thresholding t Used to highlight moving targets. The threshold formula is as follows:

[0056]

[0057] Among them, B t (i, j) is the pixel value of the thresholded image, and T is the preset threshold, that is, the preset pixel threshold.

[0058] Finally, the difference image B after thresholding (binarization) t and the current frame image (second image frame) I t Fusion is performed to enhance the saliency of the moving target. For example, a simple weighted sum operation can be used to combine the two types of image information to obtain the final processed image (i.e., the image to be identified) It′, where the fusion formula is shown as follows:

[0059] I′ t (i,j)=α·I t (i,j)+(1-α)·B t (i,j)

[0060] Among them, It′(i,j) is the fused image, that is, the image to be recognized, and α is the weight coefficient, which is used to adjust the contribution ratio of the current frame image and the difference image. Usually α takes a value between 0 and 1.

[0061] The embodiment of the present application can effectively highlight moving targets in the video, such as insect pests on crops, and reduce the interference of static background in the preprocessing stage by introducing the inter-frame difference method. The significance of moving targets can be enhanced: by calculating the inter-frame difference, the moving targets will appear as obvious features in the differential image, thereby improving the model's ability to detect dynamic targets; background interference can be reduced: thresholding can significantly reduce the impact of static background, allowing the model to focus more on moving targets; detection efficiency can be improved: the fused image combines motion information with the original image features, improving the detection accuracy of the detection algorithm for dynamic targets such as insect pests. Through these improvements, YOLOv5's ability to detect crop pests when processing video data can be improved, achieving more efficient and accurate real-time monitoring.

[0062] After obtaining the image to be identified, the target detection model can be used to extract the image to be identified. In the model of the related art (such as YOLOv5), the main task of the feature extraction module is to extract useful features from the input image for target detection. However, these feature extraction modules have certain limitations in capturing the global context information of the image. In order to overcome this problem, the embodiment of the present application introduces a non-local neural network (Non-Local Neural Networks) mechanism, which enhances the globality and context information of feature representation by calculating the correlation between all pixels in the image, thereby improving the quality of feature extraction and overall detection accuracy, wherein the non-local neural network mechanism is a technology that enhances feature representation capabilities through global context information. Different from the characteristics of the local receptive field in the traditional convolutional neural network (CNN), the non-local neural network can calculate the correlation between all pixels in the image, so it can capture richer global information, and the specific steps of feature extraction are as follows.

[0063] In some embodiments of the present application, determining the correlation parameters between each pixel point in the image to be identified and other pixel points except the pixel point, and determining the first feature map corresponding to the image to be identified based on the correlation parameters includes the following steps: using the backbone network in the target detection model to extract features of the image to be identified, and obtaining the original features corresponding to the pixels in the image to be identified, wherein the target detection model is used to identify crop pests; determining the weighted weights corresponding to the pixels based on the correlation parameters between the pixels and each other pixel point, and determining the enhanced features corresponding to the pixels in the image to be identified based on the weighted weights, wherein the enhanced features are features that capture all image context information of the image to be identified; fusing the original features and enhanced features corresponding to each pixel point in the image to be identified to obtain the first feature map.

[0064] Specifically, for each pixel x in the image to be identified i , calculate its difference with all other pixels x j In the embodiment of the present application, this correlation can be measured by the inner product (i.e., the correlation parameter), and the specific calculation formula of the correlation parameter is as follows:

[0065]

[0066] Where: φ(·) represents the function that transforms the input features through a linear transformation (weight matrix). q and W k They are the weight matrices of query and key respectively.

[0067] Then, the calculated correlation parameter is used to perform weighted averaging on the features of all pixels, where the weighted average calculation formula is as follows:

[0068]

[0069] Among them, Output i Represents the enhanced features obtained after weighted averaging, Softmax(Affinity(x i ,x j )) means converting the correlation parameter into weighted weight and normalizing it. Wv is the weight matrix of value. φ(W v x j ) represents a linear transformation of the features of each pixel.

[0070] Finally, the enhanced features after weighted averaging are fused with the original features to obtain a first feature map. In this embodiment, the feature fusion process can be achieved by simple addition or concatenation, as shown in the following formula:

[0071] Final Output i =Output i +x i

[0072] Among them: Final Output i is the feature in the first feature map obtained after fusion, x i is the original feature of the input. Output i It is the enhanced feature calculated by weighted average.

[0073] The embodiments of the present application can significantly improve the global information capture capability of the feature extraction module by introducing a non-local neural network mechanism. The specific effects include: enhancing global context information: the non-local mechanism can capture a wider range of feature correlations by considering the context information of the entire image, thereby making the feature representation richer and more comprehensive. Improving the accuracy of feature representation: by calculating the correlation between all pixels, long-distance dependencies can be accurately captured, improving the recognition ability and accuracy of features. Improving overall detection accuracy: as the quality of feature extraction is improved, the accuracy of final target detection will also be significantly improved, reducing false detections and missed detections. This makes YOLOv5 more advantageous in applications such as crop pest detection.

[0074] It should be noted that the model in the related art (such as the original YOLOv5 network) uses CSPDarknet53 as the backbone network, which improves the detection accuracy while increasing the computational complexity. In order to solve the problem of high computational complexity, the embodiment of the present application replaces the backbone network with EfficientNet. Among them, EfficientNet is an efficient convolutional neural network architecture. The design adopts a composite scaling method to optimize the depth, width and resolution of the network to achieve the best computational efficiency and model performance. Through the composite scaling method, the amount of calculation is significantly reduced while maintaining efficient feature extraction capabilities, as follows.

[0075] In some embodiments of the present application, before using the backbone network in the target detection model to perform feature extraction on the image to be identified, the method also includes the following steps: determining the data amount of the video stream to be identified and the amount of available computing resources in the system; determining a composite scaling factor based on the data amount and the amount of available computing resources, wherein the composite scaling factor is used to reduce the amount of computation of the backbone network when performing feature extraction while ensuring computational efficiency; adjusting the network structure parameters of the backbone network based on the composite scaling factor, wherein the network structure parameters include: network depth, network width and resolution, wherein the network depth is used to characterize the number of network layers in the network structure, the network width is used to characterize the number of channels of each network layer, and the resolution is used to characterize the size of the image input to the network structure.

[0076] In an embodiment of the present application, EfficientNet can use a composite scaling method to evenly adjust the depth, width and resolution of the network. Specifically, a composite scaling factor φ (including a scaling factor of α for network depth, a scaling factor of β for width, and a scaling factor of γ for resolution) is used to simultaneously adjust the depth, width and resolution, where the depth (D) is the number of layers in the network, which determines the expressive power of the model. Increasing the network depth can improve the feature extraction capability of the model, but it will also increase the amount of calculation and the complexity of the model; the width (W) is the number of channels in each layer of the network, and increasing the width can improve the feature representation capability of each layer, but it will also increase the amount of calculation; the resolution (R) is the resolution of the input image, and a higher input resolution can capture more detailed information, but the computing requirements will also increase accordingly.

[0077] The above compound scaling factor φ (including the scaling factor of network depth α, the scaling factor of width β, and the scaling factor of resolution γ) can be obtained through experiments and is used to balance the computational efficiency and performance of the network. The specific compound scaling formula is shown as follows:

[0078] EfficientNet(φ)=B(α.D,β.W,γ·R)

[0079] Among them, B represents the basic network structure, α is the depth scaling factor, β is the width scaling factor, γ is the resolution scaling factor, D represents the number of layers of the network, W represents the number of channels in each layer, and R represents the resolution of the input image.

[0080] In the embodiment of the present application, after replacing CSPDarknet53 with EfficientNet in YOLOv5, the optimization process of the network structure is as follows: First, select the EfficientNet version suitable for the target task (such as EfficientNet-B0, B1, B2, etc.), each version has different depth, width and resolution. After that, select the appropriate scaling factor φ according to the task requirements (such as the amount of data in the video stream to be identified) and computing resources (such as the amount of available computing resources in the system). For example, EfficientNet-B0 is the basic version, and EfficientNet-B1, B2, B3, etc. are extensions based on the B0 version. Since EfficientNet has optimized the volume and computational complexity of the network during design. Therefore, during the replacement process, EfficientNet can reduce the amount of computation and parameters of the model without significantly losing the feature extraction capability.

[0081] The embodiment of the present application replaces CSPDarknet53 with EfficientNet, so that the backbone network of YOLOv5 can reduce the computational complexity while maintaining the feature extraction capability: EfficientNet optimizes the network structure through compound scaling, and can achieve efficient feature extraction with less computation than CSPDarknet53; reduces the model size: the design of EfficientNet makes the network smaller and has fewer parameters, thereby reducing the storage and computing requirements of the model in actual deployment; maintains efficient feature extraction capability: despite the reduction in computation, EfficientNet can still provide efficient feature extraction capabilities, ensuring that the target detection accuracy will not be significantly reduced due to network simplification. Through these improvements, the lightweight YOLOv5 model of the present application can achieve higher efficiency and lower computing resource consumption when processing crop pest detection, while maintaining excellent detection performance.

[0082] After obtaining the first feature map, it is necessary to perform bidirectional feature fusion on the first feature maps of different scales to obtain the second feature map, which is described in detail below.

[0083] In the target detection task, the Feature Pyramid Network (FPN) is used to enhance feature expression at different scales in order to effectively detect targets of different sizes. The original version of YOLOv5 uses Path Aggregation Network (PANet) for feature fusion. However, the computational complexity of PANet is high. In order to further reduce the computational overhead, the embodiment of the present application replaces PANet with Bi-directional Feature Pyramid Network (BiFPN). BiFPN can reduce the computational burden while ensuring the detection effect through weighted feature fusion and simplified connection mode. Among them, BiFPN is an improved feature pyramid network structure, which aims to improve multi-scale feature representation through efficient feature fusion strategy while reducing computational overhead. The core idea of ​​BiFPN is to improve feature utilization efficiency and reduce redundant calculations through bidirectional feature fusion and weighting mechanism, as follows.

[0084] In some embodiments of the present application, bidirectional feature fusion is performed on first feature maps of different scales to obtain a second feature map, including the following steps: feature fusion is performed on first feature maps of different scales in order of resolution from high to low to obtain a third feature map; feature fusion is performed on first feature maps of different scales in order of resolution from low to high to obtain a fourth feature map; weight coefficients corresponding to the third feature map and the fourth feature map are respectively determined, and the second feature map is determined based on the third feature map, the fourth feature map, and the corresponding weight coefficients.

[0085] Specifically, BiFPN enhances the complementarity of features at different levels through bidirectional feature fusion. Assume that there are multiple levels L1, L2, ..., L n (L1 is a high-resolution feature map, L n is a low-resolution feature map), BiFPN considers the fusion of the upper and lower directions when fusing features:

[0086] Bottom-up fusion (from low resolution to high resolution) is as follows:

[0087] F u (L i )=Conv(ReLU(W i ·L i +B i ))

[0088] Among them, W i and B i are the weights and biases of the convolutional layer, Conv represents the convolution operation, ReLU represents the activation function, and F u (L i) represents the feature map after bottom-up fusion, that is, the fourth feature map mentioned above.

[0089] Top-down fusion (from high resolution to low resolution) is as follows:

[0090] F d (L i )=Conv(ReLU(W i ·L i +B i ))

[0091] Among them, W i are the weights and biases of the convolutional layer, Conv represents the convolution operation, ReLU represents the activation function, and F d (L i ) represents the feature map after top-down fusion, that is, the third feature map mentioned above.

[0092] The final bidirectional fused feature map (i.e. the second feature map mentioned above) F i It can be obtained by weighted fusion, as shown in the following formula:

[0093] F i =α·F u (L i )+β·F d (L i )

[0094] Among them, α and β are fusion weights (i.e., the above-mentioned weight coefficients), which are obtained through learning to optimize the fusion effect.

[0095] In addition, during the feature fusion process, BiFPN also introduces a weighting mechanism to optimize the fusion efficiency of features. Specifically, the contribution of feature maps of different scales is calculated by weighted averaging. The formula of the weighting mechanism is as follows:

[0096]

[0097] Among them, w i,j is the feature map L j For the feature map F i Weights,The weights can be learned through the network to optimize the feature fusion process.

[0098] In addition, BiFPN reduces the amount of computation by simplifying the connection between feature maps. For example, a simple addition operation can be used instead of a complex connection operation: i =L i +Conv(L i ), the feature map is processed by lightweight convolution operations instead of using multiple convolutional layers for feature fusion.

[0099] The embodiment of the present application improves the efficiency of feature utilization by adopting BiFPN: the weighted feature fusion mechanism can effectively improve the information utilization of the feature map at each scale, thereby making the feature map more accurate and comprehensive; reduces the computational overhead: the simplified connection method and the two-way fusion strategy reduce the computational complexity, while retaining rich multi-scale feature information and reducing the computational burden; ensures the detection effect: despite the reduction in computational overhead, BiFPN can maintain good detection performance, ensure that sufficient feature information can be obtained at different scales, and improve the overall detection accuracy. In summary, by replacing PANet with BiFPN, the efficiency of feature fusion can be improved while reducing the computational overhead, thereby improving the performance of YOLOv5 in target detection tasks.

[0100] After the second feature map is obtained by feature fusion, a detection frame for indicating the pest area in the image to be identified can be determined in the image to be identified based on the second feature map. The specific steps are as follows.

[0101] In some embodiments of the present application, determining a detection frame in an image to be identified based on a second feature map includes the following steps: using a detection head in a target detection model to make predictions based on the second feature map to obtain a detection frame in the image to be identified, wherein the target detection model is trained based on a training data set, and the training data set includes: a plurality of training images marked with real detection frames; wherein the training process of the target detection model includes: using a target detection model to make predictions based on the training images to obtain a predicted detection frame; determining a target loss function value based on the predicted detection frame and the real detection frame in the training data; and adjusting the model parameters of the target detection model based on the target loss function value to complete the training of the target detection model.

[0102] Specifically, in the original YOLOv5, CIoU (Complete Intersection over Union) is used as the loss function for bounding box regression, which optimizes the prediction of the bounding box by comprehensively considering factors such as the overlap of the bounding box, the center point distance, and the aspect ratio. However, for the detection of small targets, CIoU may perform poorly due to the small size of the target. In order to improve the detection accuracy of small targets, the embodiment of the present application introduces the SIoU (SiameseIoU) loss function, which further optimizes the regression effect of the bounding box by additionally considering the geometric center distance, angle, and scale matching between the target box and the prediction box, as follows.

[0103] In some embodiments of the present application, determining a target loss function value based on a predicted detection frame and a real detection frame in training data includes the following steps: determining an intersection-and-union ratio between the predicted detection frame and the real detection frame, wherein the intersection-and-union ratio is used to characterize the degree of overlap between the predicted detection frame and the real detection frame; determining the Euclidean distance between the center point of the predicted detection frame and the center point of the real detection frame; determining an angle difference value between a rotation angle of the predicted detection frame relative to a vertical direction and a rotation angle of the real detection frame relative to a vertical direction; determining first size data of the predicted detection frame and second size data of the real detection frame, and determining a size matching degree between the predicted detection frame and the real detection frame based on the first size data and the second size data; determining a target loss function value based on the intersection-and-union ratio, the Euclidean distance, the angle difference value, and the size matching degree.

[0104] The SIoU loss function aims to improve the accuracy of bounding box regression by introducing more geometric information, especially in small target detection tasks. It is an extension of the traditional IoU (Intersection over Union) loss function, which introduces the geometric center distance, angle and scale matching between the target box and the predicted box, and improves the positioning accuracy of small targets. Specifically, the SIoU loss function combines the following information to calculate the target loss function value:

[0105] 1) IoU (Intersection over Union): IoU measures the degree of overlap between the predicted box and the true box and is a basic loss calculation indicator. The calculation formula is as follows:

[0106]

[0107] Among them, Area of ​​Intersection is the intersection area of ​​the predicted detection box and the real detection box, and Area of ​​Union is their union area.

[0108] 2) Center point distance

[0109] The SIoU loss function calculates the Euclidean distance between the center points of the predicted detection box and the true detection box. If the center of the predicted detection box is (x p ,y p ), the center of the real detection box is (x t ,y t ), then the center point distance D c for:

[0110]

[0111] 3) Angle difference

[0112] The SIoU loss function takes into account the angle difference between the predicted detection box and the real detection box. If the rotation angle of the predicted detection box is θ p , the rotation angle of the real detection frame is θ t , then the angle difference Δθ is:

[0113] Δθ=|θ p -θ t |

[0114] 4) Scale matching

[0115] SIoU also considers the scale matching between the real detection box and the predicted detection box, which is usually described by the ratio of the width and height of the detection box. If the width and height of the predicted detection box are w p and h p , the width and height of the real detection box are w t and h t , then the scale matching degree S m for:

[0116]

[0117] In the implementation of this application, taking into account the above-mentioned factors such as intersection-over-union ratio, Euclidean distance, angle difference value, and size matching degree, the SIoU loss function can be expressed as:

[0118] SIoU=1-IoU+λ1·D c +λ2·Δθ+λ3·(1-S m )

[0119] Among them, λ1, λ2, and λ3 are weight coefficients used to adjust the contribution of different parts to the total loss.

[0120] By introducing the SIoU loss function in the embodiment of the present application, YOLOv5 can achieve significant improvements in small target detection tasks. Specifically, the positioning accuracy is improved: the SIoU loss function effectively improves the positioning accuracy of small targets by comprehensively considering the geometric center distance, angle difference and scale matching degree; optimizes small target detection: SIoU focuses on the precise positioning of small targets and enhances the model's detection ability for small targets, thereby performing better in scenarios such as crop pest detection. It can better handle the matching problem of the target box: by introducing additional geometric information, SIoU can better handle the matching problem of the bounding box and improve the overall detection performance. In summary, replacing CIoU with the SIoU loss function can effectively optimize the bounding box regression performance of YOLOv5, especially when dealing with small targets, significantly improving the detection accuracy and performance of the model.

[0121] It should be noted that the network element devices involved in the solutions in the above embodiments may include embedded smart cameras, mobile embedded systems, communication modules, etc. The various network element devices are connected through the communication module to achieve data transmission and interaction. Among them, the smart camera is used to obtain the image to be identified, the mobile embedded system is used to run the algorithm to achieve target detection and prediction, and the communication module is used to transmit the monitoring results. The algorithm model can be deployed on the mobile embedded system, and the image processing, target detection and communication functions can be realized through software.

[0122] This application scheme introduces the inter-frame difference method and non-local neural network mechanism, so that the model can more effectively capture dynamic targets and global context information, enhance the globality and context information of feature representation, improve the quality and accuracy of feature extraction, enhance the recognition ability of the model, enable the model to better understand and identify targets in complex backgrounds, improve the detection accuracy of dynamic targets such as crop pests, better handle various types of pests, and reduce false detection and missed detection; use EfficientNet as the backbone network and BiFPN as the feature fusion module to greatly reduce the computational complexity and resource consumption of the model. While maintaining efficient feature extraction and fusion capabilities, these improvements ensure that the model runs more efficiently and is suitable for resource-constrained environments, so that the model can be effectively deployed on resource-constrained embedded devices and reduce deployment costs; the SIoU loss function is used to optimize the positioning accuracy of small targets, so that the model performs better when detecting small targets (such as pests on crops). Reduce false detection and missed detection of small targets and improve the overall detection effect; by simplifying the feature pyramid network and optimizing the loss function, the overall efficiency of the system has been improved. The improved system can process and respond to detection results more quickly to meet the needs of real-time monitoring, improve the performance and efficiency of the system, and ensure that pests can be quickly and accurately identified in real-time video streams.

[0123] According to an embodiment of the present application, an embodiment of a crop pest identification device is also provided. Figure 4 Schematic diagram of a crop pest identification device provided according to an embodiment of the present application. Figure 4 As shown, the device comprises:

[0124] The image acquisition module 40 is used to acquire the video stream to be identified and determine the image to be identified based on the video stream to be identified, wherein the video stream to be identified is a video stream containing images of crops;

[0125] A feature extraction module 42 is used to determine a correlation parameter between each pixel point in the image to be identified and other pixel points except the pixel point, and determine a first feature map corresponding to the image to be identified based on the correlation parameter, wherein the correlation parameter is used to characterize the degree of correlation between different pixel points in the image to be identified;

[0126] The feature fusion module 44 is used to perform bidirectional feature fusion on the first feature maps of different scales to obtain a second feature map, wherein each first feature map of different scales corresponds to a different resolution, and the fusion order of the bidirectional feature fusion on the first feature maps of different scales includes: in order from high to low resolution, and in order from low to high resolution;

[0127] The target detection module 46 is used to determine a detection frame in the image to be identified based on the second feature map, wherein the detection frame is used to indicate the pest area in the image to be identified.

[0128] Optionally, determining the image to be identified based on the video stream to be identified includes: acquiring a first image frame and a second image frame in the video stream to be identified, wherein the second image frame is an image frame immediately following the first image frame in the video stream to be identified; determining a differential image based on the first image frame and the second image frame, wherein the pixel value of a pixel in the differential image is used to characterize the difference between the pixel in the first image frame and the second image frame; binarizing the differential image based on a preset pixel threshold, and fusing the binarized differential image with the second image frame to obtain the image to be identified.

[0129] Optionally, based on the correlation parameters, determining the first feature map corresponding to the image to be identified includes: using the backbone network in the target detection model to extract features of the image to be identified, and obtaining original features corresponding to pixels in the image to be identified, wherein the target detection model is used to identify crop pests; based on the correlation parameters between the pixel point and each other pixel point, determining the weighted weight corresponding to the pixel point, and based on the weighted weight, determining the enhanced features corresponding to the pixel point in the image to be identified, wherein the enhanced features are features that capture all image context information of the image to be identified; and fusing the original features and enhanced features corresponding to each pixel point in the image to be identified to obtain the first feature map.

[0130] Optionally, before using the backbone network in the target detection model to extract features from the image to be identified, the crop pest identification device is also used to: determine the data volume of the video stream to be identified and the amount of available computing resources in the system; determine a composite scaling factor based on the data volume and the amount of available computing resources, wherein the composite scaling factor is used to reduce the amount of computation of the backbone network when performing feature extraction while ensuring computational efficiency; adjust the network structure parameters of the backbone network based on the composite scaling factor, wherein the network structure parameters include: network depth, network width and resolution, wherein the network depth is used to characterize the number of network layers in the network structure, the network width is used to characterize the number of channels of each network layer, and the resolution is used to characterize the size of the image input to the network structure.

[0131] Optionally, performing bidirectional feature fusion on first feature maps of different scales to obtain a second feature map includes: performing feature fusion on first feature maps of different scales in order of resolution from high to low to obtain a third feature map; performing feature fusion on first feature maps of different scales in order of resolution from low to high to obtain a fourth feature map; respectively determining weight coefficients corresponding to the third feature map and the fourth feature map, and determining the second feature map based on the third feature map, the fourth feature map, and the corresponding weight coefficients.

[0132] Optionally, determining a detection frame in the image to be identified based on the second feature map includes: using a detection head in a target detection model to make predictions based on the second feature map to obtain a detection frame in the image to be identified, wherein the target detection model is trained based on a training data set, and the training data set includes: multiple training images marked with real detection frames; wherein the training process of the target detection model includes: using the target detection model to make predictions based on the training images to obtain a predicted detection frame; determining a target loss function value based on the predicted detection frame and the real detection frame in the training data; and adjusting model parameters of the target detection model based on the target loss function value to complete the training of the target detection model.

[0133] Optionally, determining the target loss function value based on the predicted detection frame and the real detection frame in the training data includes: determining the intersection-and-union ratio between the predicted detection frame and the real detection frame, wherein the intersection-and-union ratio is used to characterize the degree of overlap between the predicted detection frame and the real detection frame; determining the Euclidean distance between the center point of the predicted detection frame and the center point of the real detection frame; determining the angle difference value between the rotation angle of the predicted detection frame relative to the vertical direction and the rotation angle of the real detection frame relative to the vertical direction; determining first size data of the predicted detection frame and second size data of the real detection frame, and determining the size matching degree between the predicted detection frame and the real detection frame based on the first size data and the second size data; determining the target loss function value based on the intersection-and-union ratio, the Euclidean distance, the angle difference value, and the size matching degree.

[0134] It should be noted that the various modules in the above-mentioned crop pest identification device can be program modules (for example, a set of program instructions that implement a certain specific function) or hardware modules. For the latter, it can be expressed in the following forms, but is not limited to this: the expression form of each of the above-mentioned modules is a processor, or the functions of each of the above-mentioned modules are implemented by a processor.

[0135] It should be noted that the crop pest identification device provided in this embodiment can be used to perform Figure 2 The crop pest identification method shown, therefore, the relevant explanations and descriptions of the above-mentioned crop pest identification method are also applicable to the embodiments of the present application and will not be repeated here.

[0136] The embodiment of the present application also provides a non-volatile storage medium, the non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the following crop pest identification method by running the computer program: obtaining a video stream to be identified, and determining an image to be identified based on the video stream to be identified, wherein the video stream to be identified is a video stream containing images of crops; determining a correlation parameter between each pixel point in the image to be identified and other pixel points except the pixel point, and determining a first feature map corresponding to the image to be identified based on the correlation parameter, wherein the correlation parameter is used to characterize the degree of correlation between different pixels in the image to be identified; performing bidirectional feature fusion on the first feature maps of different scales to obtain a second feature map, wherein each first feature map of different scales corresponds to a different resolution, and the fusion order of bidirectional feature fusion on the first feature maps of different scales includes: in order from high to low resolution, and in order from low to high resolution; determining a detection frame in the image to be identified based on the second feature map, wherein the detection frame is used to indicate the pest area in the image to be identified.

[0137] The embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the crop pest identification method described in each embodiment of the present application: obtaining a video stream to be identified, and determining an image to be identified based on the video stream to be identified, wherein the video stream to be identified is a video stream containing images of crops; determining a correlation parameter between each pixel point in the image to be identified and other pixel points except the pixel point, and determining a first feature map corresponding to the image to be identified based on the correlation parameter, wherein the correlation parameter is used to characterize the degree of correlation between different pixels in the image to be identified; performing bidirectional feature fusion on the first feature maps of different scales to obtain a second feature map, wherein each first feature map of different scales corresponds to a different resolution, and the fusion order of bidirectional feature fusion on the first feature maps of different scales includes: in order from high to low resolution, and in order from low to high resolution; determining a detection frame in the image to be identified based on the second feature map, wherein the detection frame is used to indicate the pest area in the image to be identified.

[0138] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0139] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0140] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0141] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0142] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0143] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0144] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for identifying crop pests, characterized in that: include: Acquire a video stream to be identified, and determine an image to be identified based on the video stream to be identified, wherein the video stream to be identified is a video stream containing images of crops; Determine a correlation parameter between each pixel point in the image to be identified and other pixel points except the pixel point, and determine a first feature map corresponding to the image to be identified based on the correlation parameter, wherein the correlation parameter is used to characterize the degree of correlation between different pixel points in the image to be identified; Performing bidirectional feature fusion on the first feature maps of different scales to obtain a second feature map, wherein each of the first feature maps of different scales corresponds to a different resolution, and the fusion order of performing bidirectional feature fusion on the first feature maps of different scales includes: an order from high to low resolution and an order from low to high resolution; A detection frame is determined in the image to be identified based on the second feature map, wherein the detection frame is used to indicate an insect-infested area in the image to be identified.

2. The crop pest identification method according to claim 1, characterized in that: Determining the image to be identified according to the video stream to be identified includes: Acquire a first image frame and a second image frame in the to-be-identified video stream, wherein the second image frame is an image frame immediately following the first image frame in the to-be-identified video stream; Determine a differential image based on the first image frame and the second image frame, wherein a pixel value of a pixel in the differential image is used to represent a difference between the pixel in the first image frame and the second image frame; The differential image is binarized according to a preset pixel threshold, and the binarized differential image is fused with the second image frame to obtain the image to be identified.

3. The crop pest identification method according to claim 1, characterized in that: Determining, according to the correlation parameter, a first feature map corresponding to the image to be identified includes: Using the backbone network in the target detection model, extracting features from the image to be identified, and obtaining original features corresponding to pixels in the image to be identified, wherein the target detection model is used to identify crop pests; Determine a weighted weight corresponding to the pixel point according to the correlation parameter between the pixel point and each other pixel point, and determine an enhanced feature corresponding to the pixel point in the image to be identified according to the weighted weight, wherein the enhanced feature is a feature after capturing image context information of the image to be identified; The original features and the enhanced features corresponding to each pixel in the image to be identified are fused to obtain the first feature map.

4. The crop pest identification method according to claim 3, characterized in that: Before extracting features of the image to be identified using the backbone network in the target detection model, the method further includes: Determining the amount of data of the video stream to be identified and the amount of available computing resources in the system; Determining a composite scaling factor according to the data volume and the available computing resources, wherein the composite scaling factor is used to reduce the amount of computation of the backbone network when performing feature extraction while ensuring computing efficiency; According to the composite scaling factor, the network structure parameters of the backbone network are adjusted, wherein the network structure parameters include: network depth, network width and resolution, the network depth is used to characterize the number of network layers in the network structure, the network width is used to characterize the number of channels of each network layer, and the resolution is used to characterize the size of the image input into the network structure.

5. The crop pest identification method according to claim 1, characterized in that: Performing bidirectional feature fusion on the first feature maps of different scales to obtain a second feature map includes: According to the order of resolution of the first feature maps from high to low, feature fusion is performed on the first feature maps of different scales to obtain a third feature map; According to the order of resolution of the first feature maps from low to high, feature fusion is performed on the first feature maps of different scales to obtain a fourth feature map; The weight coefficients corresponding to the third characteristic map and the fourth characteristic map are determined respectively, and the second characteristic map is determined according to the third characteristic map, the fourth characteristic map, and the corresponding weight coefficients.

6. The crop pest identification method according to claim 1, characterized in that: Determining a detection frame in the image to be recognized according to the second feature map includes: Using a detection head in a target detection model, predicting according to the second feature map, to obtain the detection frame in the image to be identified, wherein the target detection model is trained according to a training data set, and the training data set includes: a plurality of training images marked with real detection frames; Among them, the training process of the target detection model includes: using the target detection model to make predictions based on the training image to obtain a predicted detection frame; determining a target loss function value based on the predicted detection frame and the real detection frame in the training data; and adjusting the model parameters of the target detection model based on the target loss function value to complete the training of the target detection model.

7. The crop pest identification method according to claim 6, characterized in that: Determining a target loss function value according to the predicted detection box and the real detection box in the training data includes: Determining an intersection-and-union ratio between the predicted detection frame and the real detection frame, wherein the intersection-and-union ratio is used to characterize the degree of overlap between the predicted detection frame and the real detection frame; Determine the Euclidean distance between the center point of the predicted detection box and the center point of the real detection box; Determine an angle difference value between a rotation angle of the predicted detection frame relative to the vertical direction and a rotation angle of the real detection frame relative to the vertical direction; Determine first size data of the predicted detection frame and second size data of the real detection frame, and determine a size matching degree between the predicted detection frame and the real detection frame according to the first size data and the second size data; The target loss function value is determined according to the intersection-over-union ratio, the Euclidean distance, the angle difference value, and the size matching degree.

8. A crop pest identification device, characterized in that: include: An image acquisition module, used to acquire a video stream to be identified, and determine an image to be identified based on the video stream to be identified, wherein the video stream to be identified is a video stream containing images of crops; a feature extraction module, used to determine a correlation parameter between each pixel point in the image to be identified and other pixel points except the pixel point, and determine a first feature map corresponding to the image to be identified based on the correlation parameter, wherein the correlation parameter is used to characterize the degree of correlation between different pixel points in the image to be identified; A feature fusion module, configured to perform bidirectional feature fusion on the first feature maps of different scales to obtain a second feature map, wherein each of the first feature maps of different scales corresponds to a different resolution, and the fusion order of the bidirectional feature fusion on the first feature maps of different scales includes: an order from high to low resolution and an order from low to high resolution; The target detection module is used to determine a detection frame in the image to be identified based on the second feature map, wherein the detection frame is used to indicate an insect-infested area in the image to be identified.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the processor is used to run a program stored in the memory, wherein the program executes the crop pest identification method according to any one of claims 1 to 7 when running.

10. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the crop pest identification method according to any one of claims 1 to 7 by running the computer program.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the crop pest identification method according to any one of claims 1 to 7 are implemented.

Images