A method and device for forecasting rice leaf roller moth based on AR glasses

Abstract

The application discloses a rice leaf roller moth flying forecasting method and device based on AR glasses. The method comprises the following steps: acquiring a moth video collected by the AR glasses; determining at least two moth images from the moth video, and inputting each moth image into a convolutional neural network model to obtain each moth image marked with a rice leaf roller moth; determining a flight state of the rice leaf roller moth according to all the moth images marked with the rice leaf roller moth, and counting the number of the rice leaf roller moths based on the flight state of the rice leaf roller moths; and sending the number of the rice leaf roller moths to a client. The flight state of the moths is obtained by processing the collected moth video, and the number of the moths is counted according to the flight state of the moths, so that the number of the moths does not need to be visually observed by a forecasting personnel, the visual fatigue of the forecasting personnel is reduced, the efficiency and accuracy of the forecasting are improved, and the forecasting data has traceability and is convenient for subsequent research.

Description

Technical Field

[0001] This application belongs to the field of intelligent monitoring and forecasting technology for agricultural pests, and specifically relates to a method and device for monitoring and forecasting rice leaf roller moth based on AR glasses. Background Technology

[0002] Rice is one of the most important food crops, with about half the world's population relying on it as their staple food. The Ministry of Agriculture and Rural Affairs' "List of Class I Crop Diseases and Pests" released in September 2020 lists 10 pests, including the rice leaf roller (…). Cnaphalocrocismedinalis Ranked sixth, the damage caused by the rice leaf roller to rice production should not be underestimated. The current technical specification for monitoring and forecasting the rice leaf roller, GB / T 15793-2011, details the survey methods and timing for the rice leaf roller moth. This involves surveying from the first sighting of the moth under a light or in the field until the rice reaches the heading stage. One plot of land each is selected from different growth stages and from fields with good, medium, and poor growth of the main rice varieties. Each plot has a survey area of ​​50 m². 2 ~100 m 2 Holding a 2-meter-long bamboo pole, surveyors slowly move the upper part of the rice clumps against the wind along the paddy field ridges (while simultaneously surveying surrounding weeds before the mid-tillering stage). While moving the poles, surveyors must quickly identify and count the rice leaf roller moths. This demands a high level of professional skill and reaction time from the surveyors, especially when the number of rice leaf roller moths is large and other moths (such as the rice stem borer and rice caltrop) take flight together. The manual moth-driving and counting method requires high professional skills, yields inaccurate data, and lacks traceability. Therefore, it is necessary to develop a simple and convenient survey method and device that can be used by non-professionals to improve the accuracy and timeliness of forecasting.

[0003] In some existing technologies, such as the method, apparatus, and storage medium for identifying moving objects disclosed in application number CN202010206308.7, the method includes: acquiring video; detecting the motion trajectory of a moving object in the acquired video; determining the ordered intra-class distance of the detected motion trajectory based on the motion trajectory of the moving object; for motion trajectories whose ordered intra-class distance meets predetermined conditions, using a pre-trained image classification model to identify each image corresponding to the motion trajectory in the video, and determining whether the video contains a specified object based on the identification result.

[0004] For example, the method for detecting the presence of small animals at night based on trajectory search disclosed in application number CN202011429259.X includes the following steps: background modeling; foreground calculation; merging and extracting pipelines; creating a velocity matching model; velocity pattern matching; movement direction pattern matching; and reclassification. This invention does not rely solely on the appearance features of the target, which can greatly improve the detection rate. Furthermore, by combining pattern matching and deep learning classifiers, it can significantly reduce false detections caused by factors such as lighting conditions.

[0005] Among the technologies mentioned above, it is clear that none of them laid out the corresponding scene before collecting videos or images (thus failing to produce good shooting results). Furthermore, none of them distinguished or counted the categories in the collected videos or images, thus failing to demonstrate the efficient monitoring and reporting effect that is different from manual methods. Summary of the Invention

[0006] To address the aforementioned technical shortcomings—namely, low monitoring efficiency and accuracy due to worker eye fatigue, and the unsuitability of manual counting for high insect population densities, thus affecting overall monitoring effectiveness—this application proposes a monitoring method and device for rice leaf roller moth based on AR glasses. The technical solution is as follows:

[0007] In a first aspect, embodiments of this application provide a method for monitoring and forecasting rice leaf roller moths based on AR glasses, including:

[0008] Acquire moth videos captured by AR glasses;

[0009] At least two frames of moth images are identified from the moth video, and each frame of moth image is input into a convolutional neural network model to obtain each frame of moth image labeled with the rice leaf roller moth; wherein, the convolutional neural network model is trained from multiple sample images labeled with the rice leaf roller moth.

[0010] The flight status of the rice leaf roller moth was determined based on all images of the moths marked with the rice leaf roller moth, and the number of the rice leaf roller moths was counted based on the flight status of the moths.

[0011] Send the number of rice leaf roller moths to the client.

[0012] In one alternative to the first aspect, acquiring the moth video captured by AR glasses includes:

[0013] Send a request to the AR glasses to collect video of the user using a pole to poke at the rice stalks, in order to obtain the video of the user using a pole to poke at the rice stalks captured by the AR glasses;

[0014] The system identifies and processes videos of users poking at rice clumps with a pole. When it detects that the user's hand is in a preset area in the video, it sends a request to the AR glasses to start collecting the first video of poking at the rice clumps. The AR glasses then generate corresponding prompts based on the request and collect the moth video based on the user's first voice command.

[0015] Acquire moth videos captured by AR glasses.

[0016] In another alternative to the first aspect, after detecting that the user's hand is within a preset area in the video of the user using a pole to move the rice stalks, the method further includes:

[0017] Send a request to the AR glasses to capture rice video in order to obtain the rice video captured by the AR glasses;

[0018] When the rice in the rice video is identified as being in the seedling stage, the location where the pole first moves the rice is determined, and a request to start collecting the second video of the rice clump being moved is generated based on the location where the pole first moves the rice.

[0019] Send a request to the AR glasses to start collecting a second video of the moth moving in the rice clumps. The AR glasses will then generate corresponding prompts based on this request and collect video of the moth based on the user's second voice command; or

[0020] When the rice in the rice image is identified as being in the middle to late stage, the location where the pole second-handedly moves the rice is determined, and a third video request to start collecting the moving rice clumps is generated based on the location where the pole second-handedly moves the rice.

[0021] Send a request to the AR glasses to start collecting the third video of the moth plucking the rice clumps. The AR glasses will generate corresponding prompt information based on the request and collect the moth video based on the user's third voice command.

[0022] Acquire moth videos captured by AR glasses.

[0023] In another alternative to the first aspect, at least two frames of moth images are determined from the moth video, including:

[0024] Obtain the recording time corresponding to the moth video, and determine at least two moments from the recording time according to a preset time interval;

[0025] Extract the moth image corresponding to each moment from the moth video.

[0026] In another alternative to the first aspect, the flight status of the rice leaf roller moth is determined based on all images of moths marked with the moth moth, including:

[0027] Kalman filtering was applied to the moth videos to predict the flight trajectory of each rice leaf roller moth in the videos.

[0028] Based on the flight trajectory of each rice leaf roller moth, the predicted flight position and actual flight position of each rice leaf roller moth are determined in each frame of moth images marked with rice leaf roller moths.

[0029] The predicted flight position and the distance between the actual flight position of each rice leaf roller moth in each frame of moth image are calculated, and when the distance is detected to be within a preset distance range, the flight state of the rice leaf roller moth is determined to be continuous flight.

[0030] When the detected distance is not within the preset distance range, the flight status of the rice leaf roller moth is determined to be a new takeoff.

[0031] In another alternative to the first aspect, before calculating the distance between the predicted flight position and the actual flight position of each rice leaf roller moth in each frame of moth image, the method further includes:

[0032] From all images of moths marked with rice leaf roller moth, determine the actual flight distance of each rice leaf roller moth between two adjacent frames of moth images;

[0033] Based on the actual flight location distance and the preset time interval, the flight speed corresponding to each rice leaf roller moth is calculated;

[0034] When the detected distance is within a preset distance range, the flight state of the rice leaf roller moth is determined to be continuous flight, including:

[0035] When the distance detected is within a preset distance range and the flight speed is within a preset speed range, the flight state of the rice leaf roller moth is determined to be continuous flight.

[0036] In another alternative to the first aspect, the number of rice leaf roller moths is counted based on their flight status, including:

[0037] The flight trajectory of each rice leaf roller moth in the moth video is marked according to the flight status of the rice leaf roller moth.

[0038] Count the number of flight tracks whose flight status is continuous flight and count the number of flight tracks whose flight status is newly taken off.

[0039] Secondly, embodiments of this application provide a monitoring device for rice leaf roller moth based on AR glasses, comprising:

[0040] The data acquisition module is used to acquire moth videos captured by the AR glasses;

[0041] The data processing module is used to determine at least two frames of moth images from the moth video, and input each frame of moth image into the convolutional neural network model to obtain each frame of moth image labeled with rice leaf roller moth; wherein, the convolutional neural network model is trained from multiple sample images labeled with rice leaf roller moth.

[0042] The data statistics module is used to determine the flight status of the rice leaf roller moth based on all images of moths marked with the rice leaf roller moth, and to count the number of rice leaf roller moths based on their flight status.

[0043] The data sending module is used to send the number of rice leaf roller moths to the client.

[0044] In one alternative embodiment of the second aspect, the data acquisition module is used for:

[0045] Send a request to the AR glasses to collect video of the user using a pole to poke at the rice stalks, in order to obtain the video of the user using a pole to poke at the rice stalks captured by the AR glasses;

[0046] The system identifies and processes videos of users poking at rice clumps with a pole. When it detects that the user's hand is in a preset area in the video, it sends a request to the AR glasses to start collecting the first video of poking at the rice clumps. The AR glasses then generate corresponding prompts based on the request and collect the moth video based on the user's first voice command.

[0047] Acquire moth videos captured by AR glasses.

[0048] In another alternative solution of the second aspect, the data acquisition module is also used for:

[0049] After detecting that the user's hand is in a preset area in the video of the user holding a stick and moving the rice stalks, a request to collect rice video is sent to the AR glasses to obtain the rice video collected by the AR glasses;

[0050] When the rice in the rice video is identified as being in the seedling stage, the location where the pole first moves the rice is determined, and a request to start collecting the second video of the rice clump being moved is generated based on the location where the pole first moves the rice.

[0051] Send a request to the AR glasses to start collecting a second video of the moth moving in the rice clumps. The AR glasses will then generate corresponding prompts based on this request and collect video of the moth based on the user's second voice command; or

[0052] When the rice in the rice image is identified as being in the middle to late stage, the location where the pole second-handedly moves the rice is determined, and a third video request to start collecting the moving rice clumps is generated based on the location where the pole second-handedly moves the rice.

[0053] Send a request to the AR glasses to start collecting the third video of the moth plucking the rice clumps. The AR glasses will generate corresponding prompt information based on the request and collect the moth video based on the user's third voice command.

[0054] Acquire moth videos captured by AR glasses.

[0055] In another alternative of the second aspect, the data processing module is used for:

[0056] Obtain the recording time corresponding to the moth video, and determine at least two moments from the recording time according to a preset time interval;

[0057] Extract the moth image corresponding to each moment from the moth video.

[0058] In another alternative solution to the second aspect, the data statistics module is used for:

[0059] Kalman filtering was applied to the moth videos to predict the flight trajectory of each rice leaf roller moth in the videos.

[0060] Based on the flight trajectory of each rice leaf roller moth, the predicted flight position and actual flight position of each rice leaf roller moth are determined in each frame of moth images marked with rice leaf roller moths.

[0061] The predicted flight position and the distance between the actual flight position of each rice leaf roller moth in each frame of moth image are calculated, and when the distance is detected to be within a preset distance range, the flight state of the rice leaf roller moth is determined to be continuous flight.

[0062] When the detected distance is not within the preset distance range, the flight status of the rice leaf roller moth is determined to be a new takeoff.

[0063] In another alternative solution to the second aspect, the data statistics module is also used for:

[0064] Before calculating the distance between the predicted flight position and the actual flight position of each rice leaf roller moth in each frame of moth image, the actual flight position distance between two adjacent frames of moth images is determined from all moth images marked with rice leaf roller moths.

[0065] Based on the actual flight location distance and the preset time interval, the flight speed corresponding to each rice leaf roller moth is calculated;

[0066] When the detected distance is within a preset distance range, the flight state of the rice leaf roller moth is determined to be continuous flight, including:

[0067] When the distance detected is within a preset distance range and the flight speed is within a preset speed range, the flight state of the rice leaf roller moth is determined to be continuous flight.

[0068] In another alternative solution to the second aspect, the data statistics module is also used for:

[0069] The flight trajectory of each rice leaf roller moth in the moth video is marked according to the flight status of the rice leaf roller moth.

[0070] Count the number of flight tracks whose flight status is continuous flight and count the number of flight tracks whose flight status is newly taken off.

[0071] Thirdly, this application also provides a monitoring device for rice leaf roller moth based on AR glasses, including a processor and a memory;

[0072] The processor is connected to the memory;

[0073] Memory, used to store executable program code;

[0074] The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the rice leaf roller monitoring method based on AR glasses provided in the first aspect or any implementation of the first aspect of the present application.

[0075] Fourthly, embodiments of this application provide a computer storage medium storing a computer program, which includes program instructions. When executed by a processor, the program instructions can implement the AR glasses-based method for monitoring and forecasting rice leaf rollers provided in the first aspect or any implementation of the first aspect of this application.

[0076] In this embodiment, when monitoring the rice leaf roller moth, video footage of moths captured by AR glasses can be obtained. At least two frames of moth images are identified from the video, and each frame is input into a convolutional neural network model to obtain each frame of moth image labeled with the rice leaf roller moth. The flight status of the rice leaf roller moths is determined based on all the images labeled with the moths, and the number of moths is counted based on their flight status. The number of moths is then sent to the client. By processing the acquired moth video to obtain the moths' flight status and counting them based on that status, the monitoring personnel are no longer required to visually count the moths, reducing eye strain. This also improves the efficiency and accuracy of the monitoring, and the monitoring data is traceable, facilitating subsequent research. Attached Figure Description

[0077] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0078] Figure 1 A flowchart illustrating an AR glasses-based method for monitoring and forecasting rice leaf rollers in an embodiment of this application;

[0079] Figure 2 A schematic diagram of the architecture of a rice leaf roller monitoring system based on AR glasses is provided for an embodiment of this application;

[0080] Figure 3 A schematic diagram of the flight trajectory of the rice leaf roller moth provided in an embodiment of this application;

[0081] Figure 4 A schematic diagram of the structure of a rice leaf roller monitoring device based on AR glasses provided in this application embodiment;

[0082] Figure 5 This is a schematic diagram of the structure of a server provided in an embodiment of this application. Detailed Implementation

[0083] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0084] In the following description, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The following description provides multiple embodiments of this application, which can be substituted or combined with each other. Therefore, this application can also be considered to include all possible combinations of the same and / or different embodiments described. Thus, if one embodiment includes features A, B, and C, and another embodiment includes features B and D, then this application should also be considered to include embodiments containing one or more other possible combinations of A, B, C, and D, even if such embodiments are not explicitly described in the following text.

[0085] The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made to the function and arrangement of the described elements without departing from the scope of this application. Various processes or components may be appropriately omitted, substituted, or added to the examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined into other examples.

[0086] Please see Figure 1 , Figure 1 The diagram shows an overall flowchart of a method for monitoring and forecasting rice leaf roller moths based on AR glasses, as provided in an embodiment of this application.

[0087] like Figure 1 As shown, the AR glasses-based method for monitoring and forecasting rice leaf roller moths may include at least the following steps:

[0088] Step 102: Obtain the moth video captured by the AR glasses.

[0089] In this embodiment of the application, the AR glasses-based method for monitoring and forecasting rice leaf roller moths can be applied to servers, but is not limited to this method. See here for more details. Figure 2 The diagram shown is an architectural schematic of a rice leaf roller monitoring system based on AR glasses, as provided in an embodiment of this application. Figure 2 As shown, the server can establish communication connections with both the AR glasses and the mobile terminal corresponding to the client.

[0090] The AR glasses, worn by the user, allow the user to capture videos of rice leaf rollers (a type of leaf roller) in rice paddies based on voice commands, following prompts. The user can manipulate the rice paddies with a stick to make the leaf rollers fly, and the AR glasses will then capture the video based on the user's voice commands. Specifically, when the user observes the rice paddy image in the AR glasses, they can, but are not limited to, issuing a voice command like "Start video." The AR glasses will recognize and process this command, and after confirming the start of video, begin capturing the video. During video capture, the user can continuously move (while simultaneously manipulating the rice paddies with the stick). After moving a certain distance, the user can, but is not limited to, issuing a voice command like "Stop video." The AR glasses will then recognize and process this command, and after confirming the stop of video, send the captured video to the server.

[0091] In this embodiment, the AR glasses may specifically include an eyeglass frame, a camera, a monocular display, a power board, a main control device, a communication module, a voice control module, and an integrated circuit unit. The camera, monocular display, main control device, communication module, voice control module, and integrated circuit unit are all installed inside the AR glasses. The main control device is connected to the camera, monocular display, communication module, and voice control module respectively through the integrated circuit unit. The eyeglass frame is worn by the user and can be fitted with different prescription lenses or sunglasses. The camera is used to capture videos of moths, including the rice leaf roller moth. The monocular display is used to display the images captured by the camera to the user in real time. The voice control module is used to perform operations such as taking photos, recording videos, and zooming in and out of objects according to the user's voice control, thus achieving "hands-free" operation. The power board provides power to the camera and AR imaging system. The communication module is used to establish a communication connection with the server to transmit the captured images to the server. The main control device is used to control the operation and shutdown of all components in the AR glasses. The integrated circuit unit connects different modules within the glasses' camera.

[0092] The server can receive moth videos captured by AR glasses, extract at least two frames of moth images from these videos, and input each frame into a trained convolutional neural network model to label the rice leaf roller moth in each frame. Then, by combining all the labeled moth images, the server can determine the flight state of each moth appearing in the images, count the moths based on their flight states, and send the counting results to the corresponding mobile terminal of the client for quick viewing. Alternatively, the server can send the moth video along with the counting results to the client's mobile terminal, ensuring a correlation between the counting results and the video footage and guaranteeing the traceability of the counting results.

[0093] The client-side mobile terminal may, but is not limited to, having a rice leaf roller monitoring and survey app installed. This app is used to receive counting results or technical results, along with corresponding moth videos, sent from the server, and to display them to the user. Users can log into their accounts on the app and receive the counting results or technical results, along with the moth videos, manually or automatically. It is understood that the mobile terminal can also obtain real-time time, weather, and geographic information, and users can manually enter information such as field location, rice variety, and survey personnel.

[0094] Specifically, when monitoring the rice leaf roller moth, the server receives moth videos collected by AR glasses. During the moth video collection process, the AR glasses can be used by a user who walks through the rice paddies while wearing them, using a stick to move the rice clumps. When the user sees the rice clumps on the AR glasses' display screen, they can issue a voice command to "record." The AR glasses can then recognize and process this voice command, and after determining that the video has started, they can begin recording the video. During the video recording process, the user can continue to move (while simultaneously moving the rice clumps with the stick). After the user has moved a certain distance, they can issue a voice command to "stop." The AR glasses can then recognize and process this voice command, and after determining that the video has stopped, they can send the collected video to the server.

[0095] As an optional embodiment of this application, acquiring the moth video captured by AR glasses includes:

[0096] Send a request to the AR glasses to collect video of the user using a pole to poke at the rice stalks, in order to obtain the video of the user using a pole to poke at the rice stalks captured by the AR glasses;

[0097] The system identifies and processes videos of users poking at rice clumps with a pole. When it detects that the user's hand is in a preset area in the video, it sends a request to the AR glasses to start collecting the first video of poking at the rice clumps. The AR glasses then generate corresponding prompts based on the request and collect the moth video based on the user's first voice command.

[0098] Acquire moth videos captured by AR glasses.

[0099] To ensure the effectiveness of the user's manipulation of the rice stalks, during the acquisition of moth videos captured by AR glasses, a request to acquire the user's pole-wielding video of the rice stalks can be sent to the AR glasses first. This allows for the determination of the user's pole-wielding position based on the video. After acquiring the user's pole-wielding video, the video can be processed (but is not limited to) to identify the user's handheld portion and the straight pole portion. When the user's handheld portion is detected to be within a preset area corresponding to the straight pole portion, it indicates that the user's pole-wielding position is valid. It is understood that in this embodiment, the straight pole can be, but is not limited to, a 2m long, lightweight, rigid straight pole, and the preset area can correspond to the 0-0.8m range of the straight pole to ensure an effective pole width of over 1m for moth control. Next, when the user's hand is in the preset area corresponding to the straight rod, a request to start collecting the first video of plucking the rice clump is sent to the AR glasses. The AR glasses then generate corresponding prompt information based on the request to start collecting the first video of plucking the rice clump, and collect the moth video based on the user's first voice command. Here, the user's first voice command can be, but is not limited to, a voice command with the content "record video".

[0100] It is also understandable that after obtaining the video of the user plucking the rice stalks with a pole, it is possible, but not limited to, to perform recognition processing on the video of the user plucking the rice stalks with a pole, to identify the user's plucking rate from the video of the user plucking the rice stalks with a pole, and to remind the user through AR glasses when the user's plucking rate exceeds a threshold, so as to further ensure the effectiveness and reliability of the user's plucking of the rice stalks.

[0101] As another optional embodiment of this application, after detecting that the user's hand is in a preset area in the video of the user using a pole to move the rice stalks, the method further includes:

[0102] Send a request to the AR glasses to capture rice video in order to obtain the rice video captured by the AR glasses;

[0103] When the rice in the rice video is identified as being in the seedling stage, the location where the pole first moves the rice is determined, and a request to start collecting the second video of the rice clump being moved is generated based on the location where the pole first moves the rice.

[0104] Send a request to the AR glasses to start collecting a second video of the moth moving in the rice clumps. The AR glasses will then generate corresponding prompts based on this request and collect video of the moth based on the user's second voice command; or

[0105] When the rice in the rice image is identified as being in the middle to late stage, the location where the pole second-handedly moves the rice is determined, and a third video request to start collecting the moving rice clumps is generated based on the location where the pole second-handedly moves the rice.

[0106] Send a request to the AR glasses to start collecting the third video of the moth plucking the rice clumps. The AR glasses will generate corresponding prompt information based on the request and collect the moth video based on the user's third voice command.

[0107] Acquire moth videos captured by AR glasses.

[0108] To further ensure that rice leaf rollers hidden within the rice clumps can take flight as much as possible when users move the rice, the position of moving the rice clumps can be adjusted by recognizing the growth stage of the rice. Specifically, after detecting that the user's hand is within a preset area in the video of the user moving the rice clumps with a stick, a request to collect rice video can be sent to the AR glasses to obtain the rice video captured by the AR glasses. This rice video can be understood as a video specifically used to capture the shape of the rice, and its content may only contain multiple rice shapes in the rice clump. It is possible that when the rice in the video is identified as being in the seedling stage, it indicates that the rice in this rice clump has not yet matured. To avoid damaging the rice when moving the rice clumps, the first part of the stick that is moving the rice can be determined to represent the first part of the rice clump, and a second video request to start collecting the rice clump can be generated based on the first part of the stick that is moving the rice. Here, the second video request to start collecting the rice clump can include prompts indicating that the upper part of the rice clump is being moved and that the moving force is relatively light. Next, a request to start collecting a second video of the rice stalk being moved is sent to the AR glasses. The AR glasses then generate a corresponding prompt message based on this request. This prompt message can be, but is not limited to, indicating that the top of the rice stalk is being gently moved. The prompt message can also be displayed to the user. The AR glasses then collect a video of the moth based on the user's second voice command. This second voice command can be, but is not limited to, a voice command with the content "The top of the rice stalk has been gently moved. Recording".

[0109] It is possible that when the rice in the rice video is identified as being in the middle to late stages, indicating that the rice in that rice clump has matured, in order to encourage the rice leaf roller moth hidden in the rice clump to take flight as much as possible, the location of the second part of the stick used to move the rice in the upper part of the rice clump can be determined, and a third video request to start collecting the rice clump can be generated based on the location of the second part of the stick moving the rice. Here, the request to start collecting the third video of moving the rice clump can include prompts indicating that the upper part of the rice clump is being moved and that the moving force is relatively strong. Next, the request to start collecting the third video of moving the rice clump is sent to the AR glasses, so that the AR glasses can generate corresponding prompts based on the request. These prompts can be, but are not limited to, indicating that the upper part of the rice clump is being moved forcefully, and can be displayed to the user. The AR glasses then collect the moth video based on the user's third voice command, which can be, but is not limited to, a voice command with the content "The upper part of the rice clump has been moved forcefully, record video".

[0110] Step 104: Identify at least two frames of moth images from the moth video, and input each frame of moth image into the convolutional neural network model to obtain each frame of moth image labeled with the rice leaf roller moth.

[0111] Specifically, after acquiring the moth video, it is possible, but not limited to, determining the recording time corresponding to the moth video. For example, the recording time can be the moment when the moth video starts and the moment when the video stops. At least two moments can be determined from the recording time according to a preset time interval. For example, between 13:00 and 13:10, ten moments can be determined at a time interval of 1 second.

[0112] Furthermore, after determining at least two time points, moth images corresponding to each time point can be extracted from the moth video. These images are then input into a trained convolutional neural network (CNN) model to label the rice leaf roller moth in each image. The CNN model can be trained using multiple sample images labeled with the rice leaf roller moth. The predicted target and the true target of the CNN model are determined using the Gaussian Wasserstein distance. The accuracy and recall of the rice leaf roller moth identification are used to evaluate the CNN model, and the optimal CNN model is selected. It is understood that in this embodiment, the CNN model can, but is not limited to, labeling other types of moths; it is not limited to this.

[0113] Step 106: Determine the flight status of the rice leaf roller moth based on all images of moths marked with the rice leaf roller moth, and count the number of rice leaf roller moths based on their flight status.

[0114] Specifically, after obtaining all moth images labeled with the rice leaf roller moth through a convolutional neural network model, the acquired moth video can be processed by Kalman filtering. For example, but not limited to processing the moth video according to a preset Kalman filtering algorithm, the flight trajectory map of each rice leaf roller moth in the moth video can be obtained. The flight trajectory of each rice leaf roller moth in the moth video can be understood as the flight trajectory predicted by the Kalman filtering algorithm, rather than the actual flight trajectory of each rice leaf roller moth.

[0115] See here. Figure 3 The diagram shown illustrates the flight trajectory of a rice leaf roller moth according to an embodiment of this application. Figure 3 As shown, the flight trajectory diagram marks the flight trajectory of each rice leaf roller moth in the moth video, and adjacent flight trajectories can be distinguished by different colors. Figure 3 (Not shown in the image) to effectively and quickly identify different rice leaf roller moths. It can be understood that the flight trajectory of each rice leaf roller moth can be composed of multiple location points, and each location can correspond to the predicted location in the moth image at different times.

[0116] Furthermore, based on the flight trajectory of each rice leaf roller moth in the moth video, the predicted and actual flight positions of each rice leaf roller moth can be determined in each frame of the moth image marked with rice leaf roller moths. Specifically, the flight position corresponding to each moment can be determined in the flight trajectory of each rice leaf roller moth according to at least two moments mentioned above. The flight position corresponding to each moment is also the predicted flight position. Then, each rice leaf roller moth corresponding to the predicted flight position at each moment can be mapped onto the moth image corresponding to the corresponding moment. The position of the rice leaf roller moth closest to each predicted flight position in each moth image is taken as the actual flight position of the corresponding rice leaf roller moth. It is understandable that when determining the location of the nearest rice leaf roller moth, a Cartesian coordinate system can be established based on the edge of the moth image, but is not limited to this. The coordinates corresponding to the center point of the marked rice leaf roller moth can be used as the actual flight position, and the coordinates of the center point of each rice leaf roller moth mapped in the moth image can be used as the predicted flight position. The actual flight position corresponding to the predicted flight position can be determined by calculating the Euclidean distance, and the distance between the predicted flight position and the actual flight position can also be obtained.

[0117] Furthermore, when the distance between the predicted and actual flight positions of the rice leaf roller moth in any moth image is within a preset distance range, it indicates that the moth was not hidden in the rice clump, thus confirming that the moth remained in flight while the user disturbed the rice clump. Conversely, when the distance between the predicted and actual flight positions of the rice leaf roller moth in any moth image is not within a preset distance range, it indicates that the moth had previously hidden in the rice clump, thus confirming that the moth was in a newly launched state while the user disturbed the rice clump.

[0118] As another optional embodiment of this application, before calculating the distance between the predicted flight position and the actual flight position of each rice leaf roller moth in each frame of moth image, the method further includes:

[0119] From all images of moths marked with rice leaf roller moth, determine the actual flight distance of each rice leaf roller moth between two adjacent frames of moth images;

[0120] Based on the actual flight location distance and the preset time interval, the flight speed corresponding to each rice leaf roller moth is calculated;

[0121] When the detected distance is within a preset distance range, the flight state of the rice leaf roller moth is determined to be continuous flight, including:

[0122] When the distance detected is within a preset distance range and the flight speed is within a preset speed range, the flight state of the rice leaf roller moth is determined to be continuous flight.

[0123] To further improve the accuracy of determining the flight status of the rice leaf roller moth, speed detection can also be incorporated. Specifically, before calculating the distance between the predicted and actual flight positions of each rice leaf roller moth in each frame of moth images, the actual flight position distance between two adjacent frames of moth images can be determined from all images of moths marked with rice leaf roller moths. The method for calculating this distance will not be elaborated here. Next, the flight speed of each rice leaf roller moth between any two adjacent frames can be calculated using a preset time interval. When the detected distances are all within a preset distance range, and each flight speed is within a preset speed range, it indicates that the rice leaf roller moth is not hidden in the rice clump, thus confirming that the rice leaf roller moth is continuously in flight while the user is moving the rice clump. In any case where the distances are all within the preset distance range and the flight speeds are all outside the preset speed range, it indicates that the rice leaf roller moth was hiding in the rice clump, which means that the rice leaf roller moth was in a new take-off state when the user moved the rice clump.

[0124] Furthermore, after determining the flight status of each rice leaf roller moth, the flight status of each rice leaf roller moth can be marked in the flight trajectory map of each rice leaf roller moth in the moth video. The number of all marked flight trajectories with continuous flight status can be counted, as well as the number of all marked flight trajectories with newly taken-off status can be counted.

[0125] Step 108: Send the number of rice leaf roller moths to the client.

[0126] Specifically, the number of rice leaf roller moths in continuous flight and the number of newly taken-off rice leaf roller moths can be sent together with the moth video to the corresponding mobile terminal of the client so that users can quickly view them on their mobile terminals.

[0127] Please see Figure 4 , Figure 4 This illustration shows a structural schematic diagram of a rice leaf roller monitoring device based on AR glasses, provided in an embodiment of this application.

[0128] like Figure 4 As shown, the AR glasses-based rice leaf roller monitoring device may include at least a data acquisition module 401, a data processing module 402, a data statistics module 403, and a data transmission module 404, wherein:

[0129] The data acquisition module 401 is used to acquire the moth video captured by the AR glasses;

[0130] The data processing module 402 is used to determine at least two frames of moth images from the moth video, and input each frame of moth image into the convolutional neural network model to obtain each frame of moth image labeled with rice leaf roller moth; wherein, the convolutional neural network model is trained from multiple sample images labeled with rice leaf roller moth.

[0131] The data statistics module 403 is used to determine the flight status of the rice leaf roller moth based on all images of moths marked with the rice leaf roller moth, and to count the number of rice leaf roller moths based on the flight status of the rice leaf roller moth.

[0132] The data sending module 404 is used to send the number of rice leaf roller moths to the client.

[0133] In some possible embodiments, the data acquisition module is used for:

[0134] Send a request to the AR glasses to collect video of the user using a pole to poke at the rice stalks, in order to obtain the video of the user using a pole to poke at the rice stalks captured by the AR glasses;

[0135] The system identifies and processes videos of users poking at rice clumps with a pole. When it detects that the user's hand is in a preset area in the video, it sends a request to the AR glasses to start collecting the first video of poking at the rice clumps. The AR glasses then generate corresponding prompts based on the request and collect the moth video based on the user's first voice command.

[0136] Acquire moth videos captured by AR glasses.

[0137] In some possible embodiments, the data acquisition module is also used for:

[0138] After detecting that the user's hand is in a preset area in the video of the user holding a stick and moving the rice stalks, a request to collect rice video is sent to the AR glasses to obtain the rice video collected by the AR glasses;

[0139] When the rice in the rice video is identified as being in the seedling stage, the location where the pole first moves the rice is determined, and a request to start collecting the second video of the rice clump being moved is generated based on the location where the pole first moves the rice.

[0140] Send a request to the AR glasses to start collecting a second video of the moth moving in the rice clumps. The AR glasses will then generate corresponding prompts based on this request and collect video of the moth based on the user's second voice command; or

[0141] When the rice in the rice image is identified as being in the middle to late stage, the location where the pole second-handedly moves the rice is determined, and a third video request to start collecting the moving rice clumps is generated based on the location where the pole second-handedly moves the rice.

[0142] Send a request to the AR glasses to start collecting the third video of the moth plucking the rice clumps. The AR glasses will generate corresponding prompt information based on the request and collect the moth video based on the user's third voice command.

[0143] Acquire moth videos captured by AR glasses.

[0144] In some possible embodiments, the data processing module is used for:

[0145] Obtain the recording time corresponding to the moth video, and determine at least two moments from the recording time according to a preset time interval;

[0146] Extract the moth image corresponding to each moment from the moth video.

[0147] In some possible embodiments, the data statistics module is used for:

[0148] Kalman filtering was applied to the moth videos to predict the flight trajectory of each rice leaf roller moth in the videos.

[0149] Based on the flight trajectory of each rice leaf roller moth, the predicted flight position and actual flight position of each rice leaf roller moth are determined in each frame of moth images marked with rice leaf roller moths.

[0150] The predicted flight position and the distance between the actual flight position of each rice leaf roller moth in each frame of moth image are calculated, and when the distance is detected to be within a preset distance range, the flight state of the rice leaf roller moth is determined to be continuous flight.

[0151] When the detected distance is not within the preset distance range, the flight status of the rice leaf roller moth is determined to be a new takeoff.

[0152] In some possible embodiments, the data statistics module is also used for:

[0153] Before calculating the distance between the predicted flight position and the actual flight position of each rice leaf roller moth in each frame of moth image, the actual flight position distance between two adjacent frames of moth images is determined from all moth images marked with rice leaf roller moths.

[0154] Based on the actual flight location distance and the preset time interval, the flight speed corresponding to each rice leaf roller moth is calculated;

[0155] When the detected distance is within a preset distance range, the flight state of the rice leaf roller moth is determined to be continuous flight, including:

[0156] When the distance detected is within a preset distance range and the flight speed is within a preset speed range, the flight state of the rice leaf roller moth is determined to be continuous flight.

[0157] In some possible embodiments, the data statistics module is also used for:

[0158] The flight trajectory of each rice leaf roller moth in the moth video is marked according to the flight status of the rice leaf roller moth.

[0159] Count the number of flight tracks whose flight status is continuous flight and count the number of flight tracks whose flight status is newly taken off.

[0160] Those skilled in the art will clearly understand that the technical solutions of the embodiments of this application can be implemented by means of software and / or hardware. In this specification, "unit" and "module" refer to software and / or hardware that can independently complete or cooperate with other components to complete a specific function, wherein the hardware may be, for example, a field-programmable gate array (FPGA), an integrated circuit (IC), etc.

[0161] Please see Figure 5 , Figure 5 A schematic diagram of the structure of a server provided in an embodiment of this application is shown.

[0162] like Figure 5 As shown, the server 500 may include at least a processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502.

[0163] The communication bus 502 can be used to realize the connection and communication of the above components.

[0164] The user interface 503 may include buttons, and the optional user interface may also include a standard wired interface or a wireless interface.

[0165] The network interface 504 may include, but is not limited to, Bluetooth modules, NFC modules, Wi-Fi modules, etc.

[0166] The processor 501 may include one or more processing cores. The processor 501 connects to various parts within the server 500 via various interfaces and lines, and performs various functions and processes data of the routing server 500 by running or executing instructions, programs, code sets, or instruction sets stored in the memory 505, and by calling data stored in the memory 505. Optionally, the processor 501 may be implemented using at least one hardware form of DSP, FPGA, or PLA. The processor 501 may integrate one or more of the following: CPU, GPU, and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 501 and may be implemented as a separate chip.

[0167] The memory 505 may include RAM or ROM. Optionally, the memory 505 may include a non-transitory computer-readable medium. The memory 505 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 505 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 505 may also be at least one storage device located remotely from the aforementioned processor 501. Figure 5 As shown, the memory 505, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an AR glasses-based rice leaf roller monitoring application.

[0168] Specifically, the processor 501 can be used to call the AR glasses-based rice leaf roller monitoring application stored in the memory 505, and specifically perform the following operations:

[0169] Acquire moth videos captured by AR glasses;

[0170] At least two frames of moth images are identified from the moth video, and each frame of moth image is input into a convolutional neural network model to obtain each frame of moth image labeled with the rice leaf roller moth; wherein, the convolutional neural network model is trained from multiple sample images labeled with the rice leaf roller moth.

[0171] The flight status of the rice leaf roller moth was determined based on all images of the moths marked with the rice leaf roller moth, and the number of the rice leaf roller moths was counted based on the flight status of the moths.

[0172] Send the number of rice leaf roller moths to the client.

[0173] In some possible embodiments, acquiring moth videos captured by AR glasses includes:

[0174] Send a request to the AR glasses to collect video of the user using a pole to poke at the rice stalks, in order to obtain the video of the user using a pole to poke at the rice stalks captured by the AR glasses;

[0175] The system identifies and processes videos of users poking at rice clumps with a pole. When it detects that the user's hand is in a preset area in the video, it sends a request to the AR glasses to start collecting the first video of poking at the rice clumps. The AR glasses then generate corresponding prompts based on the request and collect the moth video based on the user's first voice command.

[0176] Acquire moth videos captured by AR glasses.

[0177] In some possible embodiments, after detecting that the user's hand is within a preset area in the video of the user using a pole to move the rice stalks, the method further includes:

[0178] Send a request to the AR glasses to capture rice video in order to obtain the rice video captured by the AR glasses;

[0179] When the rice in the rice video is identified as being in the seedling stage, the location where the pole first moves the rice is determined, and a request to start collecting the second video of the rice clump being moved is generated based on the location where the pole first moves the rice.

[0180] Send a request to the AR glasses to start collecting a second video of the moth moving in the rice clumps. The AR glasses will then generate corresponding prompts based on this request and collect video of the moth based on the user's second voice command; or

[0181] When the rice in the rice image is identified as being in the middle to late stage, the location where the pole second-handedly moves the rice is determined, and a third video request to start collecting the moving rice clumps is generated based on the location where the pole second-handedly moves the rice.

[0182] Send a request to the AR glasses to start collecting the third video of the moth plucking the rice clumps. The AR glasses will generate corresponding prompt information based on the request and collect the moth video based on the user's third voice command.

[0183] Acquire moth videos captured by AR glasses.

[0184] In some possible embodiments, at least two frames of moth images are determined from the moth video, including:

[0185] Obtain the recording time corresponding to the moth video, and determine at least two moments from the recording time according to a preset time interval;

[0186] Extract the moth image corresponding to each moment from the moth video.

[0187] In some possible embodiments, the flight status of the rice leaf roller moth is determined based on all images of moths labeled with the moth, including:

[0188] Kalman filtering was applied to the moth videos to predict the flight trajectory of each rice leaf roller moth in the videos.

[0189] Based on the flight trajectory of each rice leaf roller moth, the predicted flight position and actual flight position of each rice leaf roller moth are determined in each frame of moth images marked with rice leaf roller moths.

[0190] The predicted flight position and the distance between the actual flight position of each rice leaf roller moth in each frame of moth image are calculated, and when the distance is detected to be within a preset distance range, the flight state of the rice leaf roller moth is determined to be continuous flight.

[0191] When the detected distance is not within the preset distance range, the flight status of the rice leaf roller moth is determined to be a new takeoff.

[0192] In some possible embodiments, before calculating the distance between the predicted flight position and the actual flight position of each rice leaf roller moth in each frame of moth image, the method further includes:

[0193] From all images of moths marked with rice leaf roller moth, determine the actual flight distance of each rice leaf roller moth between two adjacent frames of moth images;

[0194] Based on the actual flight location distance and the preset time interval, the flight speed corresponding to each rice leaf roller moth is calculated;

[0195] When the detected distance is within a preset distance range, the flight state of the rice leaf roller moth is determined to be continuous flight, including:

[0196] When the distance detected is within a preset distance range and the flight speed is within a preset speed range, the flight state of the rice leaf roller moth is determined to be continuous flight.

[0197] In some possible embodiments, the number of rice leaf roller moths is counted based on their flight status, including:

[0198] The flight trajectory of each rice leaf roller moth in the moth video is marked according to the flight status of the rice leaf roller moth.

[0199] Count the number of flight tracks whose flight status is continuous flight and count the number of flight tracks whose flight status is newly taken off.

[0200] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0201] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0202] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0203] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between apparatuses or units may be electrical or other forms.

[0204] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0205] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0206] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0207] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0208] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of embodiments of this disclosure upon considering the specification and practicing the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A method for monitoring and forecasting rice leaf roller moths based on AR glasses, characterized in that, include: Acquire moth videos captured by AR glasses; At least two frames of moth images are determined from the moth video, and each frame of the moth image is input into a convolutional neural network model to obtain each frame of the moth image labeled with the rice leaf roller moth; wherein, the convolutional neural network model is trained from multiple sample images labeled with the rice leaf roller moth. The flight status of the rice leaf roller moth is determined based on all images of the moths marked with the rice leaf roller moth, and the number of the rice leaf roller moths is counted based on the flight status of the rice leaf roller moths; the flight status of the rice leaf roller moths is either continuous flight or newly taken off, and the number of rice leaf roller moths includes the number of rice leaf roller moths in continuous flight and the number of rice leaf roller moths in newly taken off. Send the number of rice leaf roller moths to the client; The acquisition of the moth video captured by the AR glasses includes: Send a request to the AR glasses to collect a video of the user moving a pole through a rice clump, so as to obtain the video of the user moving a pole through a rice clump collected by the AR glasses; The video of the user holding a pole and moving the rice stalks is processed for recognition. When the user's hand is detected to be in a preset area in the video of the user holding a pole and moving the rice stalks, a request to collect rice video is sent to the AR glasses to obtain the rice video collected by the AR glasses. When the rice in the rice video is identified as being in the seedling stage, the location where the pole first moves the rice is determined, and a request to start collecting the second video of the rice clump being moved is generated based on the location where the pole first moves the rice. Send the request to the AR glasses to start collecting the second video of the plucking of the rice clumps, so that the AR glasses can generate corresponding prompt information according to the request to start collecting the second video of the plucking of the rice clumps, and collect the moth video according to the user's second voice command; Acquire moth videos captured by the AR glasses.

2. The method according to claim 1, characterized in that, After detecting that the user's hand is within a preset area in the video of the user moving the rice stalk with a pole, the method further includes: A request to start collecting the first video of plucking rice clumps is sent to the AR glasses, so that the AR glasses generate corresponding prompt information according to the request to start collecting the first video of plucking rice clumps, and the AR glasses collect the moth video according to the user's first voice command. Acquire moth videos captured by the AR glasses.

3. The method according to claim 1, characterized in that, Determining at least two frames of moth images from the moth video includes: Obtain the recording time corresponding to the moth video, and determine at least two moments from the recording time according to a preset time interval; Extract the moth image corresponding to each moment from the moth video.

4. The method according to claim 3, characterized in that, The determination of the flight status of the rice leaf roller moth based on all the images of the moths marked with the rice leaf roller moth includes: Kalman filtering was applied to the moth video to predict the flight trajectory of each rice leaf roller moth in the video. Based on the flight trajectory of each rice leaf roller moth, the predicted flight position and actual flight position of each rice leaf roller moth are determined in each frame of the moth image marked with rice leaf roller moths. The distance between the predicted flight position and the actual flight position of each rice leaf roller moth in each frame of the moth image is calculated, and when the distance is detected to be within a preset distance range, the flight state of the rice leaf roller moth is determined to be continuous flight. When the distance is detected to be outside the preset distance range, the flight state of the rice leaf roller moth is determined to be a new takeoff.

5. The method according to claim 4, characterized in that, Before calculating the distance between the predicted flight position and the actual flight position of each rice leaf roller moth in each frame of the moth image, the method further includes: From all the images of the rice leaf roller moth marked with the moth, determine the actual flight position distance of each moth between two adjacent frames of the moth images; Based on the actual flight location distance and the preset time interval, the flight speed corresponding to each rice leaf roller moth is calculated; The step of determining the flight state of the rice leaf roller moth as continuous flight when the detected distance is within a preset distance range includes: When the detected distance is within a preset distance range and the flight speed is within a preset speed range, the flight state of the rice leaf roller moth is determined to be continuous flight.

6. The method according to claim 4, characterized in that, The counting of the number of rice leaf roller moths based on their flight status includes: The flight trajectory of each rice leaf roller moth in the video is marked according to the flight status of the rice leaf roller moth. Count the number of flight tracks whose flight status is continuous flight and count the number of flight tracks whose flight status is newly taken off.

7. A monitoring device for rice leaf roller moth based on AR glasses, characterized in that, include: The data acquisition module is used to acquire moth videos captured by the AR glasses; The data processing module is used to determine at least two frames of moth images from the moth video, and input each frame of the moth image into a convolutional neural network model to obtain each frame of the moth image labeled with the rice leaf roller moth; wherein, the convolutional neural network model is trained from multiple sample images labeled with the rice leaf roller moth. The data statistics module is used to determine the flight status of the rice leaf roller moth based on all the images of the moths marked with the rice leaf roller moth, and to count the number of the rice leaf roller moths based on the flight status of the rice leaf roller moths; the flight status of the rice leaf roller moths is continuous flight and newly taken off, and the number of rice leaf roller moths includes the number of rice leaf roller moths in continuous flight and the number of rice leaf roller moths in newly taken off. The data sending module is used to send the number of rice leaf roller moths to the client; The acquisition of the moth video captured by the AR glasses includes: Send a request to the AR glasses to collect a video of the user moving a pole through a rice clump, so as to obtain the video of the user moving a pole through a rice clump collected by the AR glasses; The video of the user holding a pole and moving the rice stalks is processed for recognition. When the user's hand is detected to be in a preset area in the video of the user holding a pole and moving the rice stalks, a request to collect rice video is sent to the AR glasses to obtain the rice video collected by the AR glasses. When the rice in the rice video is identified as being in the seedling stage, the location where the pole first moves the rice is determined, and a request to start collecting the second video of the rice clump being moved is generated based on the location where the pole first moves the rice. Send the request to the AR glasses to start collecting the second video of the plucking of the rice clumps, so that the AR glasses can generate corresponding prompt information according to the request to start collecting the second video of the plucking of the rice clumps, and collect the moth video according to the user's second voice command; Acquire moth videos captured by the AR glasses.

8. A monitoring device for rice leaf roller moth based on AR glasses, characterized in that, Including the processor and memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code stored in the memory to perform the steps of the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer-readable storage medium stores instructions that, when executed on a computer or processor, cause the computer or processor to perform the steps of the method as described in any one of claims 1-6.

Images