Seedling leakage detection method, electronic equipment, storage medium and rice transplanter
By setting up a camera and target recognition model on the rice transplanter to calculate and alarm the seed leakage rate, the problem of poor seed leakage detection accuracy in complex farmland scenarios is solved, and efficient seed leakage recognition and alarm is achieved, which improves detection efficiency and operation quality.
Patent Information
- Application Number
- CN202411105469.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-12-03
AI Technical Summary
The prior art has poor detection accuracy of the seedling leakage rate of the seedling transplanter in complex farmland scenarios, making it difficult to effectively identify and alarm.
By setting the camera on the transplanter to collect seedling images, using the ROI area recognition and target recognition model to calculate the seedling leakage rate, and combining the YOLOv5s model for identification and alarm signal transmission.
The accuracy of the calculation of seed leakage rate is improved, and the identification and alarm of seed leakage in complex farmland scenarios is realized, the labor intensity of operators is reduced, and the detection efficiency and operation quality are improved.
Smart Images

Figure CN119068361A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image recognition technology, and in particular to a method for detecting missing rice seedlings, an electronic device, a storage medium and a rice transplanter. Background Art
[0002] At present, there are methods for detecting missing seedlings, such as continuously collecting seedling images through drone aerial photography, and using image morphological processing methods and image segmentation processing methods. For example, the number and location of missing seedlings can be calculated by fitting a straight line to the seedlings, or the image of the seedlings can be photographed at intervals by a camera installed above the transplanting arm of the transplanter, and image morphological methods such as image segmentation and digital image processing technologies such as pattern recognition can be used to identify missing seedlings. However, these traditional recognition methods have poor detection effects and accuracy when facing complex farmland scenes. Summary of the invention
[0003] The purpose of the embodiments of the present application is to provide a method for detecting missing seedlings, an electronic device, a storage medium and a rice transplanter, so as to improve the accuracy of identifying the missing seedling rate of the rice transplanter in complex farmland scenes.
[0004] In a first aspect, the present application provides a method for detecting missing seedlings, wherein a camera disposed on a rice transplanter collects images of rice seedlings that have been transplanted by the rice transplanter, and for each frame of the rice seedling image obtained, a missing seedling rate corresponding to the frame of the rice seedling image is calculated in the following manner:
[0005] Based on the recognition quality of the seedling image, the ROI area of the seedling image is determined; all seedling identification objects in the seedling image are detected by the target recognition model; the ratio of the number of seedling identification objects identified in the ROI area to the standard number of seedlings to be planted is used as the seedling coverage rate corresponding to the frame of seedling image, so as to calculate the seedling missing rate corresponding to the frame of seedling image.
[0006] Preferably, the step of determining the ROI area of the seedling image based on the recognition quality of the seedling image specifically includes: preprocessing the seedling image; identifying all preset connected area objects in the preprocessed seedling image; determining the target connected area object based on the height values of all preset connected area objects in the vertical direction; determining the target ROI upper boundary based on the distance between the target connected area object and all preset ROI upper boundaries; determining the ROI area of the seedling image based on the target ROI upper boundary and the lower boundary of the seedling image.
[0007] Preferably, the step of determining the upper boundary of the target ROI based on the distance between the target connected region object and all preset ROI upper boundaries specifically includes:
[0008] Calculate the difference between the ordinate of the lower left corner pixel coordinate or the lower right corner pixel coordinate of the identification box of the target connected area and the ordinate corresponding to all preset ROI upper boundaries; for the preset ROI upper boundary corresponding to the negative difference, calculate the absolute value of the difference corresponding to the preset ROI upper boundary; determine the preset ROI boundary corresponding to the minimum absolute value as the target ROI upper boundary.
[0009] Preferably, the rice seedling image is preprocessed in the following manner:
[0010] For each pixel in the seedling image, the super green index of the pixel is calculated as the gray value of the pixel; for each pixel in the seedling image, the binarization value of the pixel is determined according to the gray value of the pixel to obtain a binary seedling image; based on a preset structural element, the binary seedling image is expanded to obtain a preprocessed seedling image.
[0011] Preferably, the target recognition model is trained and generated in the following manner:
[0012] A training sample set is obtained, wherein the training sample set includes a plurality of sample images, each of which is marked with a recognition frame of a rice seedling and a category of the rice seedling; the sample images are sequentially input into a YOLOv5s model so that the YOLOv5s model outputs a recognition frame and a category of each rice seedling recognition object in the sample image; a loss value between a recognition frame and a category of each rice seedling recognition object in the sample image output by the YOLOv5s model and a true value is calculated by a loss function, and the parameters of the YOLOv5s model are adjusted to obtain a target recognition model;
[0013] The loss functions include:
[0014] L total =λ bbox L bbox +λ cls L cls ;
[0015] Among them, L bbox is the loss value between the recognition box of all seedling recognition objects in the sample image output by the model and the true value of the recognition box, L cls is the loss value between the category of all seedling recognition objects in the sample image output by the model and the true value of the category box, λ bbox , cls is the corresponding weight coefficient.
[0016] Preferably, the method further includes determining, for each frame of the acquired seedling image, whether the missing seedling rate corresponding to the seedling image reaches an alarm value, and if so, generating a language alarm signal and sending it to the player.
[0017] In a second aspect, the present application provides a rice transplanter, the rice transplanter comprising:
[0018] Rice transplanter body;
[0019] A camera and a mounting part, wherein the camera is fixed on the main body of the rice transplanter at a preset height and tilt angle through the mounting part;
[0020] The controller is electrically connected to the camera and is used to execute the aforementioned method for detecting leaking rice seedlings.
[0021] Preferably, the rice transplanter also includes a player, and the controller is further used to determine whether the missing seedling rate corresponding to each frame of the acquired seedling image reaches an alarm value, and if so, generate a language alarm signal and send it to the player.
[0022] In a third aspect, the present application provides an electronic device comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the aforementioned method for detecting leaking rice seedlings.
[0023] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the aforementioned method for detecting leaking seedlings are executed.
[0024] The present application provides a method for detecting missing seedlings, an electronic device, a storage medium and a rice transplanter, wherein the detection method includes collecting images of rice seedlings that have been planted by the rice transplanter through a camera arranged on the rice transplanter, and calculating the missing seedling rate corresponding to each frame of the rice seedling image in the following manner: determining the ROI region of the rice seedling image based on the recognition quality of the rice seedling image; detecting all rice seedling recognition objects in the rice seedling image through a target recognition model; and using the ratio of the number of identified rice seedling recognition objects identified in the ROI region to the standard number of rice seedlings to be planted as the missing seedling rate corresponding to the frame of the rice seedling image. By determining the ROI region corresponding to the rice seedling image, removing the part of the rice seedling image with poor recognition quality, and calculating the missing seedling rate of the rice transplanter within the ROI region, the accuracy of the missing seedling rate calculation is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0026] Figure 1 A flow chart of a method for calculating a missing seedling rate provided in an embodiment of the present application;
[0027] Figure 2 A flowchart of steps for determining a ROI region provided in an embodiment of the present application;
[0028] Figure 3 A flowchart of a preprocessing step of a rice seedling image provided in an embodiment of the present application;
[0029] Figure 4 A schematic diagram of a candidate line of an ROI upper boundary provided in an embodiment of the present application;
[0030] Figure 5 A schematic diagram of the structure of a target recognition model provided in an embodiment of the present application;
[0031] Figure 6 A schematic diagram of camera image acquisition provided in an embodiment of the present application;
[0032] Figure 7 A schematic diagram of the structure of a controller provided in an embodiment of the present application;
[0033] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0034] First, the application scenario of the present application is described. The present application can be used for detecting and alarming the missing rate of transplanted seedlings during the operation of a multi-row transplanter.
[0035] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.
[0036] In one embodiment of the present application, a rice transplanter is provided, which includes a rice transplanter body, a camera and a mounting part, a controller and a player. The camera is fixed to the rice transplanter body at a preset height and tilt angle by the mounting part, and is located behind the rice transplanter body to collect images of rice seedlings after the rice transplanter is operated. The mounting part here can be a bracket. The controller is electrically connected to the camera and the player, and is used to calculate the rice seedling leakage situation of the rice transplanter through an algorithm, and determine whether an alarm is required. If necessary, a preset language alarm message is sent to the player to control the playback area to play a voice alarm prompt.
[0037] Figure 1 The following is a flow chart of a method for calculating the missing seedling rate provided in an embodiment of the present application. Figure 1As shown, an embodiment of the present application provides a method for detecting missing rice seedlings, in which a controller collects images of rice seedlings that have been transplanted by the rice transplanter through a camera arranged on the rice transplanter, and the rice transplanter here may be a six-row rice transplanter.
[0038] For each frame of seedling image acquired, the controller can calculate the missing seedling rate corresponding to the frame of seedling image in the following way:
[0039] S1. Based on the recognition quality of the seedling image, determine the ROI area of the seedling image.
[0040] Figure 2 A flowchart of the steps for determining a ROI region provided in an embodiment of the present application. Figure 2 As shown, based on the recognition quality of the seedling image, the step of determining the ROI area of the seedling image specifically includes:
[0041] S10, preprocessing the rice seedling image.
[0042] Figure 3 A flowchart of the preprocessing steps of a rice seedling image provided in an embodiment of the present application. Figure 3 As shown in the figure, the seedling image can be preprocessed in the following ways:
[0043] S100, for each pixel in the rice seedling image, calculating the super green index of the pixel as the gray value of the pixel.
[0044] The Super Green Index EGI can be calculated by the following formula:
[0045] EGI=2×(GR)+B,
[0046] Among them, G, R, and B represent the corresponding pixel values of the pixel in the green, red, and blue channels respectively.
[0047] By performing super-green processing on the seedling image, the green seedling characteristics in the image can be highlighted, and the interference of background elements such as water and soil can be suppressed. In addition, the super-green index can be used as the grayscale value to obtain a grayscale image, replacing the grayscale conversion of the RGB image.
[0048] S101. For each pixel in the rice seedling image, determine a binarization value of the pixel according to the gray value of the pixel to obtain a binarized rice seedling image.
[0049] Next, the seedling image is binarized to further improve the accuracy of image analysis.
[0050] S102 . Based on a preset structural element, dilation processing is performed on the binary rice seedling image to obtain a preprocessed rice seedling image.
[0051] Since the target detection algorithm can better demonstrate excellent recognition performance when the occlusion is small, the image dilation technology is used here to process the seedling area in the binary image to segment the slightly occluded but still identifiable seedling part. This step ensures the ROI area range and the detection effect of the target detection algorithm.
[0052] S11, identifying all preset connected area objects in the preprocessed seedling image.
[0053] For the preprocessed seedling image, all black connected area objects and corresponding identification boxes are identified through the target detection algorithm.
[0054] S12: Determine a target connected area object based on the height values of all preset connected area objects in the vertical direction.
[0055] The length values or height values of the identification frames of all preset connected area objects in the vertical direction are compared to determine the longest identification frame in the vertical direction, and determine the pixel coordinates of the lower left corner or lower right corner of the identification frame.
[0056] S13: Determine the upper boundary of the target ROI based on the distance between the target connected region object and all preset ROI upper boundaries.
[0057] The upper boundary of the preset ROI here is pre-calibrated. Figure 4 A schematic diagram of a candidate line for the upper boundary of an ROI provided in an embodiment of the present application. Figure 4 As shown, in a specific embodiment, after the camera is fixedly installed, three ROI upper boundary candidate lines are preset based on its field of view, which are located at Dmax / 2, 2Dmax / 3, and 5Dmax / 6, which are Dmax meters away from the rice transplanter. Through the edge detection algorithm, the pixel point sets corresponding to these three straight lines in the image can be extracted to provide a basis for subsequent dynamic selection. Here, Dmax is the farthest distance covered by the camera field of view.
[0058] Further, based on the distance between the target connected region object and all preset ROI upper boundaries, the step of determining the target ROI upper boundary specifically includes:
[0059] Calculate the difference between the ordinate of the lower left corner pixel coordinate or the lower right corner pixel coordinate of the identification box of the target connected area and the ordinate corresponding to all preset ROI upper boundaries. For the preset ROI upper boundary corresponding to the negative difference, calculate the absolute value of the difference corresponding to the preset ROI upper boundary. The preset ROI boundary corresponding to the minimum absolute value is determined as the target ROI upper boundary.
[0060] By matching the pixel coordinates of the lower boundary of the recognition frame with the pixel point sets of the three preset ROI upper boundary candidate lines, the difference between the vertical coordinates identified on the same horizontal coordinate and the vertical coordinates corresponding to the preset ROI upper boundary is calculated, and the candidate line with the negative difference and the smallest absolute value is selected as the final ROI upper boundary. This dynamic selection mechanism ensures that the ROI area contains sufficient seedling information while avoiding severely occluded areas as much as possible.
[0061] S14. Determine the ROI region of the seedling image based on the upper boundary of the target ROI and the lower boundary of the seedling image.
[0062] Here, the left and right boundaries of the ROI area are determined based on the imaging positions of the straight line traces left by the transplanter on both sides of the land during the operation in the seedling image. The lower boundary of the seedling image is directly used as the lower boundary of the ROI area, which can simplify the processing flow and maintain the intuitiveness of the area definition.
[0063] In this way, the determined ROI area closely fits the actual operating range of the rice transplanter, and can not only contain sufficient seedling information, but also eliminate severely obstructed areas to ensure the recognition effect.
[0064] S2. Detect all rice seedling identification objects in the rice seedling image through the target recognition model.
[0065] In step S2, the rice seedling image is input as a whole into a pre-trained target recognition model to identify the rice seedling. The target recognition model here can be created based on YOLOv5.
[0066] S3. The ratio of the number of seedlings identified in the ROI area to the standard number of seedlings to be planted is used as the seedling coverage rate corresponding to the frame of seedlings image to calculate the seedling missing rate corresponding to the frame of seedlings image.
[0067] In step S3, for the identification frame of the rice seedling identification object detected by the target recognition model, the normalized coordinates of the center point of the identification frame are converted into pixel coordinates. Specifically, the normalized coordinates can be multiplied by the rice seedling image. The identification frames of all rice seedling identification objects are traversed to calculate whether the center point is located in the ROI area, and then the number of rice seedling identification objects in all ROI areas is counted S1.
[0068] The seedling hole spacing set in the rice transplanter operation is M meters. When the preset ROI upper boundary is selected as D max / 2, the number of seedlings to be planted in the ROI area The seedling coverage rate S = S1 / S2, and the seedling leakage rate is 1-S.
[0069] A method for detecting missing seedlings provided in an embodiment of the present application dynamically determines the ROI area corresponding to the seedling image, removes the portion of the seedling image with poor recognition quality, calculates the missing seedling rate of the rice transplanter within the ROI area, and improves the accuracy of the missing seedling rate calculation.
[0070] In one embodiment of the present application, the controller can also determine whether the missing seedling rate corresponding to each frame of the acquired seedling image reaches an alarm value. If so, a language alarm signal is generated and sent to the player.
[0071] In one embodiment of the present application, a YOLOv5 (You Only Look Once version 5) target detection algorithm is used. The algorithm is efficient and accurate and can achieve precise rice seedling detection in a complex farmland environment.
[0072] Specifically, the target recognition model can be trained and generated in the following ways:
[0073] A training sample set is obtained, where the training sample set includes multiple sample images, and each sample image is marked with a seedling identification frame and a seedling category.
[0074] The training sample set here can be divided into a training set and a validation set. The sample images here can include seedlings at different growth stages and under different environmental conditions, as well as various possible background interferences to ensure the generalization ability of the model.
[0075] Then use the image annotation tool to annotate the rice seedlings in each image. The label usually contains the identification box of the rice seedling (x min ,y min , x max ,y max ) and class label L, where (x min ,y min ) is the coordinate of the upper left corner of the recognition box, (x max ,y max ) are the coordinates of the lower right corner.
[0076] In order to enhance data diversity, data augmentation techniques can also be used, such as random scaling, random cropping, and random arrangement of four images before splicing to generate more diverse training samples. Let the original data set be D, and the enhanced data set be D′, then the enhancement process can be expressed as: D′=Augment(D), where Augment represents the data augmentation function.
[0077] The sample images are input into the YOLOv5s model in sequence, so that the YOLOv5s model outputs the identification box and category of each seedling identification object in the sample image.
[0078] The YOLOv5s model is selected here, which is suitable for real-time seedling detection due to its high efficiency and accuracy. Figure 5 A schematic diagram of the structure of a target recognition model provided in an embodiment of the present application. Figure 5 As shown in the figure, the network structure of YOLOv5 includes input layer, Backbone, Neck, Head and loss function. The input layer receives and processes the image data to be detected, Backbone uses BottleNeckCSP structure to extract features, Neck uses FPN+PAN structure for feature fusion, and Head is used to output the final detection information. BottleNeckCSP is an improved bottleneck layer that improves the performance and efficiency of the network by combining the bottleneck structure and cross-stage partial connection (CSP). FPN+PAN is a commonly used multi-scale feature fusion structure used to enhance the semantics and positioning capabilities of the network at different scales.
[0079] The loss function is used to calculate the loss value between the recognition box and category of each seedling recognition object in the sample image output by the YOLOv5s model and the true value, and the parameters of the YOLOv5s model are adjusted to obtain the target recognition model.
[0080] The loss functions include:
[0081] L total =λ bbox L bbox +λ cls L cls ;
[0082] Among them, L bbox is the loss value between the recognition box of all seedling recognition objects in the sample image output by the model and the true value of the recognition box, L cls is the loss value between the category of all seedling recognition objects in the sample image output by the model and the true value of the category box, λ bbox , cls is the corresponding weight coefficient.
[0083] The loss function of the target recognition model usually includes the recognition box regression loss function L bbox And the classification loss function L cls , the recognition box regression loss is used to measure the difference between the recognition box predicted by the model and the true recognition box. In seedling detection, the recognition box is usually used to locate the seedlings in the image. The recognition box regression loss function L bbox The goal is to make the recognition box predicted by the model as close as possible to the real recognition box. The classification loss function L clsIt is used to measure the accuracy of the model in classifying the target object. In seedling detection, the model not only needs to locate the seedling object, but also needs to identify the category of the seedling. The goal of the classification loss function is to enable the model to accurately classify each seedling into the correct category.
[0084] In the training phase, the model is trained using the training set D'. The training process can be formalized as the following optimization problem:
[0085]
[0086] Among them, θ represents the model parameters; |D′| represents the size of the training set, that is, the number of samples in the training set, Represents the normalization factor, dividing the total loss by the size of the training set |D′|, so that the average loss of each sample is obtained, that is, the average loss. This average loss is a more stable and meaningful indicator because it reflects the average performance of the model on the entire training set, rather than relying solely on the size of the training set. f(x; θ) represents the model function, also known as the prediction function or hypothesis function. This function receives the input data x and the current model parameters θ as input and outputs a predicted value. In the object detection task, this predicted value may be the coordinates of a recognition box and the category of the object. y represents the true label, which is the true value or target value corresponding to the input data x. During the training process, we hope that the output predicted by the model is as close to this true value as possible. The summation symbol ∑(x,y)∈D′ means that the loss function is calculated once for each sample (x,y) in the training set D′ and added. L total (f(x; θ), y) represents the total loss function, which measures the difference between the model prediction f(x; θ) and the true label y.
[0087] In summary, this formula describes an optimization problem, which is to minimize the average loss on the training set D′ by adjusting the model parameters θ until a set of parameters θ that minimizes the loss function is found. * .
[0088] After the model training is completed, the model testing and deployment phase begins. The goal of this phase is to evaluate the performance of the model, ensure that it can work stably and accurately in actual applications, and finally deploy the model to the actual production environment.
[0089] Model testing: Use an independent test set (different from the training set and validation set) to evaluate the performance of the model. The test set should contain a variety of complex scenarios to fully test the generalization ability of the model. Make predictions for each image in the test set and calculate the model's performance indicators, such as accuracy and recall. Analyze the model's prediction results, identify potential errors and deviations, and provide feedback to adjust the model or training process.
[0090] Model deployment: After the model performance meets the requirements, the model is deployed to the target hardware. Currently, the camera with built-in AI computing power is used, so according to the camera deployment configuration plan, the trained model is deployed to the AI camera to realize the task of deep learning detection of seedlings.
[0091] In one embodiment of the present application, in order to ensure that the camera field of view is able to see nearby seedlings, the height value and tilt angle value of the camera installation need to be determined.
[0092] like Figure 6 As shown in the figure, a six-row rice transplanter is used as an example. That is, the rice transplanter will simultaneously insert six rice seedlings in a row during operation. Assume that the camera is installed at a height of H, and it is hoped that the camera can cover the entire rice seedling area from near to far ground, focusing on the near area. Let the horizontal field of view (HFOV) of the camera be θ H , the vertical field of view (VFOV) is θ V , horizontal field of view angle θ H Determines the maximum range that the camera can see in the horizontal direction, the vertical field of view angle θ V Determines the maximum range that the camera can see in the vertical direction.
[0093] What needs to be calculated is the closest distance that the camera can cover under given field of view conditions. For the horizontal field of view, the camera can cover the entire area from left to right. The camera is installed in the middle of the rice transplanter, so the field of view angle is symmetrical, and the problem can be simplified to an isosceles triangle, where the camera is located at the vertex and the two points at the bottom are the closest and farthest points that the camera can see. Combining the horizontal and vertical fields of view, the coverage area of the camera can be regarded as an elliptical cylinder area, where the major axis of the ellipse is determined by the horizontal field of view angle and the minor axis is determined by the vertical field of view angle. The camera installation height H determines the vertical position of this elliptical cylinder. According to the calculation, the camera tilt angle can be obtained to cover the field of view:
[0094]
[0095] α is the angle at which the camera needs to tilt downward, θ V is the vertical field of view of the camera, find the viewing angle θ in the vertical direction when the camera covers Dmin V ,min, that is Dmin is the closest point to the bottom of the camera, Dmax is the farthest distance covered by the field of view, and H is the height at which the camera is installed. The coverage of the field of view can be adjusted by adjusting the tilt angle. Ultimately, the field of view size of the detection screen is determined by the camera field of view, installation height, and camera angle. In addition, in order to ensure image quality, the resolution of the camera capture screen is 1080x720 to capture clear and continuous image data.
[0096] Figure 7 A schematic diagram of the structure of a controller provided in an embodiment of the present application. Figure 7 As shown, in one embodiment of the present application, the CPU of the hardware part of the controller can use the GD32A503 chip, which has a rich peripheral interface, and the hardware module can be divided into a CAN module unit, a GPIO module unit, and a UART module unit. The main functions of each hardware unit are described as follows:
[0097] The CAN module unit is used to communicate with the camera and obtain alarm information to realize the CAN communication function. The GPIO is used to obtain the rice seedling tray lifting signal and the level status of the external mute speaker button. The UART is used to communicate with the speaker and send voice packet information.
[0098] Specifically, the present application provides an alarm controller based on the FreeRTOS real-time operating system, which can be implemented by the following technical solutions:
[0099] 1) System architecture: The present invention uses FreeRTOS as the core of the operating system and builds a multi-task scheduling execution environment. The system includes multiple functional modules such as CAN communication tasks, data analysis tasks, GPIO detection tasks, and voice broadcast tasks. The modules communicate efficiently through synchronization mechanisms such as message queues.
[0100] 2) CAN communication task: This task is responsible for initializing the CAN interface, receiving data frames from the CAN network, and safely placing them into the system's preset message queue for processing by other tasks. The interrupt-driven approach ensures the real-time and accuracy of data reception.
[0101] 3) Data parsing task: This task periodically reads CAN data frames from the message queue and parses them according to the preset protocol specifications. The parsing results will serve as the basis for subsequent operations, such as setting the voice package content.
[0102] 4) GPIO detection task: This task is used to regularly detect the status of external GPIO pins, such as the position of the rice tray, the mute button, etc. The detection results are notified to other related tasks through the internal mechanism to affect the system behavior.
[0103] 5) Voice broadcast task: This task determines whether the conditions for voice broadcast are met based on the results of the data analysis task and the GPIO detection task. If the conditions are met, the pre-made voice packet data is sent to the speaker through the UART interface to realize the voice feedback of the alarm information.
[0104] The specific process of voice alarm is as follows: after the controller is initialized, it receives the CAN data sent from the outside through the message queue. After receiving the message that meets the protocol requirements, it broadcasts the corresponding alarm voice package according to the parsed message. Before broadcasting, it is also necessary to detect the level state of the external GPIO pin. When the rice seedling tray lifting position is in the lowered state (i.e. high level) and the external mute speaker button is not pressed, a voice package data is sent to the speaker through UART. Otherwise, no voice broadcast is performed.
[0105] The rice transplanter provided in the embodiment of the present application realizes rapid response and alarm prompts to the situation of rice seedling leakage through the integrated cooperation of components such as a camera, a controller, and a voice speaker. When rice seedling leakage is detected, the system can immediately send a clear alarm message through the voice speaker to remind the operator to check and deal with it in time. This instant feedback mechanism not only improves the operating efficiency, but also ensures the quality and effect of the rice transplanting operation.
[0106] In addition, deep learning technology is used to develop visual algorithms for the collected video images, set dynamic ROI areas, and automatically identify the condition of seedlings to cope with complex and changeable seedling farmland scenes. This not only reduces the labor intensity of operators, but also significantly improves detection efficiency.
[0107] See also Figure 8 , Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 8 As shown in , the electronic device 800 includes a processor 810 , a memory 820 and a bus 830 .
[0108] The memory 820 stores machine-readable instructions executable by the processor 810. When the electronic device 800 is running, the processor 810 communicates with the memory 820 via the bus 830. When the machine-readable instructions are executed by the processor 810, the above-mentioned Figure 1 The steps of a method for detecting leaking seedlings in the method embodiment shown are specifically implemented in accordance with the method embodiment and will not be described in detail here.
[0109] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1The steps of a method for detecting leaking seedlings in the method embodiment shown are specifically implemented in accordance with the method embodiment and will not be described in detail here.
[0110] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0111] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.
[0112] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0113] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
[0114] It should be noted that if the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM) random access memory (RAM), disk or optical disk, and other media that can store program codes.
[0115] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.
[0116] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A method for detecting leaky seedlings, characterized in that: The camera disposed on the rice transplanter is used to collect images of rice seedlings that have been transplanted by the rice transplanter. For each frame of the rice seedling image obtained, the missing rice seedling rate corresponding to the frame of the rice seedling image is calculated in the following manner: Based on the recognition quality of the rice seedling image, determining the ROI area of the rice seedling image; Detecting all rice seedling identification objects in the rice seedling image by using a target recognition model; The ratio of the number of seedling identification objects identified in the ROI area to the standard number of seedlings to be planted is used as the seedling coverage rate corresponding to the frame of seedling image to calculate the seedling missing rate corresponding to the frame of seedling image.
2. The method according to claim 1, characterized in that The step of determining the ROI area of the rice seedling image based on the recognition quality of the rice seedling image specifically includes: Preprocessing the rice seedling image; Identify all preset connected area objects in the preprocessed seedling image; Determine the target connected area object based on the height values of all preset connected area objects in the vertical direction; Based on the distance between the target connected area object and all preset ROI upper boundaries, determine the target ROI upper boundary; Based on the upper boundary of the target ROI and the lower boundary of the seedling image, the ROI area of the seedling image is determined.
3. The method according to claim 2, characterized in that The step of determining the upper boundary of the target ROI based on the distance between the target connected region object and all preset ROI upper boundaries specifically includes: Calculate the difference between the ordinate of the lower left corner pixel coordinate or the lower right corner pixel coordinate of the identification box of the target connected area and the ordinate corresponding to the upper boundary of all preset ROIs; For a preset ROI upper boundary corresponding to a negative value of the difference, calculating the absolute value of the difference corresponding to the preset ROI upper boundary; The preset ROI boundary corresponding to the minimum absolute value is determined as the upper boundary of the target ROI.
4. The method according to claim 2, characterized in that: The rice seedling image is preprocessed in the following manner: For each pixel in the rice seedling image, a super green index of the pixel is calculated as the gray value of the pixel; For each pixel in the rice seedling image, a binarization value of the pixel is determined according to the gray value of the pixel to obtain a binarized rice seedling image; Based on the preset structural elements, the binary seedling image is expanded to obtain the preprocessed seedling image.
5. The method according to claim 1, characterized in that The target recognition model is trained and generated in the following way: Acquire a training sample set, wherein the training sample set includes a plurality of sample images, each sample image being marked with a seedling identification frame and a seedling category; The sample images are sequentially input into the YOLOv5s model so that the YOLOv5s model outputs the identification frame and category of each rice seedling identification object in the sample image; Calculate the loss value between the recognition frame and category of each rice seedling recognition object in the sample image output by the YOLOv5s model and the true value through the loss function, and adjust the parameters of the YOLOv5s model to obtain the target recognition model; The loss function include: L total =λ bbox L bbox +λ cls L cls ; Among them, L bbox is the loss value between the recognition box of all seedling recognition objects in the sample image output by the model and the true value of the recognition box, L cls is the loss value between the category of all seedling recognition objects in the sample image output by the model and the true value of the category box, λ bbox , cls is the corresponding weight coefficient.
6. The method according to claim 1, characterized in that Also includes: For each frame of the rice seedling image obtained, determine whether the missing rice seedling rate corresponding to the rice seedling image reaches an alarm value. If so, generate a language alarm signal and send it to the player.
7. A rice transplanter, characterized in that: The rice transplanter comprises: Rice transplanter body; A camera and a mounting member, wherein the camera is fixed on the rice transplanter body at a preset height and tilt angle through the mounting member; A controller is electrically connected to the camera and is used to execute the method for detecting leaking rice seedlings as described in claims 1-5.
8. The rice transplanter according to claim 7, characterized in that: The rice transplanter also includes a player, and the controller is also used to determine whether the missing seedling rate corresponding to each frame of the rice seedling image obtained has reached an alarm value. If so, a language alarm signal is generated and sent to the player.
9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of the method for detecting leaking rice seedlings as described in any one of claims 1 to 6.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for detecting leaking rice seedlings as claimed in any one of claims 1 to 6 are executed.
Citation Information
Patent Citations
An image detection method for transplanting quality of an unmanned transplanter
CN107464232A
Miss-planting rate detection method of rice transplanter
CN111932551A
Paddy field seedling leakage recognition method and system
CN113989225A
Vegetation coverage monitoring system and method
CN117523387A
Seedling missing detection method and device and storage medium
CN118015466A
