A method, device, storage medium and electronic device for detecting seedling shortage

Abstract

The present application proposes a method, device, storage medium, and electronic device for detecting missing seedlings, which processes a current frame image to determine a current suspected missing seedling position in the current frame image; wherein the current frame image displays a current target area, and the current target area includes an area where seedlings have been planted before the location where the current frame image is captured; based on historical suspected missing seedling positions in historical frame images within a first time range, it is determined whether the current suspected missing seedling position is a target missing seedling position. Compared to determining whether a seedling is missing by referring to only a single image, this embodiment combines the current frame image and historical frame images for a comprehensive judgment, thereby reducing the possibility of misidentification and improving the accuracy of identifying the missing seedling position.

Description

Technical Field

[0001] The present application relates to the field of agricultural machinery, and more specifically, to a method, device, storage medium and electronic device for detecting seedling absence. Background Art

[0002] With the development and advancement of science and technology, mechanized agriculture has become a hot field. Mechanized agriculture relies on mechanized tools, such as seed drills, rice transplanters, harvesters, and irrigation machines. Mechanized tools can significantly increase productivity while reducing labor costs, playing a vital role in the advancement of agricultural technology.

[0003] For example, rice transplanters, a mechanized tool, offer high efficiency and rapid seedling transplanting. Even brief periods of malfunction can significantly impact transplanting success. Therefore, monitoring the operating status of rice transplanters is crucial to promptly detect any missing seedlings and prevent widespread incidents. Therefore, efficiently and quickly identifying missing seedlings has become a pressing challenge for those skilled in the art. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, storage medium and electronic device for detecting seedling absence, so as to at least partially improve the above-mentioned problems.

[0005] In order to achieve the above objectives, the technical solutions adopted in the embodiments of the present application are as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for detecting seedling absence, which is applied to a control system of a rice transplanter, and the method includes:

[0007] Processing the current frame image to determine a current suspected missing seedling position in the current frame image;

[0008] The current frame image displays a current target area, and the current target area includes a planted seedling area before the location where the current frame image is captured;

[0009] Based on historical suspected seedling missing positions in historical frame images within the first time range, it is determined whether the current suspected seedling missing position is the target seedling missing position.

[0010] In a second aspect, an embodiment of the present application provides a rice transplanter control method, which is applied to a control system of the rice transplanter, and the method includes:

[0011] Based on the target seedling-missing position determined by the above-mentioned seedling-missing detection method, the rice transplanter is controlled to carry out seedling replenishment.

[0012] In a third aspect, an embodiment of the present application provides a seedling shortage detection device, which is applied to a control system of a rice transplanter, and the device includes:

[0013] a marking unit, configured to process the current frame image to determine a current suspected missing seedling position in the current frame image;

[0014] The current frame image displays a current target area, and the current target area includes a planted seedling area before the location where the current frame image is captured;

[0015] The processing unit is used to determine whether the current suspected seedling missing position is the target seedling missing position based on the historical suspected seedling missing position in the historical frame image within the first time range.

[0016] In a fourth aspect, an embodiment of the present application provides a rice transplanter control device, which is applied to a control system of the rice transplanter, and the device includes:

[0017] The control unit is used to control the rice transplanter to replenish seedlings based on the target seedling-missing position determined by the above-mentioned seedling-missing detection method.

[0018] In a fifth aspect, an embodiment of the present application provides a storage medium on which a computer program is stored, and the computer program implements the above method when executed by a processor.

[0019] In a sixth aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, wherein the memory is used to store one or more programs; when the one or more programs are executed by the processor, the above-mentioned method is implemented.

[0020] In the seventh aspect, an embodiment of the present application provides a rice transplanter, comprising a machine body, a rice transplanting mechanism arranged on the machine body, and the above-mentioned electronic device; the electronic device is used to control the rice transplanter to detect seedling shortages and / or to replenish seedlings at target seedling shortage positions.

[0021] Compared to the prior art, the embodiments of the present application provide a method, device, storage medium, and electronic device for detecting missing seedlings, which processes the current frame image to determine the current suspected missing seedling position in the current frame image; wherein the current frame image displays the current target area, and the current target area includes the area where seedlings have been planted before the location where the current frame image is captured; based on the historical suspected missing seedling positions in the historical frame images within the first time range, it is determined whether the current suspected missing seedling position is the target missing seedling position. Compared to judging whether a seedling is missing by referring to only a single image, this embodiment combines the current frame image and the historical frame images for a comprehensive judgment, thereby reducing the possibility of misidentification and improving the accuracy of identifying the missing seedling position.

[0022] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0025] Figure 2 A schematic diagram of a process for detecting seedling deficiency provided in an embodiment of the present application;

[0026] Figure 3 A reference diagram of the current frame image provided in an embodiment of the present application;

[0027] Figure 4 Schematic diagram of sub-steps S101 and S102 provided in an embodiment of the present application;

[0028] Figure 5 One of the flow charts of the seedling absence detection method provided in the embodiment of the present application;

[0029] Figure 6 A schematic diagram of the sub-steps of S104 provided in an embodiment of the present application;

[0030] Figure 7 A schematic flow chart of a rice transplanter control method provided in an embodiment of the present application;

[0031] Figure 8 A schematic diagram of a unit of a seedling shortage detection device provided in an embodiment of the present application;

[0032] Figure 9 A unit schematic diagram of the rice transplanter control device provided in an embodiment of the present application.

[0033] In the figure: 10 - processor; 11 - memory; 12 - bus; 13 - communication interface; 201 - marking unit; 202 - processing unit; 401 - acquisition unit; 402 - control unit. DETAILED DESCRIPTION

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0035] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without creative work are within the scope of protection of the present application.

[0036] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0037] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0038] In the description of this application, it should be noted that the terms "upper", "lower", "inside", "outside", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, or are the orientations or positional relationships in which the product of the application is usually placed when in use. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they should not be understood as limitations on this application.

[0039] It should also be noted that, in the description of this application, unless otherwise expressly specified or limited, the terms "disposed" and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; mechanical connections, or electrical connections; direct connections, indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in this application based on the specific circumstances.

[0040] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0041] For example, a rice transplanter can detect seedling shortages during the transplanting process. Dedicated personnel monitor the transplanter's progress in real time to identify any missing seedlings or mechanical failures. If a seedling shortage is detected, timely replacements are made and an alarm is issued for potential mechanical failures. In the event of a shortage, the transplanter operator can periodically monitor the transplanting process and address any missing seedlings.

[0042] While assigning dedicated staff to monitor the transplanter's trays and planting progress in real time can help identify problems quickly, the additional staff increases labor costs and workload. While relying solely on the driver to observe the transplanter's progress sporadically reduces labor costs, it can easily distract the driver and cause problems such as the transplanter's position shifting. Furthermore, the driver's infrequent observation makes it difficult to detect missing seedlings in real time, which can lead to long gaps in seedlings, necessitating re-planting or manual restocking later, reducing operational efficiency.

[0043] In response to the above problems, an embodiment of the present application provides a seedling shortage detection method, which can perform seedling shortage detection and fault alarm in real time during the operation of the rice transplanter, thereby replacing the observation of seedling shortage by additional staff, and assisting in problem judgment, reducing labor costs and improving work efficiency.

[0044] The embodiment of the present application provides an electronic device, which can be a control system on a rice transplanter, or an independent computer device (such as a server, a computer, or an autopilot). Figure 1 , a schematic diagram of the structure of an electronic device. The electronic device includes a processor 10, a memory 11, and a bus 12. The processor 10 and the memory 11 are connected via the bus 12. The processor 10 is used to execute executable modules stored in the memory 11, such as computer programs.

[0045] The processor 10 can be an integrated circuit chip with signal processing capabilities. During the implementation process, each step of the seedling deficiency detection method can be completed by the hardware integrated logic circuit in the processor 10 or the instructions in the form of software. The above-mentioned processor 10 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0046] The memory 11 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0047] The bus 12 may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. Figure 1 Only one bidirectional arrow is used in the figure, but it does not mean that there is only one bus 12 or one type of bus 12.

[0048] The memory 11 is used to store programs, such as a program corresponding to a seedling absence detection device. The seedling absence detection device includes at least one software function module that can be stored in the memory 11 in the form of software or firmware or embedded in the operating system (OS) of the electronic device. Upon receiving an execution instruction, the processor 10 executes the program to implement the seedling absence detection method.

[0049] Possibly, the electronic device provided in the embodiment of the present application further includes a communication interface 13. The communication interface 13 is connected to the processor 10 via a bus.

[0050] It should be noted that the electronic device is in communication with the image acquisition module, and the electronic device can obtain the image information collected by the image acquisition module (such as the current frame image and historical frame image below). Optionally, the image acquisition module can be installed directly behind the rice transplanter, directly above the center of the seedling tray, about 1.5 meters from the ground. The shooting direction of the image acquisition module is perpendicular to the ground, and it can look down and capture the entire current working area of the rice transplanter, reducing obstructions between seedlings and alleviating the impact of light.

[0051] Optionally, the processor 10 may include a first processor and a second processor, wherein the first processor is configured to execute the seedling absence detection method provided in the embodiment of the present application, and the second processor is configured to control the normal operation of the rice transplanter, such as transplanting seedlings. By decoupling the functions of the first processor and the second processor, development difficulty is reduced.

[0052] It should be understood that Figure 1 The structure shown is only a schematic diagram of a portion of the electronic device. The electronic device may also include Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0053] The present invention provides a method for detecting seedling deficiency, which can be applied to, but is not limited to, Figure 1 For detailed procedures, please refer to the electronic equipment shown in Figure 2 The seedling shortage detection method includes S101 and S102, which are described in detail as follows.

[0054] S101, processing the current frame image to determine the current suspected missing seedling position in the current frame image.

[0055] The current frame image displays a current target area, and the current target area includes a planted seedling area before the location where the current frame image is captured.

[0056] Optionally, the current frame image further displays a current seedling planting area, and the current seedling planting area includes the seedling planting area when the position where the current frame image is located is captured.

[0057] It should be understood that the current target area is the area that the image acquisition module can capture at the current position of the rice transplanter. Figure 3 , Figure 3 A reference schematic diagram of the current frame image provided in an embodiment of the present application.

[0058] Optionally, the current suspected seedling-missing position is a position where the seedling-missing confidence is greater than a preset first confidence threshold, where the first confidence threshold is, for example, 85%. It should be understood that due to the influence of possible shooting posture, light obstruction, and brightness changes, the current suspected seedling-missing position determined based on a single image may have errors or misjudgments. In order to reduce errors, it is necessary to execute S102 after S101.

[0059] S102: Determine a target missing seedling position from the current suspected missing seedling position based on the historical suspected missing seedling position in the historical frame image within the first time range.

[0060] The target seedling-missing position is a current suspected seedling-missing position that overlaps with a historical suspected seedling-missing position that exceeds a first quantity threshold.

[0061] Optionally, the first time range may be the time for continuously acquiring 15 frames of images (eg, 1 s to 1.5 s), and the first quantity threshold may be, for example, 5. It should be understood that the first quantity threshold may be adjusted according to the size of the first time range.

[0062] Optionally, when a rice transplanter is working in a field and needs to detect the missing seedling position in real time, the first time range can be a continuous time range before the acquisition time of the current image frame. It should be understood that within the first time range, the position of the rice transplanter will change continuously. The current target area will also change continuously, so there will be overlapping areas between the historical frame image and the current frame image. The overlapping areas between the historical frame image and the current frame image within the first time range can reach a preset repetition threshold, so the historical suspected missing seedling positions in the historical frame image can be used as reference data for determining the target missing seedling positions.

[0063] In one possible implementation, after completing the rice transplanting operation, the rice transplanter uploads the collected pictures to the server. When the server performs seedling missing detection based on the received pictures, the first time range can also be a continuous time range after the acquisition time of the current image frame, or a continuous time range with the acquisition time of the current image frame as the midpoint.

[0064] Optionally, the first time range can be set according to the forward speed of the rice transplanter.

[0065] Compared with judging whether seedlings are missing by referring to only a single picture, this embodiment combines the current frame image and the historical frame image for comprehensive judgment, which reduces the possibility of misidentification and improves the accuracy of identifying the missing seedling position.

[0066] In summary, the embodiment of the present application provides a method for detecting missing seedlings, which processes the current frame image to determine the current suspected missing seedling position in the current frame image; wherein the current frame image displays the current target area, and the current target area includes the area where seedlings have been planted before the position where the current frame image is captured; based on the historical suspected missing seedling positions in the historical frame images within the first time range, it is determined whether the current suspected missing seedling position is the target missing seedling position. Compared with judging whether there are missing seedlings by referring to only a single image, this embodiment combines the current frame image and the historical frame images for comprehensive judgment, thereby reducing the possibility of misidentification and improving the accuracy of identifying the missing seedling position.

[0067] exist Figure 2 On the basis of the content in S101, how to obtain the current suspected missing seedling position, the embodiment of the present application also provides several possible implementation methods, please refer to the following.

[0068] In a possible implementation, the position of seedlings in the current frame image is identified based on image recognition, and the interval between two adjacent positions of seedlings in the same row is used to determine the current suspected position of missing seedlings.

[0069] In another possible implementation, see Figure 4 , S101 includes: S101-1, which is specifically described as follows.

[0070] S101-1, processing the current frame image through the seedling absence detection neural network model to obtain the current suspected seedling absence position in the current frame image.

[0071] Optionally, the current frame image can be used as the input of the seedling absence detection neural network model, so that the seedling absence detection neural network model outputs the current suspected seedling absence position in the current frame image.

[0072] Optionally, the missing seedling detection neural network model is a fusion model of a trained crop seedling detection sub-model and a trained missing seedling detection sub-model; wherein, the crop seedling detection sub-model is trained based on the first category of image data, and the missing seedling detection sub-model is trained based on the output of the trained crop seedling detection sub-model and the second category of image data, the image content of the first category of image data is the same as that of the second category of image data, the first category of image data carries a first label, and the first label represents the confidence level of the presence of seedlings at each location in the image, and the second category of image data carries a second label, and the second label represents the confidence level of the missing seedlings at each location in the image.

[0073] It should be understood that since the image texture and other features of the missing seedling part are almost the same as those of the background part, a simple detection algorithm can easily detect the background as the missing seedling position or ignore the missing seedling position as the background. Therefore, a two-step segmented detection method is adopted. First, a simple detection algorithm is used to detect the seedling part with relatively obvious features, and the detected seedling position feature map is used as the input of the missing seedling detection sub-model. Then, based on the detected seedling position, the missing seedling position is inferred by the missing seedling detection algorithm. Since the camera angle of view is almost fixed relative to the area where the transplanter has just completed transplanting, the algorithm that uses the detected seedling position and the relative reference position to infer the missing seedling position has a relatively high accuracy. Based on this, the training of the missing seedling detection neural network model includes two parts. In the first part, the crop seedling detection sub-model is trained using the first type of image data. When the accuracy of the crop seedling detection sub-model reaches the requirement, the training of the crop seedling detection sub-model is stopped. The trained seedling detection sub-model is then excluded from the training of the seedling-absence detection sub-model. The output feature map of the seedling detection sub-model is used as the input of the seedling-absence detection sub-model. The seedling-absence detection sub-model is trained using the second-category image data and the output feature map of the seedling detection sub-model. Training is stopped when the seedling-absence detection sub-model reaches the required accuracy. Finally, the seedling detection sub-model and the seedling-absence detection sub-model are fused together.

[0074] In addition, the present application performs seedling shortage detection based on the same position, which can be understood as detecting whether there are seedlings at a certain position based on the assumption that there should be seedlings at that position, thereby obtaining the detection result. This is conducive to avoiding the situation where the detection result of a position that should be identified as seedling shortage is not seedling shortage, that is, the occurrence of seedling shortage detection omissions. Therefore, the seedling shortage detection of the present application is more targeted and accurate.

[0075] Optionally, the output feature map of the crop seedling detection sub-model includes the confidence level of the presence of a seedling at each location in the image estimated by the crop seedling detection sub-model.

[0076] Please continue to refer to Figure 4 Regarding the content in S102, how to accurately determine the target missing seedling position, the embodiment of the present application also provides a possible implementation method, such as Figure 4 As shown, S102 includes: S102-1, which is described in detail as follows.

[0077] S102-1: When the current suspected seedling missing position coincides with a number of historical suspected seedling missing positions exceeding a first threshold, determine the current suspected seedling missing position as a target seedling missing position.

[0078] Optionally, a position identifier (such as three-dimensional coordinates or two-dimensional coordinates) is determined from the current frame image. When the position identifier is the current suspected missing seedling position, it is determined whether the position identifier is a historical suspected missing seedling position in each historical image frame. If so, it means that the current suspected missing seedling position coincides with a historical suspected missing seedling position. When the number of overlaps exceeds a first number threshold, it indicates that it is the target missing seedling position.

[0079] It's important to understand that a malfunction in a rice transplanter can lead to seedling shortages. If the malfunction isn't repaired promptly, the shortage can spread. Large or continuous seedling shortages can impact the final harvest. Therefore, when large or continuous seedling shortages occur, it's crucial to diagnose the fault type and facilitate repairs.

[0080] exist Figure 2 On the basis of how to find a large area of seedling shortage or continuous seedling shortage and perform fault diagnosis, the embodiment of the present application also provides a possible implementation method, please refer to Figure 5 The seedling shortage detection method further includes: S103 and S104, which are specifically described as follows.

[0081] S103: Determine a planting row that meets the first preset condition or the second preset condition as a target row.

[0082] Among them, the first preset condition represents that the number of consecutive target missing seedling positions in the planting row is greater than a preset second number threshold, and the second preset condition represents that the total number of target missing seedling positions in the planting row is greater than a preset third number threshold.

[0083] Assuming the rice transplanter can simultaneously plant seedlings in six rows of reserved holes, the first row can be identified as the target row when it meets the first or second preset conditions. After observing the first row, the staff can determine the specific seedling shortage and make additional seedlings.

[0084] It should be understood that when there is a target row that meets the first preset condition, it indicates that there is a short-term continuous shortage of seedlings. When there is a target row that meets the second preset condition, it indicates a long-term discontinuous shortage of seedlings. Large-scale shortage of seedlings may be caused by a malfunction of the rice transplanter. It is necessary to diagnose the type of fault, that is, execute S104.

[0085] S104: Determine the fault type corresponding to the rice transplanter based on the working condition data corresponding to the target row.

[0086] Optionally, the working condition data includes one or more of the number of remaining seedlings corresponding to the target row, the vertical displacement of the actuator for inserting the crop seedlings into the holes, and the number of floating seedlings in the rice paddy. For details, please refer to the descriptions in S104-1 to S104-7 below.

[0087] In one possible implementation, Figure 5 The first preset condition and the second preset condition in the embodiment of the present application also provide a possible implementation method, please refer to the following.

[0088] The first preset condition indicates that the number of consecutive target missing seedling positions in the planting row within the second time range is greater than a preset second number threshold, and the second preset condition indicates that the total number of target missing seedling positions in the planting row within the third time range is greater than a preset third number threshold.

[0089] Optionally, the second time range and the third time range can be specifically set according to the operating conditions of the rice transplanter. For example, the second time range can be 3s, and the third time range can be 30s. It should be understood that the second quantity threshold can be adjusted according to the size of the second time range, and the third quantity threshold can be adjusted according to the size of the third time range. For example, the second quantity threshold can be 5, and the third quantity threshold can be 15. It should be understood that the second time range and the third time range can be continuous time ranges before the acquisition time of the current image frame. Compared with screening target rows that meet the first preset condition and / or the second preset condition in the entire time period, the screening range can be narrowed by limiting the second time range and the third time range, thereby improving the computing efficiency.

[0090] exist Figure 5 On the basis of the above, in the case where the working condition data includes the remaining seedling amount corresponding to the target row and the vertical displacement of the actuator for inserting the crop seedlings into the acupuncture points, when the target row meets the first preset condition, regarding the content in S104, how to determine the fault type corresponding to the rice transplanter, the embodiment of the present application also provides a possible implementation method, please refer to Figure 6 , S104 includes: S104-1, S104-2, S104-3, S104-4 and S104-5, which are described in detail as follows.

[0091] S104-1, determine whether the remaining seedling quantity is greater than a preset fourth quantity threshold. If not, execute S104-3; if so, execute S104-2.

[0092] Optionally, after determining the target row that meets the first preset condition, the remaining seedling amount corresponding to the target row can be obtained.

[0093] Optionally, the seedling tray image corresponding to the target row can be obtained, and the remaining seedling amount can be obtained based on the seedling tray image recognition. The seedling tray weight can also be obtained based on the pressure sensor installed under the seedling tray corresponding to the target row, and the remaining seedling amount can be obtained based on the seedling tray weight.

[0094] When the remaining seedling amount is less than or equal to the preset fourth quantity threshold, it means that the remaining seedling amount in the seedling tray is too small, and it is difficult for the actuator (such as a gripper) to obtain the crop seedlings, resulting in a continuous shortage of seedlings for a period of time. At this time, it is necessary to execute S104-2 to determine that the fault type is a shortage of seedlings in the seedling tray.

[0095] When the amount of remaining seedlings is greater than the preset fourth quantity threshold, it is necessary to combine other monitoring data, such as the vertical displacement of the actuator, to further determine the type of fault, that is, execute S104-3.

[0096] Alternatively, the fourth quantity threshold may be the amount of seedlings corresponding to a seedling tray weight of 250 g, or the amount of seedlings corresponding to a preset threshold (e.g., 5%) of crop seedling area in the seedling tray. It should be understood that the threshold may be specifically set based on the total area of the seedling tray.

[0097] S104-2, determining the fault type as a lack of seedlings in the seedling tray.

[0098] Optionally, after determining that the fault type is a lack of seedlings in the seedling tray, the rice transplanter can be controlled to return to replenish the seedlings, or the rice transplanter can be controlled to suspend operation and wait for the user to replenish the seedlings, or the seedling replenishment vehicle can be notified to replenish the seedlings.

[0099] S104-3: Determine whether the vertical displacement of the actuator is greater than a first depth threshold. If not, execute S104-4; if so, execute S104-5.

[0100] The actuator is used to grab the crop seedlings from the seedling tray and insert the crop seedlings into the acupuncture points.

[0101] Optionally, the vertical displacement of the actuator may be acquired by a positioning device (such as Real-time kinematic, RTK for short) installed on the actuator.

[0102] The vertical displacement may be the value most recently monitored by the positioning device, or may be the average value of the values monitored by the positioning device within a preset time range.

[0103] Optionally, the first depth threshold may be 2 cm. It should be understood that the first depth threshold may be specifically set according to the soil type.

[0104] It should be understood that vertical displacement is equivalent to the actuator's insertion depth. When the vertical displacement is too small, the seedlings are not inserted deep enough, which can easily cause them to float, i.e., missing seedlings. Therefore, when the vertical displacement is less than or equal to the first depth threshold, the insertion depth is insufficient. In this case, S104-4 should be executed to determine the fault type as shallow insertion.

[0105] On the contrary, when the vertical displacement is greater than the first depth threshold, it indicates that the actuator has failed, for example, the gripper of the actuator is loose and the crop seedlings cannot be obtained from the seedling tray. In this case, S104-5 is executed.

[0106] S104-4, determining the fault type as too shallow insertion.

[0107] Optionally, after determining that the fault type is too shallow insertion, the operating parameters of the actuator are adjusted so that the vertical displacement is greater than the first depth threshold.

[0108] S104-5: Determine the fault type as an actuator fault.

[0109] Optionally, the actuator failure is, for example, any one or more of the following: the actuator is loose, the actuator is blocked by foreign matter, and the actuator is damaged.

[0110] Please continue to refer to Figure 6 In the case where the working condition data includes the number of floating seedlings in the rice paddy and the vertical displacement of the actuator for inserting the crop seedlings into the acupuncture points, when the target row meets the second preset condition, regarding the content in S104, how to determine the fault type corresponding to the rice transplanter, the embodiment of the present application also provides a possible implementation method, such as Figure 6 As shown, S104 includes: S104-6, S104-7, S104-8, S104-9 and S104-10, which are described in detail as follows.

[0111] S104-6: Determine whether the number of floating seedlings in the rice paddy is greater than a preset fifth number threshold. If so, execute S104-8; if not, execute S104-7.

[0112] Optionally, an image of the rice paddy is obtained, and floating seedlings in the image are identified to determine the number of floating seedlings. If the number of floating seedlings is greater than a preset fifth threshold, it indicates that the actuator was able to retrieve the seedlings from the seedling tray, but the seedlings were not firmly inserted into the corresponding acupuncture points, resulting in floating seedlings. In this case, further determination of the fault type is required, i.e., executing S104-8. Otherwise, it indicates that the actuator has a fault (e.g., a loose gripper), and executing S104-7.

[0113] Optionally, the fifth quantity threshold can be specifically set according to the working time of the rice transplanter in the rice seedling field, that is, the fifth quantity threshold can be set according to the number of times the rice transplanter performs the seedling transplanting action in the rice seedling field.

[0114] S104-7: Determine the fault type as an actuator fault.

[0115] Optionally, the actuator failure is, for example, any one or more of the following: the actuator is loose, the actuator is blocked by foreign matter, and the actuator is damaged.

[0116] S104-8: Determine whether the vertical displacement of the actuator is greater than a first depth threshold. If so, execute S104-10; if not, execute S104-9.

[0117] It should be understood that when the vertical displacement is less than or equal to the first depth threshold, it means that the insertion depth is insufficient. At this time, S104-9 needs to be executed to determine that the fault type is too shallow insertion; on the contrary, when the vertical displacement is greater than the first depth threshold, it means that the insertion depth is sufficient. Then the reason for the lack of seedlings and floating is that the water level in the rice field is too deep. At this time, S104-10 is executed.

[0118] S104-9, determining the fault type is too shallow insertion.

[0119] Optionally, after determining that the fault type is too shallow insertion, the operating parameters of the actuator may be adjusted so that the vertical displacement is greater than a first depth threshold.

[0120] S104-10: Determine the fault type as too deep water level.

[0121] Optionally, the water valve corresponding to the rice paddy can be controlled to release water to lower the water level.

[0122] When there is a large area of seedling shortage or continuous seedling shortage, the final harvest will be affected. Therefore, when the fault type and / or target row are determined, an alarm should be issued to prompt the user to repair the fault and replant the seedlings. Figure 5 Regarding how to implement the alarm reminder, the embodiment of the present application also provides a possible implementation method. The seedling shortage detection method also includes: S105 and / or S106, which are specifically described as follows.

[0123] S105: Issue a fault alarm based on the fault type.

[0124] Optionally, the type of fault alarm may vary with the type of fault, so that when a user observes a fault alarm, he or she can directly and quickly determine the type of fault, thereby improving repair efficiency.

[0125] S106: If the target row exists, a seedling shortage accident alarm is issued.

[0126] In one possible implementation, an alarm is triggered when the first preset condition is met, the second preset condition is met, or both conditions are met simultaneously, but the types of alarms triggered may be different. It should be understood that the triggering of the alarm is used to prompt staff to observe the seedling shortage, replenish seedlings in a timely manner, and troubleshoot the rice transplanter to prevent the seedling shortage from expanding.

[0127] Optionally, the target row identifier (e.g., first row) and the missing seedling type can also be output for the convenience of the user to observe. A prompt message can be sent to the user terminal to prompt the user to confirm the fault.

[0128] Optionally, when there is a target row, the rice transplanter can be controlled to suspend operation, and after receiving a continue operation instruction transmitted by the user terminal, the rice transplanter can be controlled to resume operation.

[0129] It should be understood that if the first or second preset conditions are met, if the rice transplanter continues to operate, a large area of seedlings may be missing. To avoid this large area of seedlings missing, after determining the existence of the target row, it is necessary to control the rice transplanter to suspend operation. After the staff has checked and repaired and confirmed that the current seedling shortage defect has been overcome, the staff can transmit a continue operation instruction through the user terminal, where the user terminal can be a mobile phone, remote control, or handle. After receiving the continue operation instruction transmitted by the user terminal, the rice transplanter is controlled to resume operation.

[0130] The present application also provides a rice transplanter control method, please refer to Figure 7 The rice transplanter control method includes: S301 and S302, which are specifically described as follows.

[0131] S301, determining a target missing seedling position by any of the missing seedling detection methods described above.

[0132] For details, please refer to the implementation of the seedling absence detection method in the previous article.

[0133] S302: Based on the target seedling-missing position determined by any of the seedling-missing detection methods described above, control the rice transplanter to supplement the seedlings.

[0134] Optionally, after determining the target seedling-missing position, the rice transplanter can be controlled to move to the target seedling-missing position, thereby completing the seedling-filling operation.

[0135] See also Figure 8 , Figure 8 A seedling absence detection device is provided in an embodiment of the present application. Optionally, the seedling absence detection device is applied to the electronic device described above.

[0136] The control device of the rice transplanter includes a marking unit 201 and a processing unit 202 .

[0137] The marking unit 201 is used to process the current frame image to determine the current suspected missing seedling position in the current frame image;

[0138] The current frame image displays a current target area, and the current target area includes a planted seedling area before the location where the current frame image is captured;

[0139] The processing unit 202 is configured to determine a target missing seedling position from the current suspected missing seedling position based on historical suspected missing seedling positions in the historical frame images within the first time range.

[0140] Optionally, the marking unit 201 may execute the above-mentioned S101, and the processing unit 202 may execute the above-mentioned S102-S106.

[0141] It should be noted that the seedling shortage detection device provided in this embodiment can execute the method flow shown in the above method flow embodiment to achieve the corresponding technical effect. For the sake of brief description, where this embodiment is not mentioned, reference can be made to the corresponding content in the above embodiment.

[0142] See also Figure 9 , Figure 9 A rice transplanter control device is provided in an embodiment of the present application. Optionally, the rice transplanter control device is applied to the electronic device described above.

[0143] The rice transplanter control device includes: an acquisition unit 401 and a control unit 402.

[0144] The acquisition unit 401 is configured to determine a target missing seedling position by using any of the above-mentioned missing seedling detection methods.

[0145] The control unit 402 is used to control the rice transplanter to replenish seedlings based on the target seedling-missing position determined by any of the seedling-missing detection methods described above.

[0146] It should be noted that the rice transplanter control device provided in this embodiment can execute the method flow shown in the above method flow embodiment to achieve the corresponding technical effects. For the sake of brief description, any part not mentioned in this embodiment can be referred to the corresponding content in the above embodiment.

[0147] The present application also provides a storage medium storing computer instructions or programs that, when read and executed, execute the seedling absence detection method and / or rice transplanter control method of the above-described embodiments. The storage medium may include memory, flash memory, registers, or a combination thereof.

[0148] The following provides an electronic device, which can be a control system on a rice transplanter or an independent computer device. Figure 1 As shown, the above-mentioned seedling shortage detection method and / or rice transplanter control method can be implemented. Specifically, the electronic device includes: a processor 10, a memory 11, and a bus 12. The processor 10 may be a CPU. The memory 11 is used to store one or more programs. When the one or more programs are executed by the processor 10, the above-mentioned seedling shortage detection method and / or rice transplanter control method are executed.

[0149] The following provides a rice transplanter, comprising a machine body, a rice transplanting mechanism arranged on the machine body and the above-mentioned electronic device; the electronic device is used to control the rice transplanter to detect missing seedlings and / or to replenish seedlings at target missing seedling positions.

[0150] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0151] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0152] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0153] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

[0154] It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above and that the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the present application is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

Claims

1. A method for detecting seedling shortage, characterized in that: A control system applied to a rice transplanter, the method comprising: Processing the current frame image to determine a current suspected missing seedling position in the current frame image; The current frame image displays a current target area, and the current target area includes a planted seedling area before the location where the current frame image is captured; Determining whether a current suspected seedling-missing position is a target seedling-missing position based on historical suspected seedling-missing positions in historical frame images within a first time range; Determine a planting row that meets a first preset condition or a second preset condition as a target row, wherein the first preset condition indicates that the number of consecutive target missing seedling positions in the planting row is greater than a preset second number threshold, and the second preset condition indicates that the total number of target missing seedling positions in the planting row is greater than a preset third number threshold; or, the first preset condition indicates that the number of consecutive target missing seedling positions in the planting row within a second time range is greater than the preset second number threshold, and the second preset condition indicates that the total number of target missing seedling positions in the planting row within a third time range is greater than the preset third number threshold; The fault type corresponding to the rice transplanter is determined based on the operating condition data corresponding to the target row.

2. The method for detecting seedling deficiency according to claim 1, wherein The processing of the current frame image to determine the current suspected missing seedling position in the current frame image includes: The current frame image is processed by a missing seedling detection neural network model to obtain the current suspected missing seedling position in the current frame image.

3. The method for detecting seedling deficiency according to claim 2, wherein The seedling absence detection neural network model is a fusion model of the trained crop seedling detection sub-model and the trained seedling absence detection sub-model; Among them, the crop seedling detection sub-model is trained based on the first category of image data, and the missing seedling detection sub-model is trained based on the output of the trained crop seedling detection sub-model and the second category of image data. The image content of the first category of image data is the same as that of the second category of image data. The first category of image data carries a first label, and the first label represents the confidence level of the presence of seedlings at each location in the image. The second category of image data carries a second label, and the second label represents the confidence level of the missing seedlings at each location in the image.

4. The method for detecting seedling deficiency according to claim 1, wherein The determining whether the current suspected seedling-missing position is the target seedling-missing position based on the historical suspected seedling-missing position in the historical frame image within the first time range includes: When the current suspected seedling-missing position coincides with historical suspected seedling-missing positions exceeding a first quantity threshold, the current suspected seedling-missing position is determined to be a target seedling-missing position.

5. The method for detecting seedling deficiency according to claim 1, wherein The working condition data includes the remaining seedling amount corresponding to the target row and the vertical displacement of the actuator for inserting the crop seedling into the hole; When the target row satisfies the first preset condition, determining the fault type corresponding to the rice transplanter based on the working condition data corresponding to the target row includes: When the remaining seedling quantity is less than or equal to a preset fourth quantity threshold, determining that the fault type is a lack of seedlings in the seedling tray; When the remaining seedling amount is greater than the fourth quantity threshold, determining whether the vertical displacement is greater than a preset first depth threshold; If the vertical displacement is less than or equal to the first depth threshold, determining that the fault type is too shallow insertion; If the vertical displacement is greater than the first depth threshold, it is determined that the fault type is an actuator fault.

6. The method for detecting seedling deficiency according to claim 1, wherein The working condition data includes the number of floating seedlings in the rice paddy and the vertical displacement of the actuator for inserting the crop seedlings into the acupuncture points; When the target row satisfies the second preset condition, determining the fault type corresponding to the rice transplanter based on the working condition data corresponding to the target row includes: determining whether the number of floating seedlings in the rice paddy is greater than a preset fifth number threshold; If the number of floating seedlings is less than or equal to the fifth number threshold, determining that the fault type is an actuator fault; If the number of floating seedlings is greater than the fifth number threshold, determining whether the vertical displacement of the actuator is greater than a preset first depth threshold; If the vertical displacement is less than or equal to the first depth threshold, determining that the fault type is too shallow insertion; If the vertical displacement is greater than the first depth threshold, it is determined that the fault type is too deep water level.

7. The method for detecting seedling deficiency according to claim 1, wherein After determining the fault type corresponding to the rice transplanter, the method further includes: Producing a fault alarm based on the fault type; And / or when the target row exists, a seedling shortage accident alarm is issued.

8. A rice transplanter control method, characterized in that: A control system applied to a rice transplanter, the method comprising: Based on the target seedling-missing position determined by the seedling-missing detection method according to any one of claims 1 to 7, the rice transplanter is controlled to supplement the seedlings.

9. A seedling shortage detection device, characterized in that: A control system for a rice transplanter, comprising: a marking unit, configured to process the current frame image to determine a current suspected missing seedling position in the current frame image; The current frame image displays a current target area, and the current target area includes a planted seedling area before the location where the current frame image is captured; a processing unit, configured to determine whether a current suspected seedling-missing position is a target seedling-missing position based on historical suspected seedling-missing positions in historical frame images within a first time range; The processing unit is also used to determine a planting row that meets a first preset condition or a second preset condition as a target row, wherein the first preset condition indicates that the number of consecutive target missing seedling positions in the planting row is greater than a preset second quantity threshold, and the second preset condition indicates that the total number of target missing seedling positions in the planting row is greater than a preset third quantity threshold; or, the first preset condition indicates that the number of consecutive target missing seedling positions in the planting row within a second time range is greater than the preset second quantity threshold, and the second preset condition indicates that the total number of target missing seedling positions in the planting row within a third time range is greater than the preset third quantity threshold; based on the working condition data corresponding to the target row, determine the fault type corresponding to the rice transplanter.

10. A rice transplanter control device, characterized in that: A control system for a rice transplanter, comprising: A control unit is used to control the rice transplanter to replenish seedlings based on the target seedling-missing position determined by the seedling-missing detection method according to any one of claims 1 to 7.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. An electronic device, characterized in that: include: a processor and a memory, the memory being configured to store one or more programs; When the one or more programs are executed by the processor, the method according to any one of claims 1 to 8 is implemented.

13. A rice transplanter, characterized in that: The invention comprises a machine body, a rice transplanting mechanism arranged on the machine body and the electronic device according to claim 12; the electronic device is used to control the rice transplanter to detect the lack of seedlings and / or to replenish seedlings at the target lack of seedlings position.

Images