Weeding point positioning method, device and equipment

Abstract

The invention provides a weeding point positioning method, device and equipment. The method comprises the following steps: acquiring an original image formed by collecting weeds to be removed; preprocessing the original image to obtain a to-be-processed image; the to-be-processed image is identified, the image area where the to-be-removed weeds are located and the type of the to-be-removed weeds are determined, and the type of the to-be-removed weeds is related to the structure and shape of the weeds; processing the image area, and determining a skeleton of the weeds to be removed; and determining a weeding point position of the to-be-removed weed based on the type of the to-be-removed weed and the skeleton of the to-be-removed weed, wherein the weeding point position is a stem base center of the to-be-removed weed in the original image. The weeding point positioning method is used for positioning the weeding points in the area shown in the image more accurately and more efficiently according to the shot image.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer positioning technology, and in particular to a weeding point positioning method, device and equipment. Background Art

[0002] As people's demand for food supply increases, increasing food production is crucial to ensuring food security. However, the yield and quality of crops are easily affected by climate change, environment, pests, and weeds. Among them, weeds have extremely strong reproductive ability. Fast-growing weeds compete with crops for nutrients and growth space, reduce crop photosynthesis, and also threaten to breed pests and introduce crop diseases. Directly or indirectly affect crop yields. According to statistics, the annual food loss caused by weeds is about 13.2%, which is equivalent to the annual food ration of 1 billion people. There are more than 1,400 weeds, more than 130 of which cause serious damage to farmland, and the direct loss reaches 100 billion yuan. At present, the methods of field weeding are generally manual weeding, chemical weeding, biological weeding and mechanical weeding.

[0003] As the previous weeding methods were inefficient and ineffective, with the advancement of science and technology, weeding methods tend to be mechanical weeding, and weeding machinery is also developing towards automation and intelligence. The rapid development of deep learning, multimodal intelligent perception and robotics technology has provided new ideas for weeding. Traditional machine learning methods (such as random forests or support vector machines) are used to identify weed targets, but these methods are highly dependent on feature extraction methods. Subsequent deep learning algorithms automatically extract feature information of weeds through convolutional neural networks (CNNs), which can avoid the shortcomings of traditional extraction methods and are widely used in the field of image segmentation and recognition. At present, the segmentation and positioning of field crop weeds in natural environments still face great challenges. Factors such as the environment in which the photos are taken result in low accuracy in the boundary segmentation between the background and weeds, and weeds evolve towards crop mimicry, making it difficult to effectively distinguish between crops and weeds. How to effectively extract the significant differences between seedlings and grasses and achieve segmentation with the background and accurately obtain the three-dimensional coordinates of weeding points remains to be solved. Summary of the invention

[0004] The embodiments of the present invention provide a weeding point positioning method, device and equipment for more accurately and efficiently positioning a weeding point in an area shown in an image based on a captured image.

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a weeding point positioning method, comprising:

[0006] Obtaining an original image formed by collecting weeds to be removed;

[0007] Preprocessing the original image to obtain an image to be processed;

[0008] Identifying the image to be processed, determining the image area where the weeds to be removed are located and the type of the weeds to be removed, wherein the type of the weeds to be removed is related to the structure and shape of the weeds;

[0009] Processing the image area to determine the skeleton of the weeds to be removed;

[0010] The weeding point position of the weeds to be removed is determined based on the type of the weeds to be removed and the skeleton of the weeds to be removed, and the weeding point position is the center of the stem base of the weeds to be removed in the original image.

[0011] In some embodiments, the step of identifying the image to be processed and determining the image region where the weeds to be removed are located and the type of the weeds to be removed includes:

[0012] Identify the image to be processed based on a neural network model in an instance segmentation algorithm to determine the type of weeds to be removed and the image area where the stem base is located;

[0013] The image region is segmented based on a semantic segmentation model in an instance segmentation algorithm to obtain a mask image corresponding to the image region.

[0014] In some embodiments, the processing of the image area to determine the skeleton of the weeds to be removed includes:

[0015] Determine eight neighborhood pixels of each pixel based on the mask image;

[0016] The eight neighboring pixels of each pixel are sequentially queried along the specified direction to determine the number of pixels that are 1 and the number of times the pixel value changes from 0 to 1;

[0017] Determine whether the pixel point is a skeleton pixel point at the stem base of the weed to be removed based on the number, the number of times and the eight neighboring pixels, and if so, retain it; if not, mark it for deletion;

[0018] All pixel points in the mask image are traversed to obtain all skeleton pixel points forming the skeleton of the stem base.

[0019] In some embodiments, the skeleton is the stem base skeleton of the weeds to be removed;

[0020] The step of determining the weeding point position of the weeds to be removed based on the type of the weeds to be removed and the skeleton of the weeds to be removed comprises:

[0021] Determine the candidate area where the weeding point is located in the skeleton based on the type of the weeds to be removed, wherein the type of the weeds to be removed includes a multi-branched Y-type and a type where leaves are randomly dispersed in all directions;

[0022] The weeding point is identified and determined in the candidate area.

[0023] In some embodiments, the determining of the candidate area where the weeding point is located in the skeleton based on the type of the weeds to be removed includes:

[0024] When the type of the weeds to be removed is a multi-branched Y-type or a type with leaves randomly scattered in all directions, determining the intersection points of all skeleton lines and the area with the most dense pixel distribution in the skeleton;

[0025] The candidate region is determined based on the skeleton line intersections and the region with the most densely distributed pixels.

[0026] In some embodiments, identifying and determining the weeding point in the candidate area includes:

[0027] Obtaining a hyperparameter, wherein the hyperparameter is calculated based on the diameter of the circumscribed circle of the weeds to be removed;

[0028] Determine a circular area in the candidate area with an intersection point in the candidate area as the center and the hyperparameter as the radius;

[0029] Calculate and determine the area of ​​black pixels within the circular area;

[0030] Traversing all intersection points, and determining the areas of black pixels corresponding to all the intersection points;

[0031] The weeding points are determined based on area screening of black pixels corresponding to all the intersections.

[0032] In some embodiments, the step of determining the weeding point based on area screening of black pixels corresponding to all the intersections includes:

[0033] Sort the areas of all the black pixels by size;

[0034] The intersection point corresponding to the black pixel area that meets the requirements of the sorting result is determined as the weeding point.

[0035] In some embodiments, the preprocessing of the original image includes:

[0036] The original image is rotated, white noise is added, and cropped.

[0037] Another embodiment of the present invention also provides a weeding point positioning device, including:

[0038] An acquisition module, used for acquiring an original image formed by collecting weeds to be removed;

[0039] A processing module is used to pre-process the original image to obtain an image to be processed;

[0040] An identification module is used to identify the image to be processed, determine the image area where the weeds to be removed are located and the type of the weeds to be removed, wherein the type of the weeds to be removed is related to the structure and shape of the weeds;

[0041] A processing module, used for processing the image area to determine the skeleton of the weeds to be removed;

[0042] The determination module is used to determine the weeding point position of the weeds to be removed according to the type of the weeds to be removed and the skeleton of the weeds to be removed, and the weeding point position is the center of the stem base of the weeds to be removed in the original image.

[0043] Another embodiment of the present invention further provides an electronic device, comprising:

[0044] one or more processors;

[0045] a memory configured to store one or more programs;

[0046] When the one or more programs are executed by the one or more processors, the one or more processors implement the weeding point positioning method as described in any one of the embodiments above.

[0047] Based on the disclosure of the above embodiments, it can be known that the beneficial effects of the embodiments of the present invention include extracting the weed target to be removed from the collected image, and then using the semantic segmentation network to perform semantic segmentation on the weed target in the area where the weed target is located, so as to obtain the accurate pixel area of ​​the weed to be removed in the collected image. Then, the skeleton of the weed to be removed is extracted based on the pixel area, and finally the weeding point is quickly and effectively determined based on the extracted skeleton. The method in this embodiment significantly optimizes the existing weeding point positioning algorithm, improves the accuracy of weeding point positioning, and provides useful data for accurate weeding of weeding equipment, such as weeding robots, to help weeding equipment accurately weed.

[0048] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings.

[0049] The technical solution of the present application is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 The figure is a flow chart of the weeding point positioning method in an embodiment of the present invention.

[0052] Figure 2 The figure is a schematic diagram of the application process of the weeding point positioning method in the embodiment of the present invention.

[0053] Figure 3 The figure is a schematic diagram of the application flow of the weeding point positioning method in another embodiment of the present invention.

[0054] Figure 4 4 is a position relationship diagram of eight neighborhood pixels in an embodiment of the present invention.

[0055] Figure 5 Schematic diagram of skeleton line intersections in an embodiment of the present invention.

[0056] Figure 6 The figure is a schematic diagram of the application flow of the weeding point positioning method in another embodiment of the present invention.

[0057] Figure 7 Schematic diagram of weeding points in an embodiment of the present invention.

[0058] Figure 8 4 is a structural block diagram of a weeding point positioning device in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings, but are not intended to limit the present invention.

[0060] It should be understood that various modifications may be made to the embodiments disclosed herein. Therefore, the following description should not be considered as limiting, but merely as an example of an embodiment. Other modifications within the scope of the present disclosure will occur to those skilled in the art.

[0061] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the general description of the present disclosure given above and the detailed description of the embodiments given below, serve to explain the principles of the present disclosure.

[0062] These and other characteristics of the invention will become apparent from the following description of a preferred form of embodiment given as a non-limiting example, with reference to the accompanying drawings.

[0063] It should also be understood that, although the invention has been described with reference to some specific examples, those skilled in the art will be able to realize many other equivalent forms of the invention that have the characteristics recited in the claims and are therefore within the scope of protection defined thereby.

[0064] The above and other aspects, features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.

[0065] Specific embodiments of the present disclosure are described hereinafter with reference to the accompanying drawings; however, it should be understood that the disclosed embodiments are merely examples of the present disclosure, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid obscuring the present disclosure with unnecessary or redundant details. Therefore, the specific structural and functional details disclosed herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present disclosure in a variety of ways with substantially any suitable detailed structure.

[0066] This specification may use the phrases "in one embodiment," "in another embodiment," "in a further embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present disclosure.

[0067] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0068] When using traditional geometric calculation methods to estimate weed target coordinates and weeding points, due to the growth shape of weeds, the existing method simply locates the regional center of the weed plant through the center of mass and the intersection of the weed skeleton, which is inaccurate. Therefore, it cannot provide accurate weeding point location information to the execution end of weeding equipment, such as a weeding robot, so that the weeding robot can adjust its posture according to the weeding point location information and complete accurate weeding.

[0069] By summarizing the weed identification and positioning algorithms currently used in research, it is found that the limitations of the existing weeding point positioning methods are mainly reflected in the following aspects: ① When the appearance characteristics of crops and weeds of the same type are relatively small, it is easy to confuse crops with weeds, resulting in inaccurate weed target identification. ② The weed area after identification needs to further calculate the location of its weeding point, but the existing method simply locates the weeding point by the centroid, which is inaccurate and fails to consider the impact of various weed growth forms on the centroid position.

[0070] To solve the above problems, Figure 1 As shown, an embodiment of the present invention provides a weeding point positioning method, including:

[0071] S1: obtaining an original image formed by collecting weeds to be removed;

[0072] S2: preprocessing the original image to obtain an image to be processed;

[0073] S3: Identify the image to be processed, determine the image area where the weeds to be removed are located and the type of the weeds to be removed, and the type of the weeds to be removed is related to the structure and shape of the weeds;

[0074] S4: Processing the image area to determine the skeleton of the weeds to be removed;

[0075] S5: Determine a weeding point position of the weeds to be removed based on the type of the weeds to be removed and the skeleton of the weeds to be removed, wherein the weeding point position is the center of the stem base of the weeds to be removed in the original image.

[0076] First, due to the large leaves of weeds, the root position of the weeds must be accurately identified when using an instrument for intelligent weeding. Weeding by the root position can improve weeding efficiency. Based on this, in this embodiment, the weeding point is pre-positioned at the base of the weed stem, so that the weeding point can be determined by identifying the base of the weed stem.

[0077] The original image described in this embodiment is an RGB image, which can be a video frame image or a separately captured image. The method described in this embodiment realizes weeding point positioning based on RGB images and depth information estimation, including extracting the weed target to be removed from the captured image, and then using a semantic segmentation network to perform semantic segmentation on the weed target in the area where the weed target is located, thereby obtaining the accurate pixel area of ​​the weeds to be removed in the captured image. Afterwards, the skeleton of the weeds to be removed is extracted based on the pixel area, and finally the weeding point is quickly and effectively determined based on the extracted skeleton. The method in this embodiment significantly optimizes the existing weeding point positioning algorithm, improves the accuracy of weeding point positioning, and provides useful data for accurate weeding of weeding equipment, such as weeding robots, to help weeding equipment to accurately weed. For example, Figure 2 As shown, after the weeding point position is determined, the posture of the weeding robot can be controlled and adjusted based on the weeding point position, and after the posture coordinates are optimized, the weeding robot can be matched and controlled to perform weeding.

[0078] like Figure 3 As shown, after obtaining the original image, the original image needs to be preprocessed, including:

[0079] S6: rotating, adding white noise, and cropping the original image.

[0080] For example, considering the angle at which a field robot arm (the execution end of a weeding robot) or other working machinery collects images when working, this embodiment only takes the image in the natural environment of the field collected at a top-down angle as the research object, that is, the original image. After obtaining the original image, the system can, but is not limited to, rotate, add white noise, crop, and other operations on the original image to complete the data enhancement processing of the original image.

[0081] Furthermore, the step of identifying the image to be processed and determining the image region where the weeds to be removed are located and the type of the weeds to be removed includes:

[0082] S7: identifying the image to be processed based on the neural network model in the instance segmentation algorithm, and determining the type of the weeds to be removed and the image area where the stem base is located;

[0083] S8: Segment the image region based on a semantic segmentation model in an instance segmentation algorithm to obtain a mask image corresponding to the image region.

[0084] For example, in this embodiment, the instance segmentation algorithm is used to complete the identification and segmentation of the roots of the weeds to be removed, and a mask is obtained. At the same time, when performing the identification, the system observes the image to be processed and determines the type of the weeds to be removed in combination with the prior data.

[0085] The above steps, this embodiment is based on SCTNet (real-time semantic segmentation architecture) as the basis for the above steps, and the SCTNet is a single-branch CNN with transformer semantic information. With the help of CFBlock and semantic information alignment modules, rich semantic information can be captured from the transformer branch during training. The design of SCTNet takes into account the pursuit of rich long-distance context in real-time semantic segmentation methods. However, traditional methods usually adopt additional semantic branches, resulting in increased computational overhead and slow reasoning speed. In order to solve this dilemma, SCTNet uses transformer as a semantic branch used only for training, and integrates it with a single-branch CNN. In this way, while retaining the high efficiency of lightweight single-branch CNN, SCTNet can also obtain rich semantic representations. In the reasoning process, only a single-branch CNN needs to be deployed, thereby improving the reasoning speed.

[0086] SCTNet is a deep learning network focused on real-time segmentation tasks, aiming to obtain rich semantic information while reducing computational costs. To achieve this goal, SCTNet decomposes the network architecture into two key branches: a CNN branch for inference (the neural network model) and a Transformer branch for semantic alignment in the training phase (the semantic segmentation model).

[0087] In SCTNet, a typical hierarchical CNN backbone is used to accelerate the inference speed. The backbone consists of several core modules: the Stem module, which consists of two 3×3 convolutional layers. The first two stages of the convolutional layer consist of stacked residual modules, and the last two stages use the proposed CFBlock. These modules implement long-range context capture functions similar to Transformer blocks through carefully designed convolution operations. In addition, in the Decoder Head, DAPPM is composed of the segmentation head. In order to further enrich the context information, SCTNet adds an additional DAPPM after Stage4, and splices the S2 and S4 outputs and feeds them into the segmentation head.

[0088] During the training phase, SCTNet uses the Transformer branch to extract global semantic context, while supervising the convolution branch through the semantic information alignment module to align the high-quality global context from the Transformer. This design is inspired by the superiority of the Transformer in capturing global semantic context and the advantage of CNN in modeling hierarchical local information. In the inference phase, in order to avoid excessive computational costs, SCTNet only deploys the CNN branch. By leveraging the semantic information aligned by the Transformer branch, the single-branch CNN is able to generate accurate segmentation results without additional semantic information or dense fusion. Specifically, the input image is fed into a single-branch hierarchical convolutional backbone, and the decoder head extracts features from the backbone and simply splices them for pixel classification.

[0089] In addition, the semantic segmentation branch, that is, the Transformer branch, can also be replaced by other semantic segmentation algorithms, such as FCN, SegNet, Deeplab, etc.

[0090] Furthermore, due to the randomness of the weed growth direction and the randomness of the leaf length, the centroid of the weed deviates from the center of the stem base, and the center position of the stem base cannot be determined by calculating the centroid coordinates. According to the weed growth characteristics, the skeleton structure of the weed is extracted, and it is observed that the weed center is located at the bottom of the weed and near the intersection of the blade skeleton center; and the weed connected domain in the image is a shape that diverges from the center to the surroundings, and the distribution of white pixels closer to the stem base area is denser, so it can be concluded that the center of the weed stem base is located at the intersection of the skeleton center, and the area with the densest pixel distribution of the weed plant is likely to include the central positioning point, that is, the weeding point area, so this embodiment is based on this discovery to narrow the recognition range of the weeding point and optimize the positioning algorithm.

[0091] Continue to combine Figure 3 As shown, the processing of the image area to determine the skeleton of the weeds to be removed includes:

[0092] S9: determining eight neighboring pixels of each pixel point whose pixel value is 1 based on the mask image;

[0093] S10: querying eight neighboring pixels of each pixel point in sequence along a specified direction to determine the number of pixels that are 1 and the number of times the pixel value changes from 0 to 1;

[0094] S11: determining whether the pixel point is a skeleton pixel point of the stem base of the weed to be removed based on the number, the number of times and the eight neighboring pixels, and if so, retaining it; if not, marking it for deletion;

[0095] S12: Traverse all pixel points with a pixel value of 1 in the mask image to obtain all skeleton pixel points forming the skeleton of the stem base.

[0096] The principle of the method described in this embodiment is to delete unnecessary contour pixels, retain the axis pixels in the weeds to be removed, and implement skeleton extraction based on this. The skeleton extraction algorithm is to calculate the eight neighboring pixels of each pixel point in the segmented mask image to determine whether the pixel point meets the preset conditions. If it meets the conditions, it will be retained, otherwise it will be deleted.

[0097] For example, the pixel point is assumed to be (P1), and its eight neighboring pixels P2-P9 are as follows: Figure 4 As shown in the arrangement. Let A(P1) represent the number of times the pixel value changes from 0 to 1 from P2 to P9; B(P1) represents the number of pixels from P2 to P9 that are 1. The skeleton extraction algorithm specifically includes two steps:

[0098] Loop through all points with a pixel value of 1 and check whether the pixel satisfies the following four conditions: ①2≤B(P1)≤6; ②A(P1)=1; ③P2×P4×P8=0; ④P2×P6×P8=0;

[0099] The pixels that meet the above conditions are marked as deleted points, and the remaining unmarked points are retained.

[0100] The above two steps are repeated until all points with a pixel value of 1 are traversed, and the pixels retained are counted as the skeleton pixel points of the stem base of the weeds, thereby obtaining the skeleton of the stem base of the weeds to be removed.

[0101] Furthermore, in this embodiment, the skeleton is a stem base skeleton of the weeds to be removed; the determining of the weeding point position of the weeds to be removed based on the type of the weeds to be removed and the skeleton of the weeds to be removed includes:

[0102] S13: determining a candidate area where the weeding point is located in the skeleton based on the type of the weeds to be removed, wherein the type of the weeds to be removed includes a multi-branched Y-type and a type where leaves are randomly dispersed in all directions;

[0103] S14: Identify and determine the weeding point in the candidate area.

[0104] Wherein, determining the candidate area where the weeding point is located in the skeleton based on the type of the weeds to be removed includes:

[0105] S15: When the type of the weeds to be removed is a multi-branched Y-type or a type with leaves randomly scattered in all directions, determining the intersection points of all skeleton lines and the area with the densest pixel distribution in the skeleton;

[0106] S16: Determine the candidate region based on the intersection points of the skeleton lines and the region with the densest pixel distribution.

[0107] For example, based on the above content, it can be known that the center point of the stem base of the weed to be removed is located at the skeleton intersection or the area with dense black pixel distribution. Therefore, the skeleton intersection is first determined, including extracting the intersection of all skeleton lines in the skeleton map, and storing the intersection coordinates in the set C∈R nx2 The intersection point can be referred to Figure 5 shown.

[0108] Then, if Figure 6 As shown, the step of identifying and determining the weeding point in the candidate area includes:

[0109] S17: Obtaining a hyperparameter, wherein the hyperparameter is calculated based on the diameter of the circumscribed circle of the weeds to be removed;

[0110] S18: determining a circular area within the candidate area with an intersection point within the candidate area as the center and the hyperparameter as the radius;

[0111] S19: Calculate and determine the area of ​​black pixels in the circular area;

[0112] S20: traversing all intersection points, and determining the areas of black pixels corresponding to all the intersection points;

[0113] S21: Determine the weeding point based on the area screening of the black pixels corresponding to all the intersections.

[0114] The step of determining the weeding point based on the area screening of the black pixels corresponding to all the intersections includes:

[0115] S22: sorting the areas of all the black pixels;

[0116] S23: Determine the intersection point corresponding to the black pixel area that meets the requirements of the sorting result as the weeding point.

[0117] For example, for weeds to be removed, the leaves are densely distributed near the base of the stem, and the corresponding black pixels are densely distributed near the base of the stem in the binary image (mask image). Based on this, when determining the pixel-dense area, first calculate the short axis length of the minimum circumscribed rectangle of the weeds and record it as R. With R as the diameter and the diagonal center of the circumscribed rectangle as the center of the circle, draw a circular area in the mask image. This circular area can contain most of the pixels in the base area of ​​the weed stem. At this time, R can be approximately equal to the diameter of the circumscribed circle of the weed, and introduce the hyperparameter r=R / 4. Suppose the possible positioning point c to be screened is n The coordinates are (x n ,y n ), with c n Select a circular area m with radius r as the center n , m n The equation is shown in the following formula (1). Then m is calculated according to formula (2): n The area of ​​black pixels in the region S = {s1, s2, ..., s n},s n The larger the value, the greater the density of the local area where the pixel point as the center of the circle is located, and the higher the probability that the pixel point is a weeding point.

[0118] (xx n ) 2 +(yy n ) 2 =r 2 (1)

[0119]

[0120] Where N is m n The number of pixels in the area. The location of the weeding points obtained after screening can be referred to Figure 7 Based on the above method, all intersections are traversed, the areas of black pixels corresponding to all the intersections are determined, and the areas are sorted. After that, the center of the circle (pixel point) with the largest area can be selected as the weeding point, but is not limited to it. Based on the weeding point, the posture algorithm and other support are provided for the subsequent intelligent automatic mechanized weeding.

[0121] like Figure 8 As shown, another embodiment of the present invention also provides a weeding point positioning device 100, including:

[0122] An acquisition module, used for acquiring an original image formed by collecting weeds to be removed;

[0123] A processing module is used to pre-process the original image to obtain an image to be processed;

[0124] An identification module is used to identify the image to be processed, determine the image area where the weeds to be removed are located and the type of the weeds to be removed, wherein the type of the weeds to be removed is related to the structure and shape of the weeds;

[0125] A processing module, used for processing the image area to determine the skeleton of the weeds to be removed;

[0126] The determination module is used to determine the weeding point position of the weeds to be removed according to the type of the weeds to be removed and the skeleton of the weeds to be removed, and the weeding point position is the center of the stem base of the weeds to be removed in the original image.

[0127] In some embodiments, the step of identifying the image to be processed and determining the image region where the weeds to be removed are located and the type of the weeds to be removed includes:

[0128] Identify the image to be processed based on a neural network model in an instance segmentation algorithm to determine the type of weeds to be removed and the image area where the stem base is located;

[0129] The image region is segmented based on a semantic segmentation model in an instance segmentation algorithm to obtain a mask image corresponding to the image region.

[0130] In some embodiments, the processing of the image area to determine the skeleton of the weeds to be removed includes:

[0131] Determine eight neighborhood pixels of each pixel based on the mask image;

[0132] The eight neighboring pixels of each pixel are sequentially queried along the specified direction to determine the number of pixels that are 1 and the number of times the pixel value changes from 0 to 1;

[0133] Determine whether the pixel point is a skeleton pixel point at the stem base of the weed to be removed based on the number, the number of times and the eight neighboring pixels, and if so, retain it; if not, mark it for deletion;

[0134] All pixel points in the mask image are traversed to obtain all skeleton pixel points forming the skeleton of the stem base.

[0135] In some embodiments, the skeleton is the stem base skeleton of the weeds to be removed;

[0136] The step of determining the weeding point position of the weeds to be removed based on the type of the weeds to be removed and the skeleton of the weeds to be removed comprises:

[0137] Determine the candidate area where the weeding point is located in the skeleton based on the type of the weeds to be removed, wherein the type of the weeds to be removed includes a multi-branched Y-type and a type where leaves are randomly dispersed in all directions;

[0138] The weeding point is identified and determined in the candidate area.

[0139] In some embodiments, the determining of the candidate area where the weeding point is located in the skeleton based on the type of the weeds to be removed includes:

[0140] When the type of the weeds to be removed is a multi-branched Y-type or a type with leaves randomly scattered in all directions, determining the intersection points of all skeleton lines and the area with the most dense pixel distribution in the skeleton;

[0141] The candidate region is determined based on the skeleton line intersections and the region with the most densely distributed pixels.

[0142] In some embodiments, identifying and determining the weeding point in the candidate area includes:

[0143] Obtaining a hyperparameter, wherein the hyperparameter is calculated based on the diameter of the circumscribed circle of the weeds to be removed;

[0144] Determine a circular area in the candidate area with an intersection point in the candidate area as the center and the hyperparameter as the radius;

[0145] Calculate and determine the area of ​​black pixels within the circular area;

[0146] Traversing all intersection points, and determining the areas of black pixels corresponding to all the intersection points;

[0147] The weeding points are determined based on area screening of black pixels corresponding to all the intersections.

[0148] In some embodiments, the step of determining the weeding point based on area screening of black pixels corresponding to all the intersections includes:

[0149] Sort the areas of all the black pixels by size;

[0150] The intersection point corresponding to the black pixel area that meets the requirements of the sorting result is determined as the weeding point.

[0151] In some embodiments, the preprocessing of the original image includes:

[0152] The original image is rotated, white noise is added, and cropped.

[0153] Another embodiment of the present invention further provides an electronic device, comprising:

[0154] one or more processors;

[0155] a memory configured to store one or more programs;

[0156] When the one or more programs are executed by the one or more processors, the one or more processors implement the weeding point positioning method as described in any one of the embodiments above.

[0157] The electronic device may be a processing device integrated in a weeding robot or other weeding equipment, such as a central control machine.

[0158] Furthermore, an embodiment of the present application also provides a storage medium on which a computer program is stored, and when the program is executed by a processor, the weeding point positioning method described above is implemented. It should be understood that each scheme in this embodiment has the corresponding technical effects in the above method embodiment, which will not be repeated here.

[0159] Furthermore, an embodiment of the present application also provides a computer program product, which is tangibly stored on a computer-readable medium and includes computer-readable instructions, which, when executed, enable at least one processor to perform a weeding point positioning method such as that in the embodiment described above.

[0160] It should be noted that the computer storage medium of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access storage medium (RAM), a read-only storage medium (ROM), an erasable programmable read-only storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only storage medium (CD-ROM), an optical storage medium, a magnetic storage medium, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate or transmit a program configured to be used by or in conjunction with an instruction execution system, system or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, antenna, optical cable, RF, etc., or any suitable combination of the above.

[0161] In addition, it should be understood by those skilled in the art that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0162] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or boxes.

[0163] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction system, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0164] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of protection of the present application is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

Claims

1. A weeding point positioning method, characterized in that: include: Obtaining an original image formed by collecting weeds to be removed; Preprocessing the original image to obtain an image to be processed; Identifying the image to be processed, determining the image area where the weeds to be removed are located and the type of the weeds to be removed, wherein the type of the weeds to be removed is related to the structure and shape of the weeds; Processing the image area to determine the skeleton of the weeds to be removed; The weeding point position of the weeds to be removed is determined based on the type of the weeds to be removed and the skeleton of the weeds to be removed, and the weeding point position is the center of the stem base of the weeds to be removed in the original image.

2. The weeding point positioning method according to claim 1, characterized in that: The step of identifying the image to be processed and determining the image area where the weeds to be removed are located and the type of the weeds to be removed includes: Identify the image to be processed based on a neural network model in an instance segmentation algorithm to determine the type of weeds to be removed and the image area where the stem base is located; The image region is segmented based on a semantic segmentation model in an instance segmentation algorithm to obtain a mask image corresponding to the image region.

3. The weeding point positioning method according to claim 2, characterized in that: The step of processing the image region to determine the skeleton of the weeds to be removed includes: Determine eight neighboring pixels of each pixel point whose pixel value is 1 based on the mask image; The eight neighboring pixels of each pixel are sequentially queried along the specified direction to determine the number of pixels that are 1 and the number of times the pixel value changes from 0 to 1; Determine whether the pixel point is a skeleton pixel point of the stem base of the weed to be removed based on the number, the number of times and the eight neighboring pixels, and if so, retain it; if not, mark it for deletion; All pixel points with a pixel value of 1 in the mask image are traversed to obtain all skeleton pixel points forming the skeleton of the stem base.

4. The weeding point positioning method according to claim 1, characterized in that: The skeleton is the skeleton of the stem base of the weeds to be removed; The step of determining the weeding point position of the weeds to be removed based on the type of the weeds to be removed and the skeleton of the weeds to be removed comprises: Determine the candidate area where the weeding point is located in the skeleton based on the type of the weeds to be removed, wherein the type of the weeds to be removed includes a multi-branched Y-type and a type where leaves are randomly dispersed in all directions; The weeding point is identified and determined in the candidate area.

5. The weeding point positioning method according to claim 4, characterized in that: The step of determining the candidate area where the weeding point is located in the skeleton based on the type of the weeds to be removed comprises: When the type of the weeds to be removed is a multi-branched Y-type or a type with leaves randomly scattered in all directions, determining the intersection points of all skeleton lines and the area with the most dense pixel distribution in the skeleton; The candidate region is determined based on the skeleton line intersections and the region with the most densely distributed pixels.

6. The weeding point positioning method according to claim 4, characterized in that: The step of identifying and determining the weeding point in the candidate area includes: Obtaining a hyperparameter, wherein the hyperparameter is calculated based on the diameter of the circumscribed circle of the weeds to be removed; Determine a circular area in the candidate area with an intersection point in the candidate area as the center and the hyperparameter as the radius; Calculate and determine the area of ​​black pixels within the circular area; Traversing all intersection points, and determining the areas of black pixels corresponding to all the intersection points; The weeding points are determined based on area screening of black pixels corresponding to all the intersections.

7. The weeding point positioning method according to claim 6, characterized in that: The step of determining the weeding point based on area screening of black pixels corresponding to all the intersections includes: Sort the areas of all the black pixels by size; The intersection point corresponding to the black pixel area that meets the requirements of the sorting result is determined as the weeding point.

8. The weeding point positioning method according to claim 1, characterized in that: The preprocessing of the original image comprises: The original image is rotated, white noise is added, and cropped.

9. A weeding point positioning device, characterized in that: include: An acquisition module, used for acquiring an original image formed by collecting weeds to be removed; A preprocessing module, used for preprocessing the original image to obtain an image to be processed; An identification module is used to identify the image to be processed, determine the image area where the weeds to be removed are located and the type of the weeds to be removed, wherein the type of the weeds to be removed is related to the structure and shape of the weeds; A processing module, used for processing the image area to determine the skeleton of the weeds to be removed; The determination module is used to determine the weeding point position of the weeds to be removed according to the type of the weeds to be removed and the skeleton of the weeds to be removed, and the weeding point position is the center of the stem base of the weeds to be removed in the original image.

10. An electronic device comprising: one or more processors; a memory configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the weeding point locating method according to any one of claims 1 to 8.

Images