Chrysanthemum positioning method, device and equipment and storage medium

Abstract

The invention discloses a chrysanthemum positioning method and device, equipment and a storage medium. The method comprises the steps that chrysanthemum image information corresponding to chrysanthemum to be picked is obtained, and the chrysanthemum image information is obtained based on shooting of a depth camera; performing information extraction on the basis of the chrysanthemum image information, and determining chrysanthemums in the chrysanthemum image information and pixel coordinates and depth values corresponding to each chrysanthemum; determining point cloud data corresponding to each chrysanthemum based on the pixel coordinates and the depth values; and optimizing the point cloud data based on a preset optimization algorithm, determining target position information corresponding to each chrysanthemum, and picking the chrysanthemum based on the target position information. According to the invention, the chrysanthemum can be automatically identified and positioned, the labor cost is greatly reduced, and the accuracy and efficiency of chrysanthemum positioning are improved, so that the chrysanthemum picking efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and particularly to a chrysanthemum positioning method, device, equipment and storage medium. Background Art

[0002] In the agricultural field, the speed of chrysanthemum recognition and positioning directly determines the efficiency and accuracy of chrysanthemum picking. The chrysanthemum positioning technology plays a crucial role in the development of chrysanthemum picking technology.

[0003] Currently, the traditional chrysanthemum positioning method usually locates and picks chrysanthemums manually. However, the traditional chrysanthemum recognition and positioning method has a high labor cost and a low chrysanthemum positioning efficiency, thus reducing the chrysanthemum picking efficiency. Summary of the Invention

[0004] The present invention provides a chrysanthemum positioning method, device, equipment and medium to achieve automatic recognition and positioning of chrysanthemums, greatly reduce labor costs, improve the accuracy and efficiency of chrysanthemum positioning, and thus improve the chrysanthemum picking efficiency.

[0005] According to one aspect of the present invention, a chrysanthemum positioning method is provided. The method includes:

[0006] Obtaining chrysanthemum image information corresponding to the chrysanthemums to be picked, where the chrysanthemum image information is obtained by shooting based on a depth camera;

[0007] Performing information extraction based on the chrysanthemum image information to determine the chrysanthemums in the chrysanthemum image information and the pixel coordinates and depth values corresponding to each chrysanthemum;

[0008] Based on the pixel coordinates and the depth values, determining the point cloud data corresponding to each chrysanthemum;

[0009] Optimizing the point cloud data based on a preset optimization algorithm to determine the target position information corresponding to each chrysanthemum, so as to perform chrysanthemum picking based on the target position information.

[0010] According to another aspect of the present invention, a chrysanthemum positioning device is provided. The device includes:

[0011] An image information acquisition module, configured to obtain chrysanthemum image information corresponding to the chrysanthemums to be picked, where the chrysanthemum image information is obtained by shooting based on a depth camera;

[0012] An image information extraction module, configured to perform information extraction based on the chrysanthemum image information to determine the chrysanthemums in the chrysanthemum image information and the pixel coordinates and depth values corresponding to each chrysanthemum;

[0013] A point cloud data determination module, configured to determine point cloud data corresponding to each chrysanthemum based on the pixel coordinates and the depth value;

[0014] A target position determination module, configured to optimize the point cloud data based on a preset optimization algorithm, determine target position information corresponding to each chrysanthemum, and perform chrysanthemum picking based on the target position information.

[0015] According to another aspect of the present invention, there is provided an electronic device, including:

[0016] At least one processor; and

[0017] A memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor is enabled to execute the chrysanthemum positioning method according to any embodiment of the present invention.

[0019] According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing a processor to implement the chrysanthemum positioning method according to any embodiment of the present invention when executed.

[0020] The technical solution of the embodiment of the present invention, by obtaining chrysanthemum image information corresponding to the chrysanthemum to be picked, wherein the chrysanthemum image information is obtained by shooting with a depth camera, can more directly reflect the spatial position relationship of the object, and helps to accurately identify the position and posture of the chrysanthemum. Information extraction is performed based on the chrysanthemum image information to determine the chrysanthemums in the chrysanthemum image information and the pixel coordinates and depth values corresponding to each chrysanthemum, so as to accurately describe the position of the chrysanthemum in three-dimensional space. Based on the pixel coordinates and the depth value, the point cloud data corresponding to each chrysanthemum is determined, which can more intuitively reflect the spatial position relationship of the chrysanthemum. The point cloud data is optimized based on a preset optimization algorithm, and the target position information corresponding to each chrysanthemum is determined, which can remove noise and redundant information in the point cloud data, improve the reliability and stability of the data, and perform chrysanthemum picking based on the target position information, thereby improving the picking efficiency and accuracy. Through the chrysanthemum positioning based on the chrysanthemum image information, it helps to realize the automatic picking of chrysanthemums, can greatly reduce the cost of manual positioning and picking, and improve the efficiency and quality of chrysanthemum positioning and picking.

[0021] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. Description of the Drawings

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0023] Figure 1 is a flowchart of a chrysanthemum positioning method provided in Embodiment 1 of the present invention;

[0024] Figure 2 is an example diagram of chrysanthemum image information involved in Embodiment 1 of the present invention;

[0025] Figure 3 is a structural diagram of a chrysanthemum positioning device involved in Embodiment 1 of the present invention;

[0026] Figure 4 is a schematic diagram of a camera shooting structure involved in Embodiment 1 of the present invention;

[0027] Figure 5 is a schematic diagram of a moving structure involved in Embodiment 1 of the present invention;

[0028] Figure 6 is a schematic diagram of a crawler mechanism involved in Embodiment 1 of the present invention;

[0029] Figure 7 is a flowchart of a chrysanthemum positioning method provided in Embodiment 2 of the present invention;

[0030] Figure 8 is a schematic structural diagram of a chrysanthemum positioning device provided in Embodiment 3 of the present invention;

[0031] Figure 9 is a schematic structural diagram of an electronic device for implementing the chrysanthemum positioning method of the embodiments of the present invention. Detailed implementation manners

[0032] In order to enable those skilled in the art to better understand the solutions of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0033] It should be noted that the terms "first", "second", "target", etc. in the description, claims and the above-mentioned drawings of the present invention are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that such data used can be interchanged under appropriate circumstances, so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0034] Embodiment 1

[0035] Figure 1 The following is a flowchart of a chrysanthemum positioning method provided for Embodiment 1 of the present invention. This embodiment is applicable to the situation of positioning chrysanthemums. This method can be executed by a chrysanthemum positioning device, which can be implemented in the form of hardware and / or software. The chrysanthemum positioning device can be configured in an electronic device. As Figure 1 shown, the method includes:

[0036] S110. Obtain chrysanthemum image information corresponding to the chrysanthemums to be picked, where the chrysanthemum image information is obtained by shooting with a depth camera.

[0037] Among them, the chrysanthemum image information may refer to the image information including the chrysanthemums and their surrounding environment obtained by shooting the chrysanthemum planting area to be picked with a depth camera. A depth camera may refer to a camera that can simultaneously obtain the RGB image and depth (distance) of an object.

[0038] Specifically, a depth camera can be used to shoot the chrysanthemum planting area to obtain the image information including the chrysanthemums and their surrounding environment, that is, the chrysanthemum image information. The depth camera can provide rich three-dimensional information, which helps to accurately identify the position and posture of the chrysanthemums. Compared with traditional two-dimensional images, the depth image can more directly reflect the spatial position relationship of objects.

[0039] Exemplarily, the map information collected by GPS can be used to plan a route, navigate the device equipped with the camera into the roads in the chrysanthemum field, adjust the position to enter the chrysanthemum field ridge, move along the planned roads in the chrysanthemum field, and at the same time turn on the camera. The main control chip controls the fixed motor in the camera shooting structure, and the fixed motor drives the camera to return to the default position of the camera shooting structure, turn on the camera for image acquisition, and obtain the image information including the chrysanthemums and their surrounding environment.

[0040] Exemplarily, S110 may include: performing image acquisition on the area to be picked to obtain area image information corresponding to the area to be picked; in response to the absence of chrysanthemum images in the area image information, adjusting the shooting position, and based on the adjusted shooting position, re-obtaining the area image information corresponding to the area to be picked; in response to the presence of chrysanthemum images in the area image information, determining the area image information as the chrysanthemum image information corresponding to the chrysanthemum to be picked.

[0041] Among them, the area to be picked may refer to the un-picked chrysanthemum planting area. The area image information may refer to the image information obtained by shooting the area to be picked.

[0042] Specifically, aiming the shooting device at the area to be picked, performing image acquisition to obtain the area image information corresponding to this area; the collected area image information can be analyzed using an image recognition algorithm to determine whether there are chrysanthemum images. In response to the absence of chrysanthemum images in the area image information, adjust the position of the shooting device to better capture the chrysanthemum images, and re-obtain the area image information corresponding to the area to be picked; in response to the presence of chrysanthemum images in the area image information, directly determine the area image information as the chrysanthemum image information corresponding to the chrysanthemum to be picked. By adjusting the shooting position, it can ensure that the chrysanthemum images are accurately captured, thereby improving the accuracy of subsequent chrysanthemum recognition.

[0043] Exemplarily, after turning on the camera, the position of the camera in the camera shooting structure can be appropriately adjusted according to whether there are chrysanthemums in the image. If no chrysanthemums are detected in the captured image, the camera can be appropriately moved until chrysanthemums can be detected. After detecting the chrysanthemum features, the latest area image information can be directly used as the chrysanthemum image information corresponding to the chrysanthemum to be picked, and the position coordinates of the current camera in the camera shooting structure can be recorded, and the data of this coordinate can be retained for subsequent coordinate calculations.

[0044] Exemplarily, in response to the presence of chrysanthemum images in the area image information, determining the area image information as the chrysanthemum image information corresponding to the chrysanthemum to be picked further includes: performing pose analysis on the chrysanthemums in the area to be picked based on the area image information to determine whether there are chrysanthemums in the area to be picked in a lodging state; in response to the absence of chrysanthemums in a lodging state in the area to be picked, determining the area image information as the chrysanthemum image information corresponding to the chrysanthemum to be picked; in response to the presence of chrysanthemums in a lodging state in the area to be picked, performing pose adjustment on the chrysanthemums in a lodging state in the area to be picked, re-obtaining the area image information corresponding to the area to be picked after pose adjustment, and determining the re-obtained area image information as the chrysanthemum image information corresponding to the chrysanthemum to be picked.

[0045] Among them, the lodging state may refer to the phenomenon that the stems or flower branches of a plant (such as chrysanthemum) deviate from the normal upright position due to external factors (such as wind, rain, mechanical collision) or its own reasons (such as weak stems, poor growth), and partially or completely lodge on the ground or other objects.

[0046] Specifically, according to the regional image information, the pose analysis of chrysanthemums in the area to be picked is carried out. The feature information of chrysanthemums, such as shape, size, etc., is extracted by using image processing techniques (such as edge detection, contour extraction, etc.). According to the extracted feature information, the pose of the chrysanthemums is analyzed to judge whether they are in a lodging state; if the pose analysis result shows that there are no chrysanthemums in the lodging state in the area to be picked, the regional image information is directly determined as the chrysanthemum image information corresponding to the chrysanthemums to be picked; if the pose analysis result shows that there are chrysanthemums in the lodging state in the area to be picked, a mechanical device (such as a robotic arm, clamp, etc.) can be used to adjust the pose of the lodging chrysanthemums to make them return to the upright state. After the pose adjustment of the chrysanthemums is completed, the depth camera is used again to collect images of the area to be picked, and new regional image information is obtained. The newly collected regional image information is confirmed to ensure that it contains all the chrysanthemums to be picked and the chrysanthemums are in the upright state, and the confirmed regional image information is determined as the chrysanthemum image information corresponding to the chrysanthemums to be picked. By timely identifying and processing the lodging chrysanthemums, and re-collecting and confirming the image information, the picking efficiency and quality can be improved, and the accuracy and stability of the entire picking process can be ensured.

[0047] S120. Information extraction is carried out based on the chrysanthemum image information to determine the chrysanthemums in the chrysanthemum image information and the pixel coordinates and depth values corresponding to each chrysanthemum.

[0048] Among them, the pixel coordinates may refer to the position of each pixel in the image plane in the image, which can be represented by two-dimensional coordinates (x, y). The depth value may refer to the depth information corresponding to each pixel, that is, the distance between the object represented by the pixel and the camera.

[0049] Specifically, the obtained chrysanthemum image information can be preprocessed, such as denoising, enhancing contrast, etc., to improve the image quality. Information extraction is carried out on the preprocessed chrysanthemum image information to determine the contour and boundary of each chrysanthemum, and according to the contour and boundary of each chrysanthemum, the pixel coordinates and depth values corresponding to each chrysanthemum are extracted, which can accurately identify the position and shape of the chrysanthemums and precisely describe the position of the chrysanthemums in the three-dimensional space.

[0050] Exemplarily, S120 may include: performing object detection on the chrysanthemum image information based on the target detection algorithm to determine the chrysanthemums in the chrysanthemum image information and the pixel coordinates of the center point corresponding to each chrysanthemum; performing depth feature extraction on the center point corresponding to each chrysanthemum based on the pixel coordinates to determine the depth value corresponding to each chrysanthemum in the chrysanthemum image information.

[0051] Among them, the object detection algorithm can refer to a key technology in the field of computer vision, aiming to identify the location and category of specific target objects in images or videos. For example, the object detection algorithm can be based on the single-stage detection (YouOnly Look Once, YOLO) algorithm.

[0052] Specifically, the object detection algorithm can be used to perform object detection on the chrysanthemum image information, and output the detection results of each chrysanthemum in the chrysanthemum image, including the category of the chrysanthemum (the category here is "chrysanthemum") and the bounding box information. According to the bounding box information, the pixel coordinates of the center point of each chrysanthemum are calculated, without manual annotation or intervention, improving the detection efficiency. The pixel coordinates of the center point can be calculated from the upper left and lower right coordinates of the bounding box, for example, by taking the average of the upper left and lower right coordinates of the bounding box. As Figure 2 shown, the chrysanthemum image information can include RGB images and depth images. Each pixel value in the depth image represents the distance (depth value) of the object corresponding to the pixel from the camera. The calculated pixel coordinates of the center point of each chrysanthemum are mapped onto the depth image. On the depth image, according to the mapped pixel coordinates, the depth value of the center point of each chrysanthemum is extracted. This depth value represents the distance of the chrysanthemum from the camera. By extracting the depth value, the position information of each chrysanthemum in three-dimensional space can be obtained, providing a basis for subsequent three-dimensional reconstruction, positioning, or picking operations.

[0053] Exemplarily, the object detection algorithm can be the YOLO algorithm. The YOLO algorithm is used to identify all chrysanthemums in the image, and the identified chrysanthemum features are framed. For each detected chrysanthemum bounding box, each bounding box contains the center coordinates (x, y), width (W), and height (H). These coordinates are relative coordinates with respect to the grid cell. Subsequently, these coordinates are normalized to obtain the coordinates in the camera coordinate system, and the formula is as follows:

[0054] x center-pixel = x × W

[0055] y center-pixel = x × H

[0056] The calculated pixel coordinates of the center point of each chrysanthemum are mapped onto the depth image. On the depth image, according to the mapped pixel coordinates, the depth value z of the center point of each chrysanthemum is extracted. This depth value represents the distance of the chrysanthemum from the camera.

[0057] According to the pixel coordinates of the center point detected by YOLO and the corresponding depth value z extracted from the depth image, combined with the calibrated internal parameter values of the camera, the focal length and the position of the optical center, using the camera internal parameters and the depth value, the pixel coordinates are converted into real space coordinates, and the conversion rule is as follows:

[0058]

[0059] Z = kz

[0060] Where k is the scale factor of the depth camera, and subsequently the calculated (X, Y, Z) coordinates are combined into the real - space coordinates of each chrysanthemum.

[0061] S130. Based on the pixel coordinates and depth values, determine the point - cloud data corresponding to each chrysanthemum.

[0062] Where the point - cloud data may refer to the three - dimensional coordinates corresponding to each chrysanthemum.

[0063] Specifically, according to the pixel coordinates and depth values corresponding to each extracted chrysanthemum, combined with the internal and external parameters of the camera (such as focal length, optical center position, etc.), convert the two - dimensional pixel coordinates into horizontal and vertical coordinates in the three - dimensional camera coordinate system, convert the depth value into the vertical coordinate in the three - dimensional camera coordinate system, and combine the converted three - dimensional coordinate parameters to obtain the point - cloud data corresponding to each chrysanthemum, so as to more intuitively reflect the three - dimensional shape and spatial position relationship of the chrysanthemum.

[0064] Exemplarily, S130 may include: based on the pixel coordinates and depth values corresponding to each chrysanthemum, combined with the camera calibration parameters, perform coordinate transformation on the pixel coordinates and depth values to determine the point - cloud data corresponding to each chrysanthemum.

[0065] Where the camera calibration parameters may refer to a set of parameters used to describe the imaging characteristics of the camera, and these parameters are crucial for converting the two - dimensional pixel coordinates in the image into three - dimensional world coordinates.

[0066] Specifically, convert the pixel coordinates and depth values corresponding to each chrysanthemum into three - dimensional point - cloud coordinates through the camera calibration parameters to determine the point - cloud data corresponding to each chrysanthemum. Through coordinate transformation, convert the two - dimensional image information into three - dimensional space coordinates, accurately determine the position of each chrysanthemum in the camera coordinate system, thereby providing accurate three - dimensional position information for the picking device and improving the picking accuracy.

[0067] Exemplarily, based on the pixel coordinates and depth values corresponding to each chrysanthemum, combined with the camera calibration parameters, perform coordinate transformation on the pixel coordinates and depth values to determine the point - cloud data corresponding to each chrysanthemum, including: based on the pixel coordinates and camera calibration parameters, perform three - dimensional coordinate transformation on the pixel coordinates to determine the three - dimensional horizontal coordinates corresponding to the pixel coordinates; based on the depth value and camera calibration parameters, perform coordinate transformation on the depth value to determine the three - dimensional vertical coordinates corresponding to the depth value; merge the three - dimensional horizontal coordinates and three - dimensional vertical coordinates corresponding to each chrysanthemum in the chrysanthemum image information to obtain the point - cloud data corresponding to each chrysanthemum.

[0068] Specifically, perform three-dimensional coordinate transformation on the pixel coordinates according to the camera calibration parameters, normalize the pixel coordinates, and determine the three-dimensional horizontal coordinates corresponding to the pixel coordinates, that is, obtain X and Y (horizontal coordinates) in the three-dimensional coordinates (X, Y, Z); perform coordinate transformation on the depth value according to the camera calibration parameters, and directly determine the depth value as the three-dimensional vertical coordinate, that is, obtain Z (vertical coordinate) in the three-dimensional coordinates (X, Y, Z); merge the obtained three-dimensional horizontal coordinates (X, Y) with the obtained three-dimensional vertical coordinate Z to form a complete three-dimensional point cloud coordinate (X, Y, Z), and obtain the point cloud data corresponding to each chrysanthemum, so as to realize the conversion of two-dimensional image information into three-dimensional space information and provide the real spatial position of the object.

[0069] S140. Optimize the point cloud data based on a preset optimization algorithm, and determine the target position information corresponding to each chrysanthemum, so as to perform chrysanthemum picking based on the target position information.

[0070] Among them, the preset optimization algorithm can refer to an algorithm that is preset for processing point cloud data to improve the quality and accuracy of the data. The target position information can refer to the accurate position of each chrysanthemum in three-dimensional space.

[0071] Specifically, the point cloud data can be processed according to the preset optimization algorithm, such as denoising, thinning, smoothing, etc., to improve the quality and accuracy of the point cloud data. According to the optimized point cloud data, the precise position of the chrysanthemum in three-dimensional space can be determined, and the target position information corresponding to each chrysanthemum can be obtained, so that the chrysanthemum picking device (such as a picking robot) can perform chrysanthemum picking according to the target position information, thereby greatly improving the positioning and picking efficiency of the chrysanthemum.

[0072] In this embodiment, by obtaining the chrysanthemum image information corresponding to the chrysanthemum to be picked, where the chrysanthemum image information is obtained by shooting with a depth camera, it can more directly reflect the spatial position relationship of the object and helps to accurately identify the position and posture of the chrysanthemum. Perform information extraction based on the chrysanthemum image information, and determine the chrysanthemums in the chrysanthemum image information and the pixel coordinates and depth values corresponding to each chrysanthemum, so as to accurately describe the position of the chrysanthemum in three-dimensional space. Based on the pixel coordinates and depth values, determine the point cloud data corresponding to each chrysanthemum, which can more intuitively reflect the spatial position relationship of the chrysanthemum. Optimize the point cloud data based on a preset optimization algorithm, and determine the target position information corresponding to each chrysanthemum, which can remove the noise and redundant information in the point cloud data, improve the reliability and stability of the data, and perform chrysanthemum picking based on the target position information, thereby improving the picking efficiency and accuracy. Through the chrysanthemum positioning based on the chrysanthemum image information, it helps to realize the automatic picking of chrysanthemums, can greatly reduce the cost of manual positioning and picking, and improve the efficiency and quality of chrysanthemum positioning and picking.

[0073] Exemplarily, the above chrysanthemum positioning method can be executed by a chrysanthemum positioning device, such as Figure 3 shown. The chrysanthemum positioning device may include a solar panel 1, a support platform 2, a camera shooting structure 3, a chrysanthemum guiding plate 4, a chrysanthemum fixing fixture 5, a moving structure 6, a vehicle frame 7, and a control box 8. The solar panel 1 is installed on the top of the vehicle frame 7, the chrysanthemum guiding plate 4 is installed on the front side, and the chrysanthemum fixing fixture 5 and the moving device 6 are installed below. The support platform 3 is a rectangular plate-like structure. The control box 8 is installed above the support platform, and the camera shooting structure is installed below. The control box 8 is connected to the camera shooting structure 3, the chrysanthemum fixing fixture 5, and the moving structure 6.

[0074] Such as Figure 4 shown, the camera shooting structure 3 includes a connecting rod 301. Sliding grooves 302 are installed on both sides of the connecting rod 301. A lead screw A303 and a fixed motor A305 are installed on one side of the sliding groove 302. The other side of the lead screw A303 is fixed on the sliding groove 302. A slider A304 is installed on the lead screw A303. The slider A304 is connected through a connecting rod on a cross plate 306. A fixed motor B307 and a lead screw B308 are installed below the cross plate 306. A slider B309 is installed on the lead screw B308. A camera bracket 310 is installed below the slider B309.

[0075] Such as Figure 5 shown, the moving structure 6 includes an electric control box 601, four transmission rods 602, a crawler housing 603, and a crawler mechanism 604; as Figure 6 shown, the crawler mechanism internally includes a shock-absorbing spring 605, a connecting frame 606, and a crawler wheel 607. The electric control box 601 is installed at the bottom of the vehicle frame 7. Four transmission rods 602 are installed on both sides of the electric control box 601. The transmission rods 602 are connected to the crawler wheels 607 at both ends of the crawler structure 604. By driving the transmission rods 602 through the electric control box 601, the transmission rods 602 drive the crawler wheels to complete the driving of the moving structure 3.

[0076] The working principle of the device is as follows: First, calibrate the internal parameters and the position of the optical center of the camera device on flat ground, and set the angle reference and the initial position of the camera. Subsequently, the main control chip in the control box 8 drives the device into the farmland through the electric control box 601 in the moving structure 6. At the same time, the main control chip navigates to the ridge in the chrysanthemum field according to the GPS module in the control box, and the steering function can be realized by controlling the reverse drive of the crawler mechanisms 604 on both sides. After moving above the chrysanthemums, first turn on the camera. In the camera shooting structure 3, the fixed motor A305 and the fixed motor B307 drive the camera bracket 310 to move to the initial position. The camera equipped with a gyroscope measures the current angle. If the difference from the current angle reference is too large, the angle is leveled by adjusting the camera bracket 310 in the camera shooting structure 3. At the same time, the camera shoots the current image to detect whether there are chrysanthemum features in the image. If there are no chrysanthemum features, the position of the camera bracket 310 is further adjusted, and when the chrysanthemum features are captured, the coordinates of the current camera bracket 310 in the camera moving structure 3 are read. Subsequently, the center point of the chrysanthemum features is extracted through an image recognition algorithm, and the horizontal coordinates of the chrysanthemums in the real space are obtained by combining the center point coordinates with the current position of the camera. The depth image of the chrysanthemums is read to obtain the depth value of each chrysanthemum feature. The height coordinates of the chrysanthemum features in the real space are obtained according to the depth value and the internal parameter information calibrated by the camera, and the real three-dimensional coordinates of the chrysanthemums are generated and the point cloud data is sent to the upper computer, and the upper computer analyzes to obtain the growth state and height information of the chrysanthemums.

[0077] Embodiment 2

[0078] Figure 7 The flowchart of a chrysanthemum positioning method provided by Embodiment 2 of the present invention. On the basis of the above embodiments, the step of "optimizing the point cloud data based on a preset optimization algorithm to determine the target position information corresponding to each chrysanthemum" is optimized. The explanations of the same or corresponding terms in the above embodiments are not repeated here.

[0079] See Figure 7 , another chrysanthemum positioning method provided by this embodiment specifically includes the following steps:

[0080] S210. Obtain the chrysanthemum image information corresponding to the chrysanthemums to be picked, where the chrysanthemum image information is obtained based on the shooting of a depth camera.

[0081] S220. Perform information extraction based on the chrysanthemum image information to determine the chrysanthemums in the chrysanthemum image information and the pixel coordinates and depth values corresponding to each chrysanthemum.

[0082] S230. Determine the point cloud data corresponding to each chrysanthemum based on the pixel coordinates and the depth values.

[0083] S240. Based on a preset optimization algorithm, perform local plane fitting on each point within the preset neighborhood radius corresponding to each point to determine the local plane corresponding to each point.

[0084] Among them, the local plane can refer to a plane fitted within a certain local area of the point cloud.

[0085] Specifically, extract all points within the preset neighborhood radius corresponding to each point, and use these neighborhood points for plane fitting. For example, the local plane corresponding to each point can be obtained through the least squares method, so as to capture the local geometric features in the point cloud data and have stronger robustness to noise and outliers in the point cloud.

[0086] S250. Based on the local plane, determine the spatial weight and geometric weight corresponding to each point within each neighborhood.

[0087] Among them, the spatial weight can refer to the weight value assigned according to the spatial distance from the point to the center of the local plane or other points. The geometric weight can refer to the weight value assigned according to the geometric relationship between the point and the local plane (such as the distance from the point to the plane, the angle between the point and the normal vector of the plane).

[0088] Specifically, calculate the spatial weight of each point according to the distance from the point to the center of the local plane or the distance distribution from the point to other points within the neighborhood. Points closer in distance are usually assigned higher weights, which can reflect the spatial distribution characteristics of points within the neighborhood and help smooth the noise in the point cloud data; calculate the geometric weight of each point according to the fitting degree between the point and the local plane, such as the distance from the point to the plane or the angle between the point and the normal vector of the plane. Points with better fitting degrees are assigned higher weights, which can emphasize the points with good fitting degrees to the local plane and improve the accuracy and robustness of plane fitting.

[0089] S260. Based on the coordinates, spatial weights, and geometric weights of each point within the corresponding neighborhood of each point, determine the weighted average position corresponding to each point, and determine the weighted average position as the target position information corresponding to each chrysanthemum, so as to perform chrysanthemum picking based on the target position information.

[0090] Among them, the weighted average position can refer to the weighted average value calculated according to the coordinates of each point within the neighborhood and the corresponding weights, which is used as the final position of the point.

[0091] Specifically, for each point, the weighted average position is calculated using the coordinates, spatial weights, and geometric weights of each point within its neighborhood, and the calculated weighted average position is used as the target position information corresponding to each chrysanthemum, so that the chrysanthemum picking device (such as a picking robot) picks chrysanthemums according to the target position information, thereby greatly improving the positioning and picking efficiency of chrysanthemums. Through weighted averaging, the influence of noise and outliers on the target position information can be reduced, and the accuracy of positioning can be improved.

[0092] Exemplarily, after confirming the real-space coordinates of the chrysanthemums, point cloud data of the chrysanthemums is generated, and bilateral filtering is used to optimize the point cloud data. The input is the original point cloud data, the normal vector of each point is calculated, and the points within its neighborhood are selected. A local plane is fitted to the neighborhood points, and the normal vector of the plane is calculated. The spatial weight and geometric weight of each point are calculated, and the weighted average position is calculated. For each point p in the point cloud, its weighted average position BF(p) after fitting can be expressed as:

[0093]

[0094] where s is the set of neighborhood points of point p, d(p,q) is the Euclidean distance between point p and q, is the depth difference between point p and q, w is the normalization factor, and I(q) represents the depth value of point q, ensuring that the sum of the weights is 1. Among them, w s is the spatial weight, and w r is the geometric weight, which can be expressed as:

[0095]

[0096] where σ s and σ r are the standard deviations of the spatial and geometric ranges respectively, which can be set according to requirements.

[0097] The technical solution of this embodiment passes. Through the present invention, based on a preset optimization algorithm, local plane fitting is performed on each point within the preset neighborhood radius corresponding to each point to determine the local plane corresponding to each point, and the local geometric features in the point cloud data can be captured. Based on the local plane, the spatial weight and geometric weight corresponding to each point within each neighborhood are determined, thereby improving the accuracy and robustness of plane fitting. Based on the coordinates, spatial weights, and geometric weights of each point within the neighborhood corresponding to each point, the weighted average position corresponding to each point is determined, and the weighted average position is determined as the target position information corresponding to each chrysanthemum, improving the accuracy of positioning. Through local fitting, the algorithm has stronger robustness to noise and outliers in the point cloud, can improve the accuracy and robustness of plane fitting, and through weighted averaging, the influence of noise and outliers on the target position information can be reduced, improving the accuracy of positioning.

[0098] Embodiment III

[0099] Figure 8 This is a schematic structural diagram of a chrysanthemum positioning device provided in Embodiment 3 of the present invention. As Figure 8 shown, the device includes: an image information acquisition module 310, an image information extraction module 320, a point cloud data determination module 330, and a target position determination module 340;

[0100] Among them, the image information acquisition module is used to acquire chrysanthemum image information corresponding to the chrysanthemums to be picked, where the chrysanthemum image information is obtained by shooting with a depth camera;

[0101] The image information extraction module is used to perform information extraction based on the chrysanthemum image information to determine the chrysanthemums in the chrysanthemum image information and the pixel coordinates and depth values corresponding to each chrysanthemum;

[0102] The point cloud data determination module is used to determine the point cloud data corresponding to each chrysanthemum based on the pixel coordinates and the depth values;

[0103] The target position determination module is used to optimize the point cloud data based on a preset optimization algorithm to determine the target position information corresponding to each chrysanthemum, so as to pick chrysanthemums based on the target position information.

[0104] In this embodiment, by acquiring chrysanthemum image information corresponding to the chrysanthemums to be picked, where the chrysanthemum image information is obtained by shooting with a depth camera, it can more directly reflect the spatial position relationship of the object, which helps to accurately identify the position and posture of the chrysanthemums. Information extraction is performed based on the chrysanthemum image information to determine the chrysanthemums in the chrysanthemum image information and the pixel coordinates and depth values corresponding to each chrysanthemum, so as to accurately describe the position of the chrysanthemums in three-dimensional space. Based on the pixel coordinates and the depth values, the point cloud data corresponding to each chrysanthemum is determined, which can more intuitively reflect the spatial position relationship of the chrysanthemums. The point cloud data is optimized based on a preset optimization algorithm to determine the target position information corresponding to each chrysanthemum, which can remove the noise and redundant information in the point cloud data, improve the reliability and stability of the data, and pick chrysanthemums based on the target position information, thereby improving the picking efficiency and accuracy. Through the chrysanthemum positioning based on the chrysanthemum image information, it helps to realize the automatic picking of chrysanthemums, can greatly reduce the cost of manual positioning and picking, and improve the efficiency and quality of chrysanthemum positioning and picking.

[0105] Optionally, the image information acquisition module 310 includes:

[0106] The first image acquisition unit is used to perform image acquisition on the area to be picked to acquire the area image information corresponding to the area to be picked;

[0107] A second image acquisition unit, configured to adjust the shooting position in response to the absence of a chrysanthemum image in the regional image information, and re-acquire the regional image information corresponding to the to-be-picked area based on the adjusted shooting position;

[0108] A third image acquisition unit, configured to determine the regional image information as the chrysanthemum image information corresponding to the to-be-picked chrysanthemum in response to the presence of a chrysanthemum image in the regional image information.

[0109] Optionally, the third image acquisition unit is specifically configured to: perform pose analysis on the chrysanthemums in the to-be-picked area based on the regional image information to determine whether there are any chrysanthemums in a lodging state in the to-be-picked area; in response to the absence of chrysanthemums in a lodging state in the to-be-picked area, determine the regional image information as the chrysanthemum image information corresponding to the to-be-picked chrysanthemum; in response to the presence of chrysanthemums in a lodging state in the to-be-picked area, adjust the pose of the chrysanthemums in a lodging state in the to-be-picked area, re-acquire the regional image information corresponding to the to-be-picked area after pose adjustment, and determine the re-acquired regional image information as the chrysanthemum image information corresponding to the to-be-picked chrysanthemum.

[0110] Optionally, the image information extraction module 320 is specifically configured to: perform object detection on the chrysanthemum image information based on an object detection algorithm to determine the chrysanthemums in the chrysanthemum image information and the pixel coordinates of the center point corresponding to each chrysanthemum; perform depth feature extraction on the center point corresponding to each chrysanthemum based on the pixel coordinates to determine the depth value corresponding to each chrysanthemum in the chrysanthemum image information.

[0111] Optionally, the point cloud data determination module 330 includes:

[0112] A point cloud data determination unit, configured to perform coordinate transformation on the pixel coordinates and the depth values based on the pixel coordinates and the depth values corresponding to each chrysanthemum and in combination with camera calibration parameters to determine the point cloud data corresponding to each chrysanthemum.

[0113] Optionally, the point cloud data determination unit is specifically configured to: perform three-dimensional coordinate transformation on the pixel coordinates based on the pixel coordinates and the camera calibration parameters to determine the three-dimensional horizontal coordinates corresponding to the pixel coordinates; perform coordinate transformation on the depth values based on the depth values and the camera calibration parameters to determine the three-dimensional vertical coordinates corresponding to the depth values; and combine the three-dimensional horizontal coordinates and the three-dimensional vertical coordinates corresponding to each chrysanthemum in the chrysanthemum image information to obtain the point cloud data corresponding to each chrysanthemum.

[0114] Optionally, the target position determination module 340 is specifically configured to: perform local plane fitting on each point within a preset neighborhood radius corresponding to each point based on a preset optimization algorithm to determine a local plane corresponding to each point; determine a spatial weight and a geometric weight corresponding to each point within each neighborhood based on the local plane; determine a weighted average position corresponding to each point based on the coordinates of each point within the neighborhood corresponding to each point, the spatial weight, and the geometric weight, and determine the weighted average position as the target position information corresponding to each chrysanthemum.

[0115] The above device can execute the chrysanthemum positioning method provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution of the chrysanthemum positioning method.

[0116] Embodiment 4

[0117] Figure 9 It is a schematic structural diagram of an electronic device for implementing the chrysanthemum positioning method according to an embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processors, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0118] As Figure 9 shown, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. The memory stores a computer program executable by at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0119] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disc, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0120] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the daisy positioning method.

[0121] In some embodiments, the daisy positioning method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the daisy positioning method described above can be executed. Alternatively, in other embodiments, the processor 11 can be configured to execute the daisy positioning method by any other suitable means (e.g., by means of firmware).

[0122] In particular, according to the embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network through the communication unit 19, or installed from the storage unit 18, or installed from the ROM 12. When the computer program is executed by the processor 11, the above-mentioned functions defined in the methods of the embodiments of the present invention are executed.

[0123] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0124] The computer programs for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, a special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The computer programs can be executed entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

[0125] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0126] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the electronic device. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0127] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0128] The computing system can include a client and a server. The client and the server are generally far from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system and solves the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0129] It should be understood that various forms of the processes shown above can be used, reordering, adding, or deleting steps. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is imposed herein.

[0130] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A chrysanthemum positioning method, characterized in that, Including: Obtain the chrysanthemum image information corresponding to the chrysanthemums to be picked, where the chrysanthemum image information is obtained based on the shooting of a depth camera; Perform information extraction based on the chrysanthemum image information to determine the chrysanthemums in the chrysanthemum image information and the pixel coordinates and depth values corresponding to each chrysanthemum; Based on the pixel coordinates and the depth values, determine the point cloud data corresponding to each chrysanthemum; Optimize the point cloud data based on a preset optimization algorithm to determine the target position information corresponding to each chrysanthemum, so as to pick chrysanthemums based on the target position information.

2. The method according to claim 1, wherein The obtaining of the chrysanthemum image information corresponding to the chrysanthemums to be picked includes: Perform image acquisition on the area to be picked to obtain the area image information corresponding to the area to be picked; In response to the non-existence of chrysanthemum images in the area image information, adjust the shooting position, and based on the adjusted shooting position, re-obtain the area image information corresponding to the area to be picked; In response to the existence of chrysanthemum images in the area image information, determine the area image information as the chrysanthemum image information corresponding to the chrysanthemums to be picked.

3. The method according to claim 2, wherein The determining of the area image information as the chrysanthemum image information corresponding to the chrysanthemums to be picked in response to the existence of chrysanthemum images in the area image information further includes: Perform pose analysis on the chrysanthemums in the area to be picked based on the area image information to determine whether there are chrysanthemums in a lodging state in the area to be picked; In response to the non-existence of chrysanthemums in a lodging state in the area to be picked, determine the area image information as the chrysanthemum image information corresponding to the chrysanthemums to be picked In response to the existence of chrysanthemums in a lodging state in the area to be picked, perform pose adjustment on the chrysanthemums in a lodging state in the area to be picked, re-obtain the area image information corresponding to the area to be picked after pose adjustment, and determine the re-obtained area image information as the chrysanthemum image information corresponding to the chrysanthemums to be picked.

4. The method according to claim 1, characterized in that, The performing of information extraction based on the chrysanthemum image information to determine the chrysanthemums in the chrysanthemum image information and the pixel coordinates and depth values corresponding to each chrysanthemum includes: Perform target detection on the chrysanthemum image information based on a target detection algorithm to determine the chrysanthemums in the chrysanthemum image information and the pixel coordinates of the center points corresponding to each chrysanthemum; Perform depth feature extraction on the center points corresponding to each chrysanthemum based on the pixel coordinates to determine the depth values corresponding to each chrysanthemum in the chrysanthemum image information.

5. The method according to claim 1, characterized in that The determining of the point cloud data corresponding to each chrysanthemum based on the pixel coordinates and the depth values includes: Based on the pixel coordinates and the depth values corresponding to each chrysanthemum, in combination with the camera calibration parameters, perform coordinate transformation on the pixel coordinates and the depth values to determine the point cloud data corresponding to each chrysanthemum.

6. The method according to claim 5, wherein The performing of coordinate transformation on the pixel coordinates and the depth values based on the pixel coordinates and the depth values corresponding to each chrysanthemum, in combination with the camera calibration parameters, to determine the point cloud data corresponding to each chrysanthemum includes: Based on the pixel coordinates and the camera calibration parameters, perform three-dimensional coordinate transformation on the pixel coordinates to determine the three-dimensional horizontal coordinates corresponding to the pixel coordinates; Based on the depth value and the camera calibration parameters, perform coordinate transformation on the depth value to determine the three-dimensional vertical coordinate corresponding to the depth value; Combine the three-dimensional horizontal coordinate and the three-dimensional vertical coordinate corresponding to each chrysanthemum in the chrysanthemum image information to obtain the point cloud data corresponding to each chrysanthemum.

7. The method according to claim 1, wherein The optimizing the point cloud data based on a preset optimization algorithm to determine the target position information corresponding to each chrysanthemum includes: Based on a preset optimization algorithm, perform local plane fitting on each point within a preset neighborhood radius corresponding to each point to determine the local plane corresponding to each point; Based on the local plane, determine the spatial weight and geometric weight corresponding to each point within each neighborhood; Based on the coordinates, the spatial weight, and the geometric weight of each point within the neighborhood corresponding to each point, determine the weighted average position corresponding to each point, and determine the weighted average position as the target position information corresponding to each chrysanthemum.

8. A chrysanthemum positioning device, characterized in that, including: An image information acquisition module, configured to acquire chrysanthemum image information corresponding to the chrysanthemum to be picked, where the chrysanthemum image information is obtained by shooting with a depth camera; An image information extraction module, configured to perform information extraction based on the chrysanthemum image information to determine the chrysanthemums in the chrysanthemum image information and the pixel coordinates and depth values corresponding to each chrysanthemum; A point cloud data determination module, configured to determine the point cloud data corresponding to each chrysanthemum based on the pixel coordinates and the depth values; A target position determination module, configured to optimize the point cloud data based on a preset optimization algorithm to determine the target position information corresponding to each chrysanthemum, so as to pick the chrysanthemum based on the target position information.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the chrysanthemum positioning method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a processor to implement the chrysanthemum positioning method according to any one of claims 1-7 when executed.

Images