Method and device for identifying hopper height of grass receiving vehicle of silage maize harvester

Abstract

The invention provides a hopper height identification method and device for a grass receiving vehicle of a silage maize harvester. The method comprises the following steps: acquiring a target depth map of a grass receiving vehicle to be identified acquired by a binocular camera, and converting the target depth map into a three-dimensional point cloud; the binocular camera is installed on the silage machine, and plane fitting processing is carried out to determine the target near side face of the to-be-recognized grass receiving vehicle; segmenting the target near side surface along the vertical direction to obtain a plurality of segments; based on the number of the data points in each segment, determining an edge line of the target near side surface; and based on the three-dimensional coordinate values of the data points belonging to the sideline of the target near side surface, determining the hopper height of the grass receiving vehicle to be identified, thereby improving the identification precision of the hopper height of the grass receiving vehicle of the silage maize harvester.

Description

Technical Field

[0001] The present application relates to the field of agricultural machinery, and more specifically, to a method and device for identifying the height of a silage truck bucket. Background Art

[0002] Silage machines are widely used to harvest corn silage as livestock feed. Silage feed can keep nutrients from being lost while enriching nutrition, which is beneficial to the digestion and absorption of livestock such as cattle and sheep, and improves economic benefits. During the operation, the silage machine needs to unload the harvested silage into the grass receiving vehicle in real time, and ensure that the silage will not be sprayed outside the bucket of the grass receiving vehicle. This process requires a silage machine operator to drive the silage machine, and another person to control the sprayer to spray the grass to the area where the grain is not full in the bucket, ensuring that the silage will not be sprayed outside the bucket of the grass receiving vehicle. At present, there is an automatic unloading function for silage machines. This function uses the camera installed on the silage machine to identify the bucket range of the grass receiving vehicle, and judges the full state of the bucket based on the bucket range and its height, and automatically controls the sprayer to spray the area where the grain is not full. After turning on this function, only one silage machine operator is needed to drive the silage machine, and there is no need to add additional personnel to control the sprayer, which greatly reduces labor costs and reduces the fatigue of silage machine operators.

[0003] It can be seen that an important part of realizing the automatic unloading function of the silage machine is the recognition of the bucket height. Only when the bucket height is accurately recognized can the grain fullness status of each area in the bucket be accurately judged based on the bucket height. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a method and device for identifying the height of a forage truck bucket, which can improve the accuracy of identifying the forage truck bucket.

[0005] The present application provides a method for identifying the height of a silage truck bucket, the method comprising:

[0006] Obtaining a target depth map of the forage truck to be identified collected by a binocular camera, and converting the target depth map into a three-dimensional point cloud in a world coordinate system; the binocular camera is installed on the forage truck; the three-dimensional point cloud includes a plurality of data points;

[0007] Performing plane fitting processing on the three-dimensional point cloud, extracting data points belonging to the near side of the grass receiving vehicle to be identified, and determining the target near side of the grass receiving vehicle to be identified; wherein the near side is the side of the grass receiving vehicle close to the silage machine;

[0008] Segmenting the near side surface of the target of the three-dimensional point cloud of the grass-catching vehicle to be identified along the vertical direction of the world coordinate system to obtain a plurality of segments of the near side surface of the target;

[0009] Determine the edge of the target near side surface based on the number of data points in each segment of the target near side surface; wherein the change in the number of data points between the segment where the edge is located and the segment above the edge meets the preset change condition;

[0010] The height of the bucket of the grass receiving vehicle to be identified is determined based on the three-dimensional coordinate values ​​of the data points belonging to the edge line of the near side surface of the target in the world coordinate system.

[0011] In some embodiments, in the method for identifying the height of a forage truck bucket, the edge line of the target near side surface is determined based on the number of data points in each segment of the target near side surface; the method comprises:

[0012] Based on the number of data points in each segment of the near side surface of the target, a target line graph is generated; the target line graph represents the relationship between different segments of the near side surface of the target and the number of data points corresponding to each segment;

[0013] The slope of each segment is calculated, the slopes are sorted and the segment positions where the slopes are located are recorded; the segment position where the maximum slope is located is determined as the target segment, and the edge line of the target near side surface is determined based on the target segment.

[0014] In some embodiments, in the forage truck bucket height recognition method, based on the number of data points in each segment of the target near side surface, a target line graph is generated, including: taking the vertical coordinate value of each segment of the target near side surface in the world coordinate system as the x value of the segment;

[0015] The number of data points in each segment of the near side of the target is taken as the y value of the segment;

[0016] Generate a target line chart based on the x and y values ​​of each segment.

[0017] In some embodiments, in the method for identifying the height of a forage truck bucket, before segmenting the target near side surface of the three-dimensional point cloud of the forage truck to be identified along the vertical direction of the world coordinate system, the method further includes:

[0018] The height of the segment is determined based on the accuracy requirements.

[0019] In some embodiments, in the method for identifying the height of the bucket of a forage truck, plane fitting processing is performed on the three-dimensional point cloud to extract data points belonging to the near side of the forage truck to be identified, and the target near side of the forage truck to be identified is determined, including:

[0020] Randomly extract a sample subset from the three-dimensional point cloud, and determine a sample plane model corresponding to the sample subset;

[0021] Calculating the deviation between the data points in the three-dimensional point cloud and the sample plane model, determining the inner sample points and the outer sample points in the data points, and recording the number of the inner sample points corresponding to the sample plane model;

[0022] Repeat the step of determining the sample plane model until a preset iteration stop condition is met, and obtain multiple sample plane models;

[0023] The sample plane model with the largest number of sample points is selected as the target near side surface of the grass picking vehicle to be identified.

[0024] In some embodiments, in the method for identifying the height of a forage truck bucket, after determining the edge line of the target near side surface based on the number of data points in each segment of the target near side surface, the method further includes:

[0025] Based on the distance between the proximal side and the outer side of the bucket of the grass receiving vehicle, the edge line of the target proximal side is translated to determine the edge line of the target outer side of the bucket of the grass receiving vehicle;

[0026] The edge line of the near side surface of the target and the edge line of the outer side surface of the target are used to determine the position of the bucket of the grass receiving truck.

[0027] In some embodiments, a device for identifying the height of a silage truck bucket is also provided, the device comprising:

[0028] An acquisition module is used to acquire a target depth map of the forage truck to be identified collected by a binocular camera, and convert the target depth map into a three-dimensional point cloud; the binocular camera is installed on the forage truck; the three-dimensional point cloud includes a plurality of data points, each of which corresponds to a three-dimensional coordinate value in a world coordinate system;

[0029] An extraction module is used to perform plane fitting processing on the three-dimensional point cloud, extract data points belonging to the near side of the grass receiving vehicle to be identified, and determine the target near side of the grass receiving vehicle to be identified; wherein the near side is the side of the grass receiving vehicle close to the silage machine;

[0030] A segmentation module, which segments the near side surface of the target of the three-dimensional point cloud of the grass-catching vehicle to be identified along the vertical direction of the world coordinate system to obtain multiple segments of the near side surface of the target;

[0031] A first determination module is used to determine the edge of the target near side surface based on the number of data points in each segment of the target near side surface; wherein the change in the number of data points between the segment where the edge is located and the segment above the edge meets the preset change condition;

[0032] The second determination module is used to determine the height of the bucket of the grass receiving vehicle to be identified based on the three-dimensional coordinate values ​​of the data points belonging to the edge line of the near side surface of the target in the world coordinate system.

[0033] In some embodiments, in the forage truck bucket height recognition device, the first determination module, when determining the edge line of the target near side surface based on the number of data points in each segment of the target near side surface, is specifically used to:

[0034] Based on the number of data points in each segment of the near side surface of the target, a target line graph is generated; the target line graph represents the relationship between different segments of the near side surface of the target and the number of data points corresponding to each segment;

[0035] A target segment with the largest slope in the target line graph is found, and a sideline of the near side of the target is determined based on the target segment.

[0036] In some embodiments, in the forage truck bucket height recognition device, the first determination module, when determining the position with the largest slope in the target broken line graph and determining the edge line of the near side surface of the target based on the target segmentation, is specifically used to:

[0037] Calculate the slope of each segment, sort the slopes and record the segment position of the slope;

[0038] The segment position where the maximum slope is located is determined as the target segment.

[0039] In an embodiment of the present application, a method and device for identifying the height of a forage truck bucket are provided. The method obtains a target depth map of a forage truck to be identified collected by a binocular camera, and converts the target depth map into a three-dimensional point cloud; performs plane fitting processing on the three-dimensional point cloud, extracts data points belonging to the near side of the forage truck to be identified, and determines the target near side of the forage truck to be identified; wherein the near side is the side of the forage truck close to the forage truck; the target near side of the three-dimensional point cloud of the forage truck to be identified is segmented along the vertical direction of the world coordinate system to obtain multiple segments of the target near side; based on the number of data points in each segment of the target near side, the edge line of the target near side is determined; wherein the segment where the edge line is located and the change in the number of data points between the segments above the edge line meets the preset change conditions; based on the three-dimensional coordinate values ​​of the data points belonging to the edge line of the near side of the target in the world coordinate system, the height of the bucket of the grass-catching truck to be identified is determined. In this way, processing is performed in three-dimensional space, which has one more dimension of distance information than a two-dimensional image, thereby improving the recognition accuracy. The edge of the grass-catching truck is identified based on the change law of the data points near the edge of the three-dimensional point cloud of the bucket of the grass-catching truck, and the recognition accuracy is high again; and the recognition process does not depend on the number of data sets, but only needs to use the CPU for calculation, which greatly reduces resource consumption and hardware costs, and also solves the problem of poor model generalization ability caused by limited data in the data set, thereby leading to the problem of recognition accuracy of the bucket of the grass-catching truck. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 A flow chart of the method for identifying the height of a silage truck bucket according to an embodiment of the present application is shown;

[0042] Figure 2 Another flow chart of the method for identifying the height of a silage truck bucket of a silage truck according to an embodiment of the present application is shown;

[0043] Figure 3 shows a two-dimensional depth map before conversion according to an embodiment of the present application;

[0044] Figure 4 The converted three-dimensional point cloud image described in the embodiment of the present application is shown;

[0045] Figure 5 A flow chart of a method for determining the edge line of the near side surface of the target according to an embodiment of the present application is shown;

[0046] Figure 6 The figure showing the height identification diagram of the bucket according to the embodiment of the present application is shown;

[0047] Figure 7 A schematic diagram of the structure of the silage truck bucket height recognition device described in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0048] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0049] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0050] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0051] Silage machines are widely used to harvest corn silage as livestock feed. Silage feed can keep nutrients from being lost while enriching nutrition, which is beneficial to the digestion and absorption of livestock such as cattle and sheep, and improves economic benefits. During the operation, the silage machine needs to unload the harvested silage into the grass receiving vehicle in real time, and ensure that the silage will not be sprayed outside the bucket of the grass receiving vehicle. This process requires a silage machine operator to drive the silage machine, and another person to control the sprayer to spray the grass to the area where the grain is not full in the bucket, ensuring that the silage will not be sprayed outside the bucket of the grass receiving vehicle. At present, there is an automatic unloading function for silage machines. This function uses the camera installed on the silage machine to identify the bucket range of the grass receiving vehicle, and judges the full state of the bucket based on the bucket range and its height, and automatically controls the sprayer to spray the area where the grain is not full. After turning on this function, only one silage machine operator is needed to drive the silage machine, and there is no need to add additional personnel to control the sprayer, which greatly reduces labor costs and reduces the fatigue of silage machine operators.

[0052] It can be seen that an important part of realizing the automatic unloading function of the silage machine is the recognition of the bucket height. Only when the bucket height is accurately recognized can the grain fullness status of each area in the bucket be accurately judged based on the bucket height.

[0053] In order to realize the height recognition of the truck bucket, we can consider performing traditional image processing on the RGB image and depth image and using deep learning to train the model to recognize the truck bucket. The traditional image processing method uses algorithms such as canny edge extraction, grayscale binarization and Hough line detection to extract the edge of the truck bucket. Deep learning uses a large amount of collected data to train the model and then recognize the truck bucket.

[0054] However, in traditional edge extraction methods, two-dimensional image processing does not contain distance values, which makes it impossible to directly process three-dimensional spatial information; and it is difficult to correctly extract edges in areas with repeated textures and weak textures. In addition, due to interferences such as reinforcing ribs and tarpaulins on the bucket of the hay truck in actual production, the bucket edge captured in the two-dimensional image is easily confused. Moreover, when the bucket is too full of grain, it is easy to cause occlusion and confusion between silage and the upper edge of the bucket, which has a great impact on the recognition of the bucket height.

[0055] Deep learning requires a large amount of data and powerful computing power, and has high hardware requirements and high costs. The model design of deep learning is also very complex, and a lot of manpower, material resources and time are required to develop new algorithms and models. In addition, deep learning is highly dependent on data. The larger the amount of data, the better the performance, but the performance is limited when the amount of data is limited. The silage harvest season is only two months a year, September and October. The data collected during each harvest season is limited, and the number of grass receiving trucks at a work site is small, which makes the diversity of the collected data far from meeting the requirements. This makes the generalization ability of the trained model too poor, but there are many types of grass receiving trucks in China, resulting in a low recognition accuracy rate for the grass receiving truck bucket.

[0056] Based on this, in an embodiment of the present application, a method and device for identifying the height of a forage truck bucket are provided, the method obtaining a target depth map of a forage truck to be identified collected by a binocular camera, and converting the target depth map into a three-dimensional point cloud; performing plane fitting processing on the three-dimensional point cloud, extracting data points belonging to the near side of the forage truck to be identified, and determining the target near side of the forage truck to be identified; wherein the near side is the side of the forage truck close to the forage truck; segmenting the target near side of the three-dimensional point cloud of the forage truck to be identified along the vertical direction of the world coordinate system to obtain multiple segments of the target near side; determining the edge line of the target near side based on the number of data points in each segment of the target near side; wherein the edge line is located The change in the number of data points between the segments and the segments above the edge line meets the preset change conditions; based on the three-dimensional coordinate values ​​of the data points belonging to the edge line of the near side of the target in the world coordinate system, the height of the bucket of the grass-catching truck to be identified is determined. In this way, processing is performed in three-dimensional space, which has one more dimension of distance information than a two-dimensional image, thereby improving the recognition accuracy. The edge of the grass-catching truck is identified based on the change law of the data points near the edge of the three-dimensional point cloud of the bucket of the grass-catching truck, and the recognition accuracy is high again; and the recognition process does not depend on the number of data sets, but only needs to use the CPU for calculation, which greatly reduces resource consumption and hardware costs, and also solves the problem of poor model generalization ability caused by limited data in the data set, thereby leading to the problem of recognition accuracy of the bucket of the grass-catching truck.

[0057] Please refer to Figure 1 , Figure 1A flow chart of a method for identifying the height of a silage truck bucket according to an embodiment of the present application is shown; the method comprises the following steps S101-S105:

[0058] S101, obtaining a target depth map of the forage truck to be identified collected by a binocular camera, and converting the target depth map into a three-dimensional point cloud; the binocular camera is installed on the forage truck; the three-dimensional point cloud includes a plurality of data points, each data point corresponding to a three-dimensional coordinate value in a world coordinate system;

[0059] S102, performing plane fitting processing on the three-dimensional point cloud, extracting data points belonging to the near side of the grass receiving vehicle to be identified, and determining the target near side of the grass receiving vehicle to be identified; wherein the near side is the side of the grass receiving vehicle close to the silage machine;

[0060] S103, segmenting the target near side surface of the three-dimensional point cloud of the to-be-identified grass-catching vehicle along the vertical direction of the world coordinate system to obtain a plurality of segments of the target near side surface;

[0061] S104, determining the edge of the target near side surface based on the number of data points in each segment of the target near side surface; wherein the change in the number of data points between the segment where the edge is located and the segment above the edge meets the preset change condition;

[0062] S105: Determine the height of the bucket of the grass receiving vehicle to be identified based on the three-dimensional coordinate values ​​of the data points belonging to the edge line of the near side surface of the target in the world coordinate system.

[0063] The method for recognizing the height of the bucket of a forage machine described in the embodiment of the present application is mainly divided into two parts: the near side recognition of the bucket of the forage machine and the upper edge extraction of the near side of the bucket. Please refer to Figure 2 , Figure 2 Another flow chart of the method for identifying the height of the silage truck bucket described in an embodiment of the present application is shown.

[0064] The steps S101-S102 are used to realize the near side recognition of the grass receiving truck bucket.

[0065] In the step S101, a target depth map of the forage truck to be identified collected by a binocular camera is obtained, and the target depth map is converted into a three-dimensional point cloud in a world coordinate system; the binocular camera is installed on the forage truck; and the three-dimensional point cloud includes a plurality of data points.

[0066] A depth map is an image in which the value of each pixel represents the depth of the pixel in the scene (i.e., how far away it is from the camera). The depth information of each point in the scene is obtained by calculating the parallax of the binocular camera to generate a depth map.

[0067] A 3D point cloud is a set of points in a 3D space, each point has an (x, y, z) coordinate representing its position in the 3D space.

[0068] Each data point in the three-dimensional point cloud corresponds to a three-dimensional coordinate value in the world coordinate system.

[0069] Convert the target depth map into a three-dimensional point cloud in the world coordinate system. For details, please refer to Figure 2 , based on the internal parameters of the binocular camera, the target depth map is converted into a three-dimensional point cloud in the world coordinate system.

[0070] The intrinsic parameters of the binocular camera include focal length (fx, fy) and optical center (cx, cy). These parameters define the imaging characteristics of the camera. Each pixel value in the depth map represents the distance from the camera to the corresponding point (depth value). This depth value needs to be converted according to the camera's intrinsic parameters to obtain the actual three-dimensional space coordinates. The camera's intrinsic parameters can be obtained by consulting the manual or other materials or manually calibrating the camera.

[0071] For each pixel in the depth map, use the following formula to calculate its coordinates in three-dimensional space:

[0072]

[0073] Z=d, (3);

[0074] Among them, u, v are the coordinates of the pixel point in the image coordinate system, d is the depth value, cx, cy are the coordinates of the camera optical center, fx, fy are the focal lengths of the camera; X, Y, Z are the coordinates of the pixel point in the world coordinate system.

[0075] The comparison chart before and after the conversion is as follows Figure 3 and Figure 4 As shown; Figure 3 Characterizes the two-dimensional depth map before conversion described in the embodiment of the present application; Figure 4 Characterizes the converted three-dimensional point cloud image described in the embodiment of the present application.

[0076] In step S102, the three-dimensional point cloud is subjected to plane fitting processing to extract data points belonging to the near side of the grass receiving vehicle to be identified, and the target near side of the grass receiving vehicle to be identified is determined; wherein the near side is the side of the grass receiving vehicle close to the silage machine.

[0077] Correspondingly, the side opposite to the entry side of the grass receiving vehicle is the far side.

[0078] Performing plane fitting processing on the three-dimensional point cloud, extracting data points belonging to the near side surface of the grass receiving vehicle to be identified, and determining the target near side surface of the grass receiving vehicle to be identified, including:

[0079] In some embodiments, a sample subset is randomly extracted from the three-dimensional point cloud, and a sample plane model corresponding to the sample subset is determined;

[0080] Calculating the deviation between the data points in the three-dimensional point cloud and the sample plane model, determining the inner sample points and the outer sample points in the data points, and recording the number of the inner sample points corresponding to the sample plane model;

[0081] Repeat the step of determining the sample plane model until a preset iteration stop condition is met, and obtain multiple sample plane models;

[0082] The sample plane model with the largest number of sample points is selected as the target near side surface of the grass picking vehicle to be identified.

[0083] Please refer to Figure 2 In some embodiments, after conversion into a three-dimensional point cloud, a plane fitting process is performed to select the near side surface of the grass receiving truck bucket.

[0084] After extracting the side of the grass receiving truck bucket, detect the upper edge of the bucket; please refer to Figure 2 ,The edge detection part on the truck bed is mainly composed of grid statistics.

[0085] In the step S103, the target near side surface of the three-dimensional point cloud of the to-be-identified grass-catching vehicle is segmented along the vertical direction of the world coordinate system to obtain a plurality of segments of the target near side surface.

[0086] The target near side surface of the three-dimensional point cloud of the to-be-identified grass-catching vehicle is segmented along the vertical direction of the world coordinate system to obtain a plurality of segments of the target near side surface for grid statistics.

[0087] Before segmenting the target near side surface of the three-dimensional point cloud of the to-be-identified grass-catching vehicle along the vertical direction of the world coordinate system, the method further comprises:

[0088] The height of the segment is determined based on the accuracy requirements.

[0089] For example, if the recognition accuracy requirement for the height of the grass-catching machine bucket is 1 cm, the near side surface of the target of the three-dimensional point cloud is segmented according to 1 cm. Here, 1 cm refers to the coordinate value in the world coordinate system.

[0090] The target near side surface of the three-dimensional point cloud is segmented along the vertical direction of the world coordinate system, that is, segmented along the height direction of the grass picker bucket, and each segment corresponds to a height.

[0091] After segmenting the point cloud, the number of point clouds in each segment is counted.

[0092] In step S104, the edge of the target near side surface is determined based on the number of data points in each segment of the target near side surface; wherein the change in the number of data points between the segment where the edge line is located and the segment above the edge line meets the preset change condition.

[0093] Here, the segment where the edge line is located and the segment above the edge line has the largest change in the number of data points.

[0094] Because the upper edge of the bucket of the grass receiving truck is the length of the bucket, although there are data points above the upper edge due to the hanging tarpaulin, grass hanging on the wall, shooting angle, etc., the number of data points will drop suddenly. Based on this rule, the changes in the number of data points in different segments of the near side of the target are analyzed to find the segment where the edge line with the maximum slope is located.

[0095] For details, please refer to Figure 5 , based on the number of data points in each segment of the near side surface of the target, determining the edge line of the near side surface of the target; comprising the following steps S501-S502:

[0096] S501, obtaining the number of data points in each segment of the near side surface of the target, and generating a target line graph; the target line graph represents the relationship between different segments of the near side surface of the target and the number of data points corresponding to each segment;

[0097] S502, calculating the slope of each segment of the target line graph, sorting the slopes and recording the segment positions where the slopes are located; determining the segment position where the maximum slope is located as the target segment, and determining the edge line of the target near side surface based on the target segment.

[0098] The step S502 is to find the target segment with the largest slope in the target line graph, and determine the edge line of the near side surface of the target based on the target segment.

[0099] In some embodiments, generating a target line graph based on the number of data points in each segment of the target proximal side includes:

[0100] Taking the vertical coordinate value of each segment of the near side surface of the target in the world coordinate system as the x value of the segment;

[0101] The number of data points in each segment of the near side of the target is taken as the y value of the segment;

[0102] Generate a target line chart based on the x and y values ​​of each segment.

[0103] That is to say, specifically, the number of points in each segment needs to be plotted on a two-dimensional coordinate axis, where x is the z value of the world coordinate system corresponding to the segment, and y is the number of data points in the segment. These points are connected in sequence with broken lines to form a line graph.

[0104] In the step S105, the height of the bucket of the grass receiving vehicle to be identified is determined based on the three-dimensional coordinate values ​​of the data points belonging to the edge line of the near side surface of the target in the world coordinate system.

[0105] Please refer to Figure 2 , returning the three-dimensional coordinate value of the edge line (target segment) belonging to the near side surface of the target in the world coordinate system as the height of the bucket of the grass receiving vehicle.

[0106] Because the z value of the edge line in the world coordinate system at this time is the x value of the position with the maximum slope in the target line graph, the x value corresponding to the edge line is returned as the height of the bucket of the grass receiving truck.

[0107] In some embodiments, in the method for identifying the height of a forage truck bucket, after determining the edge line of the target near side surface based on the number of data points in each segment of the target near side surface, the method further includes:

[0108] Based on the distance between the proximal side and the outer side of the bucket of the grass receiving vehicle, the edge line of the target proximal side is translated to determine the edge line of the target outer side of the bucket of the grass receiving vehicle;

[0109] The edge line of the near side surface of the target and the edge line of the outer side surface of the target are used to determine the position of the bucket of the grass receiving truck.

[0110] Since the distance between the near side surface and the outer side surface of the bucket of the grass receiving truck is fixed, the edge line of the target outer side surface of the bucket of the grass receiving truck is determined by translating the edge line of the target near side surface.

[0111] Please refer to Figure 6 , Figure 6 The truck bed height identification diagram described in the embodiment of the present application is shown, wherein the two black side lines are the side lines of the near side of the target and the side lines of the outer side of the target respectively.

[0112] The method for identifying the height of the forage truck bed described in the embodiment of the present application, compared with deep learning that relies too much on data and is prone to overfitting and high cost, and two-dimensional image processing that has low recognition accuracy due to occlusion, etc., converts the depth map generated by a binocular camera into a three-dimensional point cloud map, operates on the data in three dimensions, and adds one-dimensional distance information, which increases the processability of the data and improves the recognition accuracy; uses plane fitting to operate only on the point cloud belonging to the near side, greatly reducing the influence of other noise points on the recognition accuracy; uses grid statistics to identify the height of the bed, fully utilizing the characteristics of the bed edge to accurately detect the bed edge.

[0113] Based on the same inventive concept, an embodiment of the present application also provides a device for identifying the height of a silage truck bucket corresponding to the task dispatching method. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the method for identifying the height of a silage truck bucket above the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0114] Please refer to Figure 7 , Figure 7 The structure diagram of the device for identifying the height of the bucket of a silage truck according to an embodiment of the present application is shown; Figure 7 As shown, the silage machine grass truck bucket height recognition device comprises:

[0115] The acquisition module 701 is used to acquire a target depth map of the forage truck to be identified collected by a binocular camera, and convert the target depth map into a three-dimensional point cloud; the binocular camera is installed on the forage truck; the three-dimensional point cloud includes a plurality of data points, each of which corresponds to a three-dimensional coordinate value in a world coordinate system;

[0116] The extraction module 702 is used to perform plane fitting processing on the three-dimensional point cloud, extract data points belonging to the near side of the grass receiving vehicle to be identified, and determine the target near side of the grass receiving vehicle to be identified; wherein the near side is the side of the grass receiving vehicle close to the silage machine;

[0117] The segmentation module 703 is used to segment the near side surface of the target of the three-dimensional point cloud of the grass-catching vehicle to be identified along the vertical direction of the world coordinate system to obtain multiple segments of the near side surface of the target;

[0118] A first determination module 704 is used to determine the edge of the target near side surface based on the number of data points in each segment of the target near side surface; wherein the change in the number of data points between the segment where the edge is located and the segment above the edge meets a preset change condition;

[0119] The second determination module 705 is used to determine the height of the bucket of the grass receiving vehicle to be identified based on the three-dimensional coordinate values ​​of the data points belonging to the edge line of the near side surface of the target in the world coordinate system.

[0120] In some embodiments, in the forage truck bucket height recognition device, the first determination module, when determining the edge line of the target near side surface based on the number of data points in each segment of the target near side surface, is specifically used to:

[0121] Based on the number of data points in each segment of the near side surface of the target, a target line graph is generated; the target line graph represents the relationship between different segments of the near side surface of the target and the number of data points corresponding to each segment;

[0122] The slope of each segment of the target line graph is calculated, the slopes are sorted and the segment positions of the slopes are recorded; the segment position of the maximum slope is determined as the target segment, and the edge line of the target near side surface is determined based on the target segment.

[0123] In some embodiments, in the forage truck bucket height recognition device, the first determination module, when generating the target line graph based on the number of data points in each segment of the target near side surface, is specifically used to:

[0124] Taking the vertical coordinate value of each segment of the near side surface of the target in the world coordinate system as the x value of the segment;

[0125] The number of data points in each segment of the near side of the target is taken as the y value of the segment;

[0126] Generate a target line chart based on the x and y values ​​of each segment.

[0127] In some embodiments, the device for identifying the height of a silage truck bucket further includes:

[0128] The third determination module is used to determine the height of the segment based on the accuracy requirement before segmenting the target near side surface of the three-dimensional point cloud of the grass-catching vehicle to be identified along the vertical direction of the world coordinate system.

[0129] In some embodiments, in the device for identifying the height of a forage truck bucket, the extraction module performs plane fitting processing on the three-dimensional point cloud to extract data points belonging to the near side of the forage truck to be identified, and determines the target near side of the forage truck to be identified, specifically for:

[0130] Randomly extract a sample subset from the three-dimensional point cloud, and determine a sample plane model corresponding to the sample subset;

[0131] Calculating the deviation between the data points in the three-dimensional point cloud and the sample plane model, determining the inner sample points and the outer sample points in the data points, and recording the number of the inner sample points corresponding to the sample plane model;

[0132] Repeat the step of determining the sample plane model until a preset iteration stop condition is met, and obtain multiple sample plane models;

[0133] The sample plane model with the largest number of sample points is selected as the target near side surface of the grass picking vehicle to be identified.

[0134] In some embodiments, the silage machine grass receiving truck bucket height recognition device further includes:

[0135] A fourth determination module is used for translating the edge line of the target near side surface based on the distance between the near side surface and the outer side surface of the bucket of the grass receiving vehicle to determine the edge line of the target outer side surface of the bucket of the grass receiving vehicle after determining the edge line of the target near side surface based on the number of data points in each segment of the target near side surface;

[0136] The edge line of the near side surface of the target and the edge line of the outer side surface of the target are used to determine the position of the bucket of the grass receiving truck.

[0137] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0138] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0139] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0140] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a platform server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0141] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for identifying the height of a silage truck bucket, characterized in that: The method comprises: Obtaining a target depth map of the forage truck to be identified collected by a binocular camera, and converting the target depth map into a three-dimensional point cloud in a world coordinate system; the binocular camera is installed on the forage truck; the three-dimensional point cloud includes a plurality of data points; Performing plane fitting processing on the three-dimensional point cloud, extracting data points belonging to the near side of the grass receiving vehicle to be identified, and determining the target near side of the grass receiving vehicle to be identified; wherein the near side is the side of the grass receiving vehicle close to the silage machine; Segmenting the near side surface of the target of the three-dimensional point cloud of the grass-catching vehicle to be identified along the vertical direction of the world coordinate system to obtain a plurality of segments of the near side surface of the target; Determine the edge of the target near side surface based on the number of data points in each segment of the target near side surface; wherein the change in the number of data points between the segment where the edge is located and the segment above the edge meets the preset change condition; The height of the bucket of the grass receiving vehicle to be identified is determined based on the three-dimensional coordinate values ​​of the data points belonging to the edge line of the near side surface of the target in the world coordinate system.

2. The method for identifying the height of a silage truck bucket according to claim 1, characterized in that: Determining a boundary line of the near side surface of the target based on the number of data points in each segment of the near side surface of the target; comprising: Based on the number of data points in each segment of the near side surface of the target, a target line graph is generated; the target line graph represents the relationship between different segments of the near side surface of the target and the number of data points corresponding to each segment; A target segment with the largest slope in the target line graph is found, and a sideline of the near side of the target is determined based on the target segment.

3. The method for identifying the height of a silage truck bucket according to claim 2, characterized in that: Determining the position with the largest slope in the target line graph, and determining the edge line of the near side surface of the target based on the target segmentation, including: Calculate the slope of each segment, sort the slopes and record the segment position of the slope; The segment position where the maximum slope is located is determined as the target segment.

4. The method for identifying the height of a silage truck bucket according to claim 2, characterized in that: Based on the number of data points in each segment of the near side of the target, a target line graph is generated, including: Taking the vertical coordinate value of each segment of the near side surface of the target in the world coordinate system as the x value of the segment; The number of data points in each segment of the near side of the target is taken as the y value of the segment; Generate a target line chart based on the x and y values ​​of each segment.

5. The method for identifying the height of a silage truck bucket according to claim 1 or 2, characterized in that: Before segmenting the target near side surface of the three-dimensional point cloud of the to-be-identified grass-catching vehicle along the vertical direction of the world coordinate system, the method further includes: The height of the segment is determined based on the accuracy requirements.

6. The method for identifying the height of a silage truck bucket according to claim 1, characterized in that: Performing plane fitting processing on the three-dimensional point cloud, extracting data points belonging to the near side surface of the grass receiving vehicle to be identified, and determining the target near side surface of the grass receiving vehicle to be identified, including: Randomly extract a sample subset from the three-dimensional point cloud, and determine a sample plane model corresponding to the sample subset; Calculating the deviation between the data points in the three-dimensional point cloud and the sample plane model, determining the inner sample points and the outer sample points in the data points, and recording the number of the inner sample points corresponding to the sample plane model; Repeat the step of determining the sample plane model until a preset iteration stop condition is met, and obtain multiple sample plane models; The sample plane model with the largest number of sample points is selected as the target near side surface of the grass picking vehicle to be identified.

7. The method for identifying the height of a silage truck bucket according to claim 6, characterized in that: After determining the edge of the near side surface of the target based on the number of data points in each segment of the near side surface of the target, the method further includes: Based on the distance between the proximal side and the outer side of the bucket of the grass receiving vehicle, the edge line of the target proximal side is translated to determine the edge line of the target outer side of the bucket of the grass receiving vehicle; The edge line of the near side surface of the target and the edge line of the outer side surface of the target are used to determine the position of the bucket of the grass receiving truck.

8. A device for identifying the height of a silage truck bucket, characterized in that: The device comprises: An acquisition module is used to acquire a target depth map of the forage truck to be identified collected by a binocular camera, and convert the target depth map into a three-dimensional point cloud; the binocular camera is installed on the forage truck; the three-dimensional point cloud includes a plurality of data points, each of which corresponds to a three-dimensional coordinate value in a world coordinate system; An extraction module is used to perform plane fitting processing on the three-dimensional point cloud, extract data points belonging to the near side of the grass receiving vehicle to be identified, and determine the target near side of the grass receiving vehicle to be identified; wherein the near side is the side of the grass receiving vehicle close to the silage machine; A segmentation module, which segments the near side surface of the target of the three-dimensional point cloud of the grass-catching vehicle to be identified along the vertical direction of the world coordinate system to obtain multiple segments of the near side surface of the target; A first determination module is used to determine the edge of the target near side surface based on the number of data points in each segment of the target near side surface; wherein the change in the number of data points between the segment where the edge is located and the segment above the edge meets the preset change condition; The second determination module is used to determine the height of the bucket of the grass receiving vehicle to be identified based on the three-dimensional coordinate values ​​of the data points belonging to the edge line of the near side surface of the target in the world coordinate system.

9. The device for identifying the height of a silage truck bucket according to claim 8, characterized in that: The first determination module, when determining the edge line of the target near side surface based on the number of data points in each segment of the target near side surface, is specifically configured to: Based on the number of data points in each segment of the near side surface of the target, a target line graph is generated; the target line graph represents the relationship between different segments of the near side surface of the target and the number of data points corresponding to each segment; A target segment with the largest slope in the target line graph is found, and a sideline of the near side of the target is determined based on the target segment.

10. The device for identifying the height of a silage truck bucket according to claim 9, characterized in that: The first determination module, when determining the position with the largest slope in the target line graph and determining the edge line of the near side surface of the target based on the target segmentation, is specifically used to: Calculate the slope of each segment, sort the slopes and record the segment position of the slope; The segment position where the maximum slope is located is determined as the target segment.

Images