Flat grain height calculation method, control device, storage medium and flat grain system

Abstract

This invention discloses a method, control device, storage medium, and grain leveling system for calculating grain leveling height. The method involves acquiring unit images of multiple calibration areas; horizontally stitching these unit images to form multiple elongated images; obtaining multiple template depth matrices; and, based on these template depth matrices, obtaining a combined depth matrix for the corresponding combined images, ultimately acquiring the horizontal depth matrix of the multiple elongated images. The multiple horizontal depth matrices are then sequentially stitched vertically according to the overlapping boundary positions to obtain an overall depth matrix. The overall depth matrix is ​​used to obtain the spatial volume between the plane where the grain leveling device is located and the top surface of the grain in the grain silo. The grain leveling height is calculated based on the spatial volume and the plane area of ​​the grain silo. By combining image stitching with image synthesis to form an overall depth matrix, and then calculating the spatial volume using the overall depth matrix to further determine the grain leveling height, this method can directly obtain a relatively accurate grain leveling depth, improving grain leveling efficiency.

Description

Technical Field

[0001] This invention relates to the field of grain leveling technology, specifically to a method for calculating grain leveling height, a control device, a storage medium, and a grain leveling system. Background Technology

[0002] Grain storage is a crucial component of grain circulation, and grain storage technology has a significant impact on the quality and quantity of food. Currently, grain warehouse management in China faces extensive management practices, characterized by poor conditions, low-quality personnel, and a lack of scientific and technological content. Further improving grain reserve conditions, ensuring the construction of a green and scientific grain reserve system, and guaranteeing the safe storage and quality of food are of paramount importance. In recent years, the scale of grain storage warehouses has been increasing, and the grain layers inside have become thicker. However, the leveling work in my country's grain storage industry still largely relies on manual labor. This is not only slow and labor-intensive, but also largely repetitive. The warehouses are dusty, and when localized grain heating or abnormalities occur, deep excavation and leveling are required. If this also relies on manual labor, the work is unsafe, physically demanding, and extremely inefficient.

[0003] Currently, there are relatively few robots used for grain leveling both domestically and internationally. They can be broadly categorized into three types: fixed, mobile, and hybrid. Fixed robots typically operate on the top of grain silos. For example, truss-type grain leveling devices are highly efficient and produce high-quality results, but some corners are blind spots. Mobile robots are small and relatively flexible, but their efficiency is not high. Hybrid robots combine the advantages of both types, but mobile robots are prone to tipping over. Currently, regardless of the type of grain leveling device used, a problem urgently needs to be addressed: the target leveling height based on the overall condition of the grain pile in the silo has not been proposed or resolved. Taking truss-type grain leveling devices as an example, how much should the robot descend in the Z-axis direction to achieve overall flatness during grain surface leveling? Currently, this is done manually by visually inspecting the robot and remotely controlling it. Due to human subjectivity, the overall flatness cannot be considered, easily leading to small-angle tilts in the grain surface. This results in significant errors at both ends along the length of large grain silos. Summary of the Invention

[0004] The main objective of this invention is to propose a method for calculating grain leveling height, a control device, a storage medium, and a grain leveling system, aiming to solve the problem that it is difficult to determine the grain leveling height in existing large grain warehouses, which leads to the tilting of the grain surface after leveling.

[0005] To achieve the above objectives, this invention proposes a method for calculating grain leveling height, used to calculate the grain leveling height when the top surface of the grain in a grain silo is irregular, so as to guide the grain leveling device to level the top surface of the grain in the grain silo. The method for calculating grain leveling height includes the following steps:

[0006] Acquire unit images of multiple calibration areas, wherein the multiple calibration areas are multiple regions formed by dividing the top surface of the grain along the horizontal and vertical directions;

[0007] Multiple unit images are sequentially stitched together horizontally to form multiple strip-shaped images that extend horizontally and are distributed vertically. Each strip-shaped image is sequentially formed into multiple combined images during the formation process. Each combined image is formed by stitching together the unit image with adjacent unit images or adjacent combined images.

[0008] Multiple template depth matrices are obtained sequentially based on each of the combined images, and a combined depth matrix corresponding to the combined image is obtained based on each of the template depth matrices, finally obtaining the horizontal depth matrix of multiple strip-shaped images;

[0009] Obtain the overlapping boundary of two adjacent horizontal depth matrices, and then stitch together the multiple horizontal depth matrices along the vertical direction according to the position of the overlapping boundary to obtain the overall depth matrix.

[0010] The spatial volume between the plane where the grain leveling device is located and the top surface of the grain in the grain warehouse is obtained based on the overall depth matrix.

[0011] The grain leveling height is calculated based on the volume of the space and the floor area of ​​the grain warehouse.

[0012] Optionally, before the step of "sequentially stitching together multiple unit images along the horizontal direction to form multiple strip-shaped images that extend horizontally and are distributed vertically", the method further includes:

[0013] The multiple unit images are converted to grayscale to obtain grayscale images of the multiple unit images;

[0014] Binarize the multiple grayscale images to obtain the binarized images of the multiple unit images;

[0015] Obtain the distribution curve of the number of white pixels in each binarized image in the vertical or horizontal direction;

[0016] The distribution curves of two adjacent binarized images are compared to perform a first match on the two adjacent unit images, wherein the result of the first match includes similar regions of the two adjacent unit images;

[0017] The step of "separating multiple unit images horizontally to form multiple strip-shaped images that extend horizontally and are distributed vertically" includes:

[0018] Based on the result of the first matching, a second matching is performed on two adjacent unit images, wherein the result of the second matching includes feature points within similar regions of the two adjacent unit images;

[0019] Based on feature points obtained from similar regions, multiple unit images are sequentially stitched together horizontally to form the elongated image;

[0020] Repeat the above steps longitudinally to obtain multiple strip-shaped images.

[0021] Optionally, the step of "separating multiple unit images horizontally to form multiple horizontally extending and vertically distributed strip-shaped images" includes:

[0022] Multiple feature points in each pair of adjacent unit images are obtained by calculating using the SURF algorithm, and multiple initial transformation matrices are obtained based on the multiple feature points, wherein the multiple feature points are mutually matched pixels in the two adjacent unit images;

[0023] The RANSAC algorithm is used to calculate multiple initial transformation matrices, and the matrix with the largest number of corresponding points among the multiple initial transformation matrices is determined as the transformation matrix H. The calculation formula is as follows:

[0024]

[0025] Where x and y are the row and column coordinates of one of the pixels in two adjacent unit images, x' and y' are the row and column coordinates of the other pixel in two adjacent unit images, and h0~h7 are the coefficients of the transformation matrix H;

[0026] The two adjacent unit images are stitched together to form a combined image according to the image transformation matrix H;

[0027] Repeat the above steps to stitch the next horizontal unit image with the combined image, so as to stitch multiple unit images sequentially along the horizontal direction to form the long strip image.

[0028] Optionally, the unit image includes unit pixel information and a unit depth matrix, and the scaling factor k is determined by the number of pixels in the unit pixel information and the number of elements in the unit depth matrix;

[0029] The step of "establishing a template depth matrix and obtaining multiple horizontal depth matrices based on multiple strip-shaped images" includes:

[0030] Obtain the dimensions of each of the combined images;

[0031] The size of the template depth matrix corresponding to each of the combined images is calculated based on the scaling factor k, so as to establish the corresponding template depth matrix;

[0032] The overlapping portion of the two unit depth matrices is calculated based on the overlapping portion when two adjacent unit images are stitched together and the scaling factor k.

[0033] The average value of the element points in the overlapping portion of the two unit depth matrices is taken and then concatenated to form a combined depth matrix corresponding to the combined image;

[0034] Repeat the above steps to concatenate the next horizontal unit depth matrix with the combined depth matrix, thereby concatenating multiple unit depth matrices sequentially along the horizontal direction to form the horizontal depth matrix.

[0035] Optionally, the vertical direction is used as the column direction of the horizontal depth matrix, and the horizontal direction is used as the row direction of the horizontal depth matrix;

[0036] The step of “obtaining the overlapping boundary of two adjacent horizontal depth matrices in the vertical direction, and sequentially splicing multiple horizontal depth matrices along the vertical direction according to the position of the overlapping boundary to obtain the overall depth matrix” includes:

[0037] Obtain the boundary row of the current horizontal depth matrix located on one side of the adjacent horizontal depth matrix;

[0038] Subtract the boundary row of the current horizontal depth matrix from the adjacent horizontal depth matrix, and take the absolute value of the result;

[0039] Sum the result rows, find the row containing the minimum value in the row summation, and determine it as the overlapping boundary;

[0040] The two lateral depth matrices are joined together according to the overlapping boundary.

[0041] Repeat the above steps to sequentially stitch together the multiple horizontal depth matrices to form the overall depth matrix.

[0042] Optionally, the step of "obtaining the spatial volume between the plane where the grain leveling device is located and the top surface of the grain in the grain silo based on the overall depth matrix" includes:

[0043] Obtain the area of ​​each element in the overall depth matrix. The volume between the plane where the robot is located and the top surface of the grain in the grain silo is calculated using the infinitesimal element method. The calculation formula is as follows:

[0044] Where D represents the plane where the robot is located; V represents the volume between the plane where the robot is located and the top surface of the grain in the grain warehouse; x and y represent the row and column in the overall depth matrix, respectively.

[0045] Optionally, the grain silo has an entrance and exit at its edge. Before the step of "the robot moves along a plane above the grain silo and sequentially acquires unit images of the top surface of the grain in multiple grain silos," the method further includes:

[0046] Determine if a global update is needed;

[0047] If so, execute the step of "dividing the grain warehouse into multiple calibration areas along the longitudinal and transverse directions, and sequentially acquiring unit images of multiple calibration areas";

[0048] If not, obtain a row of multiple calibration areas adjacent to the inlet and outlet, and sequentially obtain the unit images of multiple calibration areas in each row towards the inside of the grain warehouse in a row-by-row manner. Compare each unit image in each row with the previously obtained unit image to determine whether there has been a change. If there has been a change, update the unit image of the corresponding calibration area until it is determined that there has been no change in multiple unit images in a row.

[0049] Based on the updated multiple unit images and the multiple unit images that have not changed, perform the step of "sequentially stitching the multiple unit images along the horizontal direction to form multiple strip-shaped images that extend horizontally and are distributed vertically".

[0050] The present invention also provides a grain leveling height control device, the grain leveling height control device including a memory, a processor, and a grain leveling height calculation program for a grain warehouse stored in the memory and executable on the processor, the grain leveling height calculation program for the grain warehouse being configured to implement the steps of the grain leveling height calculation method as described in any of the above claims.

[0051] The present invention also provides a storage medium storing a grain leveling height calculation program for a grain silo, wherein when the grain leveling height calculation program is executed by a processor, it implements the steps of the grain leveling height calculation method as described in any of the preceding claims.

[0052] The present invention also provides a grain leveling system for installation in a grain warehouse, the grain leveling system comprising:

[0053] Grain leveling device, which can be movably installed inside the grain warehouse, is used for grain leveling operations;

[0054] A depth camera is installed at the lower end of the grain leveling device. The depth camera is used to photograph the grain surface of the grain warehouse to obtain unit image information.

[0055] A driving device, located inside the grain silo, drives and connects to the grain leveling device, enabling the grain leveling device to be movably positioned along the longitudinal, lateral, and vertical directions; and...

[0056] A grain leveling height control device is installed on the grain leveling device and is electrically connected to the grain leveling device, the depth camera and the drive device. The grain leveling height control device is the grain leveling height control device as described above.

[0057] In the technical solution of this invention, multiple unit images of calibration areas are acquired, wherein the multiple calibration areas are multiple regions formed by dividing the top surface of the grain along the horizontal and vertical directions; the multiple unit images are sequentially spliced ​​along the horizontal direction to form multiple elongated images extending horizontally and distributed vertically, wherein each elongated image sequentially forms multiple combined images during the formation process, and each combined image is formed by splicing the unit image with adjacent unit images or adjacent combined images; multiple template depth matrices are sequentially obtained according to each combined image, and a combined depth matrix corresponding to the combined image is obtained according to each template depth matrix, finally obtaining the horizontal depth matrix of the multiple elongated images; the overlapping boundary of two adjacent horizontal depth matrices is obtained, and the multiple horizontal depth matrices are sequentially spliced ​​along the vertical direction according to the position of the overlapping boundary to obtain an overall depth matrix; the spatial volume between the plane where the grain leveling device is located and the top surface of the grain in the grain warehouse is obtained according to the overall depth matrix; the grain leveling height is calculated according to the spatial volume and the plane area of ​​the grain warehouse. Multiple unit images are stitched together, and the depth matrices of multiple units are stitched together according to the stitching position of the unit images to finally establish an overall depth matrix. The spatial volume between the plane where the grain leveling device is located and the top surface of the grain is calculated through the overall depth matrix. The distance between the grain leveling device and the target grain leveling height is obtained by dividing the spatial volume by the area, which is also the target grain leveling height. This solves the technical problem that the grain leveling height is difficult to determine in the past. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0059] Figure 1 This is a schematic diagram of the structure of an embodiment of the grain leveling system provided by the present invention;

[0060] Figure 2 A schematic diagram of the structure of the grain leveling height control device in the hardware operating environment involved in the embodiments of the present invention;

[0061] Figure 3 A schematic diagram of the structure of the first embodiment of the grain leveling height calculation method provided by the present invention;

[0062] Figure 4 A schematic diagram of the structure of a second embodiment of the grain leveling height calculation method provided by the present invention;

[0063] Figure 5 for Figure 4 A simplified schematic diagram of the two binarized images to be stitched together in the embodiment;

[0064] Figure 6 for Figure 5 The distribution curves of white pixels in the two binarized images;

[0065] Figure 7 This is a schematic diagram of the third embodiment of the grain leveling height calculation method provided by the present invention.

[0066]

[0067] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0068] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0069] It should be noted that if the embodiments of the present invention involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of the components in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0070] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the meaning of "and / or" throughout the text includes three parallel solutions; for example, "A and / or B" includes solution A, solution B, or a solution where both A and B are satisfied simultaneously. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0071] Currently, there are relatively few robots used for grain leveling both domestically and internationally. They can be broadly categorized into three types: fixed, mobile, and hybrid. Fixed robots typically operate on the top of grain silos. For example, truss-type grain leveling devices are highly efficient and produce high-quality results, but some corners are blind spots. Mobile robots are small and relatively flexible, but their efficiency is not high. Hybrid robots combine the advantages of both types, but mobile robots are prone to tipping over. Currently, regardless of the type of grain leveling device used, a problem urgently needs to be addressed: the target leveling height based on the overall condition of the grain pile in the silo has not been proposed or resolved. Taking truss-type grain leveling devices as an example, how much should the robot descend in the Z-axis direction to achieve overall flatness during grain surface leveling? Currently, this is done manually by visually inspecting the robot and remotely controlling it. Due to human subjectivity, the overall flatness cannot be considered, easily leading to small-angle tilts in the grain surface. This results in significant errors at both ends along the length of large grain silos.

[0072] Please see Figure 1 This invention proposes a grain leveling system for installation within a grain silo. The grain leveling system includes a grain leveling device 1, a depth camera 2, a drive device 3, and a grain leveling height control device. The grain leveling device 1 is movably installed within the grain silo for grain leveling operations. The depth camera 2 is located at the lower end of the grain leveling device 1 and is used to capture images of the grain surface in the grain silo to obtain unit image information. The drive device 3 is located within the grain silo and is connected to the grain leveling device 1, enabling the grain leveling device 1 to be movably installed in the longitudinal, lateral, and vertical directions. The grain leveling height control device is installed on the grain leveling device 1 and is electrically connected to the grain leveling device 1, the depth camera 2, and the drive device 3. The present invention acquires grain surface of grain warehouse by taking pictures of the grain warehouse by the depth camera 2 to obtain multiple unit image information, and transmits the multiple unit image information to the grain leveling height control device. The grain leveling height control device calculates the grain leveling height data to guide the drive device 3 to drive the grain leveling device 1 to move to the plane corresponding to the grain leveling height, and moves along the plane at the corresponding grain leveling height to level the grain.

[0073] In one embodiment of the present invention, the grain leveling device 1 is a truss-type grain leveling robot 11. The truss-type grain leveling robot 11 includes two longitudinal beams 13, a crossbeam 12, and the grain leveling robot 11. The two longitudinal beams 13 extend laterally and are spaced apart longitudinally on both sides of the grain silo. The crossbeam 12 extends longitudinally, and its two ends are movably mounted on the two longitudinal beams 13 respectively, and are slidably mounted along the extension direction of the two longitudinal beams 13. The grain leveling robot 11 is movably mounted on the crossbeam 12 longitudinally. The driving device 3 includes a telescopic rod 33, two lateral driving mechanisms 32, and a longitudinal driving mechanism 31. Multiple telescopic rods 33 are provided, extending vertically and respectively disposed at the lower ends of the two longitudinal beams 13, to drive the two longitudinal beams 13 to move movably in the vertical direction, thereby enabling the grain leveling robot 11 to move vertically. The two lateral driving mechanisms 32 are respectively mounted on the longitudinal beams 13 to drive the transverse beams 12 to move laterally. The longitudinal driving mechanism 31 is mounted on the transverse beams 12 to drive the grain leveling robot 11 to move longitudinally, thus enabling the grain leveling robot 11 to be movable in the longitudinal, lateral, and vertical directions.

[0074] In another embodiment of the present invention, the driving device 3 includes a telescopic rod, two lateral driving mechanisms 32, and a longitudinal driving mechanism 31. The telescopic rod is provided and correspondingly positioned between the grain leveling robot 11 and the crossbeam 12. The two lateral driving mechanisms 32 are respectively mounted on the longitudinal beam 13. The longitudinal driving mechanism 31 is drivenly connected to the upper end of the telescopic rod to drive the telescopic rod to move longitudinally. The lower end of the telescopic rod is fixedly connected to the grain leveling robot 11 to drive the grain leveling robot 11 to move longitudinally and vertically, thereby enabling the grain leveling robot 11 to be movable in the longitudinal, lateral, and vertical directions. It is understood that multiple support columns are respectively provided on the lower side of the two longitudinal beams 13 to maintain the longitudinal beams 13 at a certain height.

[0075] It should be noted that the specific forms of the lateral drive mechanism 32 and the longitudinal drive mechanism 31 are not limited here. In this embodiment, the lateral drive mechanism 32 and the longitudinal drive mechanism 31 are servo motors. In other embodiments, the lateral drive mechanism 32 and the longitudinal drive mechanism 31 can also be hydraulic components. Their specific connection methods are well known to those skilled in the art and will not be described here.

[0076] It should be further noted that the telescopic rod 33 is specifically a hydraulic telescopic rod 33, which has a large load-bearing capacity, sufficient power, and stable movement. In other embodiments, the telescopic rod 33 can also be a gear and rack mechanism or a threaded screw mechanism combined with a high-power motor to realize the telescopic movement of the telescopic rod 33. The specific configuration is also well known to those skilled in the art and will not be described here.

[0077] Furthermore, the depth camera 2 is fixedly installed at the lower end of the grain leveling robot 11, which is simple and convenient to modify and makes it easy to obtain the height of the plane where the grain leveling robot 11 is located.

[0078] In this embodiment of the invention, the grain leveling robot 11 is a conventional device available on the market, and its specific form is not limited here. In this embodiment, the grain leveling robot 11 specifically includes multiple grain leveling rods extending laterally. The multiple grain leveling rods are evenly distributed around the circumference of the grain leveling robot 11. The grain leveling robot 11 also includes a rotating component for driving the multiple grain leveling rods to rotate along the vertical axis. When the driving device 3 moves the multiple grain leveling rods to the corresponding grain leveling height, the grain is leveled by moving the grain with the multiple grain leveling rods.

[0079] Reference Figure 2 , Figure 2 This is a schematic diagram of the structure of the grain leveling height control device in the hardware operating environment involved in the embodiment of the present invention.

[0080] like Figure 2 As shown, the grain leveling height control device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0081] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the grain leveling height control device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0082] like Figure 2 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a grain leveling height calculation program.

[0083] exist Figure 2 In the grain leveling height control device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the grain leveling height control device of the present invention can be set in the grain leveling height control device. The grain leveling height control device calls the grain leveling height calculation program stored in the memory 1005 through the processor 1001 and executes the grain leveling height calculation method provided in the embodiment of the present invention.

[0084] This invention provides a method for calculating grain leveling height, used to calculate the leveling height when the top surface of the grain in a grain silo is irregular, so as to guide the grain leveling device in leveling the top surface of the grain in the grain silo. Figure 3 , Figure 3 This is a flowchart illustrating the first embodiment of the grain leveling height calculation method of the present invention.

[0085] In this embodiment, the method for calculating the grain leveling height includes the following steps:

[0086] S20. Obtain unit images of multiple calibration areas, wherein the multiple calibration areas are multiple regions formed by dividing the top surface of the grain along the horizontal and vertical directions;

[0087] In this step, since this calculation method is applicable to large-area grain silos, it is difficult to obtain a complete image in a single shot, and it would reduce accuracy. Therefore, the entire top surface of the grain is divided into multiple calibration areas along the longitudinal and transverse directions, and a unit image of each calibration area is obtained.

[0088] Specifically, the leveling device is driven to move by the driving device, thereby causing the depth camera to move along the longitudinal and lateral directions in the plane, and the depth camera acquires the unit images of multiple calibration areas.

[0089] It is understandable that, in order to ensure the integrity of image stitching, the unit images acquired by the depth camera will partially overlap with the adjacent unit images.

[0090] S30. Multiple unit images are sequentially spliced ​​together in the horizontal direction to form multiple strip-shaped images that extend in the horizontal direction and are distributed in the vertical direction. Each strip-shaped image is sequentially formed into multiple combined images during the formation process. Each combined image is formed by splicing the unit image with adjacent unit images or adjacent combined images.

[0091] In this step, the multiple unit images are further processed. The multiple unit images are arranged in an array with multiple columns and rows along the vertical and horizontal directions. The multiple unit images located in the same row are sequentially spliced ​​together along the horizontal direction to form multiple strip-shaped images.

[0092] Specifically, in the process of forming each of the elongated images, the first unit image of each row and the second unit image adjacent to it are first spliced ​​together to form the first combined image, the first combined image and the third unit image adjacent to it are spliced ​​together to form the second combined image, and so on until the last unit image of each row is spliced ​​together to form the last combined image, which is also the elongated image.

[0093] It should be noted that the grain silo is generally rectangular, but it can also be square or rectangular. In this embodiment, if the grain silo is square, there are no specific restrictions on the vertical and horizontal directions. If the grain silo is rectangular, because the image stitching speed is faster, the length direction of the grain silo is used as the horizontal direction and the width direction of the grain silo is used as the vertical direction to increase the length of the long strip image, thereby improving the stitching speed and reducing the amount of computation.

[0094] Further, step S30 includes the following steps:

[0095] S31. Calculate multiple feature points in every two adjacent unit images using the SURF (Speeded Up Robust Features) algorithm, and obtain multiple initial transformation matrices based on the multiple feature points, wherein the multiple feature points are mutually matched pixel points in two adjacent unit images.

[0096] In this step, the SURF algorithm can be used to extract and match feature points in the overlapping parts of two adjacent unit images, and the initial transformation matrix is ​​obtained based on the extracted feature points. Since there are many feature points, multiple initial transformation matrices are generally obtained.

[0097] S32. The RANSAC (Random Sample Consensus) algorithm is used to calculate the transformation matrix H among the multiple initial transformation matrices, and the matrix with the largest number of corresponding points is selected. The calculation formula is as follows:

[0098]

[0099] Where x and y are the row and column coordinates of one of the pixels in two adjacent unit images, x' and y' are the row and column coordinates of the other pixel in two adjacent unit images, and h0~h7 are the coefficients of the transformation matrix H;

[0100] Since multiple initial transformation matrices were obtained in step S31, it is necessary to filter these initial transformation matrices. In this step, the RANSAC algorithm is used to filter the multiple initial transformation matrices. Specifically, by verifying each initial transformation matrix, the number of corresponding pixels in two adjacent unit images is calculated when each initial transformation matrix is ​​applied, and the matrix with the largest number of corresponding pixels is selected as the transformation matrix H.

[0101] It should be noted that the RANSAC algorithm is also a commonly used technique in the field of machine vision. Its specific calculation method can be found in existing technologies and will not be described in detail here.

[0102] S33. Combine two adjacent unit images to form a combined image according to the image transformation matrix H;

[0103] In this step, the SURF algorithm is used to stitch two adjacent unit images together to form a combined image based on the obtained transformation matrix H.

[0104] It should be noted that the SURF algorithm is a commonly used technique in the field of image stitching. Its specific calculation method can be found in existing technologies and will not be described in detail here.

[0105] S34. Repeat the above steps to stitch the next horizontal unit image with the combined image to stitch multiple unit images sequentially along the horizontal direction to form the long strip image.

[0106] S40. Obtain multiple template depth matrices sequentially based on each of the combined images, obtain a combined depth matrix corresponding to the combined image based on each of the template depth matrices, and finally obtain the horizontal depth matrix of multiple strip-shaped images.

[0107] In this step, as multiple unit images are sequentially stitched together to form various combined images, multiple template depth matrices are established for each combined image. The template depth matrix only contains the size of the matrix and does not contain depth data. The depth information in each unit image acquired by the depth camera is filled into the template depth matrix to form the combined depth matrix. When the last combined image is stitched together, the last combined depth matrix is ​​formed, which is also the horizontal depth matrix.

[0108] Furthermore, the unit image includes unit pixel information and a unit depth matrix, and the scaling factor k is determined by the number of pixels in the unit pixel information and the number of elements in the unit depth matrix;

[0109] Step S40 includes the following steps:

[0110] S41. Obtain the dimensions of each of the combined images;

[0111] It should be noted that the number of pixels captured by the depth camera and the number of elements in the depth matrix are known values, but the size of the overlapping part of each unit image and the adjacent unit image is unknown. The size of the overlapping part can be known through the combined images obtained by the above steps. The size of each combined image can be obtained by subtracting the overlapping part. The size of the combined image includes the image size and the number of pixels.

[0112] S42. Calculate the size of the template depth matrix corresponding to each of the combined images according to the scaling factor k, so as to establish the corresponding template depth matrix;

[0113] It should be noted that the dimensions of the template depth matrix include both the dimensions of the template matrix and the number of elements.

[0114] S43. Calculate the overlapping portion of the two unit depth matrices based on the overlapping portion when two adjacent unit images are stitched together and the scaling factor k;

[0115] S44. Take the average value of the element points of the overlapping part of the two unit depth matrices and stitch them together to form a combined depth matrix corresponding to the combined image;

[0116] In this step, to improve the accuracy of the stitched combined depth matrix, the elements of the overlapping portions of the two unit depth matrices are added together and averaged to form the stitched depth matrix. The specific calculation process is performed using MATLAB software, and the formula in MATLAB is as follows:

[0117]

[0118] Where depth represents the combined depth matrix; and This represents the depth matrices of the two cells to be spliced; It is the column where the splicing boundary of the two unit images in the combined image is located; A represents the total number of columns in the corresponding matrix; i is the column number, specifically, i ranges from 1 to... The column number it belongs to.

[0119] Specifically, This expresses the combined matrix. All rows in the (Ai)th column; This expresses the unit matrix. All rows in the (Ai)th column; This expresses the unit matrix. All rows in the () )List.

[0120] S45. Repeat the above steps to concatenate the next horizontal unit depth matrix with the combined depth matrix, so as to concatenate multiple unit depth matrices sequentially along the horizontal direction to form the horizontal depth matrix.

[0121] S50. Obtain the overlapping boundary of two adjacent horizontal depth matrices, and stitch the multiple horizontal depth matrices together along the longitudinal direction according to the position of the overlapping boundary to obtain the overall depth matrix.

[0122] In this step, when splicing two adjacent horizontal depth matrices, the length of the overlapping boundary is relatively long, and using the above method alone would result in excessive computation. Therefore, by obtaining the overlapping boundary of the two horizontal depth matrices, the matrices are spliced ​​directly according to the position of the overlapping boundary, thereby obtaining the overall depth matrix.

[0123] Further, step S50 includes the following steps:

[0124] S51. Obtain the boundary row of the current horizontal depth matrix located on one side of the adjacent horizontal depth matrix;

[0125] S52. Subtract the boundary row of the current horizontal depth matrix from the adjacent horizontal depth matrix, and take the absolute value of the result;

[0126] S53. Sum the result rows, obtain the row containing the minimum value in the row summation, and determine it as the overlapping boundary;

[0127] S54. Join the two horizontal depth matrices according to the overlapping boundary;

[0128] S55. Repeat the above steps to sequentially stitch together the multiple horizontal depth matrices to form the overall depth matrix.

[0129] It should be noted that during stitching, the first horizontal depth matrix is ​​taken from the vertical end, and the adjacent horizontal depth matrix is ​​taken as the second horizontal depth matrix. A difference matrix is ​​obtained by subtracting the boundary rows of the first horizontal depth matrix from the second horizontal depth matrix. The absolute value of this difference matrix is ​​then taken, and the rows with the minimum sum are identified. The row containing the minimum sum indicates that the row in the second horizontal depth matrix has the smallest difference from the boundary rows of the first horizontal depth matrix. This is thus determined as the overlapping boundary between the second and first horizontal depth matrices, and stitching can be performed using this overlapping boundary as the standard. This process is repeated vertically to stitch together the subsequent horizontal depth matrices to obtain the overall depth matrix. This method significantly reduces the computational load of image stitching.

[0130] Specifically, the above steps are performed using MATLAB software, and the formula in MATLAB is as follows:

[0131]

[0132]

[0133]

[0134] in, It is the difference matrix that stores the difference between the two horizontal depth matrices; and The two horizontal depth matrices to be spliced ​​are represented by B, where B is the total number of rows in the horizontal depth matrices and i is the row number, specifically i ranging from 1 to B.

[0135] The first column of the formula represents subtracting the i-th row of the second horizontal depth matrix from the boundary column of the first horizontal depth matrix to obtain... The i-th row of the matrix, where i ranges from 1 to B.

[0136] The formula in the second column indicates that the entire... The absolute values ​​of all elements in the matrix are taken first, and then the sum of each row is obtained to obtain matrix c.

[0137] The formula in the third column indicates that the minimum value in matrix c is found and its position is assigned to the variable bianyuan, which is the overlapping boundary of the second horizontal depth matrix.

[0138] S60. Obtain the spatial volume between the plane where the grain leveling device is located and the top surface of the grain in the grain warehouse according to the overall depth matrix.

[0139] Further, step S60 includes the following steps:

[0140] S61. Obtain the area of ​​each element in the overall depth matrix. The volume between the plane where the robot is located and the top surface of the grain in the grain silo is calculated using the infinitesimal element method. The calculation formula is as follows:

[0141] Where D represents the plane where the robot is located; V represents the volume between the plane where the grain leveling robot is located and the top surface of the grain in the grain warehouse; x and y represent the row and column in the overall depth matrix, respectively.

[0142] In this embodiment, the infinitesimal method is used to calculate the double integral. The plane D where the depth camera is located is divided into n small rectangles, and the element value represents the depth f(x, y) of the small rectangle. By calculating the integral between the grain surface and the plane where the depth camera is located, the total volume V of the space between the grain leveling robot and the grain surface is obtained.

[0143] In another embodiment of the invention, to improve calculation accuracy, interpolation processing can be performed on the surface drawn from the overall depth matrix, and the interpolated value can be returned. The surface always passes through the data points before interpolation, forming a new depth matrix after interpolation, and creating a visualized 3D surface graphic. Through interpolation processing, the rows and columns of the overall depth matrix are expanded, that is, the area of ​​the rectangular region corresponding to each depth data is reduced. This makes the total volume V closer to the actual volume, and then the integration in the above steps is performed to calculate the volume.

[0144] S70. Calculate the grain leveling height based on the volume of the space and the planar area of ​​the grain warehouse.

[0145] In this step, the area of ​​the plane where the depth camera is located is taken as the base area, which is the floor area of ​​the grain silo. Dividing the base area by the base area yields the average height between the grain surface and the plane where the depth camera is located, i.e., the grain leveling height. This distance is the target distance for leveling the grain surface, which is the working target of the grain leveling robot. The specific calculation formula is as follows:

[0146]

[0147] in, V represents the height of the grain level; V represents the total volume; S represents the height of the grain level. D The area of ​​the bottom of the plane where the depth camera is located.

[0148] Please see Figure 4 Based on the first embodiment described above, a second embodiment of the grain leveling height calculation method of the present invention is proposed.

[0149] Before step S30, which involves "sequentially stitching together multiple unit images horizontally to form multiple elongated images that extend horizontally and are distributed vertically," the method further includes:

[0150] S201. Perform grayscale processing on the multiple unit images to obtain grayscale images of the multiple unit images;

[0151] S202. Binarize the multiple grayscale images to obtain the binarized images of the multiple unit images;

[0152] S203. Obtain the distribution curve of the number of white pixels in each of the binarized images in the vertical or horizontal direction;

[0153] S204. Compare the distribution curves of two adjacent binarized images to perform a first match on the two adjacent unit images, wherein the result of the first match includes similar regions of the two adjacent unit images.

[0154] Please see Figure 5 and Figure 6 In this embodiment, multiple unit images are grayscaled and binarized to obtain binarized images of the unit images. Each binarized image includes only black and white pixels. By accumulating the white pixels in each binarized image vertically or horizontally, the distribution curves of the number of white pixels in each binarized image in the vertical direction and horizontal direction can be obtained. When stitching horizontally, the distribution curves of two adjacent binarized images in the horizontal direction can be compared to achieve a first match. By comparing the distribution curves, similar regions of the two unit images in the horizontal direction can be obtained. Similarly, when stitching vertically, the distribution curves of two adjacent binarized images in the vertical direction can also be compared to obtain similar regions of the two unit images in the vertical direction.

[0155] Step S30, which involves "sequentially stitching together multiple unit images along the horizontal direction to form multiple elongated images that extend horizontally and are distributed vertically," includes:

[0156] S35. Based on the result of the first matching, perform a second matching on two adjacent unit images, wherein the result of the second matching includes feature points within similar regions of the two adjacent unit images;

[0157] S36. Based on the feature points obtained in similar regions, multiple unit images are sequentially stitched together horizontally to form the elongated image;

[0158] S37. Repeat the above steps in the longitudinal direction to obtain a plurality of the elongated images.

[0159] After obtaining similar regions through the first matching, a rough matching is performed. In the subsequent stitching process, identification and calculation can be directly performed within the similar regions of the two stitched unit images to conduct a second matching. The specific calculation process after the second matching can refer to the first embodiment described above. This embodiment greatly reduces the amount of data calculation during stitching by using the first matching, thereby improving stitching efficiency.

[0160] Please see Figure 7 Based on the first or second embodiment described above, a third embodiment of the grain leveling height calculation method of the present invention is proposed.

[0161] In this embodiment, the edge of the grain silo is provided with an inlet and outlet. Before step S20, which involves "the robot moving along a plane above the grain silo and sequentially acquiring unit images of the top surface of the grain in multiple grain silos," the method further includes:

[0162] S10. Determine if a global update is needed;

[0163] It should be noted that grain silos typically have grain inlets and outlets for storage and retrieval. In large grain silos, after grain is retrieved or replenished, only the inlet / outlet positions may change, while other areas remain unchanged. If every grain level change were to trigger a full check, it would result in a large amount of invalid calculations. Therefore, it's advisable to determine whether a global update is necessary before performing any checks.

[0164] S11. If so, execute step S20, which involves "dividing the grain warehouse into multiple calibration areas along the longitudinal and transverse directions and sequentially acquiring unit images of multiple calibration areas".

[0165] If a global update is required, it indicates that the overall grain level in the granary has changed significantly and a complete re-statistic is needed, which would require data from all regions.

[0166] S12. If not, obtain a row of multiple calibration areas adjacent to the inlet and outlet, and sequentially obtain the unit images of the multiple calibration areas in each row towards the inside of the grain warehouse in a row-by-row manner. Compare each unit image in each row with the previously obtained unit image to determine whether there has been a change. If there has been a change, update the unit image of the corresponding calibration area until it is determined that there has been no change in the multiple unit images in a row.

[0167] If a global update is not required, the areas that have changed are generally located near the inlet and outlet. Therefore, multiple calibration areas in a row adjacent to the inlet and outlet are acquired, and multiple unit images in each row are acquired sequentially. When multiple images in a row are found to be completely unchanged, it can be determined that no changes have occurred in any subsequent rows, and the detection stops. By using a local detection method for changed areas, areas that have not changed do not need to be detected, thus improving data acquisition efficiency. The unit images of the changed areas are acquired to update the data of the unit images stored during the previous grain leveling. For areas that have not changed, the data of the unit images stored during the previous grain leveling is directly retrieved.

[0168] It is understood that the row mentioned in this embodiment can be either vertical or horizontal.

[0169] S13. Based on the updated multiple unit images and the multiple unit images that have not changed, perform step S30 of “sequentially stitching the multiple unit images along the horizontal direction to form multiple strip-shaped images that extend horizontally and are distributed vertically”.

[0170] In one embodiment of the present invention, to improve detection efficiency, it is not necessary to detect every single calibration area in a row. For ease of description, taking the inlet / outlet located at one end of the grain silo's transverse direction as an example, the row adjacent to the inlet / outlet is the first column, and among the multiple calibration areas in the first column, the end closest to the inlet / outlet is the first row.

[0171] During detection, starting from the first row of the first column, detection proceeds sequentially along the column direction. It is determined whether each calibration region is consistent with previously stored data, and two detection results are output: consistent or inconsistent. If inconsistent, the previously stored data needs to be updated. It is also determined whether the detection result of each calibration region differs from the previous calibration region. If a change occurs, it is recorded as a mutation, indicating that the calibration region containing the boundary of the changed region has been detected. It can be understood that when the first mutation occurs, it means that the calibration region where the mutation occurred is the first boundary of the first column of the changed region; when the second mutation occurs, it means that the previous calibration region of the mutated calibration region is the second boundary of the first column of the changed region.

[0172] When a second mutation is detected, the first column can be considered complete, and the second column can be tested, thus reducing the number of tests. The testing method for the second column is the same as for the first column: testing row by row until the second mutation occurs, and testing each column horizontally. When testing each column, the row where the second mutation occurs needs to be counted, and the row with the most occurrences is determined as the farthest boundary of the changed region in the vertical direction. As each column is tested, if no mutation occurs in the row containing the farthest boundary of a column, it indicates that this column represents the farthest boundary of the changed region in the horizontal direction, and testing can be stopped at this point.

[0173] In this embodiment, only a certain area needs to be detected during the detection process, which greatly reduces the amount of detection and computation compared to performing global detection every time, and improves the update efficiency.

[0174] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention's specification and drawings under the inventive concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A method for calculating grain leveling height, used to calculate the grain leveling height when the top surface of the grain in a grain silo is irregular, so as to guide the grain leveling device to level the top surface of the grain in the grain silo, characterized in that, The method for calculating the height of the grain leveling area includes the following steps: Acquire unit images of multiple calibration areas, wherein the multiple calibration areas are multiple regions formed by dividing the top surface of the grain along the horizontal and vertical directions; Multiple unit images are sequentially stitched together horizontally to form multiple strip-shaped images that extend horizontally and are distributed vertically. Each strip-shaped image is sequentially formed into multiple combined images during the formation process. Each combined image is formed by stitching together the unit image with adjacent unit images or adjacent combined images. Multiple template depth matrices are obtained sequentially based on each of the combined images, and a combined depth matrix corresponding to the combined image is obtained based on each of the template depth matrices, finally obtaining the horizontal depth matrix of multiple strip-shaped images; Obtain the overlapping boundary of two adjacent horizontal depth matrices, and then stitch together the multiple horizontal depth matrices along the vertical direction according to the position of the overlapping boundary to obtain the overall depth matrix. The spatial volume between the plane where the grain leveling device is located and the top surface of the grain in the grain warehouse is obtained based on the overall depth matrix. The grain leveling height is calculated based on the space volume and the floor area of ​​the grain silo. The step of "separating multiple unit images horizontally to form multiple horizontally extending and vertically distributed strip-shaped images" includes: Multiple feature points in each pair of adjacent unit images are obtained by calculating using the SURF algorithm, and multiple initial transformation matrices are obtained based on the multiple feature points, wherein the multiple feature points are mutually matched pixels in the two adjacent unit images; The RANSAC algorithm is used to calculate multiple initial transformation matrices, and the matrix with the largest number of corresponding points among the multiple initial transformation matrices is determined as the transformation matrix H. The calculation formula is as follows: Where x and y are the row and column coordinates of one of the pixels in two adjacent unit images, x' and y' are the row and column coordinates of the other pixel in two adjacent unit images, and h0~h7 are the coefficients of the transformation matrix H; The two adjacent unit images are stitched together to form a combined image according to the image transformation matrix H; Repeat the above steps to stitch the next horizontal unit image with the combined image, so as to stitch multiple unit images sequentially along the horizontal direction to form the long strip image.

2. The method for calculating the height of leveled grain as described in claim 1, characterized in that, Before the step of "separating multiple unit images horizontally to form multiple horizontally extending and vertically distributed strip-shaped images", the method further includes: The multiple unit images are converted to grayscale to obtain grayscale images of the multiple unit images; Binarize the multiple grayscale images to obtain the binarized images of the multiple unit images; Obtain the distribution curve of the number of white pixels in each binarized image in the vertical or horizontal direction; The distribution curves of two adjacent binarized images are compared to perform a first match on the two adjacent unit images, wherein the result of the first match includes similar regions of the two adjacent unit images; The step of "separating multiple unit images horizontally to form multiple strip-shaped images that extend horizontally and are distributed vertically" includes: Based on the result of the first matching, a second matching is performed on two adjacent unit images, wherein the result of the second matching includes feature points within similar regions of the two adjacent unit images; Based on feature points obtained from similar regions, multiple unit images are sequentially stitched together horizontally to form the elongated image; Repeat the above steps longitudinally to obtain multiple strip-shaped images.

3. The method for calculating the height of leveled grain as described in claim 1, characterized in that, The unit image includes unit pixel information and a unit depth matrix. The scaling factor k is determined by the number of pixels in the unit pixel information and the number of elements in the unit depth matrix. The "obtaining the horizontal depth matrix of multiple elongated images" includes: Obtain the dimensions of each of the combined images; The size of the template depth matrix corresponding to each of the combined images is calculated based on the scaling factor k, so as to establish the corresponding template depth matrix; The overlapping portion of the two unit depth matrices is calculated based on the overlapping portion when two adjacent unit images are stitched together and the scaling factor k. The average value of the element points in the overlapping portion of the two unit depth matrices is taken and then concatenated to form a combined depth matrix corresponding to the combined image; Repeat the above steps to concatenate the next horizontal unit depth matrix with the combined depth matrix, thereby concatenating multiple unit depth matrices sequentially along the horizontal direction to form the horizontal depth matrix.

4. The method for calculating the height of leveled grain as described in claim 1, characterized in that, The vertical direction is the column direction of the horizontal depth matrix, and the horizontal direction is the row direction of the horizontal depth matrix; The step of "obtaining the overlapping boundary of two adjacent horizontal depth matrices, and sequentially concatenating multiple horizontal depth matrices along the vertical direction according to the position of the overlapping boundary to obtain the overall depth matrix" includes: Obtain the boundary row of the current horizontal depth matrix located on one side of the adjacent horizontal depth matrix; Subtract the boundary row of the current horizontal depth matrix from the adjacent horizontal depth matrix, and take the absolute value of the result; Sum the result rows, find the row containing the minimum value in the row summation, and determine it as the overlapping boundary; The two lateral depth matrices are joined together according to the overlapping boundary. Repeat the above steps to sequentially stitch together the multiple horizontal depth matrices to form the overall depth matrix.

5. The method for calculating the height of leveled grain as described in claim 1, characterized in that, The step of "obtaining the spatial volume between the plane where the grain leveling device is located and the top surface of the grain in the grain warehouse based on the overall depth matrix" includes: Obtain the area of ​​each element in the overall depth matrix. The volume between the plane where the robot is located and the top surface of the grain in the grain silo is calculated using the infinitesimal element method. The calculation formula is as follows: Where D represents the plane where the robot is located; V represents the volume between the plane where the robot is located and the top surface of the grain in the grain warehouse; x and y represent the row and column in the overall depth matrix, respectively.

6. The method for calculating the height of leveled grain as described in claim 1, characterized in that, The grain silo has an entrance and exit at its edge. Before the step of "acquiring unit images of multiple calibration areas", the following steps are also included: Determine if a global update is needed; If so, proceed with the step of "acquiring unit images of multiple calibration areas"; If not, obtain a row of multiple calibration areas adjacent to the inlet and outlet, and sequentially obtain the unit images of multiple calibration areas in each row towards the inside of the grain warehouse in a row-by-row manner. Compare each unit image in each row with the previously obtained unit image to determine whether there has been a change. If there has been a change, update the unit image of the corresponding calibration area until it is determined that there has been no change in multiple unit images in a row. Based on the updated multiple unit images and the multiple unit images that have not changed, perform the step of "sequentially stitching the multiple unit images along the horizontal direction to form multiple strip-shaped images that extend horizontally and are distributed vertically".

7. A grain leveling height control device, characterized in that, The system includes a memory, a processor, and a grain leveling height calculation program for a grain silo stored in the memory and executable on the processor, the grain leveling height calculation program being configured to implement the steps of the grain leveling height calculation method as described in any one of claims 1 to 6.

8. A storage medium, characterized in that, The storage medium stores a grain leveling height calculation program for the grain warehouse. When the grain leveling height calculation program for the grain warehouse is executed by the processor, it implements the steps of the grain leveling height calculation method as described in any one of claims 1 to 6.

9. A grain leveling system for installation within a grain silo, characterized in that, include: Grain leveling device, which can be movably installed inside the grain warehouse, is used for grain leveling operations; A depth camera is installed at the lower end of the grain leveling device. The depth camera is used to photograph the grain surface of the grain warehouse to obtain unit image information. A driving device, located inside the grain silo, drives and connects to the grain leveling device, enabling the grain leveling device to be movably positioned along the longitudinal, lateral, and vertical directions; and... A grain leveling height control device is installed on the grain leveling device and electrically connected to the grain leveling device, the depth camera and the drive device. The grain leveling height control device is the grain leveling height control device as described in claim 7.

Images