A garbage grabbing method and system based on temperature and depth detection

Abstract

The present invention proposes a garbage grabbing method and system based on temperature and depth detection. The method includes: setting a temperature threshold, measuring the temperature of the upper surface of a garbage pool, and obtaining a coordinate set of planar areas of garbage exceeding the temperature threshold; collecting binocular images of the garbage in the garbage pool and obtaining depth information from the binocular images; using the depth information from the binocular images to perform a three-dimensional reconstruction of the garbage distribution within the garbage pool, obtaining a three-dimensional coordinate model of the garbage distribution within the garbage pool; and transmitting the three-dimensional coordinates of garbage exceeding the temperature threshold to an overhead crane control unit. The overhead crane control unit then grabs the garbage exceeding the temperature threshold based on the three-dimensional coordinates. By monitoring the temperature of the garbage in the garbage pool, high-temperature garbage is identified, quickly and accurately grabbed, and used for waste incineration power generation, reducing energy loss and improving the efficiency and utilization rate of waste incineration.

Description

Technical Field

[0001] The present invention relates to the field of automation, and more particularly to a garbage grabbing method and system based on temperature and depth detection. Background Art

[0002] Waste-to-energy (WTE) is one of the most effective methods for treating municipal solid waste both domestically and internationally. It is an environmentally friendly renewable energy industry that significantly impacts achieving carbon peak and carbon neutrality. After entering the waste pool, waste undergoes a fermentation process, during which excessively hot waste is captured and then incinerated to generate electricity.

[0003] There is currently an automatic garbage grabbing system and method, which includes a gantry crane, a depth sensor, a visible light vision QR code ranging and positioning device, a QR code, a computing device and a control device; multiple depth sensors and visible light vision QR code ranging and positioning devices are respectively connected to the computing device, the computing device is connected to the control device, and the control device is connected to the gantry crane; multiple depth sensors are used to detect the depth information of garbage in the garbage room scene; the visible light vision QR code ranging and positioning device is used to complete the positioning of the gripper by visible light vision QR code ranging; the computing device is used to calculate the volume of the garbage and send a grabbing instruction to the control device; the control device is used to control the gripper to grab the garbage at the position with the largest garbage volume.

[0004] The above method calculates the volume of garbage and grabs large garbage, ignoring the temperature generated by the fermentation of garbage itself, and is not suitable for garbage incineration power generation. Summary of the Invention

[0005] The present invention aims to solve the problem of how to utilize the characteristic of rising temperature during garbage fermentation to improve the efficiency of garbage incineration power generation, and provides a garbage grabbing method and system based on temperature and depth detection.

[0006] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0007] In a first aspect, the present invention proposes a garbage grabbing method based on temperature and depth detection, comprising the following steps:

[0008] S1: Set a temperature threshold, measure the temperature of the upper surface of the garbage pool, and obtain the coordinate set of the garbage plane area that exceeds the temperature threshold;

[0009] S2: Collect binocular images of garbage in the garbage pool and obtain depth information of the binocular images. Use the depth information of the binocular images to perform three-dimensional reconstruction of the garbage distribution and obtain a three-dimensional coordinate model of the garbage distribution in the garbage pool;

[0010] S3: combining the coordinate set of the plane area of ​​the garbage exceeding the temperature threshold, obtaining the corresponding three-dimensional coordinates of the garbage exceeding the temperature threshold from the three-dimensional coordinate model of the garbage distribution, and sending the three-dimensional coordinates of the garbage exceeding the temperature threshold to the overhead crane control unit;

[0011] S4: The overhead crane control unit grabs the garbage that exceeds the temperature threshold according to the three-dimensional coordinates of the garbage that exceeds the temperature threshold.

[0012] In this technical solution, during the garbage fermentation process, the characteristic that objects with high temperatures contain high energy is utilized. By monitoring the temperature of the garbage in the garbage pool, high-temperature garbage is found, and the high-temperature garbage is quickly and accurately captured and used for garbage incineration power generation, reducing energy loss and improving the efficiency and utilization rate of garbage incineration.

[0013] Preferably, the method further comprises: displaying the real-time status of the garbage pool in a virtual manner according to the three-dimensional coordinates of the garbage distribution, and displaying the garbage on a display device.

[0014] Preferably, S1 specifically includes the following steps:

[0015] S1.1: Set a temperature threshold and use an infrared camera to measure the temperature of the upper surface of the garbage pool in real time.

[0016] S1.2: When the temperature of the garbage in the garbage pool exceeds a temperature threshold, the infrared camera collects the temperature distribution of the garbage in the garbage pool and generates a temperature distribution map of the garbage pool.

[0017] S1.3: Based on the temperature distribution map of the garbage pool, obtain a coordinate set of the garbage plane area that exceeds the temperature threshold.

[0018] Preferably, S2 specifically includes the following steps:

[0019] S2.1: Use a binocular camera to collect binocular images of the garbage pool and the garbage in the garbage pool;

[0020] S2.2: Perform image processing on the binocular image to obtain depth information of the binocular image;

[0021] S2.3: Obtain the three-dimensional coordinates of the garbage distribution from the depth information of the garbage, and map the three-dimensional coordinate set of the garbage distribution to the constructed three-dimensional model of the empty garbage pool to obtain a three-dimensional coordinate model of the garbage distribution in the garbage pool.

[0022] Preferably, S2.2 specifically includes:

[0023] S2.2.1: Calibrate the binocular camera to obtain its intrinsic and extrinsic parameters and homography matrix;

[0024] S2.2.2: Correct the two original binocular images based on the binocular camera calibration results. The corrected images are located in the same plane and are parallel to each other.

[0025] S2.2.3: Perform pixel matching on the two corrected binocular images;

[0026] S2.2.4: Calculate the depth of each pixel based on the pixel matching results to obtain the depth information of the garbage binocular image.

[0027] Preferably, in S4, after receiving the three-dimensional coordinates of the garbage exceeding the temperature threshold, the overhead crane control unit controls the overhead crane to move to the plane coordinate position of the garbage exceeding the temperature threshold, and lowers the overhead crane to a specified depth to grab the garbage.

[0028] Preferably, the coordinates of the point with the highest surface temperature on the garbage pool are calculated from the coordinates of the garbage plane area exceeding the temperature threshold, and the overhead crane preferentially grabs the garbage at the point with the highest surface temperature on the garbage pool.

[0029] Preferably, an upper limit for storing garbage in the garbage pool is set, and the overhead crane gives priority to grabbing garbage that exceeds a temperature threshold in the garbage pool that exceeds the upper limit.

[0030] In a second aspect, the present invention proposes a garbage grabbing system based on temperature and depth detection, comprising:

[0031] The temperature detection module is used to measure the temperature of the upper surface of the garbage pool and obtain the coordinate set of the garbage plane area that exceeds the temperature threshold;

[0032] A depth information acquisition module is used to collect binocular images of garbage in the garbage pool and obtain depth information of the binocular images, and obtain the three-dimensional coordinates of the garbage distribution in the garbage pool from the depth information of the binocular images;

[0033] A modeling module is used to use the three-dimensional coordinates of the garbage distribution in the garbage pool to perform three-dimensional reconstruction of the garbage distribution and obtain a three-dimensional coordinate model of the garbage distribution in the garbage pool;

[0034] The control module is used to combine the coordinate set of the plane area of ​​garbage exceeding the temperature threshold, obtain the corresponding three-dimensional coordinates of the garbage exceeding the temperature threshold from the three-dimensional coordinate model of the garbage distribution, and control the overhead crane to grab the garbage exceeding the temperature threshold based on the three-dimensional coordinates of the garbage exceeding the temperature threshold.

[0035] Preferably, the system further comprises a display module, which is used to display the status of the garbage pool in real time and display the garbage on a display device.

[0036] Preferably, the temperature detection module includes an infrared camera and a first processor, wherein the infrared camera collects a temperature distribution image of the garbage in the garbage pool, and the first processor generates a temperature distribution map of the garbage pool based on the image collected by the infrared camera, and outputs a coordinate set of a garbage plane area exceeding a temperature threshold;

[0037] The depth information acquisition module includes a binocular camera and a second processor. The binocular camera collects binocular images of the garbage pool and the garbage in the garbage pool. The second processor performs image processing on the binocular images to obtain depth information of the binocular images and outputs the three-dimensional coordinates of the garbage distribution in the garbage pool.

[0038] The control system includes an overhead crane control unit, which controls the overhead crane to grab the garbage according to the three-dimensional coordinates of the garbage exceeding the temperature threshold.

[0039] Compared with the existing technology, the beneficial effect of the technical solution of the present invention is: during the garbage fermentation process, the characteristic that objects with high temperatures contain high energy is utilized, the temperature of the garbage in the garbage pool is monitored, the high-temperature garbage is found, the high-temperature garbage is quickly and accurately captured and used for garbage incineration power generation, thereby reducing energy loss and improving the efficiency and utilization rate of garbage incineration. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flow chart of the garbage grabbing method based on temperature and depth detection.

[0041] Figure 2 This is a three-dimensional coordinate model diagram of garbage distribution in the garbage pool.

[0042] Figure 3 Schematic diagram of the binocular camera taking the binocular image depth information of garbage.

[0043] Figure 4 This is the simulation result diagram of the garbage pool.

[0044] Figure 5 This is the temperature distribution diagram of garbage in the garbage pool.

[0045] Figure 6 This is the architecture diagram of the garbage grabbing system based on temperature and depth detection.

[0046] Figure 7 Schematic diagram of the garbage grabbing system structure based on temperature and depth detection.

[0047] Among them, 1-infrared camera, 2-binocular camera, 3-overhead crane. DETAILED DESCRIPTION

[0048] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting this patent;

[0049] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0050] Example 1

[0051] See also Figure 1 This embodiment proposes a garbage grabbing method based on temperature and depth detection, including the following steps:

[0052] S1: Set a temperature threshold, measure the temperature of the upper surface of the garbage pool, and obtain the coordinate set of the garbage plane area that exceeds the temperature threshold;

[0053] S2: Collect binocular images of garbage in the garbage pool and obtain depth information of the binocular images. Use the depth information of the binocular images to perform three-dimensional reconstruction of the garbage distribution and obtain a three-dimensional coordinate model of the garbage distribution in the garbage pool;

[0054] S3: combining the coordinate set of the plane area of ​​the garbage exceeding the temperature threshold, obtaining the corresponding three-dimensional coordinates of the garbage exceeding the temperature threshold from the three-dimensional coordinate model of the garbage distribution, and sending the three-dimensional coordinates of the garbage exceeding the temperature threshold to the overhead crane control unit;

[0055] S4: The overhead crane control unit grabs the garbage that exceeds the temperature threshold according to the three-dimensional coordinates of the garbage that exceeds the temperature threshold.

[0056] In the specific implementation process, a temperature threshold is first set. The temperature of the garbage in the garbage pool is monitored in real time. When the temperature of the garbage in the garbage pool exceeds the temperature threshold, a picture of the garbage temperature in the garbage pool is immediately collected and processed accordingly to obtain a temperature distribution map of the garbage in the garbage pool. The plane coordinate set (x k ,y k ) and the coordinates of the highest point on the surface temperature of the garbage pool. Secondly, the binocular image of the garbage in the garbage pool is collected and the depth information of the binocular image is obtained through the binocular vision algorithm, where the depth information obtained includes the three-dimensional coordinate values ​​corresponding to the image pixels; the three-dimensional coordinate set (x n ,y n ,z n ); The three-dimensional coordinate set (x n ,y n ,z n ) is mapped to the constructed three-dimensional model of the empty garbage pool to obtain the three-dimensional coordinate model of the garbage pool distribution, such as Figure 2 As shown, Figure 2 The three-dimensional coordinate model diagram of the garbage distribution in the garbage pool. Finally, the coordinate set of the garbage plane area exceeding the temperature threshold (x k ,y k), obtain the corresponding three-dimensional coordinates (x k ,y k ,z k ); According to the three-dimensional coordinates (x k ,y k ,z k ), use the garbage crawler to crawl garbage that exceeds the temperature threshold.

[0057] During the garbage fermentation process, we take advantage of the fact that objects with high temperatures contain high energy. By monitoring the temperature of the garbage in the garbage pool, we can find the garbage with high temperatures, quickly and accurately grab the high-temperature garbage and use it for garbage incineration power generation, reducing energy loss and improving the efficiency and utilization rate of garbage incineration.

[0058] Example 2

[0059] This embodiment proposes a garbage grabbing method based on temperature and depth detection, including the following steps:

[0060] S1: Set a temperature threshold, measure the temperature of the upper surface of the garbage pool, and obtain the coordinate set of the garbage plane area that exceeds the temperature threshold. The specific steps include:

[0061] S1.1: Set the temperature threshold and use infrared camera 1 to measure the temperature of the upper surface of the garbage pool in real time;

[0062] S1.2: When the temperature of the garbage in the garbage pool exceeds the temperature threshold, the infrared camera 1 collects the temperature distribution of the garbage in the garbage pool and generates a temperature distribution map of the garbage pool;

[0063] S1.3: Based on the temperature distribution map of the garbage pool, obtain the coordinate set of the garbage plane area that exceeds the temperature threshold:

[0064] In this embodiment, infrared temperature photos taken by the infrared camera 1 at four angles are first obtained, and then the infrared camera 1 is rotated to the same angle (the specified upper left corner). Then, the infrared temperature photos are further used as input and image processing is performed to obtain a temperature distribution map with the upper left corner as the origin. The temperature distribution map is completely mapped to the set plane coordinate axis to obtain a plane coordinate map with temperature parameters. Each coordinate point has a temperature in the format of (x, y, temperature), for example (1, 5, 70°C).

[0065] S2: Collect binocular images of garbage in the garbage pool and obtain depth information of the binocular images, use the depth information of the binocular images to perform three-dimensional reconstruction of the garbage distribution, and obtain a three-dimensional coordinate model of the garbage distribution in the garbage pool, which specifically includes the following steps:

[0066] S2.1: Use binocular camera 2 to collect binocular images of garbage in the garbage pool;

[0067] S2.2: Perform image processing on the binocular image to obtain depth information of the binocular image, specifically including:

[0068] S2.2.1: Calibrate the binocular camera 2 to obtain the internal and external parameters and homography matrix of the binocular camera 2;

[0069] S2.2.2: Correct the two original binocular images based on the calibration results of binocular camera 2. The two corrected images are located in the same plane and are parallel to each other.

[0070] S2.2.3: Perform pixel matching on the two corrected binocular images. This embodiment uses a regional stereo matching algorithm. Given a point in one binocular image, a subwindow is selected within the neighborhood of that point. Similarities are found within a region in the other binocular image. The window that is most similar to the subwindow image is found. The corresponding pixel in the matching window is the matching point of the pixel, and a dense disparity map is obtained.

[0071] S2.2.4: Calculate the depth of each pixel based on the pixel matching results to obtain the depth information of the garbage binocular image; Figure 3 As shown, Figure 3 Schematic diagram of binocular image depth information obtained by binocular camera 2. The baseline distance between binocular cameras 2 is b, and the camera focal length is f, u L and u R are the coordinates of the images respectively. For the depth information z of point P, according to the principle of similar triangles, we can get:

[0072]

[0073]

[0074] S2.3: Obtain the three-dimensional coordinates of the garbage distribution from the depth information of the binocular image of the garbage, map the three-dimensional coordinate set of the garbage distribution to the constructed three-dimensional model of the empty garbage pool, and obtain a three-dimensional coordinate model of the garbage distribution in the garbage pool.

[0075] S3: combining the coordinate set of the plane area of ​​garbage exceeding the temperature threshold, obtaining the corresponding three-dimensional coordinates of the garbage exceeding the temperature threshold from the three-dimensional coordinate model of the garbage distribution, and sending the three-dimensional coordinates of the garbage exceeding the temperature threshold to the overhead crane control unit.

[0076] S4: The overhead crane control unit grabs the garbage that exceeds the temperature threshold according to the three-dimensional coordinates of the garbage that exceeds the temperature threshold.

[0077] In the specific implementation process, the garbage pool is divided into several areas, and corresponding infrared cameras 1 and binocular cameras 2 are configured in several areas. A temperature threshold is set, and the infrared camera 1 is used to monitor the temperature of the garbage in the garbage pool in real time. When the temperature of the garbage in the garbage pool exceeds the temperature threshold, the infrared camera 1 immediately collects pictures of the garbage temperature in the garbage pool, performs corresponding image processing on the pictures, obtains the temperature distribution map of the garbage in the garbage pool, and calculates the plane coordinate set (x k ,y k ) and the coordinates of the highest point on the surface temperature of the garbage pool. Secondly, the binocular camera 2 collects the binocular image of the garbage in the garbage pool and obtains the depth information of the binocular image through the binocular vision algorithm, wherein the acquired depth information includes the three-dimensional coordinate values ​​corresponding to the image pixels; the three-dimensional coordinate set (x n ,y n ,z n ); The three-dimensional coordinate set (x n ,y n ,z n ) is mapped to the constructed three-dimensional model of the empty garbage pool to obtain the three-dimensional coordinate model of the garbage pool distribution. Combined with the coordinate set of the garbage plane area exceeding the temperature threshold, the corresponding three-dimensional coordinates (x k ,y k ,z k ), and the three-dimensional coordinates (x k ,y k ,z k ) is sent to the crane control unit. The crane control unit determines the three-dimensional coordinates (x k ,y k ,z k ), the overhead crane 3 moves to the plane coordinate position of the garbage that exceeds the temperature threshold, and the overhead crane 3 is lowered to the specified depth to grab the garbage.

[0078] In this embodiment, the coordinates of the point with the highest surface temperature on the garbage pool are calculated from the coordinates of the garbage plane area that exceeds the temperature threshold; the upper limit of the garbage storage capacity of the garbage pool is set, and when the overhead crane control unit receives instructions from multiple small garbage pools, the overhead crane 3 gives priority to grabbing the garbage that exceeds the temperature threshold and the garbage with the highest temperature in the garbage pool that exceeds the upper limit of the garbage storage capacity.

[0079] In this embodiment, an upper limit for the storage of garbage in the garbage pool is set, and overhead crane 3 prioritizes grabbing garbage that exceeds the temperature threshold in garbage pools that exceed the upper limit. Specifically, when overhead crane 3 receives multiple grab requests simultaneously, it prioritizes the garbage based on the difference between the garbage pool height and the garbage height. The smaller the difference, the higher the priority.

[0080] This embodiment uses Unity3D simulation technology to virtualize and display the real-time status of the garbage pool, and displays the garbage in the form of diamonds on the display. Specifically, the garbage pool is subdivided into 0.1×0.1 small squares, and the coordinates of the highest point of each small square are selected as diamond display points according to the three-dimensional coordinates of the garbage distribution; the coordinates of the diamond display points are used to construct a representative coordinate set for each small square, and the representative coordinate set is imported into Unity3D. Using the interface given by Unity, the representative coordinates of each small square are filled with white diamonds. If there is garbage with a temperature exceeding the threshold in the small square, it will be displayed with red diamonds instead. Figure 4 As shown, Figure 4 This is the simulation result diagram of the garbage pool.

[0081] During the garbage fermentation process, the characteristic that objects with high temperatures contain high energy is utilized. By using an infrared camera 1 to monitor the temperature of the garbage in the garbage pool, garbage with high temperatures is found, and the binocular camera 2 and overhead crane 3 are used to quickly and accurately grab the high-temperature garbage and use it for garbage incineration power generation, thereby reducing energy loss and improving the efficiency and utilization rate of garbage incineration.

[0082] Example 3

[0083] This embodiment proposes a garbage grabbing method based on temperature and depth detection. First, the length, width, and height of the garbage pit are measured, resulting in a length, width, and height of 80 meters, 20 meters, and 18 meters, respectively. The garbage pit is then divided into eight smaller pits, each with a length, width, and height of 10 meters, 2.5 meters, and 18 meters, respectively. These pits are designated as Garbage Pit 1, 2, 3, 4, 5, 6, 7, and 8. Each smaller pit is equipped with four infrared cameras 1, positioned at the four corners, to monitor temperature changes in the pit in real time. Two binocular cameras 2 are located directly above the center of each pit. All pits share a common overhead crane 3.

[0084] The spam crawling process is as follows:

[0085] S1: Set the temperature threshold to 50°C and use infrared camera 1 to measure the temperature of the upper surface of the garbage pool in real time. When the temperature of the garbage in the garbage pool exceeds the temperature threshold, infrared camera 1 collects the temperature distribution of the garbage in the garbage pool, generates a temperature distribution map of the garbage pool, calculates the coordinate set of the garbage plane area that exceeds the temperature threshold, and sends instructions to binocular camera 2 to collect depth information. Figure 5 As shown, Figure 5 is the temperature distribution diagram of garbage in the garbage pool, from Figure 7 It can be seen that there are two areas exceeding the temperature threshold, with temperatures of 73°C and 51°C respectively.

[0086] In this embodiment, the coordinates of the point with the highest temperature on the upper surface of the garbage pool (5, 7) are calculated from the coordinates of the garbage plane area exceeding the temperature threshold.

[0087] S2: Use binocular camera 2 to collect binocular images of garbage in the garbage pool and obtain depth information of the binocular images, use the depth information of the binocular images to perform three-dimensional reconstruction of the garbage distribution, and obtain a three-dimensional coordinate model of the garbage distribution in the garbage pool.

[0088] S3: According to the coordinate set of the plane area of ​​garbage exceeding the temperature threshold and the three-dimensional coordinate model of garbage distribution, the three-dimensional coordinates (5, 7, 10) of the garbage exceeding the temperature threshold in garbage pool No. 1 are determined, and the three-dimensional coordinates (5, 7, 10) of the garbage exceeding the temperature threshold are sent to the overhead crane control unit.

[0089] S4: After the overhead crane control unit obtains the three-dimensional coordinate signal, it controls the overhead crane 3 to move to the plane coordinate (5,7) of the garbage in the No. 1 garbage pool, lowers the overhead crane 3 to the specified depth 10 to grab the garbage, and records the current location of the overhead crane 3 in the No. 1 garbage pool and the specific plane coordinate (5,7), waiting for the next grab.

[0090] During the garbage fermentation process, the characteristic that objects with high temperatures contain high energy is utilized. By using an infrared camera 1 to monitor the temperature of the garbage in the garbage pool, garbage with high temperatures is found, and the binocular camera 2 and overhead crane 3 are used to quickly and accurately grab the high-temperature garbage and use it for garbage incineration power generation, thereby reducing energy loss and improving the efficiency and utilization rate of garbage incineration.

[0091] Example 4

[0092] See also Figure 6 ,This embodiment proposes a garbage grabbing system based on temperature and depth ,detection, including a temperature detection module, a depth information ,acquisition module, a modeling module and a control module.

[0093] In the specific implementation process, the infrared camera in the temperature detection module monitors the temperature of the garbage in the garbage pool in real time. When the temperature of the garbage in the garbage pool exceeds the temperature threshold, the temperature of the garbage in the garbage pool is immediately captured. The first processor performs corresponding image processing on the image to obtain the temperature distribution map of the garbage in the garbage pool, and calculates the plane coordinate set (x k ,y k) and the coordinates of the highest point on the surface temperature of the garbage pool. The depth information acquisition module collects binocular images of the garbage in the garbage pool, and the second processor obtains the depth information of the binocular image through the binocular vision algorithm, and outputs the three-dimensional coordinates of the garbage distribution in the garbage pool; the modeling module uses the three-dimensional coordinates of the garbage distribution in the output garbage pool to perform three-dimensional reconstruction of the garbage distribution in the garbage pool, and obtains a three-dimensional coordinate model of the garbage distribution in the garbage pool; the control module combines the coordinate set of the garbage plane area that exceeds the temperature threshold, and obtains the corresponding three-dimensional coordinates (x k ,y k ,z k ), and the three-dimensional coordinates (x k ,y k ,z k ) is sent to the overhead crane control unit, which controls overhead crane 3 to move to the appropriate position to grab garbage that exceeds the temperature threshold. The display module fully virtualizes the real-time status of the garbage pool, displaying the garbage in the shape of a diamond on the display.

[0094] During the garbage fermentation process, we take advantage of the fact that objects with high temperatures contain high energy. By monitoring the temperature of the garbage in the garbage pool, we can find the garbage with high temperatures, quickly and accurately grab the high-temperature garbage and use it for garbage incineration power generation, reducing energy loss and improving the efficiency and utilization rate of garbage incineration.

[0095] Example 5

[0096] This embodiment proposes a garbage grabbing system based on temperature and depth detection, including a temperature detection module, a depth information acquisition module, a modeling module, and a control module.

[0097] The temperature detection module includes an infrared camera 1, the modeling module includes a binocular camera 2, and the control system includes an overhead crane control unit.

[0098] like Figure 7 As shown, Figure 7 This is a schematic diagram of the structure of a garbage grabbing system based on temperature and depth detection. During implementation, the infrared camera 1 collects the temperature distribution of garbage in the garbage pool, generating a temperature distribution map of the pool. The binocular camera 2 captures binocular images of the garbage pool and the garbage within it. The overhead crane control unit controls the overhead crane 3 to grab the garbage based on the three-dimensional coordinates of garbage exceeding the temperature threshold.

[0099] The terms used in the drawings to describe positional relationships are for illustrative purposes only and should not be construed as limiting this patent;

[0100] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A garbage grabbing method based on temperature and depth detection, characterized in that: The following steps are involved: S1: Set a temperature threshold, measure the temperature of the upper surface of the garbage pool, and obtain the coordinate set of the garbage plane area that exceeds the temperature threshold; S2: Collect binocular images of the garbage in the garbage pool and obtain depth information of the binocular images. Use the depth information of the binocular images to perform three-dimensional reconstruction of the garbage distribution to obtain a three-dimensional coordinate model of the garbage distribution in the garbage pool, including: S2.1: Use a binocular camera to collect binocular images of the garbage pool and the garbage in the garbage pool; S2.2: Perform image processing on the binocular image to obtain the depth information of the binocular image: S2.2.1: Calibrate the binocular camera to obtain its intrinsic and extrinsic parameters and homography matrix; S2.2.2: Correct the two original binocular images based on the binocular camera calibration results. The corrected images are located in the same plane and are parallel to each other. S2.2.3: Perform pixel matching on the two corrected binocular images; S2.2.4: Calculate the depth of each pixel based on the pixel matching results to obtain the depth information of the garbage binocular image; S2.3: Obtain the three-dimensional coordinates of the garbage distribution from the depth information of the binocular image of the garbage, and map the three-dimensional coordinate set of the garbage distribution to the constructed three-dimensional model of the empty garbage pool to obtain a three-dimensional coordinate model of the garbage distribution in the garbage pool; S3: combining the coordinate set of the plane area of ​​the garbage exceeding the temperature threshold, obtaining the corresponding three-dimensional coordinates of the garbage exceeding the temperature threshold from the three-dimensional coordinate model of the garbage distribution, and sending the three-dimensional coordinates of the garbage exceeding the temperature threshold to the overhead crane control unit; S4: The overhead crane control unit grabs the garbage that exceeds the temperature threshold according to the three-dimensional coordinates of the garbage that exceeds the temperature threshold.

2. The garbage grabbing method based on temperature and depth detection according to claim 1 is characterized in that: The method also includes: based on the three-dimensional coordinates of the garbage distribution, virtualizing and displaying the real-time status of the garbage pool, and displaying the garbage on a display device.

3. The garbage grabbing method based on temperature and depth detection according to claim 1 is characterized in that: S1 specifically includes the following steps: S1.1: Set a temperature threshold and use an infrared camera to measure the temperature of the upper surface of the garbage pool in real time; S1.2: When the temperature of the garbage in the garbage pool exceeds the temperature threshold, the infrared camera collects the temperature distribution of the garbage in the garbage pool and generates a temperature distribution map of the garbage pool; S1.3: Based on the temperature distribution map of the garbage pool, obtain a coordinate set of the garbage plane area that exceeds the temperature threshold.

4. The garbage grabbing method based on temperature and depth detection according to claim 1 is characterized in that: In S4, after receiving the three-dimensional coordinates of the garbage exceeding the temperature threshold, the overhead crane control unit controls the overhead crane to move to the plane coordinate position of the garbage exceeding the temperature threshold, and lowers the overhead crane to the specified depth to grab the garbage.

5. The garbage grabbing method based on temperature and depth detection according to any one of claims 1 to 4, characterized in that: Also includes: The coordinates of the point with the highest temperature on the surface of the garbage pool are calculated from the coordinates of the garbage plane area that exceeds the temperature threshold. The overhead crane control unit gives priority to grabbing the garbage at the point with the highest temperature on the surface of the garbage pool. The upper limit of garbage storage in the garbage pool is set, and the overhead crane control unit gives priority to grabbing garbage that exceeds the temperature threshold in the garbage pool that exceeds the upper limit.

6. A garbage grabbing system based on temperature and depth detection, applied to the garbage grabbing method based on temperature and depth detection according to any one of claims 1 to 5, characterized in that: include: The temperature detection module is used to measure the temperature of the upper surface of the garbage pool and obtain the coordinate set of the garbage plane area that exceeds the temperature threshold; A depth information acquisition module is used to collect binocular images of garbage in the garbage pool and obtain depth information of the binocular images, and obtain the three-dimensional coordinates of the garbage distribution in the garbage pool from the depth information of the binocular images; A modeling module is used to use the three-dimensional coordinates of the garbage distribution in the garbage pool to perform three-dimensional reconstruction of the garbage distribution and obtain a three-dimensional coordinate model of the garbage distribution in the garbage pool; The control module is used to combine the coordinate set of the plane area of ​​garbage exceeding the temperature threshold, obtain the corresponding three-dimensional coordinates of the garbage exceeding the temperature threshold from the three-dimensional coordinate model of the garbage distribution, and control the overhead crane to grab the garbage exceeding the temperature threshold based on the three-dimensional coordinates of the garbage exceeding the temperature threshold.

7. The garbage grabbing system based on temperature and depth detection according to claim 6, characterized in that: The system further comprises a display module, which is used to display the status of the garbage pool in real time and display the garbage on a display device.

8. The garbage grabbing system based on temperature and depth detection according to claim 6, characterized in that: The temperature detection module comprises an infrared camera (1) and a first processor, wherein the infrared camera (1) collects a garbage temperature distribution image of the garbage pool, and the first processor generates a temperature distribution map of the garbage pool based on the image collected by the infrared camera (1), and outputs a coordinate set of a garbage plane area exceeding a temperature threshold; The depth information acquisition module comprises a binocular camera (2) and a second processor, wherein the binocular camera (2) collects binocular images of the garbage pool and the garbage in the garbage pool, and the second processor performs image processing on the binocular images to obtain depth information of the binocular images and outputs the three-dimensional coordinates of the garbage distribution in the garbage pool; The control system comprises an overhead crane control unit, which controls the overhead crane (3) to grab the garbage based on the three-dimensional coordinates of the garbage exceeding the temperature threshold.

Images