Garbage detection and cleaning method and system and storage medium

Abstract

The embodiment of the invention discloses a garbage detection and cleaning method and system and a storage medium. According to the technical scheme provided by the embodiment of the invention, the first image obtained by shooting is acquired through the first camera device, and object recognition processing is performed on the first image; when a preset type of target garbage object is identified, determining a target geographic coordinate of the target garbage object according to the pose of the first camera device and the pixel position of the target garbage object in the first image; generating a garbage discovery event according to the target geographic coordinate, and reporting the garbage discovery event to a scheduling platform, so that the scheduling platform schedules a corresponding cleaning robot to a position corresponding to the target geographic coordinate for garbage cleaning according to the garbage discovery event; the technical problem that an existing garbage sweeping mode is low in intelligent degree can be solved, and the intelligent degree of garbage detection and sweeping is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular, to a garbage detection and cleaning method, system and storage medium. Background Art

[0002] With the continuous development of artificial intelligence technology, cleaning robots (also called driverless cleaning vehicles) are gradually used to replace traditional manually driven cleaning vehicles for efficient cleaning.

[0003] Currently, for the task planning of cleaning robots, a timed cleaning task is usually set to control the corresponding cleaning robot to clean a fixed area at a fixed time. In this way, when the corresponding cleaning robot finishes the timed cleaning task, if there is a large amount of garbage in the corresponding area, it cannot be recognized and cleaned, affecting the cleanliness of the corresponding area.

[0004] Therefore, the existing method of garbage cleaning through timed cleaning tasks has a low degree of intelligence, resulting in a poor user experience. Summary of the Invention

[0005] The embodiments of the present application provide a garbage detection and cleaning method, system and storage medium, which can solve the technical problem of low intelligence of the existing garbage cleaning method, improve the intelligence of garbage detection and cleaning, and thus improve the user experience.

[0006] In a first aspect, the embodiments of the present application provide a garbage detection and cleaning method for a first imaging device. The garbage detection and cleaning method includes:

[0007] Obtain a first image obtained by shooting, and perform object recognition processing on the first image;

[0008] When a target garbage object of a preset category is recognized, determine the target geographical coordinates of the target garbage object according to the pose of the first imaging device and the pixel position of the target garbage object in the first image;

[0009] Generate a garbage discovery event according to the target geographical coordinates, and report the garbage discovery event to a scheduling platform for the scheduling platform to dispatch the corresponding cleaning robot to the position corresponding to the target geographical coordinates for garbage cleaning according to the garbage discovery event.

[0010] In an embodiment, the pose includes height information, horizontal field of view angle, pitch angle and first geographical coordinates. When a target garbage object of a preset category is recognized, determining the target geographical coordinates of the target garbage object according to the pose of the first imaging device and the pixel position of the target garbage object in the first image includes:

[0011] When a target garbage object of a preset category is recognized, determine the field of view width of the first imaging device according to the height information and horizontal field of view angle of the first imaging device;

[0012] Calculate according to the field of view width and the pixel position of the target garbage object in the first image to obtain the horizontal distance between the target garbage object and the first imaging device;

[0013] Calculate according to the horizontal distance, the height information and the pitch angle of the first imaging device to determine the relative position information between the target garbage object and the first imaging device;

[0014] Determine the target geographical coordinates of the target garbage object according to the relative position information and the first geographical coordinates of the first imaging device.

[0015] In one embodiment, the first imaging device is a binocular camera;

[0016] When a target garbage object of a preset category is recognized, determine the target geographical coordinates of the target garbage object according to the pose of the first imaging device and the pixel position of the target garbage object in the first image, including:

[0017] When a target garbage object of a preset category is recognized, determine a second image corresponding to the first image according to the binocular camera, where the second image is another image obtained when the binocular camera captures the first image;

[0018] Determine the camera coordinates of the target garbage object in the camera coordinate system according to the first pixel position of the target garbage object in the first image, the second pixel position in the second image, and the pose of the first imaging device;

[0019] Perform rotation and translation conversion processing on the camera coordinates according to the pose of the first imaging device to determine the target geographical coordinates of the target garbage object in the world coordinate system.

[0020] In one embodiment, obtain the captured first image and perform object recognition processing on the first image, including:

[0021] Obtain the captured first image and perform object recognition processing on the first image. When an object of a preset category is recognized, determine that a garbage object is recognized;

[0022] Determine the number of garbage objects. When the number of garbage objects exceeds the preset quantity threshold corresponding to the category, determine that a target garbage object is recognized.

[0023] In a second aspect, an embodiment of the present application provides a garbage detection and cleaning method for a scheduling platform. The garbage detection and cleaning method includes:

[0024] Receive a garbage discovery event, which is generated by the first imaging device based on object recognition processing of the captured first image. When a target garbage object is recognized, it is generated according to the target geographical coordinates determined by the pose of the first imaging device and the pixel position of the target garbage object in the first image. The garbage discovery event includes the target geographical coordinates of the target garbage object;

[0025] Determine the area where the target garbage object is located according to the target geographical coordinates;

[0026] Query the scheduled task list according to the area where the target garbage object is located to determine the scheduled cleaning task time for the corresponding area;

[0027] Determine the time interval between the current time and the scheduled cleaning task time. When the time interval exceeds the preset time threshold, determine the target cleaning robot according to the target geographical coordinates and the status information of the cleaning robot. The status information includes the current geographical coordinates;

[0028] Perform path planning based on the current geographical coordinates and the target geographical coordinates of the target cleaning robot to determine the target path and generate a cleaning task instruction;

[0029] Send the cleaning task instruction to the target cleaning robot to control the target cleaning robot to go to the position corresponding to the target geographical coordinates according to the target path to perform the cleaning task.

[0030] In one embodiment, the status information further includes the current status and the remaining battery power. The current status includes the working status and the idle status;

[0031] Determine the target cleaning robot according to the target geographical coordinates and the status information of the cleaning robot, including:

[0032] Obtain the status information of all cleaning robots;

[0033] Determine the first type of cleaning robots that are currently in the idle state according to the status information;

[0034] Perform path planning based on the current geographical coordinates and the target geographical coordinates of the first type of cleaning robots to determine the travel time for each first type of cleaning robot;

[0035] Select the second type of cleaning robots that meet the task power requirements from the first type of cleaning robots according to the travel time and the remaining battery power;

[0036] Determine the cleaning robot with the least travel time among the second type of cleaning robots as the target cleaning robot.

[0037] In one embodiment, determining the area where the target garbage object is located according to the target geographical coordinates includes:

[0038] Determine the device ID of the first imaging device for photographing the target garbage object according to the target geographical coordinates, and there is a preset correspondence between the device ID and the corresponding area ID;

[0039] Determine the area ID of the area where the target garbage object is located according to the device ID and the preset correspondence;

[0040] Query the timed task list according to the area where the target garbage object is located, and determine the timed cleaning task time for the corresponding area, including:

[0041] Query the timed task list according to the area ID, and determine the timed cleaning task time for the corresponding area.

[0042] In one embodiment, after sending the cleaning task instruction to the target cleaning robot, it includes:

[0043] Send a cleaning monitoring instruction to the corresponding first imaging device, so that the first imaging device monitors the area corresponding to the target geographical coordinates within a preset time according to the cleaning monitoring instruction, to determine whether the corresponding target garbage object is cleared, and generate a monitoring result;

[0044] Receive the monitoring result fed back by the first imaging device.

[0045] In a third aspect, an embodiment of the present application provides a garbage detection and cleaning system, including: a scheduling platform, a cleaning robot, and a first imaging device. A second imaging device is provided on the cleaning robot, and the first imaging device is set at a preset position in the corresponding street area;

[0046] The scheduling platform is communicatively connected to the cleaning robot and the first imaging device;

[0047] The first imaging device is used to obtain the first image obtained by photographing, perform object recognition processing on the first image, when recognizing a target garbage object of a preset category, determine the target geographical coordinates of the target garbage object according to the pose of the first imaging device and the pixel position of the target garbage object in the first image, generate a garbage discovery event according to the target geographical coordinates, and report the garbage discovery event to the scheduling platform;

[0048] The scheduling platform is used to query the timed task list according to the area where the target garbage object of the received garbage discovery event is located, and determine the timed cleaning task time for the corresponding area; determine the time interval between the current time and the timed cleaning task time, when the time interval exceeds a preset time threshold, determine the target cleaning robot according to the target geographical coordinates and the status information of the cleaning robot, and the status information includes the current geographical coordinates; perform path planning according to the current geographical coordinates and the target geographical coordinates of the target cleaning robot, determine the target path, and generate a cleaning task instruction; send the cleaning task instruction to the target cleaning robot;

[0049] A cleaning robot is used to travel to the location corresponding to the target geographical coordinates according to the received cleaning task instruction and the corresponding target path. After arriving at the location, while cleaning, it performs real-time detection through a second camera device and cleans all preset types of garbage objects according to the real-time detection.

[0050] In a fourth aspect, an embodiment of the present application provides a storage medium storing computer-executable instructions, and the computer-executable instructions are used to execute the garbage detection and cleaning method as in the first aspect or the second aspect when executed by a computer processor.

[0051] In the embodiment of the present application, object recognition processing is performed on the first image captured by the first camera device. When a target garbage object of a preset type is recognized, according to the pose of the first camera device and the pixel position of the target garbage object in the first image, the target geographical coordinates corresponding to the target are determined, a garbage discovery event is generated according to the target geographical coordinates, and the garbage discovery event is reported to the scheduling platform, so that the scheduling platform schedules the corresponding cleaning robot to travel to the location corresponding to the target geographical coordinates for garbage cleaning according to the garbage discovery event. By adopting the above technical means, garbage corresponding recognition can be performed on the corresponding area through the first camera device, and when the target garbage object is recognized, the corresponding cleaning robot can be scheduled to clean, thereby avoiding the technical problem of low intelligence caused by regular cleaning, being able to automatically recognize the target garbage object and schedule the corresponding cleaning robot to clean, improving the intelligence of garbage automatic recognition and the intelligence of scheduling the cleaning robot, so that when a target garbage object appears within the shooting range of the first camera device, whether before or after the regular cleaning task, timely cleaning can be performed based on the cleaning robot, avoiding garbage accumulation and poor user experience, and thus improving the user experience.

[0052] The beneficial effects of the above-provided garbage detection and cleaning system and storage medium can refer to the beneficial effects of the garbage detection and cleaning method. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a flowchart of a garbage detection and cleaning method provided by an embodiment of the present application;

[0054] Figure 2 is a flowchart of another garbage detection and cleaning method provided by an embodiment of the present application;

[0055] Figure 3 is a schematic structural diagram of a garbage detection and cleaning system provided by an embodiment of the present application;

[0056] Figure 4 is a schematic structural diagram of a garbage detection and cleaning device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0057] In order to make the objectives, technical solutions, and advantages of the present application clearer, the following further describes specific embodiments of the present application in detail with reference to the accompanying drawings. It can be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Additionally, it should be noted that for ease of description, only parts related to the present application rather than all content are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the operations can be rearranged. When the operations are completed, the process can be terminated, but there may also be additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, and so on.

[0058] Currently, for the task planning of a cleaning robot, a timed cleaning task is usually set to control the corresponding cleaning robot to clean a fixed area at a fixed time. In this way, when the corresponding cleaning robot finishes the timed cleaning task, if there is a large amount of garbage in the corresponding area, it cannot be recognized and cleaned, which affects the cleanliness of the corresponding area. Therefore, the existing method of cleaning garbage through timed cleaning tasks has a low degree of intelligence, resulting in a poor user experience.

[0059] Based on this, the present application provides an embodiment of a garbage detection and cleaning method, system, and storage medium. When cleaning garbage, the corresponding area is recognized for garbage through a first imaging device. When a target garbage object is recognized, the corresponding cleaning robot can be dispatched to clean it. In this way, the technical problem of low intelligence caused by timed cleaning can be avoided. It can automatically recognize the target garbage object and dispatch the corresponding cleaning robot to clean it, improving the intelligence of automatic garbage recognition and the intelligence of dispatching the cleaning robot. When a target garbage object appears within the shooting range of the first imaging device, whether before or after the timed cleaning task, cleaning can be promptly performed based on the cleaning robot, avoiding garbage accumulation and resulting in a poor user experience, thereby enhancing the user experience.

[0060] Figure 1The flowchart of a garbage detection and cleaning method provided by an embodiment of the present application is given. In this embodiment, the garbage detection and cleaning method can be executed by a garbage detection and cleaning device, which can be implemented in software and / or hardware. The garbage detection and cleaning device can be composed of two or more physical entities or one physical entity. Generally, the garbage detection and cleaning device can be a camera device or a terminal device with scheduling functions, such as terminal devices like computers, interactive tablets, tablet computers, smart TVs, learning machines, and mobile phones.

[0061] The following takes the camera device as the main body for executing the garbage detection and cleaning method as an example for description. Refer to Figure 1 , this garbage detection and cleaning method is used for the first camera device. The first camera device can be a security camera device set in the street area, a camera device specially set for garbage detection in the corresponding street, or a camera device set in the corresponding street area for performing other functions. The garbage detection and cleaning method specifically includes:

[0062] S101. Obtain the first image obtained by shooting and perform object recognition processing on the first image.

[0063] On the basis of implementing the functions corresponding to the garbage detection and cleaning method provided in this embodiment, the first camera device can also perform other functions based on the captured images, such as the security monitoring function.

[0064] When the first camera device implements the functions corresponding to the garbage detection and cleaning method provided in this embodiment, it obtains the first image obtained by shooting and performs recognition processing on the first image. The first image can be recognized and processed through a preset image recognition model. The preset image recognition model can be trained through an image training set containing preset categories of garbage objects. When performing object recognition processing on the first image, the first image to be recognized is input into the trained image recognition model, and the corresponding recognition result can be output, that is, it is determined whether the first image contains a target garbage object of a preset category.

[0065] As described above, the first camera device shoots and performs object recognition processing on the area within its shooting range to real-time identify whether there is a garbage object of a preset category in the corresponding area, thereby realizing the intelligence and automation of garbage recognition.

[0066] S102. When a target garbage object of a preset category is recognized, determine the target geographical coordinates of the target garbage object according to the pose of the first camera device and the pixel position of the target garbage object in the first image.

[0067] The preset categories can be understood as the preset categories that can be cleaned, such as fallen leaves, plastic water bottles, fruit peels, and paper, etc.; other categories can be understood as the categories that cannot be cleaned, such as trees, vehicles, and pedestrians, etc. When performing object recognition processing on the first image, the objects and categories existing in the first image are recognized. When an object of the preset category is recognized, it is determined that there is a target garbage object in the first image, that is, a target garbage object that needs to be cleaned appears in the shooting area corresponding to the first camera device. Then, according to the pose of the first camera device and the pixel position of the target garbage object in the first image, the target geographical coordinates of the target garbage object are determined. Subsequently, the cleaning robot can be dispatched to clean according to the target geographical coordinates, improving the intelligent level of garbage recognition and the intelligent level of garbage cleaning.

[0068] In one embodiment, in order to improve the intelligence of the cleaning robot dispatching and maximize the benefits, it is not possible to dispatch the cleaning robot as soon as a garbage object of the preset category is recognized, so as to avoid wasting energy. For example, assume that the preset category is fallen leaves. If the cleaning robot is dispatched to clean when one or two fallen leaves are recognized, it will lead to multiple dispatches and reduce the overall benefits. Therefore, when performing object recognition processing on the first image, when an object of the preset category is recognized, it is determined that a garbage object is recognized. At this time, it is not yet possible to determine whether the garbage object is a target garbage object. Then, the number of garbage objects is determined. When the number of garbage objects exceeds the preset quantity threshold of the preset category, it is determined that a target garbage object is recognized. The target garbage object can be understood as a garbage object whose quantity reaches the preset quantity threshold corresponding to the preset category. For example, if the preset category is leaves and the corresponding preset threshold is 20 pieces, when it is recognized that there are more than 20 leaves in the first image, it is determined that a target garbage object is recognized, that is, the leaves that need to be cleaned are recognized. It can also be understood that if the number of recognized leaves is 2 - 3 pieces (that is, not exceeding the preset threshold), it means that the current quantity will not seriously affect the cleaning hygiene and user experience, and the cleaning robot can be not dispatched to clean first. Another example, if the preset category is plastic water bottles and the corresponding preset threshold is 3, when it is recognized that there are more than 3 plastic water bottles in the first image, it is determined that a target garbage object is recognized, that is, the plastic water bottles that need to be cleaned are recognized.

[0069] As described above, by determining the garbage object as the target garbage object only when the number of recognized garbage objects of the preset category exceeds the preset quantity threshold of the corresponding category, the waste of resources caused by immediately dispatching the cleaning robot when the number of garbage objects is small is avoided, thereby improving the intelligent level of garbage detection and the benefits of garbage cleaning, and avoiding the waste of resources caused by over-cleaning.

[0070] In one embodiment, an implementation manner for determining the target geographical coordinates of a target garbage object according to the pose of a first imaging device is provided. The pose includes height information, horizontal field of view angle, pitch angle, and first geographical coordinates. The first geographical coordinates can be understood as the geographical coordinates of the first imaging device. When the first imaging device performs object recognition on the first image and recognizes a target garbage object of a preset category, according to the height information and horizontal field of view angle in the pose of the first imaging device, the field of view width of the first imaging device is determined; based on the field of view width and the pixel position of the target garbage object in the first image, calculations are performed to obtain the horizontal distance of the target garbage object from the first imaging device; based on the horizontal distance, the height information of the first imaging device, and the pitch angle, calculations are performed, and the relative position information between the target garbage object and the first imaging device can be determined through geometric calculations; based on this relative position information and the first geographical coordinates of the first imaging device, the target geographical coordinates of the target garbage object are determined. It should be noted that the first geographical coordinates and the target geographical coordinates can be longitude and latitude coordinates.

[0071] In one embodiment, an implementation manner for determining the target geographical coordinates of a target garbage object according to the pose of a first imaging device is provided. When a target garbage object of a preset category is recognized in the first image captured by the first imaging device, a second image corresponding to the first image is determined according to a binocular camera, where the second image is another image obtained when the binocular camera captures the first image, and the first image and the second image are two images captured by the binocular camera simultaneously. According to the first pixel position of the target garbage object in the first image, the second pixel position in the second image, and the pose of the first imaging device, the camera coordinates of the target garbage object in the camera coordinate system are determined. Through rotation and translation conversion processing of the camera coordinates of the target garbage object according to the pose of the first imaging device, the target geographical coordinates of the target garbage object in the world coordinate system are determined.

[0072] Exemplarily, pixel point matching is performed on the first image and the second image captured by the binocular camera. For a pixel point in the first image, a corresponding pixel point needs to be found in the second image. This can be achieved through a sliding window or a feature matching algorithm, such as the SSD (Sum of Squared Differences) algorithm or the SIFT (Scale-Invariant Feature Transform) algorithm, etc. The result of the matching is the coordinate positions of the two pixel points in their respective images. For example, the first pixel position of the target garbage object in the first image, and the corresponding matching point is the second pixel position of the target garbage object in the second image. Therefore, the disparity can be calculated based on the first pixel position and the second pixel position. Disparity refers to the difference in pixel positions of the same spatial point on the imaging planes of the left and right cameras. The magnitude of the disparity is inversely proportional to the distance from the spatial point to the camera (i.e., the binocular camera), that is, the closer the distance, the greater the disparity; the farther the distance, the smaller the disparity. After that, based on the principle of triangulation, that is, using the two images of an object captured by two cameras (i.e., the binocular camera) at the same time, namely the first image and the second image, by calculating the disparity between the first image and the second image, combined with the geometric relationship of the cameras, the three-dimensional coordinates of the object (i.e., the target garbage object) can be restored, that is, the camera coordinates in the camera coordinate system. For example, the depth is calculated according to the disparity, the focal length of the camera, and the baseline distance, and is calculated through the following formula:

[0073] z = fB / d

[0074] where z represents the depth, f represents the focal length of the camera, B represents the baseline distance, and d represents the disparity.

[0075] According to the depth z of the camera, the points in the pixel coordinate system can be converted into camera coordinate points in the camera coordinate system. For example, first convert the pixel coordinates into image coordinates. Given the point (x, y) in the image coordinate system (note that (x, y) here is the image coordinate scaled to millimeters, rather than pixel coordinates) and the corresponding depth z (in millimeters) of this point, the pixel coordinates of the target garbage object can be converted into camera coordinates through the following formula.

[0076]

[0077] where, (X c , Y c , Z c ) are the camera coordinates in the camera coordinate system, and f is the focal length of the camera (in millimeters). Among them and They are the normalized coordinates of the points in the image coordinate system on the X-axis and Y-axis in the camera coordinate system (i.e., the coordinates without considering depth), and then multiplied by the depth z to obtain the final camera coordinates in the camera coordinate system. Therefore, all the camera coordinate points of the target garbage object can be obtained through the above conversion.

[0078] For the camera coordinate points of the target garbage object, a homogeneous transformation matrix is used to describe the rotation and translation transformation processing to obtain the world coordinates of the target garbage object in the world coordinate system. This world coordinate can be understood as the target geographical coordinate. The homogeneous transformation matrix includes a rotation matrix R and a translation vector t, and the conversion formula is as follows:

[0079] P w = R·P c + t

[0080] where, P W is the point coordinate in the world coordinate system, R is the rotation matrix, P c is the point coordinate in the camera coordinate system, and t is the translation vector. Exemplarily, the rotation matrix R is a 3×3 matrix that describes the rotation of the camera coordinate system relative to the world coordinate system. The specific values in the rotation matrix R depend on the pose (i.e., the rotation angle and direction) of the camera (i.e., the first imaging device). The translation vector t is a 3×1 matrix that describes the position of the origin of the camera coordinate system in the world coordinate system. The specific values of the translation vector t depend on the position of the camera. Through the conversion of the aforementioned rotation matrix R and translation vector t, all the point coordinates of the target garbage object in the world coordinate system can be converted, and the point coordinates in the world coordinate system are integrated to determine its center point or centroid point to obtain the target geographical coordinate of the target garbage object.

[0081] It should be noted that the target geographical coordinate can be longitude and latitude coordinates.

[0082] As described above, the target geographical coordinate of the target garbage object is directly determined by the first imaging device, and subsequently, the corresponding cleaning robot can be scheduled based on the clear target geographical coordinate. Compared with the existing method that can only identify the area identifier where the target garbage object is located, this embodiment can determine a more accurate target geographical coordinate, improve the accuracy of garbage detection, and thus improve the working efficiency of the subsequent cleaning robot for garbage cleaning based on the clear target geographical coordinate.

[0083] S103. Generate a garbage discovery event according to the target geographical coordinate and report the garbage discovery event to the scheduling platform for the scheduling platform to schedule the corresponding cleaning robot to go to the position corresponding to the target geographical coordinate for garbage cleaning.

[0084] Generate a garbage discovery event according to the target geographical coordinates, and report the garbage discovery event to the dispatching platform for the dispatching platform to dispatch the corresponding cleaning robot to the location corresponding to the target geographical coordinates according to the garbage discovery event for garbage cleaning. As described above, the corresponding area is identified for garbage through the first camera device. When the target garbage object is identified, the corresponding cleaning robot can be dispatched to clean it, thereby avoiding the technical problem of low intelligence caused by regular cleaning. It can automatically identify the target garbage object and dispatch the corresponding cleaning robot to clean it, improving the intelligence of garbage automatic identification and the intelligence of dispatching cleaning robots. When the target garbage object appears within the shooting range of the first camera device, whether before or after the regular cleaning task, it can be cleaned in a timely manner based on the cleaning robot, avoiding garbage accumulation and poor user experience, and thus improving the user experience.

[0085] To further improve the garbage cleaning effect, after the dispatching platform dispatches the corresponding target cleaning robot to the location corresponding to the target geographical coordinates for cleaning, the dispatching platform will send a cleaning monitoring instruction to the corresponding first camera device. The first camera device receives the cleaning monitoring instruction sent by the dispatching platform and monitors the area corresponding to the target geographical coordinates within a preset time according to the monitoring instruction to determine whether the corresponding target garbage object has been cleaned up, and generates a monitoring result, and feeds back the monitoring result to the dispatching platform. Exemplarily, assuming that the monitoring result is that the target garbage object has not been cleaned up, the dispatching platform dispatches other cleaning robots to the location corresponding to the target geographical coordinates for garbage cleaning based on this monitoring result to clean up the corresponding target garbage object. Another example, assuming that the monitoring result is that the target garbage object has been cleaned up, the dispatching platform does not need to dispatch a new cleaning robot for secondary cleaning.

[0086] Based on the above embodiments, Figure 2 is a flowchart of another garbage detection and cleaning method provided by an embodiment of the present application. Refer to Figure 2 , this garbage detection and cleaning method is used for the dispatching platform, and the device corresponding to the dispatching platform can be terminal devices such as computers, interactive tablets, tablet computers, smart TVs, learning machine mobile phones, etc. This garbage detection and cleaning method specifically includes:

[0087] S201. Receive a garbage discovery event, which is generated by the first camera device according to object recognition processing of the first image captured. When the target garbage object is recognized, it is generated according to the target geographical coordinates determined by the pose of the first camera device and the pixel position of the target garbage object in the first image. The garbage discovery event includes the target geographical coordinates of the target garbage object.

[0088] The first imaging device detects target garbage objects in the area within its imaging field of view through the aforementioned S101 - S103. When a target garbage object is found in the corresponding area, it reports the corresponding garbage discovery event. The scheduling platform receives the garbage discovery event reported by the corresponding first imaging device. The garbage discovery event includes information such as the target geographical coordinates of the target garbage object and the device ID of the corresponding first imaging device. Subsequently, the corresponding cleaning robot can be scheduled according to the target geographical coordinates in the garbage discovery event.

[0089] S202. Determine the area where the target garbage object is located according to the target geographical coordinates.

[0090] Determine the area where the target garbage object corresponding to the garbage discovery event is located according to the target geographical coordinates. Since a timed cleaning task is set for each area, and the garbage detection and cleaning method provided in this embodiment is implemented based on the timed cleaning task, after the scheduling platform receives the garbage discovery event reported by the corresponding first imaging device, it is necessary to first determine the area where the target garbage object corresponding to the garbage discovery event is located. Compared with the time of the timed cleaning task corresponding to this area, if the time is relatively close, there is no need to additionally schedule a cleaning robot, and the cleaning robot based on the timed cleaning task can directly clean the target garbage object. If the current time is far from the time of the timed cleaning task corresponding to the area, it is impossible to wait for the execution of the timed cleaning task, and it is necessary to additionally schedule a cleaning robot to go to the location corresponding to the target geographical coordinates for timely cleaning.

[0091] S203. Query the timed task list according to the area where the target garbage object is located, and determine the time of the timed cleaning task for the corresponding area.

[0092] Query the timed task list according to the area where the target garbage object is located to determine the time of the timed cleaning task for this area, so as to determine whether it is necessary to additionally schedule a cleaning robot to go to perform the cleaning.

[0093] Exemplarily, according to the target geographical coordinates, determine the device ID of the first imaging device that captures the target garbage object. There is a preset correspondence between the device ID and the corresponding area ID. According to the device ID and the preset correspondence, determine the area ID of the area where the target garbage object is located. Query the timed task list according to the area ID to determine the time of the timed cleaning task for the corresponding area.

[0094] S204. Determine the time interval between the current time and the time of the timed cleaning task. When the time interval exceeds the preset time threshold, determine the target cleaning robot according to the target geographical coordinates and the status information of the cleaning robot. The status information includes the current geographical coordinates.

[0095] Determine the time interval between the current time and the scheduled cleaning task time. When the time interval exceeds a preset time threshold, for example, exceeds 30 minutes, it means that it is necessary to additionally dispatch the corresponding cleaning robot to the corresponding area to clean the target garbage object. Therefore, when the time interval exceeds the preset time threshold, determine the target cleaning robot according to the target geographical coordinates and the status information of the cleaning robot. The status information includes the current geographical coordinates, the current status, and the remaining battery power. The current status includes the working status and the idle status.

[0096] When determining the target cleaning robot, the status information of all cleaning robots can be obtained first, and the first type of cleaning robots that are currently in the idle state can be determined according to the status information. Path planning is carried out according to the current geographical coordinates and the target geographical coordinates of the first type of cleaning robots to determine the travel time consumed by each first type of cleaning robot. This travel time consumed can be understood as the travel time consumed by the corresponding cleaning robot from its current geographical coordinates along the corresponding path to the target geographical coordinates. According to the travel time consumed and the remaining battery power, select the second type of cleaning robots that meet the task power requirements from the first type of cleaning robots. The task power requirements can be understood as the travel time consumed for the round trip and the total power required for the cleaning task. Finally, determine the cleaning robot with the least travel time consumed from the second type of cleaning robots as the target cleaning robot. As described above, by using the status information of the cleaning robot, the cleaning robot that is currently in the idle state, has the remaining battery power to meet the task power requirements, and has the least travel time consumed is selected as the target cleaning robot, so that it is expected to complete the cleaning task and can rush to the target geographical coordinates for the cleaning task as soon as possible, thereby improving the work efficiency of garbage cleaning and further enhancing the user experience.

[0097] S205. Perform path planning according to the current geographical coordinates and the target geographical coordinates of the target cleaning robot to determine the target path and generate a cleaning task instruction.

[0098] After determining the target cleaning robot, perform path planning according to the current geographical coordinates and the target geographical coordinates of the target cleaning robot to determine the target path. It is also possible to directly confirm the path during the path planning when screening the target cleaning robot as the target path. Generate a cleaning task instruction according to the target path.

[0099] S206. Send the cleaning task instruction to the target cleaning robot to control the target cleaning robot to travel to the position corresponding to the target geographical coordinates along the target path to perform the cleaning task.

[0100] Send the cleaning task instruction to the corresponding target cleaning robot to control the target cleaning robot and the target path to travel to the position corresponding to the target geographical coordinates to perform the cleaning task, that is, clean the target garbage object.

[0101] In one embodiment, a second imaging device is provided on each cleaning robot. The target cleaning robot can perform corresponding obstacle avoidance during the target path according to the second imaging device, and when performing the cleaning task, it can also identify garbage objects based on the second imaging device to perform cleaning more quickly and accurately. After the cleaning is completed, it can be confirmed based on the second imaging device to confirm whether the cleaning is truly completed, and the confirmation result is fed back to the scheduling platform.

[0102] In one embodiment, after the target cleaning robot reaches the area corresponding to the target geographical coordinates, it first cleans the target garbage object. After cleaning the target garbage object, it performs real-time detection based on its own second imaging device to identify whether there are garbage objects of a preset category in the corresponding area. If there are garbage objects of a preset category, they are cleaned together to ensure that there are no garbage objects of a preset category in the corresponding area before the cleaning task is considered completed.

[0103] As described above, after the scheduling platform receives a garbage discovery event, it confirms the timing cleaning task time for the area corresponding to the garbage discovery event. When the time interval between the current time and the timing cleaning task time exceeds a preset time threshold, it schedules the corresponding target cleaning robot to go to the location corresponding to the target geographical coordinates for garbage cleaning, thereby avoiding the target garbage object staying in the corresponding area for a long time and affecting the user experience. After the target garbage object is discovered, the target cleaning robot can be scheduled in time for cleaning. Compared with the existing method of timing cleaning tasks, it improves the intelligence and automation of garbage detection, as well as the flexibility of garbage cleaning, and thus enhances the user experience.

[0104] Based on the above embodiments Figure 3 is a schematic structural diagram of a garbage detection and cleaning system provided by an embodiment of the present application. Refer to Figure 3, the garbage detection and cleaning system includes: a first camera device 11, a scheduling platform 12, and a cleaning robot 13. A second camera device 131 is provided on the cleaning robot 13, and the first camera device 11 is arranged at a preset position corresponding to a street area. The scheduling platform 12 is communicatively connected to the cleaning robot 13 and the first camera device 11. The first camera device 11 is configured to execute the foregoing S101 - S103. By obtaining a first image captured, and performing object recognition processing on the first image, when a target garbage object of a preset category is recognized, according to the pose of the first camera device 11 and the pixel position of the target garbage object in the first image, determine the target geographical coordinates of the target garbage object, generate a garbage discovery event based on the target geographical coordinates, and report the garbage discovery event to the scheduling platform 12. The scheduling platform 12 is configured to execute the foregoing S201 - S206. Query the scheduled task list according to the area where the target garbage object of the received garbage discovery event is located, and determine the scheduled cleaning task time for the corresponding area; determine the time interval between the current time and the scheduled cleaning task time, and when the time interval exceeds a preset time threshold, according to the target geographical coordinates and the status information of the cleaning robot 13, determine the target cleaning robot 13, and the status information includes the current geographical coordinates; perform path planning according to the current geographical coordinates and the target geographical coordinates of the target cleaning robot 13, determine the target path, and generate a cleaning task instruction; send the cleaning task instruction to the target cleaning robot 13. The cleaning robot 13 is configured to travel to the position corresponding to the target geographical coordinates according to the received cleaning task instruction along the corresponding target path, and after arriving at the position, while performing cleaning, perform real-time detection through the second camera device 131, and clean all garbage objects of the preset category according to the real-time detection.

[0105] In one embodiment, as Figure 3 shown, each first camera device 11 is correspondingly arranged at a preset position, and its corresponding camera field of view corresponds to an area. For example, the field of view of the first camera device 11A corresponds to area A, the field of view of the first camera device 11B corresponds to area B, and the field of view of the first camera device 11C corresponds to area C. The first camera device 11A, the first camera device 11B, and the first camera device 11C are all communicatively connected to the scheduling platform 12. The scheduling platform 12 is communicatively connected to a corresponding plurality of cleaning robots 13. For example, the scheduling platform 12 is communicatively connected to the cleaning robots 13A, 13B, 13C, 13D, and 13E.

[0106] The garbage detection and cleaning system provided by the embodiment of the present application can be used to execute the garbage detection and cleaning method provided by the above embodiment, and has corresponding functions and beneficial effects.

[0107] The embodiment of the present application provides a garbage detection and cleaning device. Refer toFigure 4 The garbage detection and cleaning device includes: a processor 31, a memory 32, a communication module 33, an input device 34, and an output device 35. The number of processors in the garbage detection and cleaning device may be one or more, and the number of memories in the garbage detection and cleaning device may be one or more. The processor, memory, communication module, input device, and output device of the garbage detection and cleaning device may be connected through a bus or other means.

[0108] The memory 32, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the garbage detection and cleaning method described in any embodiment of the present application. The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the device, etc. In addition, the memory may include high-speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some instances, the memory may further include a memory remotely set relative to the processor, and these remote memories can be connected to the device through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0109] The communication module 33 is used for data transmission.

[0110] The processor 31 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory, that is, implements the above-mentioned garbage detection and cleaning method.

[0111] The input device 34 can be used to receive input digital or character information, and generate key signal inputs related to the user settings and function control of the device. The output device 35 may include a display device such as a display screen.

[0112] The above-provided garbage detection and cleaning device can be used to execute the garbage detection and cleaning method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0113] An embodiment of the present application further provides a storage medium storing computer-executable instructions, and the computer-executable instructions are used to execute the foregoing garbage detection and cleaning method when executed by a computer processor.

[0114] Storage medium - Any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media such as CD-ROMs, floppy disks or magnetic tape devices; computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory such as flash memory, magnetic media (such as hard disks or optical storage); registers or other similar types of memory elements, etc. The storage medium may also include other types of memory or combinations thereof. Additionally, the storage medium may be located in a first computer system in which the program is executed, or may be located in a different second computer system that is connected to the first computer system via a network (such as the Internet). The second computer system may provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media residing in different locations (such as in different computer systems connected via a network). The storage medium may store program instructions executable by one or more processors (e.g., embodied as a computer program).

[0115] Of course, for a storage medium provided in an embodiment of the present application that stores computer-executable instructions, the computer-executable instructions are not limited to the garbage detection and cleaning method as described above, and may also execute related operations in the garbage detection and cleaning method provided in any embodiment of the present application.

[0116] The garbage detection and cleaning device, storage medium, and garbage detection and cleaning equipment provided in the above embodiments may execute the garbage detection and cleaning method provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, reference may be made to the garbage detection and cleaning method provided in any embodiment of the present application.

[0117] The above is only the preferred embodiment of the present application and the technical principles applied. The present application is not limited to the specific embodiments described herein. Various obvious changes, re-adjustments, and substitutions that can be made by those skilled in the art will not depart from the protection scope of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments. Without departing from the concept of the present application, it may also include more other equivalent embodiments, and the scope of the present application is determined by the scope of the claims.

Claims

1. A garbage detection and cleaning method, characterized in that: For a first camera device, the method comprises: Acquire a first image obtained by shooting, and perform object recognition processing on the first image; When a target garbage object of a preset category is identified, determining the target geographic coordinates of the target garbage object according to the posture of the first camera device and the pixel position of the target garbage object in the first image; A garbage discovery event is generated according to the target geographic coordinates, and the garbage discovery event is reported to the scheduling platform, so that the scheduling platform can dispatch the corresponding cleaning robot to the location corresponding to the target geographic coordinates to clean up the garbage according to the garbage discovery event.

2. The method according to claim 1, characterized in that: The posture includes altitude information, horizontal field of view angle, pitch angle and first geographic coordinates; When a target garbage object of a preset category is identified, determining the target geographic coordinates of the target garbage object according to the posture of the first camera device and the pixel position of the target garbage object in the first image includes: When a target garbage object of a preset category is identified, determining a field of view width of the first camera device according to the height information and the horizontal field of view angle of the first camera device; Calculating according to the field of view width and the pixel position of the target garbage object in the first image to obtain a horizontal distance between the target garbage object and the first camera device; Calculate the relative position information between the target garbage object and the first camera device according to the horizontal distance and the height information and the pitch angle of the first camera device; The target geographical coordinates of the target garbage object are determined according to the relative position information and the first geographical coordinates of the first camera device.

3. The method according to claim 1, characterized in that The first camera device is a binocular camera; When a target garbage object of a preset category is identified, determining the target geographic coordinates of the target garbage object according to the posture of the first camera device and the pixel position of the target garbage object in the first image includes: When a target garbage object of a preset category is identified, a second image corresponding to the first image is determined according to the binocular camera, where the second image is another image obtained when the binocular camera captures the first image; Determine the camera coordinates of the target garbage object in a camera coordinate system according to a first pixel position of the target garbage object in the first image and a second pixel position of the target garbage object in the second image and a position and posture of the first camera device; The camera coordinates are subjected to rotation and translation conversion processing according to the position and posture of the first camera device to determine the target geographic coordinates of the target garbage object in the world coordinate system.

4. The method according to claim 1, characterized in that: The acquiring the captured first image and performing object recognition processing on the first image includes: Acquire the captured first image, perform object recognition processing on the first image, and when an object of a preset category is recognized, determine that a garbage object is recognized; The number of the garbage objects is determined, and when the number of the garbage objects exceeds a preset number threshold of a corresponding category, it is determined that a target garbage object is identified.

5. A garbage detection and cleaning method, characterized in that: For a scheduling platform, the method comprises: receiving a garbage discovery event, wherein the garbage discovery event is generated by performing object recognition processing by the first camera device according to the first image captured, and when a target garbage object is recognized, target geographic coordinates determined according to the posture of the first camera device and the pixel position of the target garbage object in the first image, and the garbage discovery event includes the target geographic coordinates of the target garbage object; Determining the area where the target garbage object is located according to the target geographic coordinates; Query the scheduled task list according to the area where the target garbage object is located, and determine the scheduled cleaning task time of the corresponding area; Determine a time interval between the current time and the scheduled cleaning task time, and when the time interval exceeds a preset time threshold, determine a target cleaning robot according to the target geographic coordinates and status information of the cleaning robot, wherein the status information includes the current geographic coordinates; Performing path planning according to the current geographic coordinates of the target cleaning robot and the target geographic coordinates, determining the target path, and generating a cleaning task instruction; The cleaning task instruction is sent to the target cleaning robot to control the target cleaning robot to go to the position corresponding to the target geographic coordinates according to the target path to perform the cleaning task.

6. The method according to claim 5, characterized in that The state information also includes the current state and the remaining power, and the current state includes the working state and the idle state; The step of determining the target cleaning robot according to the target geographic coordinates and the state information of the cleaning robot comprises: Get the status information of all cleaning robots; Determine, according to the state information, a first type of cleaning robot that is currently in an idle state; Perform path planning according to the current geographic coordinates of the first type of cleaning robots and the target geographic coordinates to determine the distance time corresponding to each of the first type of cleaning robots; Selecting a second type of cleaning robot that meets the power requirement of the task from the first type of cleaning robots according to the journey time and the remaining power; The cleaning robot that takes the least time to travel the distance is determined from the second category of cleaning robots as the target cleaning robot.

7. The method according to claim 5, characterized in that Determining the area where the target garbage object is located according to the target geographic coordinates includes: Determine, according to the target geographic coordinates, a device ID of a first camera device that photographs the target garbage object, wherein a preset comparison relationship exists between the device ID and a corresponding area ID; Determine the area ID of the area where the target garbage object is located according to the device ID and a preset comparison relationship; The querying of the scheduled task list according to the area where the target garbage object is located to determine the scheduled cleaning task time of the corresponding area includes: The scheduled task list is queried according to the area ID to determine the scheduled cleaning task time of the corresponding area.

8. The method according to claim 5, characterized in that After sending the cleaning task instruction to the target cleaning robot, the method includes: Sending a cleaning monitoring instruction to a corresponding first camera device, so that the first camera device monitors the area corresponding to the target geographic coordinates within a preset time according to the cleaning monitoring instruction to determine whether the corresponding target garbage object is cleared, and generates a monitoring result; Receive the monitoring result fed back by the first camera device.

9. A garbage detection and cleaning system, characterized in that: include: A dispatching platform, a cleaning robot and a first camera device, wherein the cleaning robot is provided with a second camera device, and the first camera device is provided at a preset position corresponding to a street area; The scheduling platform is communicatively connected with the cleaning robot and the first camera device; The first camera device is used to acquire a first image obtained by shooting, and perform object recognition processing on the first image. When a target garbage object of a preset category is recognized, the target geographic coordinates of the target garbage object are determined according to the posture of the first camera device and the pixel position of the target garbage object in the first image, a garbage discovery event is generated according to the target geographic coordinates, and the garbage discovery event is reported to the scheduling platform; The scheduling platform is used to query the scheduled task list according to the area where the target garbage object of the received garbage discovery event is located, and determine the scheduled cleaning task time of the corresponding area; Determine the time interval between the current time and the scheduled cleaning task time, and when the time interval exceeds a preset time threshold, determine the target cleaning robot according to the target geographic coordinates and the state information of the cleaning robot, wherein the state information includes the current geographic coordinates; perform path planning according to the current geographic coordinates of the target cleaning robot and the target geographic coordinates, determine the target path, and generate a cleaning task instruction; and send the cleaning task instruction to the target cleaning robot; The cleaning robot is used to go to the position corresponding to the target geographic coordinates according to the corresponding target path according to the received cleaning task instructions. After arriving at the position, it performs real-time detection through the second camera device while cleaning, and cleans all garbage objects of preset categories based on the real-time detection.

10. A storage medium storing computer executable instructions, characterized in that: The computer executable instructions are used to perform the method according to any one of claims 1 to 8 when executed by a processor.

Images