Image intelligent analysis and automatic cleaning trigger system

Abstract

The invention relates to an image intelligent analysis and automatic cleaning trigger system, and belongs to the technical field of monitoring systems, the system comprises a monitoring module, a control module and an alarm module, the monitoring module comprises a plurality of cameras, and the plurality of cameras are used for shooting images and uploading the images to the control module; the control module is used for drawing contours of all objects in the image, judging irregularity of all the contours in the image, judging whether the irregularity of all the contours exceeds a threshold value or not, marking a camera area corresponding to the image where a certain contour is located as a to-be-cleaned area when the judgment result of the contour is yes, and marking the camera area corresponding to the image where the contour is located as the to-be-cleaned area; the alarm module is instructed to give an alarm and the coordinates of the to-be-cleaned area; the computing power requirement is low, and the pertinence to an office scene is high.

Description

Technical Field

[0001] The invention belongs to the technical field of monitoring systems, and in particular relates to an image intelligent analysis and automatic cleaning triggering system. Background Art

[0003] The general solution requires employees or cleaning staff to patrol in person, which is more troublesome. For this reason, the existing technology usually adopts the method of taking pictures of the environment and using deep learning image recognition to identify garbage in the image. The general image recognition and analysis method, such as CN110263675B, a garbage target recognition system and recognition method of a community security robot, the recognition system includes a mapping module and an odometer module and a target recognition module respectively connected to the mapping module signal. The recognition method includes the steps of posture calculation, removal of laser radar data motion distortion, target positioning and point cloud map establishment. The odometer module includes a wheel odometer and a laser odometer for calculating the current posture of the robot and sending the corresponding posture data to the mapping module. The target recognition module includes a camera for using a target recognition algorithm to find the position of garbage waste in the image and publish the position and depth information of the target to the mapping module. The mapping module is used to process the information from the odometer module and the target recognition module and construct a point cloud map corresponding to the garbage waste, which reduces the monitoring pressure of the management personnel and improves the work efficiency of the cleaning staff;

[0004] However, domestic waste is generated in a wide range of areas and in different classification scenarios. For the specific office scenario, the above scheme has a large number of parameters and high computational complexity, and is often difficult to be applied to low-computing power devices. At the same time, lightweight models often find it difficult to achieve a high level of classification accuracy, and they have not made targeted improvements for office scenarios. For example, there are many obstructions such as tables and chairs in office scenarios, and there is a probability that tables, chairs and other objects will be identified as garbage, and the scene-targeting ability is poor. Therefore, a method with low computing power requirements and strong targeting for office scenarios is needed. Summary of the invention

[0005] In order to solve the above-mentioned problems existing in the prior art, the present invention provides an image intelligent analysis and automatic cleaning triggering system, which has the characteristics of low computing power requirements and strong targeting for office scenes.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] An image intelligent analysis and automatic cleaning triggering system, comprising a monitoring module, a control module and an alarm module, wherein the monitoring module comprises a plurality of cameras, and the plurality of cameras are used to capture images and upload the images to the control module;

[0008] The control module is used to draw the contours of each object in the image and determine the irregularity of each contour in the image. The control module determines whether the irregularity of each contour exceeds a threshold value, and when the judgment result of a certain contour is yes, the camera area corresponding to the image where the contour is located is marked as an area to be cleaned, and the alarm module is instructed to issue an alarm and the coordinates of the area to be cleaned.

[0009] As a preferred technical solution of the present invention, the control module is pre-input with a threshold value a0, the control module is used to draw the contours of each object in the image, and judge the irregularity a of each contour in the image, the control module judges whether the irregularity a of each contour exceeds the threshold value a0, and when the judgment result of a certain contour is yes, marks the camera area corresponding to the image where the contour is located as the area to be cleaned, and instructs the alarm module to issue an alarm and the coordinates of the area to be cleaned.

[0010] As a preferred technical solution of the present invention, the control module is used to determine the brightness of the image. The control module lowers the threshold when the brightness exceeds the threshold, and increases the threshold when the brightness is lower than the threshold.

[0011] As a preferred technical solution of the present invention, the control module is used to determine the image brightness L, and the control module determines whether the irregularity a of each contour exceeds the threshold a0×A1, where A1=L0 / L×c, L0 and c are pre-input constants.

[0012] As a preferred technical solution of the present invention, it also includes an input panel, and the input panel is used to input the values ​​of a0, L0 and c.

[0013] As a preferred technical solution of the present invention, the input panel is also used to adjust the judgment result of the contour, and the control module records the contour shape with the adjusted judgment result as an easily mixed shape. Before judging the irregularity of each contour in the image, the control module first judges whether the contour shape belongs to an easily mixed shape, and when the judgment result is yes, the control module corrects the threshold upward.

[0014] As a preferred technical solution of the present invention, some of the cameras are provided with anti-glare films.

[0015] As a preferred technical solution of the present invention, the control module determines whether the irregularity of each contour exceeds a threshold value, and when the judgment result of a certain contour is no, the camera area corresponding to the image where the contour is located is marked as a judgment-free area, and the judgment of the contours in the judgment-free area is stopped.

[0016] The beneficial effects of the present invention are:

[0017] (1) By setting up a monitoring module including several cameras and collecting pictures within the field of view through the cameras, the control module then determines whether the object is garbage based on the irregularity of the contour of the object in the picture, thereby completing the automatic analysis of whether there is garbage in the office scene, and excluding common office supplies with low irregularity from the garbage identification results, thereby reducing the computing power requirements and improving the scene targeting;

[0018] (2) By making the control module judge the brightness of the image, and making the control module lower the threshold when the brightness exceeds the threshold, and raise the threshold when the brightness is lower than the threshold before judging the contour irregularity, the judgment standard is relaxed when the lighting conditions are poor and the confidence level of the contour irregularity in judging whether the object is garbage is low, thereby reducing the false alarm rate. When the lighting conditions are better and the contour is more representative of the object shape, the judgment standard is raised, thereby improving the efficiency of garbage recognition;

[0019] (3) By using the input panel to adjust the judgment result of the contour, registering the easily confusable shapes, and correcting the threshold downward when judging the easily confusable shapes, the probability of missed judgment is reduced and the recognition accuracy is improved;

[0020] (4) By enabling the control module to determine whether the irregularity of each contour exceeds a threshold, and when the judgment result of a certain contour is negative, marking the camera area corresponding to the image where the contour is located as a judgment-free area, and stopping the judgment of the contours in the judgment-free area, the system is prevented from judging irregularly shaped personal items placed on the desk as garbage, thereby reducing the probability of misjudgment and improving recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0022] Figure 1 is a control loop block diagram of the present invention; DETAILED DESCRIPTION

[0023] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0024] See also Figure 1 , an image intelligent analysis and automatic cleaning triggering system, including a monitoring module, a control module and an alarm module, the monitoring module includes a plurality of cameras, and the plurality of cameras are used to capture images and upload the images to the control module;

[0025] Specifically, several cameras are set up in advance at various locations in the office to ensure that the cameras are aimed at several locations where garbage is likely to appear. Each camera is electrically connected to a control module. The control module instructs the camera to start and stop. The camera collects images within its own field of view and uploads them to the control module.

[0026] At the same time, the control module is used to draw the contours of each object in the image and determine the irregularity of each contour in the image. The control module determines whether the irregularity of each contour exceeds a threshold value, and when the determination result of a certain contour is yes, marks the camera area of ​​the image corresponding to the contour as the area to be cleaned, and instructs the alarm module to issue an alarm and the coordinates of the area to be cleaned;

[0027] Since there are fewer types of garbage in the office area, most of them are paper balls, food scraps, food packaging, spilled liquids, etc. In a few cases, they are fallen leaves of green plants used for office decoration. Most of them are entities with certain irregular shapes. However, the shapes of common office supplies such as desks, computers and folders are relatively regular. Therefore, most of the garbage in the camera monitoring area can be identified directly by judging the irregularity of the object contour in the image, without the need to use deep learning target recognition algorithms such as R-CNN, which require high computing power, for complex calculations;

[0028] In order to reduce the amount of calculation of the control module, in this embodiment, the camera uploads a photo every 10 seconds. Under the premise of timely cleaning, the types of garbage in the office environment that stay on the ground within 10 seconds will only have a slight impact on the perception. Therefore, the frequency of the camera monitoring and uploading environmental images can be appropriately reduced, thereby reducing the amount of calculation of the control module per unit time;

[0029] In actual use, the time points for uploading pictures by each camera can be staggered according to the situation to avoid multiple pictures to be processed being piled up in the waiting queue of the control module;

[0030] Specifically, the irregularity in this embodiment refers to the irregularity relative to the shape of a circle, a rectangle, a parallelogram and a common seat contour. When it is specifically executed, the control module pre-inputs the six-view contours of the circle, rectangle and parallelogram, as well as the common seat contour, as reference graphics, and then draws a rectangular area on each side of the reference graphics with a width of 2% of the length of the side and a length equal to the side. The annular or frame-shaped area enclosed by the rectangular area along the standard graphic is used as the standard graphic;

[0031] When processing each contour, first compare the contour with a circle, a rectangle, and a parallelogram in turn. When comparing, scale the contour size deformation to be similar to the standard figure. Then find the standard figure that is closest to the contour. Subsequently, compare the contour with the closest standard figure and calculate the ratio of the total length of the part where the contour exceeds the standard figure to the total length of the edge. This ratio is the irregularity;

[0032] According to the definition of the above irregularity, it is easy to prove that the interval of the irregularity is [0, 1);

[0033] To cooperate with the judgment of the irregularity, the control module pre-enters a threshold value a0, 0 < a0 < 1, and a0 is a constant;

[0034] Compared with items such as desks and chairs, the shapes of common office garbage such as paper balls, food residues, food packaging, and spilled liquids are relatively irregular. When the contours of such garbage are compared with the standard figure, there is a greater probability that the longer sides of the contour exceed the standard figure. For example, the shapes of office supplies such as ordinary desks and monitors are regular. After the contours in the captured pictures are scaled and deformed to be similar to the standard figure, the entire contour falls within the range of the standard figure. However, the shapes of the garbage are more irregular. After the contours in the captured pictures are scaled and deformed to be similar to the standard figure, some contours fall outside the range of the standard figure. Therefore, when the irregularity of a certain contour is larger and exceeds the threshold, there is a greater probability that it is garbage. At this time, the control module determines that there is garbage in the shooting range of this camera and instructs the alarm module to issue an alarm and the coordinates of the area to be cleaned;

[0035] Specifically, the control module pre-numbers several cameras. When each camera uploads an image, it synchronously uploads its own number. When a certain camera is determined to have garbage in the shooting range, the control module reads the corresponding number of this picture and sends it to the alarm module. The alarm module displays which numbered camera's corresponding area is the area to be cleaned;

[0036] The control module calculates the irregularity a of each contour in the image. When the irregularity a is close to or equal to 0, it means that the proportion of the total length of the part where the contour exceeds the standard figure in the total length of the contour edge is small, and the contour shape is close to the standard shape to a large extent. The probability of being garbage is low, and there is a greater probability of being office supplies;

[0037] When the irregularity a is close to 1 or greater than a0, it means that the proportion of the total length of the part where the contour exceeds the standard figure in the total length of the contour edge is large, and the contour shape deviates from the standard shape to a large extent. The probability of being garbage is large, and the probability of being office supplies is low. At this time, the control module determines that there is garbage in the field of view of the camera that captured this image and needs to be cleaned. The control module reads the corresponding number of this picture and sends it to the alarm module. The alarm module displays which numbered camera's corresponding area is the area to be cleaned;

[0038] By setting up a monitoring module including several cameras and collecting pictures within the field of view through the cameras, the control module then determines whether the object is garbage based on the irregularity of the object's contour in the picture, thereby completing an automated analysis of whether there is garbage in the office scene, and excluding common office supplies with lower irregularities from the garbage identification results, thereby reducing the computing power requirements while improving the scene targeting.

[0039] In the above process, the lighting conditions in the office are different in different time periods and under different circumstances. When the lighting conditions are relatively good, the color difference of each object in the picture taken by the camera relative to other objects is more obvious, the outline of each object is clearer, and the outline of each object can more accurately reflect the true shape of the object. Therefore, the irregularity of the outline can reflect the irregularity of the object to a greater extent, and the irregularity of the outline can prove to a greater extent whether the object is garbage. On the contrary, when the lighting conditions are poor, the color difference of each object in the picture taken by the camera relative to other objects is relatively unobvious, and the clarity of the outline of each object is low. The control module may not be able to accurately identify the outline, and the irregularity of the outline is less representative of the irregularity of the object. Therefore, the irregularity of the outline has a low confidence in the judgment of whether the object is garbage.

[0040] Therefore, the worse the lighting conditions in the office, the lower the brightness of the picture, and the higher the probability of misreporting normal items as garbage. It is necessary to appropriately relax the judgment criteria for whether it is garbage and increase the irregularity threshold so that the irregularity of the contour needs to be higher before it can be judged as garbage, thereby reducing the false alarm rate. When the lighting conditions are better, the brightness of the picture is higher, and the contour is clearer and more representative of the true shape, the threshold judgment criteria should be increased to improve the efficiency of garbage recognition.

[0041] To this end, the control module is used to determine the brightness of the image, the control module lowers the threshold when the brightness exceeds the threshold, and the control module increases the threshold when the brightness is below the threshold;

[0042] Specifically, after the control module determines the image brightness L, the control module determines whether the irregularity a of each contour exceeds a threshold value a0×A1, where A1=L0 / L×c, and L0 and c are pre-input constants;

[0043] When the brightness L of the image is small, less than L0, it means that the lighting conditions are poor, and the probability of misreporting normal items as garbage is higher. It is necessary to appropriately relax the judgment standard of whether it is garbage. At this time, the value of A1=L0 / L×c and the value of a0×A1 are higher, and a is more difficult to be greater than the judgment standard. When the control module uses whether the irregularity a exceeds the threshold a0×A1 to determine whether the object corresponding to each contour is garbage, it is completed to relax the judgment standard of whether it is garbage when the lighting conditions are poor;

[0044] Similarly, when the brightness L of the image is larger than L0, it means that the lighting conditions are good, the contours are clearer, and the probability of misreporting normal objects as garbage is low. It is necessary to appropriately narrow the judgment standard for whether it is garbage. At this time, the value of A1=L0 / L×c and the value of a0×A1 are low, and a is more likely to be greater than the judgment standard. When the control module uses whether the irregularity a exceeds the threshold a0×A1 to determine whether the object corresponding to each contour is garbage, the judgment standard for whether it is garbage is narrowed when the lighting conditions are good.

[0045] By making the control module used to judge the brightness of the image, and making the control module lower the threshold when the brightness exceeds the threshold, and increase the threshold when the brightness is lower than the threshold before judging the contour irregularity, the judgment standard is relaxed when the lighting condition is poor and the confidence level of the contour irregularity in judging whether the object is garbage is low, thereby reducing the false alarm rate. When the lighting condition is better and the contour can better represent the shape of the object, the judgment standard is increased, thereby improving the efficiency of identifying garbage.

[0046] In order to facilitate the input of the values ​​of a0, L0 and c, an input panel is also included. The input panel is used to input the values ​​of a0, L0 and c. Specifically, the input panel includes at least a computer electrically connected to the control module.

[0047] In the process of automatically performing garbage identification, there is a probability that the judgment is inaccurate, non-garbage is judged as garbage, or garbage exists but cannot be judged and an alarm is issued. At this time, manual intervention is required in the judgment process to correct the judgment result and improve the judgment accuracy. For this purpose, the input panel is also used to adjust the judgment result of the contour. The control module records the contour shape with the adjusted judgment result as an easily mixed shape. Before the control module judges the irregularity of each contour in the image, it first judges whether the contour shape belongs to an easily mixed shape. When the judgment result is yes, the control module corrects the threshold upward;

[0048] Specifically, after performing automatic judgment on a certain number of images, the control module draws the contour of the image currently performing contour irregularity judgment and uploads it to the input panel. The operator observes the contours in the input panel and judges whether each contour in the image is garbage, generates a manual judgment result, and transmits the manual judgment result back to the control module. The control module compares the manual judgment result with the judgment result automatically performed by itself to check whether there is any inconsistency;

[0049] If there is an inconsistency, it means that it is garbage but is ignored by the automatic recognition process. The appearance of such garbage is easily missed. At this time, the control module determines the inconsistent contour shape as an easily mixed shape and stores it in the database;

[0050] When the control module judges the contour in the subsequent image, the control module first judges whether the contour belongs to an easily mixed shape. The judgment method is to compare whether the similarity between the contour to be judged and the easily mixed shapes in the database is within the threshold. If the judgment result is yes, it means that this contour is easy to be misjudged. The control module corrects the threshold downward to reduce the probability of missed judgment.

[0051] By using the input panel to adjust the judgment result of the contour, registering the easily mixed shapes, and correcting the threshold downward when judging the easily mixed shapes, the probability of missed judgment is reduced and the recognition accuracy is improved.

[0052] In the process of the camera acquiring images, there is a probability that the imaging will be interfered with by the glare of the lens, which will eventually interfere with the generation and judgment of the contour. For this reason, each camera is provided with an anti-glare film.

[0053] In the above judgment process, the irregularity of the contour is judged to eliminate the conventional items with relatively regular shape contours. However, in the actual office process, various personal items, such as fans or decorations, are placed on the desk. Such items cannot be judged for irregularity using reference graphics. Such items usually appear on the desk. From the perspective of the camera usually shooting from a high place, such sundries are within the regular graphic contour of the desk within the camera's field of view. In order to eliminate such sundries on the desk from garbage judgment and prevent false alarms, the control module judges whether the irregularity of each contour exceeds the threshold, and when the judgment result of a certain contour is negative, the camera area corresponding to the image where this contour is located is marked as a judgment-free area, and the contours in the judgment-free area are stopped from being judged;

[0054] Specifically, when calculating a certain uploaded image, the control module calculates the irregularity a of each contour in the image. When the irregularity of a certain contour is less than or equal to the threshold, it means that the object corresponding to this contour is an office supply. Regardless of the size of the contour irregularity, the items on it are most likely personal items or sundries. At this time, such items need to be removed from the garbage judgment to prevent false alarms. The control module determines the area covered by the contour in this image as a judgment-free area. Then, before the control module judges the contours of other objects, it first determines whether this contour is in the judgment-free area in the image. When the judgment result is yes, the judgment is not performed. When the judgment result is no, the irregularity judgment is performed. At the same time, for the contours that are judged as having an irregularity exceeding the threshold before the judgment-free area, it is determined whether this or such contours are in the judgment-free area. When they are in the judgment-free area, the control module deletes the previous judgment result.

[0055] By enabling the control module to determine whether the irregularity of each contour exceeds a threshold, and when a contour is determined to be negative, marking the camera area corresponding to the image of the contour as a judgment-free area, and stopping the judgment of the contours in the judgment-free area, the system is prevented from judging irregularly shaped personal items placed on the desk as garbage, thereby reducing the probability of misjudgment and improving recognition accuracy.

[0056] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. An image intelligent analysis and automatic cleaning triggering system, characterized in that: It includes a monitoring module, a control module and an alarm module. The monitoring module includes a plurality of cameras, and the plurality of cameras are used to take images and upload the images to the control module; The control module is used to draw the contours of each object in the image and determine the irregularity of each contour in the image. The control module determines whether the irregularity of each contour exceeds a threshold value, and when the judgment result of a certain contour is yes, the camera area corresponding to the image where the contour is located is marked as an area to be cleaned, and the alarm module is instructed to issue an alarm and the coordinates of the area to be cleaned.

2. The image intelligent analysis and automatic cleaning triggering system according to claim 1 is characterized by: The control module is pre-input with a threshold value a0. The control module is used to draw the contours of each object in the image and determine the irregularity a of each contour in the image. The control module determines whether the irregularity a of each contour exceeds the threshold value a0, and when the judgment result of a certain contour is yes, marks the camera area corresponding to the image where the contour is located as the area to be cleaned, and instructs the alarm module to issue an alarm and the coordinates of the area to be cleaned.

3. The image intelligent analysis and automatic cleaning triggering system according to claim 2 is characterized by: The control module is used to determine the brightness of the image. When the brightness exceeds the threshold, the control module lowers the threshold, and when the brightness is lower than the threshold, the control module increases the threshold.

4. The image intelligent analysis and automatic cleaning triggering system according to claim 3 is characterized by: The control module is used to determine the image brightness L. The control module determines whether the irregularity a of each contour exceeds a threshold value a0×A1, wherein A1=L0 / L×c, and L0 and c are pre-input constants.

5. The image intelligent analysis and automatic cleaning triggering system according to claim 4 is characterized by: An input panel is also included, and the input panel is used to input the values ​​of a0, L0 and c.

6. The image intelligent analysis and automatic cleaning triggering system according to claim 5 is characterized by: The input panel is also used to adjust the judgment result of the contour. The control module records the contour shape with the adjusted judgment result as an easily mixed shape. Before judging the irregularity of each contour in the image, the control module first judges whether the contour shape belongs to an easily mixed shape. When the judgment result is yes, the control module corrects the threshold upward.

7. The image intelligent analysis and automatic cleaning triggering system according to claim 1 is characterized by: Some of the cameras are provided with anti-glare films.

8. The image intelligent analysis and automatic cleaning triggering system according to claim 1 is characterized by: The control module determines whether the irregularity of each contour exceeds a threshold value, and if the judgment result of a certain contour is no, marks the camera area corresponding to the image where the contour is located as a judgment-free area, and stops judging the contours in the judgment-free area.

Images