A gis early warning method based on cleaning result analysis of a cleaning robot

Abstract

The application belongs to the technical field of image processing, and particularly relates to a GIS early warning method based on cleaning result analysis of a cleaning robot, which comprises the following steps: acquiring original image data collected by a camera of the cleaning robot; analyzing the influence of perspective tilt according to the original image data, determining feature point deviation conditions, and generating distortion distribution information; correcting the original image data according to the distortion distribution information, and generating first image data after correction; extracting edge features for the first image data, and acquiring contour information related to cleaning area division; dividing the cleaning area according to the contour information, and determining cleaning state distribution information of each area block; and generating a GIS blind area distribution map according to the cleaning state distribution information. The application effectively solves the problems of image distortion and inaccurate area division in the vision system of the cleaning robot, improves the cleaning efficiency and quality, and provides a reliable basis for subsequent cleaning path planning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a GIS early warning method based on cleaning result analysis of a cleaning robot. BACKGROUND

[0002] In the prior art, although some cleaning robots are equipped with image acquisition devices to assist cleaning, most of these devices can only simply record image data during the cleaning process and cannot perform in-depth analysis and processing on the image data. For example, a Chinese patent with publication number CN116128801A proposes an image cleanliness evaluation method based on multi-task learning and related devices, which includes inputting a to-be-evaluated image into a feature extraction module in the evaluation network model, determining a plurality of feature maps of the to-be-evaluated image through the feature extraction module, inputting the plurality of feature maps into a prediction module in the evaluation network model, and determining a mask map and a cleanliness category corresponding to the to-be-evaluated image through the prediction module. The application extracts a plurality of feature maps through the feature extraction module, and then determines a mask map and a cleanliness category based on the plurality of feature maps through the prediction module. The prediction module uses the clean feature performance of the instance feature enhanced image for determining the mask map, focuses on the division of the organ to which the image belongs and the cleanliness evaluation of the image, so as to quickly determine a video image whose cleanliness meets a predetermined requirement, thereby preventing the subsequent lesion segmentation network from being disturbed by a noise scene, and reducing the calculation complexity of lesion detection. A similar prior art is a Korean patent application with publication number KR20220160912A, which proposes a device for estimating a dispensing path using image analysis, acquires an image of a sample to be analyzed from a camera according to an image acquisition unit, processes image noise included in the acquired image, and determines the outline of the sample using geometric information, a path extractor is used to extract the movement path of the dispenser, divide the extracted movement path of the dispenser into a plurality of regions, and track the inspection direction of each divided region to detect the boundary line of the movement path of the dispenser, a coordinate estimation unit estimates the coordinates of the movement path of the dispenser using the detected boundary line, corrects the image by performing a calibration operation on the sample to execute a target region, minimizes the error on the movement path of the dispensing nozzle, and determines the dispensing path using the obtained image. Therefore, there is a lack of a GIS early warning method for cleaning effect analysis in the prior art, which can accurately evaluate the cleaning effect, identify the cleaning blind area, and correct the image distortion caused by the tilt of the viewing angle, so that the cleaning effect of the cleaning robot in a complex environment is difficult to guarantee. SUMMARY

[0003] The application provides a GIS early warning method based on cleaning result analysis of a cleaning robot, which is used for accurately evaluating cleaning effect, identifying cleaning blind area, and correcting image deformation caused by visual angle tilt, thereby improving cleaning efficiency and quality and providing reliable basis for subsequent cleaning path planning.

[0004] In a first aspect, the application provides a GIS early warning method based on cleaning result analysis of a cleaning robot, which comprises:

[0005] obtaining original image data collected by a camera of the cleaning robot;

[0006] analyzing visual angle tilt influence according to the original image data, determining feature point deviation condition, and generating distortion distribution information;

[0007] correcting the original image data according to the distortion distribution information to generate first image data after correction;

[0008] extracting edge features for the first image data to obtain contour information related to cleaning area division;

[0009] dividing the cleaning area according to the contour information to determine cleaning state distribution information of each area block, fusing the cleaning state distribution information with geographic space information to generate a GIS blind area distribution map, and marking early warning information.

[0010] As a preferred technical solution of the application, the analysis of visual angle tilt influence according to the original image data, the determination of feature point deviation condition, and the generation of distortion distribution information comprise:

[0011] detecting feature points for the original image data to obtain position information of multiple feature points, analyzing deviation of the position information of the feature points from a preset standard position to determine influence degree of visual angle tilt on the feature points, calculating deviation amount of each feature point according to the influence degree to generate corresponding deviation distribution data, constructing the distortion distribution information according to the deviation distribution data, and recording image deformation condition caused by visual angle tilt through the distortion distribution information to provide basis for correction.

[0012] As a preferred technical solution of the application, the correction of the original image data according to the distortion distribution information to generate first image data after correction comprises:

[0013] According to the distortion distribution information, perspective transformation parameters are determined; the original image data is geometrically corrected by using the perspective transformation parameters to adjust the positions of each pixel point in the image; the deformation caused by the perspective tilt is eliminated through the correction processing to generate image data conforming to the actual scene scale; the generated image data is defined as the first image data; the quality verification is performed on the first image data to confirm whether the correction effect reaches the preset standard.

[0014] As a preferred technical solution of the present application, the edge features are extracted from the first image data, and the contour information related to the cleaning area division is obtained, including:

[0015] The first image data is preprocessed to enhance the edge features in the image, and an edge detection algorithm is used to extract the edge information in the first image data.

[0016] According to the edge information, contour data related to the cleaning area is constructed, and continuity analysis is performed on the contour data to determine whether there is a contour break phenomenon, and the boundary features of the cleaning area are recorded through the contour information.

[0017] As a preferred technical solution of the present application, the cleaning area is divided according to the contour information, including:

[0018] If there is a break phenomenon in the contour information, the broken part is repaired to generate continuous contour data, and the cleaning area is divided by using an image segmentation technology according to the repaired contour data to generate a plurality of region blocks.

[0019] The boundary of each region block is verified to confirm the accuracy of the division result, and the accurate distribution data of the cleaning area is generated through the combination of the contour information and the segmentation result, and the position information of each region block is recorded according to the distribution data.

[0020] As a preferred technical solution of the present application, the cleaning state distribution information of each region block is determined, including:

[0021] The gray scale analysis is performed on each region block after division to obtain the gray scale distribution data of the region block, the cleaning state of the region block is determined according to the gray scale distribution data, and the corresponding state information is generated; the cleaning degree of each region block is recorded through the state information, and the region block that does not reach the standard is determined by comparing the cleaning degree with a preset threshold, and the cleaning state distribution information is summarized as a state data set of the region block.

[0022] As a preferred technical solution of the present application, the GIS blind area distribution map is generated according to the cleaning state distribution information, including:

[0023] Based on the cleanliness distribution information, the location data of areas that have not reached the preset threshold are statistically analyzed. Spatial mapping is performed on the location data to generate corresponding blind spot distribution data. A visual distribution map is constructed using the blind spot distribution data.

[0024] The location and extent of cleaning blind spots are recorded according to the distribution map. The GIS blind spot distribution map is then verified to confirm its consistency with the actual cleaning status.

[0025] As a preferred embodiment of the present invention, the step of generating a GIS blind spot distribution map based on the cleanliness status distribution information further includes:

[0026] Data filtering is performed on the cleaning status distribution information to extract area block information that has not reached a preset threshold. The location and area of ​​blind spots are determined based on the area block information. A preliminary structure of the GIS blind spot distribution map is generated using the location and area data. The preliminary structure is optimized and adjusted to generate the final GIS blind spot distribution map. The locations of areas requiring special attention in the cleaning area are recorded based on the GIS blind spot distribution map, and warning information is marked.

[0027] Secondly, the present invention also provides a GIS early warning system based on cleaning result analysis of a cleaning robot, the system comprising:

[0028] A camera is used to capture raw image data;

[0029] The analysis module is used to analyze the effect of viewing angle tilt based on the original image data, determine the offset of feature points, and generate distortion distribution information;

[0030] The correction module is used to correct the original image data according to the distortion distribution information to generate corrected first image data;

[0031] The extraction module is used to extract edge features from the first image data and obtain contour information related to the division of the clean area;

[0032] The segmentation module is used to segment the cleaning area according to the contour information and determine the cleaning status distribution information of each area block;

[0033] The generation module is used to generate a GIS blind spot distribution map based on the cleanliness status distribution information.

[0034] Thirdly, the present invention also provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the above-described method.

[0035] The beneficial effects of this invention are as follows:

[0036] The application discloses a GIS early warning method based on cleaning result analysis of a cleaning robot, and realizes accurate division of a cleaning area and generation of a GIS blind area distribution map by performing perspective tilt correction and edge feature extraction on original images collected by a camera; the method first analyzes offset conditions of feature points in the original images, generates distortion distribution information, and performs perspective transformation correction according to the distortion distribution information to eliminate deformation caused by perspective tilt; then, edge features of the corrected images are extracted, a cleaning area contour is obtained, and region division is performed; by analyzing gray scales of each region block, a cleaning state is judged, and finally, a visual GIS blind area distribution map fused with geographic information is generated. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort based on these drawings.

[0038] Figure 1 The flow chart of the GIS early warning method based on cleaning result analysis of a cleaning robot in the embodiment of the present application;

[0039] Figure 2 The flow chart of the acquisition method of the first corrected image in the embodiment of the present application;

[0040] Figure 3 The flow chart of the acquisition method of the contour information related to the cleaning area division in the embodiment of the present application;

[0041] Figure 4 The structural diagram of the GIS early warning system based on cleaning result analysis of a cleaning robot in the embodiment of the present application. DETAILED DESCRIPTION

[0042] The embodiments of the present application provide a GIS early warning method and system based on cleaning result analysis of a cleaning robot and a storage medium. The terms "first", "second", "third", "fourth" and the like (if any) in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" or "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0043] For ease of understanding, the specific flow of the embodiments of the present application is described as follows, and a GIS early warning method based on cleaning result analysis of a cleaning robot in the embodiments of the present application includes: Figure 1

[0044] Step S1: Obtain original image data collected by a camera of a cleaning robot;

[0045] Step S2: According to the original image data, analyze the influence of the perspective tilt, determine the feature point offset situation, and generate distortion distribution information;

[0046] ​Specifically, the original image data collected by the robot camera, for example, when the cleaning robot is working, its camera captures the surrounding environment images at a frequency of 30 frames per second, covering a field of view of about 120 degrees in front. These image data are usually stored in RGB format, with a resolution of 1280x720 pixels. Through the built-in image sensor, the robot can record the ground conditions of the cleaning area in real time, including obstacles, stain distribution, etc. It should be noted that the purpose of obtaining the original image data is to provide a basis for subsequent perspective correction and feature analysis, ensuring the integrity and accuracy of the data. This can effectively avoid analysis deviation caused by missing or blurred images, and improve the accuracy of cleaning path planning. According to the analysis of the influence of perspective tilt on the original image data, the feature point offset condition is determined, and the distortion distribution information is generated, for example, during the movement of the cleaning robot, due to uneven ground or changes in the robot's posture, the camera perspective may tilt 10 to 15 degrees, causing the position of objects in the image to shift. In one possible implementation, key feature points in the image can be identified, such as straight edges of the ground or corners of fixed objects, the position changes of these feature points in different frames of images are analyzed to determine the degree of influence of the tilt angle on the image. Specifically, assuming that a straight line should be horizontal under ideal perspective, but in the actual image it presents a 5-degree tilt, then by comparing the difference between the ideal position and the actual position, the offset of the feature point is calculated to be about 20 pixels. Further, the offset data of multiple feature points are aggregated to form distortion distribution information covering the entire image area. This distribution information can intuitively reflect which areas in the image are more affected by perspective tilt, providing a basis for subsequent correction. The above technical solution can accurately locate the distortion area, avoid unnecessary computational burden caused by global correction, and improve the targeting of correction effect, ensuring that the cleaning robot can plan the path based on accurate image data, reducing the risk of misjudgment or omission of cleaning areas.

[0047] Step S3: correcting the original image data according to the distortion distribution information to generate corrected first image data;

[0048] Step S4: extracting edge features for the first image data to obtain contour information related to cleaning area division;

[0049] Specifically, the original image data is corrected according to the distortion distribution information to generate corrected first image data. This process is mainly realized by perspective transformation technology to correct the geometry of the image, for example, in the image collected by the cleaning robot, due to the tilt of the camera perspective, the original image may present trapezoidal distortion, with near objects appearing larger and distant objects appearing smaller. By analyzing the distortion distribution information, the offset of the feature points in the image can be determined, and a perspective transformation matrix can be constructed to correct the trapezoidal image to a rectangular image.

[0050] Specifically, assuming that a feature point in the original image is shifted from the coordinates (100, 150) to (120, 160), it can be corrected back to the correct position through the transformation matrix adjustment, which can effectively eliminate the influence of the perspective tilt and ensure the accuracy of subsequent edge feature extraction. The above technical solution provides a reliable data basis for cleaning area division to make the image closer to the geometric shape of the real scene, thereby improving the accuracy of robot path planning. The edge features are extracted from the corrected first image data to obtain contour information related to cleaning area division, and this process is usually implemented using edge detection technology.

[0051] Step S5: dividing the cleaning area according to the contour information, determining the cleaning state distribution information of each region block, fusing the cleaning state distribution information with the geographic space information to generate a GIS blind area distribution map, and marking the warning information.

[0052] Specifically, the cleaning area is divided according to the contour information, and the cleaning state distribution information of each region block is determined. This process first needs to analyze the extracted contour information, which usually refers to the boundary features of objects or ground in the cleaning area, such as corners, furniture edges, etc. Through these boundary features, the entire cleaning area can be divided into multiple small region blocks. The purpose of division is to more finely assess the cleaning state of each region block, such as dust accumulation degree or stain coverage range. After division, the cleaning state distribution information of each region block can be determined through image gray scale analysis or sensor data acquisition, such as the dust coverage rate of a region block may be 30%, while another region block may be as high as 70%. This helps to identify the key cleaning areas and ensures reasonable resource allocation, improving cleaning efficiency. The cleaning state distribution information and the corresponding geographic location information are fused to generate a GIS blind area distribution map. The positions of region blocks that do not meet the preset cleaning threshold are marked as warning information and marked in the above GIS blind area distribution map. For example, areas with a cleaning coverage rate of less than 50% can be defined as blind areas. The process of generating a GIS blind area distribution map can include mapping the cleaning state data with the geographic space position of the region block, thereby intuitively showing which areas need to be paid attention to. For example, in a 100 square meter cleaning area, there may be 10 region blocks marked as blind areas, of which 5 are located in the corners, 3 are under objects, and 2 are in the central zone. The above technical solution helps cleaning equipment or personnel quickly locate problem areas, optimizes cleaning paths, and reduces repetitive work. More importantly, the GIS blind area distribution map can also provide data support for subsequent cleaning strategy adjustment, such as increasing cleaning frequency for corner blind areas or using smaller cleaning tools for areas under furniture, thereby improving overall cleaning effect.

[0053] Further, the feature point offset condition is determined according to the analysis of the influence of the perspective tilt on the original image data, and distortion distribution information is generated, comprising:

[0054] The feature point detection is performed on the original image data to obtain the position information of a plurality of feature points. The deviation of the position information of the feature points from a preset standard position is analyzed to determine the influence degree of the perspective tilt on the feature points. The offset amount of each feature point is calculated according to the influence degree, and corresponding offset distribution data is generated. The distortion distribution information is constructed according to the offset distribution data. The image deformation caused by the perspective tilt is recorded through the distortion distribution information, thereby providing a basis for correction.

[0055] Specifically, feature point detection is performed on the original image data to obtain position information of multiple feature points. For example, in an image captured by a cleaning robot camera, key points in the image such as corner points, object edges, and the like can be identified through corner point detection technology. These feature points are parts of the image that have significant features, usually regions with large pixel value changes. By detecting, the specific position information of these points in the image coordinate system can be obtained, such as the coordinates of a certain corner point being 100 on the horizontal axis and 150 on the vertical axis. Such position information provides basic data for subsequent analysis, with the goal of determining the distribution of key regions in the image and laying the foundation for evaluating the impact of perspective tilt. The deviation of the position information of the feature points from the preset standard position is determined to determine the degree of influence of perspective tilt on the feature points. For example, under an ideal non-tilted perspective, a certain feature point should be located at coordinates 100 on the horizontal axis and 100 on the vertical axis, but in actual detection, its position is found to be 110 on the horizontal axis and 90 on the vertical axis, indicating that there is a certain deviation. By comparing the actual positions of multiple feature points with the standard positions, the direction and size of the deviation can be analyzed to determine the specific impact of perspective tilt, such as the tilt angle possibly causing the feature points to shift to the upper right as a whole. This analysis helps to quantify the impact of tilt on the image content and provides a basis for subsequent correction. The offset of each feature point is calculated according to the degree of influence to generate corresponding offset distribution data. For example, in the above analysis, if it is found that the feature points shift to the upper right as a whole, the specific offset of each point can be calculated, such as a certain point having an offset of 10 on the horizontal axis and -10 on the vertical axis. By statistically analyzing the offset of all feature points, an offset distribution data table can be generated to record the offset trend of each region. This data table can directly reflect the deformation of different regions in the image and provide a direct basis for constructing distortion distribution information. Distortion distribution information is also constructed according to the offset distribution data for subsequent correction processing. For example, the offset distribution data is further processed to generate a distortion distribution map that marks the deformation degree of each region in the image, such as the central region having less distortion and the edge region having more distortion. This distortion distribution information can guide perspective transformation to ensure that different regions are adjusted according to their deformation during correction, thereby improving the correction effect. The distortion distribution information records the image deformation caused by perspective tilt to provide a basis for correction. For example, based on the distortion distribution map, the image stretching or compression region caused by perspective tilt can be clearly recorded, such as the top of the image being compressed by 20%. The above technical solution not only helps to understand the deformation reason but also provides accurate parameters for subsequent correction algorithms to ensure that the corrected image is closer to the real scene, thereby improving the perception accuracy of the cleaning robot for the environment.

[0056] Further, the distortion distribution information is used to correct the original image data to generate corrected first image data, as shown in Figure 2 The method comprises the following steps:

[0057] According to the distortion distribution information, the perspective transformation parameters are determined; the original image data is geometrically corrected by using the perspective transformation parameters to adjust the positions of each pixel point in the image; the deformation caused by the visual angle tilt is eliminated through the correction processing to generate image data conforming to the actual scene scale; the generated image data is defined as the first image data; the quality verification is performed on the first image data to confirm whether the correction effect reaches the preset standard.

[0058] Specifically, according to the distortion distribution information, the perspective transformation parameters are determined, for example, in the original image obtained by the camera of the cleaning robot, due to the visual angle tilt, the straight line in the image may present as a curved or trapezoidal deformation, by analyzing the offset of the feature points, the distortion distribution information can be obtained, and then the perspective transformation parameters are calculated, these parameters usually include the mapping relationship of the four corner points in the image, which is used to adjust the deformed image to the normal scale, assuming that the camera is set at a preset visual angle tilt angle and a preset installation height, by analyzing the feature point offset, the parameter value of the transformation matrix is determined, which is used for subsequent correction, this method can effectively quantify the distortion degree, and provide a basis for correction, the original image data is geometrically corrected by using the perspective transformation parameters to adjust the positions of each pixel point in the image, for example, in an image with a resolution of a set value, the determined transformation parameters are used to remap the coordinates of each pixel point, and the trapezoidal area under the tilted visual angle is adjusted to a rectangular area, this process ensures that the object scale in the image is close to the real scene, and avoids distance misjudgment caused by the visual angle problem, such correction is crucial for the cleaning robot to identify the position of the obstacle, the deformation caused by the visual angle tilt is eliminated through the correction processing to generate image data conforming to the actual scene scale, for example, after correction, the originally narrow corridor area due to the visual angle tilt is restored to the real width scale in the image, the width scale displayed by the corrected image is consistent with the actual one, such processing helps the robot to more accurately plan the cleaning path and improve the work efficiency, the generated image data is defined as the first image data, for example, the corrected image is stored as a new data file and named as the first image data, which is used for subsequent edge feature extraction and region division analysis, this naming and definition facilitate the internal data management of the system and ensure the clear process, the quality verification is performed on the first image data to confirm whether the correction effect reaches the preset standard, by comparing the straightness and scale accuracy of the feature points in the corrected image, it is checked whether it conforms to the preset error range, if the correction effect is found to be poor, the transformation parameters can be adjusted again, the above technical solution can ensure that the image quality meets the needs of the cleaning region division, and effectively improves the reliability of the subsequent analysis.

[0059] Further, the edge features of the first image data are extracted to obtain contour information related to the cleaning region division, such as Figure 3 As shown in the figure, it includes:

[0060] preprocessing the first image data, enhancing edge features in the image, and extracting edge information in the first image data using an edge detection algorithm;

[0061] constructing contour data related to the cleaning area according to the edge information, performing continuity analysis on the contour data, determining whether there is a contour breaking phenomenon, and recording boundary features of the cleaning area through the contour information.

[0062] Specifically, the preprocessing of the first image data enhances the edge features in the image. For example.

[0063] In one possible implementation, the edge details can be highlighted by adjusting the contrast and brightness of the image. Assuming that the gray value range of the original image is 0 to 255, it can be adjusted to a more concentrated range such as 100 to 200, so that the gray value change of the edge part is more obvious. The purpose of this is to make it easier for the subsequent edge detection algorithm to capture clear boundary information and effectively improve the detection accuracy. An edge detection algorithm is used to extract the edge information in the first image data. For example, a classic edge detection method can be used to identify areas with sharp gray value changes by performing gradient calculation on the image. Assuming that the gray value in a certain area changes from 100 to 200, this change will be marked as a potential edge point. This method can quickly locate the boundary line in the image, laying the foundation for subsequent contour construction, and has the advantage of being able to adapt to image feature extraction under different lighting conditions. According to the edge information, contour data related to the cleaning area is constructed. In one possible implementation, the detected edge points are connected into continuous lines to form a closed or semi-closed contour. Assuming that the edge points of a certain area are distributed in the shape of an approximate rectangle, the boundary of a cleaning area can be defined. This construction method helps to clearly divide the cleaning range, avoids missing or repeated areas, and improves the rationality of the division. Continuity analysis is performed on the contour data to determine whether there is a contour breaking phenomenon. The breaking point is identified by checking the connectivity of the contour lines. Assuming that a certain segment of the contour line is interrupted by more than 5 pixel points, it is determined to be broken. Otherwise, the contour is continuous and does not need to be processed. The significance of this analysis is to discover incomplete contours in a timely manner and provide a basis for subsequent repair, ensuring the accuracy of the cleaning area division. The boundary features of the cleaning area are recorded through the contour information, providing data support for subsequent area division. The contour data is converted into boundary coordinate points. Assuming that the boundary of a certain area is defined by 10 key points, these points can be input into the segmentation algorithm as a division basis. The above technical solution can provide a reliable data basis for accurate division, ensure efficient execution of the cleaning task, and reduce the generation of blind areas.

[0064] Further, the dividing the cleaning area according to the contour information comprises:

[0065] If there is a broken phenomenon in the contour information, the broken part is repaired to generate continuous contour data, and the cleaned area is divided by using an image segmentation technique according to the repaired contour data to generate a plurality of area blocks;

[0066] The boundary of each area block is verified to confirm the accuracy of the division result, the accurate distribution data of the cleaned area is generated by combining the contour information and the segmentation result, and the position information of each area block is recorded according to the distribution data.

[0067] Specifically, for the repair of the broken phenomenon in the contour information, for example, if it is found that the contour line is obviously broken at some positions in the image data of the cleaned area, the broken part can be repaired by interpolation or curve fitting technique. Specifically, assuming that the length of the broken area is 10 pixels, a smooth connecting line can be generated by analyzing the direction of the contour points at both ends of the broken area to ensure the continuity of the contour. The purpose of this is to avoid errors in area division caused by the broken line and to improve the accuracy of subsequent segmentation. According to the repaired contour data, the cleaned area can be divided by using an image segmentation technique. The segmentation method based on region growing can be used to divide the cleaned area into a plurality of area blocks by using the contour data as a boundary constraint. Assuming that a cleaned area is divided into N area blocks, and the area of each area block is within a certain range, such division helps to accurately locate the cleaning range and provides a basis for subsequent verification. The boundary of each area block is verified, for example, by comparing the boundary of each area block with the coincidence degree of the original contour data to confirm whether the division is accurate. Assuming that the coincidence degree of the boundary of a certain area block with the contour data is less than 80%, the segmentation parameters need to be adjusted for re-division. The advantage of this is to ensure that the boundary of each area block meets the actual cleaning requirements and avoids misjudgment. The accurate distribution data is generated by combining the contour information and the segmentation result. The repaired contour data and the segmented area blocks are superimposed and analyzed to generate distribution data containing position and area. Assuming that the center point coordinates of a certain area block are A(x1, y1), the specific position can be recorded. This way can provide accurate guidance for cleaning tasks. The position information of each area block is recorded according to the distribution data, for example, the center point coordinates and boundary range of each area block are stored in a database, which is convenient for subsequent cleaning equipment to work by area. The above technical solution can improve the cleaning efficiency and ensure that there is no omission.

[0068] Further, the determination of the cleaning state distribution information of each area block comprises:

[0069] The gray scale analysis is performed on each divided area block to obtain gray scale distribution data of the area block, the cleaning state of the area block is judged according to the gray scale distribution data, and corresponding state information is generated; the cleaning degree of each area block is recorded through the state information, the cleaning degree is compared with a preset threshold value, the area block that does not reach the standard is determined, and the cleaning state distribution information is summarized as a state data set of the area block.

[0070] Specifically, gray scale analysis is performed on each divided region block to obtain gray scale distribution data of the region block. In a possible implementation, the image data of the region block can be converted into a gray scale image through image processing technology, and the gray scale value range of each pixel point is analyzed, which is usually between 0 and 255. Assuming that the gray scale values of a certain region block are concentrated in 200 to 230, it indicates that the region is relatively bright and the cleaning degree is relatively high. If the gray scale values are concentrated in 50 to 80, it indicates that there are more stains or dust. Such analysis is helpful to preliminarily judge the cleaning state of the region and provide a data basis for subsequent evaluation. In this way, the surface condition of the region block can be quantified directly, and then a basis for the cleaning strategy is provided. According to the gray scale distribution data, the cleaning state of the region block is judged, and corresponding state information is generated. For example, in actual application, a region block with an average gray scale value greater than a first brightness is marked as a good cleaning state, and a region block with an average gray scale value less than a second brightness and greater than a third brightness is marked as a poor cleaning state. Assuming that the average gray scale value of a certain region block is less than the third brightness, it is determined that the cleaning state of the region block is unqualified, and state information of “to be cleaned” is generated. The first brightness, the second brightness and the third brightness decrease in turn. Such a judgment mechanism can quickly screen out problem areas, ensure the pertinence of cleaning work, and also provide data support for subsequent resource allocation. The cleaning degree of each region block is recorded through the state information, and the cleaning degree can be divided into multiple levels, such as excellent, general, poor, etc., and each region block is labeled with a specific level. Such a recording method not only facilitates data management, but also provides a reference for optimizing the cleaning plan, thereby improving cleaning efficiency. According to the comparison between the cleaning degree and the preset threshold, the region block that does not meet the standard is determined. For example, if the threshold of the cleaning degree is set as “good”, the region block with a cleaning degree of “general” or “poor” will be listed as a non-standard region. Assuming that 3 of the 10 region blocks are non-standard, these regions will be highlighted for subsequent cleaning arrangement. This comparison method can accurately locate the problem area and ensure efficient use of cleaning resources. The cleaning state distribution information is summarized as a state data set of the region block for subsequent analysis. For example, the state information of all region blocks can be integrated into a data set, including the position, average gray scale value, cleaning level and other information of each region block. Assuming that the data set shows that the cleaning state of a certain region is generally poor, it can be inferred that the region may need more frequent cleaning and maintenance. The above technical solution obtains the cleaning state through gray scale analysis of each region block, provides comprehensive data support for the formulation of the overall cleaning strategy, and helps to improve the scientificity and effectiveness of cleaning management.

[0071] Further, the generating of the GIS blind area distribution map according to the cleaning state distribution information comprises:

[0072] According to the cleaning state distribution information, the position data of the area block not reaching the preset threshold is counted, spatial mapping is performed on the position data, corresponding blind area distribution data is generated, and a visual GIS blind area distribution map is constructed through the blind area distribution data.

[0073] According to the distribution map, the position and range of the cleaning blind area are recorded, and the GIS blind area distribution map is verified to confirm its consistency with the actual cleaning state.

[0074] Specifically, according to the cleaning state distribution information, the position data of the area block not reaching the preset threshold is counted, and exemplarily, after the cleaning robot completes a ground cleaning, the dust coverage rate of each area block is analyzed, assuming that the preset threshold is 80% cleaning degree, if the cleaning degree of a certain area block is only 60%, the position coordinates of the area block are recorded, such as the 5 square meter area in the northwest corner of the room. This process aims to screen out areas with poor cleaning effect and provide data basis for subsequent processing. Spatial mapping is performed on the position data to generate corresponding blind area distribution data, specifically, the position coordinates of the above substandard area block are mapped to the two-dimensional plan of the room, for example, the 5 square meter area in the northwest corner is marked as a red area, indicating a cleaning blind area. Through spatial mapping, the distribution of the blind area can be intuitively reflected, providing support for visualization and laying a foundation for path planning. The visual distribution map is constructed through the blind area distribution data, in one possible implementation, the mapped data is converted into a heat map using a graphics tool, the red area represents a cleaning blind area, and the green area represents a qualified area, for example, the heat map shows that there is a red area in the northwest corner and under the object, with an area of 5 square meters and 3 square meters respectively. This visualization method facilitates users or systems to quickly identify problem areas and improve cleaning efficiency. According to the distribution map, the position and range of the cleaning blind area are recorded, for example, the position of the northwest corner blind area is marked as geographic information coordinates on the heat map, and recording these information helps to accurately locate the problem area and provides a clear target for subsequent cleaning. The GIS blind area distribution map is verified to confirm its consistency with the actual cleaning state, in one embodiment, the cleaning robot is dispatched to re-inspect the northwest corner and the area under the object, images are collected through the camera to confirm the dust coverage, and the verification result is consistent with the distribution Figure 1 For example, there is still obvious dust accumulation in the northwest corner, the verification result is consistent, the above technical solution ensures the accuracy of the distribution map, provides a reliable basis for subsequent path planning, and avoids resource waste.

[0075] Further, the GIS blind area distribution map generated according to the cleaning state distribution information further comprises:

[0076] Data screening is performed on the cleaning state distribution information, region block information that does not reach a preset threshold is extracted, the position and area of a blind area are determined according to the region block information, a preliminary structure of a GIS blind area distribution map is generated through the position and area data, the preliminary structure is optimized and adjusted, and a final GIS blind area distribution map is generated, and the position of a region that needs to be focused on in a cleaning region is recorded according to the GIS blind area distribution map, and early warning information is marked.

[0077] Specifically, data screening is performed on the cleaning state distribution information, and region block information that does not reach a preset threshold is extracted. For example, in a running scene of a smart cleaning device, assuming that the cleaning state distribution information is floor dust density data collected by a sensor, and the preset threshold is a preset dust coverage rate. The data screening process can be to compare the dust density values of all region blocks with the threshold, and extract region block information whose dust density exceeds the set threshold. Such a screening method can quickly lock the areas that do not meet the cleaning standard, providing a data basis for subsequent analysis. The above technical solution can combine the size and position information of the region block during screening to ensure comprehensive and targeted data. The purpose of this is to accurately locate the problem area and improve cleaning efficiency. The position and area of the blind area are determined according to the region block information. For example, in the above scenario, assuming that three non-compliant region blocks are screened out, located at the upper left corner, the central right, and the lower right corner of the room, with areas of 2 square meters, 1.5 square meters, and 3 square meters, respectively. By analyzing the position and area of these region blocks, the distribution of the blind area can be determined, which helps to understand which areas the cleaning device is insufficient in running, which may be due to obstacles or unreasonable device path planning. The significance of determining the position and area of the blind area lies in providing a basis for subsequent optimization to ensure reasonable allocation of cleaning resources. The preliminary structure of the GIS blind area distribution map is generated through position and area data. For example, the room plan can be divided into grids, and each grid is labeled with the geographical position and area size of the blind area to form a visual distribution map. The preliminary structure can visually display the concentration and range of the blind area distribution, providing a reference for subsequent optimization. The preliminary structure can also represent the severity of the blind area by color depth, such as using dark colors to mark areas with larger areas or higher dust density. This can more intuitively reflect the severity of the problem and improve the practicality of the distribution map. The preliminary structure is optimized and adjusted to generate the final GIS blind area distribution map. Based on the preliminary structure, the display method of the distribution map can be adjusted according to actual cleaning needs, such as adding path guide marks around the blind area or merging overlapping blind areas to form a more concise view. The purpose of optimization and adjustment is to make the distribution map more in line with actual use needs, making it easier for cleaning devices or manual operations to quickly understand key areas. This can improve the execution efficiency of the cleaning plan and reduce resource waste. The GIS blind area distribution map records the positions of the areas that need to be focused on in the cleaning area, such as marking the upper left and lower right blind areas as key areas in the final distribution map, and as warning information, because they have larger areas and are located in remote locations, they are easy to be overlooked. After recording these positions, the cleaning device can be re-planned or manual cleaning can be reminded to prioritize these areas. Through the cooperation of the above technical solutions, the pertinence and completeness of the cleaning work can be ensured, and key areas can be avoided, thereby improving the overall cleaning effect.

[0078] The application further provides a GIS early warning system based on cleaning result analysis of a cleaning robot, as shown in the accompanying drawings, the system comprises: Figure 4

[0079] a camera for collecting original image data;

[0080] an analysis module for analyzing perspective tilt influence according to the original image data, determining feature point deviation, and generating distortion distribution information;

[0081] a correction module for correcting the original image data according to the distortion distribution information, and generating corrected first image data;

[0082] an extraction module for extracting edge features from the first image data, and obtaining contour information related to cleaning area division;

[0083] a division module for dividing cleaning areas according to the contour information, and determining cleaning state distribution information of each area block;

[0084] a generation module for generating a GIS blind area distribution map according to the cleaning state distribution information.

[0085] The application further provides a computer readable storage medium, wherein instructions are stored on the computer readable storage medium, and the instructions are executed by a processor to implement the above method.

[0086] ​In summary, the present application provides raw data for the entire process through image data acquisition, ensures a reliable data source for subsequent steps, and analyzes and corrects the feature points in the image to correct the image distortion caused by the tilt of the cleaning robot itself, the environment, and other factors. This process ensures accurate subsequent data and avoids misjudgment due to perspective problems. Image correction is a perspective transformation correction of the generated distortion distribution information, further eliminating image distortion caused by tilt, ensuring that the image is consistent with the actual scene, providing a better basis for accurate analysis. Edge feature extraction and contour information acquisition are based on the corrected image, extract the boundary information of the cleaning area, and ensure the accuracy of the contour information. If the contour is broken, it is also repaired to ensure the complete division of the cleaning area. Cleaning area division and GIS blind area distribution map generation are based on accurate contour information to accurately divide the cleaning area, and according to the cleaning state distribution, that is, the corresponding geographic spatial position information, a GIS blind area distribution map is generated, and the cleaning blind area is labeled to provide cleaning path optimization and early warning function. The above image correction ensures the accuracy of the image, the above perspective transformation further eliminates distortion, and the accuracy of edge extraction is enhanced. Finally, through cleaning state analysis, accurate cleaning state data is provided to generate a GIS blind area distribution map. Through the generation of the GIS blind area distribution map, the cleaning blind area and insufficient area can be identified, the path planning of the cleaning robot can be optimized, the repeated cleaning can be reduced, and the resource utilization efficiency can be improved. Finally, through the GIS blind area distribution map, the cleaning blind area can be intuitively displayed, and through the labeling of the early warning information, the user or cleaning equipment can timely pay attention to the area where cleaning is not in place, so as to ensure that the cleaning work is efficiently and accurately completed. The technical solution cooperates through multiple steps to ensure accurate evaluation of the cleaning effect and provides a reliable basis for subsequent path planning and blind area warning of the cleaning robot.

[0087] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, system and unit can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0088] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or all or part of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0089] The above description and the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features. These modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A GIS early warning method based on a cleaning result analysis of a cleaning robot, characterized by, The method comprises: Obtaining original image data collected by a cleaning robot camera; The robot records the ground conditions of the cleaning area in real time through the built-in camera, and the ground conditions include obstacles and stain distribution; According to the original image data, the influence of the perspective angle tilt is analyzed, the displacement of the feature points is determined, and distortion distribution information is generated; The distortion distribution information includes the displacement of objects in the image caused by the 10-15 degree tilt of the camera perspective angle due to uneven ground or changes in the robot's posture during movement. By comparing the differences between the ideal position and the actual position, the displacement of the feature points is calculated, which is in pixels. The displacement data of multiple feature points is summarized to form the distortion distribution information covering the entire image area; According to the distortion distribution information, the original image data is corrected to generate corrected first image data; Edge features are extracted from the first image data to obtain contour information related to the division of the cleaning area; According to the contour information, the cleaning area is divided, the cleaning state distribution information of each region block is determined, the GIS blind area distribution map is generated by fusing the cleaning state distribution information and the geographic space information, and the warning information is marked; According to the original image data, the influence of the perspective angle tilt is analyzed, the displacement of the feature points is determined, and distortion distribution information is generated; The distortion distribution information includes the displacement of objects in the image caused by the 10-15 degree tilt of the camera perspective angle due to uneven ground or changes in the robot's posture during movement. By comparing the differences between the ideal position and the actual position, the displacement of the feature points is calculated, which is in pixels. The displacement data of multiple feature points is summarized to form the distortion distribution information covering the entire image area; According to the distortion distribution information, the original image data is corrected to generate corrected first image data; According to the cleaning state distribution information and the geographic space information, the GIS blind area distribution map is generated by fusing the cleaning state distribution information and the geographic space information, which comprises: according to the cleaning state distribution information, the position data of the region block that does not reach the preset threshold is counted, the space mapping is carried out for the position data, the corresponding blind area distribution data is generated, and the visual GIS blind area distribution map is constructed through the blind area distribution data; According to the GIS blind area distribution map, the position and range of the cleaning blind area are recorded, and the GIS blind area distribution map is verified to confirm its consistency with the actual cleaning state.

2. The method of claim 1, wherein, The edge features are extracted from the first image data to obtain contour information related to the division of the cleaning area, which comprises: The first image data is pre-processed to enhance edge features in the image, and an edge detection algorithm is used to extract edge information in the first image data; According to the edge information, contour data related to the cleaning area is constructed, and continuity analysis is performed on the contour data to determine whether there is a contour break phenomenon. The boundary features of the cleaning area are recorded through the contour information.

3. The method of claim 1, wherein, The division of the cleaning area according to the contour information includes: If there is a break phenomenon in the contour information, the broken part is repaired to generate continuous contour data, and the cleaning area is divided using image segmentation technology according to the repaired contour data to generate multiple region blocks; Boundary verification is performed on each region block to confirm the accuracy of the division result. The accurate distribution data of the cleaning area is generated through the combination of the contour information and the segmentation result, and the position information of each region block is recorded according to the distribution data.

4. The method of claim 3, wherein, The determination of the cleaning state distribution information of each region block includes: Gray scale analysis is performed on each region block after division to obtain gray scale distribution data of the region block, and the cleaning state of the region block is determined according to the gray scale distribution data to generate corresponding state information. The cleaning degree of each region block is recorded through the state information, and the cleaning degree is compared with a preset threshold to determine the region block that does not meet the standard. The cleaning state distribution information is summarized as a state data set of the region block.

5. The method of claim 1, wherein, The fusion of the cleaning state distribution information and geographic spatial information to generate a GIS blind area distribution map further includes: Data filtering is performed on the cleaning state distribution information to extract region block information that does not meet the preset threshold. The position and area of the blind area are determined according to the region block information, and the preliminary structure of the GIS blind area distribution map is generated through the position and area data. The preliminary structure is optimized and adjusted to generate the final GIS blind area distribution map. The region position that needs to be focused on in the cleaning area is recorded according to the GIS blind area distribution map, and warning information is marked.

6. A GIS early warning system based on the GIS early warning method of analyzing the cleaning result of a cleaning robot according to any one of claims 1-5, characterized in that, The system includes: A camera for collecting raw image data; An analysis module for analyzing the influence of perspective tilt and determining the displacement of feature points to generate distortion distribution information; A correction module for correcting the raw image data according to the distortion distribution information to generate corrected first image data; An extraction module for extracting edge features from the first image data to obtain contour information related to the division of the cleaning area; A division module for dividing the cleaning area according to the contour information to determine the cleaning state distribution information of each region block; A generation module for generating a GIS blind area distribution map according to the cleaning state distribution information.

7. A computer-readable storage medium having stored thereon instructions, the computer-readable storage medium comprising: The instructions are executed by the processor to implement the method of any one of claims 1-5.

Images