Stain identification and detection system based on image processing

Abstract

The invention relates to the technical field of image recognition, and discloses an image processing-based stain recognition and detection system, which comprises an image acquisition module, an environment sensing module, an optimization adjustment module and a stain recognition module, wherein the image acquisition module is used for acquiring a ground image; the environment sensing module is used for acquiring environment illumination conditions; an optimization adjustment module performs illumination compensation control on collection of the ground image based on the environment illumination condition; the optimization adjustment module performs shadow recognition and correction on the ground image based on the environment illumination condition; reference images of grounds made of different materials are stored in the stain identification module; the stain identification module performs stain identification detection based on the ground image and the reference image of the ground of the corresponding material; according to the invention, the recognition and detection efficiency of stains in the ground image is improved, and the image processing cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a stain recognition and detection system based on image processing. Background Art

[0002] Currently, some ride-on floor scrubbers are experimenting with using cameras to capture floor images to assist in assessing cleaning effectiveness. Simple vision systems can identify visible stains on the floor and roughly determine changes in floor condition before and after cleaning, providing data support for cleaning operations. This improves the targeted nature of cleaning, allowing operators to promptly identify areas that have not been cleaned and perform secondary cleaning. However, existing floor stain detection systems still have numerous shortcomings. Lighting adaptability presents challenges for existing stain recognition systems. Lighting conditions vary significantly across different working environments, from indoor lighting to natural light, with fluctuating light intensity and direction. Under direct sunlight, captured floor images are prone to overexposure, resulting in a loss of detail and inaccurate identification. In dimly lit environments, images become blurred and noise increases, similarly impacting stain identification. Current technology struggles to automatically and accurately adjust image acquisition parameters based on real-time lighting changes, significantly compromising the accuracy and stability of stain recognition under complex lighting conditions. Shadow interference also impacts the reliability of cleanliness monitoring. During operation, ride-on floor scrubbers inevitably create shadows. Existing stain recognition systems struggle to accurately distinguish between shadowed and stained areas, often misjudging shadows as stains or missing detection of real stains in shadowed areas. Existing technologies have significant shortcomings when it comes to monitoring the cleanliness of floors of varying materials. Floors come in a variety of materials, such as ceramic tiles, marble, and concrete, each with its own unique ability to adsorb stains, the way they present themselves, and their surface texture. Existing stain recognition systems lack the ability to adapt to different floor materials. When switching between multiple floor materials, they are unable to accurately adjust the parameters of the recognition algorithm, resulting in decreased monitoring accuracy. Furthermore, stain recognition becomes even more difficult when the floor has complex patterns or textures. These patterns and textures interfere with the identification of stains, making it easy for the system to misjudge patterns as stains or ignore real stains hidden within them, leading to misjudgments.

[0003] For example, the patent application with publication number CN114723767A discloses a stain detection method, device, electronic device, and sweeping robot system. The method includes: obtaining an image to be detected containing the ground; grouping each pixel in the image to be detected based on the similarity between the target attributes of the pixels to obtain a segmented image; wherein, pixels belonging to different groups in the segmented image are marked with different colors; the target attribute includes the color value of the pixel; based on the difference between the pixel values ​​of each pixel in the segmented image, determining the edge of the object in the segmented image; if the edge of an object determined meets the preset stain edge feature, determining the presence of a stain on the ground corresponding to the object. In this way, the use complexity of the sweeping robot can be reduced, and the storage space occupied by the sweeping robot can be reduced.

[0004] For example, the patent application with publication number CN115049733A discloses an intelligent stain cleaning system and cleaning method based on a depth camera. The system includes: a sweeping robot, a depth camera, a stain recognition module, a navigation module and a cleaning device; the depth camera obtains a color image and a depth image, and obtains a color point cloud in the camera coordinate system based on the color image and the depth image, thereby obtaining a color image containing only the ground and sending it to the stain recognition module; the stain recognition module extracts the color and contour features of the stain itself from the color image containing only the ground through the GMM algorithm, and generates the stain coordinates in the world coordinate system and sends them to the navigation module; the navigation module generates a navigation route for the sweeping robot based on the stain coordinates in the world coordinate system, and guides the sweeping robot to the stain location to use the cleaning device to clean the stain.

[0005] The above existing technologies all have the problem raised by this background technology: when there is interference from shadows or complex patterns on the ground, the accuracy of stain recognition and detection is difficult to guarantee.

[0006] The information disclosed in this background section is only intended to enhance understanding of the overall background of the invention and should not be considered as an admission or any form of suggestion that the information constitutes the prior art already known to a person of ordinary skill in the art. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to overcome the defects of the existing technology and provide a stain recognition and detection system based on image processing, so as to improve the efficiency of stain recognition and detection in ground images and reduce the image processing cost.

[0008] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0009] A stain recognition and detection system based on image processing includes an image acquisition module, an environment perception module, an optimization and adjustment module, and a stain recognition module; wherein:

[0010] The image acquisition module is used to collect ground images;

[0011] The environmental perception module is used to obtain the ambient lighting conditions;

[0012] The optimization and adjustment module performs illumination compensation control on the acquisition of the ground image based on the ambient illumination conditions; the optimization and adjustment module also performs shadow recognition and correction on the ground image based on the ambient illumination conditions;

[0013] The stain recognition module stores reference images of floors of different materials; the stain recognition module performs stain recognition detection based on the floor image and the reference images of floors of corresponding materials.

[0014] As a preferred solution of the stain identification and detection system based on image processing of the present invention, the image acquisition module includes a camera unit, a fill light unit, and a control unit; wherein:

[0015] The camera unit is equipped with a camera; the camera collects ground images based on set camera parameters; the camera parameters include exposure time, gain coefficient, and aperture size;

[0016] The fill light unit is equipped with a fill light; the fill light is used for fill light when collecting ground images;

[0017] The control unit is used to control the camera parameters of the camera unit and the fill light intensity of the fill light.

[0018] As a preferred solution of the stain identification and detection system based on image processing of the present invention, wherein: the environmental lighting condition includes the camera light intensity;

[0019] The environment perception module includes a first perception unit; the first perception unit is used to obtain the camera light intensity;

[0020] The first sensing unit includes a light intensity sensor arranged at the camera; the first sensing unit obtains the ambient light intensity at the camera based on the light intensity sensor as the camera light intensity.

[0021] As a preferred embodiment of the stain identification and detection system based on image processing of the present invention, the ambient light condition further includes a dominant light direction; the ambient sensing module further includes a second sensing unit; the second sensing unit is configured to obtain the dominant light direction; the second sensing unit includes light intensity sensors installed at different positions on the ride-on floor scrubber, wherein the light intensity sensor installed at any position only collects the ambient light intensity in one direction;

[0022] The second sensing unit is configured with a light intensity difference threshold; the method for obtaining the dominant lighting direction is as follows: collect the light intensity in each direction; if the difference in light intensity between the two directions with the largest light intensity is greater than the light intensity difference threshold, then the direction corresponding to the larger light intensity is the dominant lighting direction; otherwise, there is no dominant lighting direction.

[0023] As a preferred solution of the stain identification and detection system based on image processing of the present invention, the optimization and adjustment module includes an illumination compensation unit; the illumination compensation unit performs illumination compensation control on the ground image based on the camera light intensity, specifically including:

[0024] Establish a mapping relationship between camera light intensity, camera parameters, fill light intensity and average brightness of ground images;

[0025] Set the target brightness of the ground image;

[0026] Collecting camera light intensity; calculating the optimal solution of camera parameters and fill light intensity based on the mapping relationship and the camera light intensity;

[0027] The optimal solutions of the camera parameters and the fill light illumination intensity are sent to the control unit; and the control unit controls the camera parameters of the camera unit and the fill light illumination intensity of the fill light unit based on the optimal solutions.

[0028] As a preferred solution of the stain identification and detection system based on image processing of the present invention, the optimization and adjustment module further includes a shadow correction unit; the shadow correction unit performs shadow identification and correction on the ground image based on the dominant lighting direction, specifically including:

[0029] If there is a dominant lighting direction, shadow recognition and correction are performed on the ground image;

[0030] Obtaining the angle between the dominant light direction and the forward direction of the ride-on floor scrubber;

[0031] pre-marking a shadow area in the ground image based on the direction angle;

[0032] Convert the ground image from RGB space to HSV space and obtain the brightness of each pixel;

[0033] Refer to the pre-marked shadow area and perform shadow detection based on the brightness of the pixels; segment the ground image into shadow and non-shadow areas;

[0034] Performing compensation correction on the shadow area based on the average brightness of the non-shadow area;

[0035] Convert the compensated ground image back to RGB space.

[0036] As a preferred solution of the stain recognition and detection system based on image processing of the present invention, the shadow detection based on the brightness of the pixel specifically includes:

[0037] Select M pixels from the pre-marked shadow area and calculate the average brightness of the M pixels; set a brightness difference threshold range and calculate the difference between the brightness of each pixel and the average brightness of the M pixels in turn; if the difference between the brightness of any pixel and the average brightness of the M pixels is within the brightness difference threshold range, mark the pixel as a shadow point; and connect the shadow points into a shadow area.

[0038] As a preferred solution of the stain identification and detection system based on image processing described in the present invention, wherein: compensating and correcting the shadow area based on the average brightness of the non-shadow area specifically includes: calculating the average brightness of all pixels in the non-shadow area as the corrected brightness; compensating and correcting the shadow area based on the corrected brightness specifically includes: adjusting the pixel value of each pixel in the shadow area to the corrected brightness.

[0039] As a preferred solution of the stain identification and detection system based on image processing of the present invention, wherein: the stain identification module includes a comparison unit; the comparison unit is used to perform stain identification and detection;

[0040] The comparison unit stores reference images of floors of different materials; the reference image of any floor material is a floor image captured by the camera unit under conditions of no stains, no shadows, and uniform lighting, and the average brightness of the reference image is the target brightness;

[0041] The stain identification detection specifically includes:

[0042] Identify the ground material corresponding to the ground image collected in real time;

[0043] Get a reference image of the ground with the corresponding material;

[0044] Based on a feature point matching algorithm, the ground image collected in real time is aligned with the reference image;

[0045] Identify the soiled area in the ground image based on the aligned ground image and the reference image.

[0046] As a preferred solution of the stain identification and detection system based on image processing described in the present invention, the identification of the stain area in the ground image based on the aligned ground image and the reference image specifically includes: calculating the pixel difference between the pixel points at corresponding positions in the ground image and the reference image point by point; the comparison unit is also configured with a pixel difference threshold; in the ground image, the pixels whose pixel difference is greater than the pixel difference threshold are marked as stain points; and the stain points in the ground image are connected to form a stain area.

[0047] As a preferred solution of the stain recognition and detection system based on image processing of the present invention, wherein: the identification of the stain area in the ground image based on the aligned ground image and the reference image further includes:

[0048] Convert the aligned ground image and reference image into HSV space and record the brightness of each pixel in the ground image and reference image;

[0049] Comparing the brightness of pixels at corresponding positions in the ground image and the reference image point by point, and constructing a brightness residual matrix; the brightness residual matrix has the same dimension as the pixel matrix of the ground image; each element in the brightness residual matrix corresponds to a pixel at the same position in the pixel matrix of the ground image, and the element value is the brightness difference between the pixel at the corresponding position in the ground image and the reference image;

[0050] A stain area in the ground image is identified based on the element values ​​in the brightness residual matrix.

[0051] As a preferred solution of the stain recognition and detection system based on image processing of the present invention, the stain area in the ground image is identified based on the element values ​​in the brightness residual matrix, specifically including:

[0052] Slidingly intercepting at least N sub-matrices from the luma residual matrix through a sliding window;

[0053] Calculate the variance of all element values ​​in each submatrix;

[0054] If the variance of the element values ​​of all submatrices is less than the preset variance threshold, there is no stain area in the ground image; if the variance of the element values ​​of at least one submatrix is ​​greater than the preset variance threshold, the corresponding submatrix is ​​marked as a potential stain area;

[0055] determining the gradient direction of the element value in the potential stain area;

[0056] The gradient direction concentration of the potential stain area is calculated based on the gradient direction of the element value; if the gradient direction concentration is greater than a preset concentration threshold, the potential stain area is a non-stained area; if the gradient direction concentration is less than or equal to the preset concentration threshold, the potential stain area is a stained area.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] The environmental perception module obtains the ambient lighting conditions, and the optimization and adjustment module establishes a mapping relationship based on this, and calculates the optimal solution for the camera parameters and fill light parameters to achieve lighting compensation control, ensuring that clear, high-quality ground images can be collected under various lighting conditions, thereby improving the stability and accuracy of stain recognition and detection.

[0059] Identify and correct shadows in ground images based on the dominant lighting direction. By pre-marking shadow areas, detecting and segmenting shadows in HSV space, compensating and correcting them based on the average brightness of non-shadow areas, and then converting back to RGB space, we prevent shadows from being misidentified as stains and improve visual recognition accuracy.

[0060] By pre-collecting reference images of different ground materials, identifying the ground material and obtaining the corresponding reference image, aligning the reference image with the real-time ground image through the feature point matching algorithm and comparing the pixel difference point by point, it is possible to accurately identify the stained area, reduce the interference of complex patterns on stain identification, and make the detection results of the stained area more reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort. Among them:

[0062] Figure 1 A schematic structural diagram of a stain recognition and detection system based on image processing provided by the present invention;

[0063] Figure 2 This is an example diagram of aligning a ground image with a reference image based on feature point matching provided by the present invention. DETAILED DESCRIPTION

[0064] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0065] This embodiment introduces a stain recognition and detection system based on image processing. Figure 1 The system includes an image acquisition module, an environment perception module, an optimization and adjustment module, and a stain recognition module; wherein:

[0066] The image acquisition module is used to collect ground images;

[0067] The image acquisition module includes a camera unit, a fill light unit, and a control unit; wherein: the camera unit is configured with a camera; the camera acquires ground images based on set camera parameters; the camera parameters include exposure time, gain coefficient, and aperture size;

[0068] The fill light unit is equipped with a fill light; the fill light is used to provide fill light when capturing ground images; the fill light is an LED light with adjustable brightness and has a fast response characteristic, and can quickly adjust the light brightness according to the control; the position and angle of the fill light should be designed and tested to avoid shadows and reflections, and ensure that the light is evenly illuminated on the ground area where the image needs to be captured.

[0069] The control unit is used to control the camera parameters of the camera unit and the fill light intensity of the fill light.

[0070] The environmental perception module is used to obtain the ambient lighting conditions;

[0071] The ambient lighting conditions include camera light intensity and dominant lighting direction;

[0072] The environment perception module includes a first perception unit and a second perception unit; wherein the first perception unit is used to obtain the camera light intensity;

[0073] The first sensing unit includes a light intensity sensor arranged at the camera; the first sensing unit obtains the ambient light intensity at the camera based on the light intensity sensor as the camera light intensity.

[0074] The second sensing unit is used to obtain the dominant light direction; the second sensing unit includes light intensity sensors installed in different positions on the ride-on floor scrubber. The light intensity sensor installed in any position only collects the ambient light intensity in one direction. For example, four light intensity sensors are installed on the ride-on floor scrubber (excluding the light intensity sensor of the first sensing unit), respectively used to collect the ambient light intensity in front, behind, left, and right of the ride-on floor scrubber (with the forward direction of the ride-on floor scrubber as the front); each light intensity sensor shields the ambient light in other directions by shading.

[0075] The second sensing unit is configured with a light intensity difference threshold; the method for the second sensing unit to obtain the dominant lighting direction is as follows: collect the light intensity in each direction; if the difference in light intensity between the two directions with the largest light intensity is greater than the light intensity difference threshold, then the direction corresponding to the larger light intensity is the dominant lighting direction; otherwise, there is no dominant lighting direction.

[0076] The optimization and adjustment module performs illumination compensation control on the acquisition of the ground image based on the ambient illumination conditions; the optimization and adjustment module also performs shadow recognition and correction on the ground image based on the ambient illumination conditions;

[0077] The optimization and adjustment module includes an illumination compensation unit. The illumination compensation unit performs illumination compensation control on the acquisition of the ground image based on the camera light intensity, specifically including:

[0078] Establish a mapping relationship between camera light intensity, camera parameters, fill light intensity and average brightness of ground images;

[0079] For example, a linear regression model is constructed to record the mapping relationship by using the camera light intensity, camera parameters (including exposure time, gain coefficient, aperture size), and fill light intensity as independent variables, and the average brightness of the ground image as the dependent variable. Experiments are conducted to capture ground images using a series of different camera light intensities, with a range of camera parameter and fill light intensity values ​​set. The experimental data is then recorded. The average brightness of the ground image is calculated as follows: the ground image is converted into a grayscale image; the average grayscale value of all pixels in the grayscale image is calculated as the average brightness of the ground image. The linear regression model is then fitted to the experimental data to obtain the mapping relationship.

[0080] Set the target brightness for ground images; maintaining consistent and stable brightness across all captured ground images is crucial. Without parameter adjustments, images can be overly bright or dark under varying ambient lighting conditions, resulting in loss of detail. By establishing a mapping relationship, setting a target brightness, and adjusting camera parameters and fill light intensity, you can ensure that ground image brightness remains within an appropriate range. For example, lowering exposure in bright light can prevent overbrightness, while increasing exposure or fill light in low light can prevent overbrightness. This ensures uniform overall image brightness and clear display.

[0081] The camera light intensity is collected; the optimal solution for the camera parameters and fill light intensity is calculated based on the mapping relationship and the camera light intensity. An algorithm such as linear quadratic programming can be used to calculate the optimal solution for the camera parameters and fill light intensity. Based on the fitted linear regression model, the camera light intensity is known, and the target brightness is given, the linear quadratic programming can find the optimal solution for the camera parameters and fill light intensity. When the camera parameters and fill light intensity are set according to the optimal solution, the brightness of the captured ground image can be guaranteed to be the set target brightness.

[0082] The optimal solution for camera parameters and fill light intensity is sent to the control unit, which then controls the camera parameters and fill light intensity based on the optimal solution. By analyzing ambient light conditions in real time and automatically adjusting camera parameters, the system ensures clear, high-quality ground images in all lighting conditions, improving the stability and accuracy of soiled area monitoring.

[0083] The optimization and adjustment module further includes a shadow correction unit; the shadow correction unit performs shadow recognition and correction on the ground image based on the dominant illumination direction, specifically including:

[0084] If there is a dominant lighting direction, shadow recognition and correction are performed on the ground image;

[0085] Obtaining the angle between the dominant light direction and the forward direction of the ride-on floor scrubber;

[0086] Based on the directional angle, shadow areas are pre-marked in the ground image. Experiments are conducted to determine the range of shadows appearing in the ground image corresponding to the angle between the dominant light direction and the forward direction, and a corresponding relationship between the angle and the shadow area is established. In practical applications, based on this correspondence, the approximate area where the shadow appears is determined based on the directional angle. For example, when a scrubber is traveling straight ahead and the dominant light is incident from the front, the scrubber's own shadow will appear on the cleaning path behind the machine.

[0087] Convert the ground image from RGB space to HSV space and obtain the brightness of each pixel;

[0088] Refer to the pre-marked shadow area and perform shadow detection based on the brightness of the pixels; divide the ground image into shadow area and non-shadow area; for example, select M pixels from the pre-marked shadow area and calculate the average brightness of these M pixels; M is a positive integer; set the brightness difference threshold range, and calculate the difference between the brightness of each pixel and the average brightness of the M pixels in turn. If the difference is within the brightness difference threshold range, mark the pixel as a shadow point; connect the shadow points into a shadow area, and eliminate isolated shadow points; the pixels outside the shadow area constitute the non-shadow area.

[0089] The shadow area is compensated and corrected based on the average brightness of the non-shadow area; specifically, the method includes: calculating the average brightness of all pixels in the non-shadow area as the corrected brightness; and compensating and correcting the shadow area based on the corrected brightness, specifically, the method includes: adjusting the pixel value of each pixel in the shadow area to the corrected brightness.

[0090] Convert the compensated ground image back to RGB space.

[0091] In the RGB color space, the color of each pixel is composed of three components: R, G, and B. When ambient light intensity increases, the values ​​of all three color components theoretically increase, resulting in a brighter overall image. Shadows, however, reduce the values ​​of each color component, darkening the image. However, the effects of light and shadow on the R, G, and B components are nonlinear, making accurate shadow compensation difficult directly in RGB space. HSV space describes each pixel using lightness, saturation, and hue. Changes in ambient light primarily significantly affect lightness, while theoretically having less significant effects on saturation and hue. Therefore, in HSV space, using lightness to compensate for shadows can more accurately eliminate the effects of shadows on the ground image. RGB space, on the other hand, provides richer color information and is more suitable for stain detection. Therefore, after shadow compensation, the ground image is converted back to RGB space.

[0092] When there is a dominant lighting direction, the ambient light intensity in one direction is significantly stronger than in other directions for a ride-on floor scrubber. In this case, the scrubber's shadow is likely to be left on the floor image. The shadow cast by the ride-on floor scrubber while operating may be mistaken for stains. Shadow detection and compensation can prevent shadow interference in stain identification and improve visual recognition accuracy.

[0093] The stain recognition module stores reference images of floors of different materials; the stain recognition module performs stain recognition detection based on the floor image and the reference images of floors of corresponding materials.

[0094] The stain recognition module includes a comparison unit; the comparison unit is used to perform stain recognition detection;

[0095] The comparison unit stores reference images of floors of different materials; the reference image of any floor material is a floor image captured by the camera unit under conditions of no stains, no shadows, and uniform lighting, and the average brightness of the reference image is the target brightness;

[0096] The stain identification detection specifically includes:

[0097] Identify the ground material corresponding to the ground image collected in real time;

[0098] Get a reference image of the ground with the corresponding material;

[0099] Based on the feature point matching algorithm, the ground image collected in real time is aligned with the reference image; first, the feature points are detected and marked in the ground image and the reference image respectively through the feature point detection algorithm, and then the feature points in the ground image and the reference image are matched based on the feature point matching algorithm; the ground image and the reference image are aligned based on the matched feature points so that a pixel-by-pixel comparison can be performed between the two to determine the cleanliness of the ground position corresponding to each pixel.

[0100] Identify the stain area in the ground image based on the aligned ground image and the reference image, specifically including:

[0101] The pixel difference between corresponding pixels in the ground image and the reference image is calculated point by point. The comparison unit is also configured with a pixel difference threshold. In the ground image, pixels with a pixel difference greater than the pixel difference threshold are marked as stain points. The stain points in the ground image are connected to form a stain area. Isolated stain points that cannot be connected to other stain points are removed.

[0102] Preferably, this embodiment provides another method for identifying a stained area in a ground image based on the aligned ground image and the reference image, specifically including:

[0103] Convert the aligned ground image and reference image into HSV space and record the brightness of each pixel in the ground image and reference image;

[0104] Comparing the brightness of pixels at corresponding positions in the ground image and the reference image point by point, and constructing a brightness residual matrix; the brightness residual matrix has the same dimension as the pixel matrix of the ground image; each element in the brightness residual matrix corresponds to a pixel at the same position in the pixel matrix of the ground image, and the element value is the brightness difference between the pixel at the corresponding position in the ground image and the reference image;

[0105] Identifying the stained area in the ground image based on the element values ​​in the brightness residual matrix; specifically comprising:

[0106] Slidingly intercepting at least N submatrices from the luma residual matrix through a sliding window, where N is a positive integer;

[0107] Calculate the variance of all element values ​​in each submatrix;

[0108] If the variance of the element values ​​of all submatrices is less than the preset variance threshold, there is no stain area in the ground image; if the variance of the element values ​​of at least one submatrix is ​​greater than the preset variance threshold, the corresponding submatrix is ​​marked as a potential stain area; the position of the potential stain area in the brightness residual matrix is ​​recorded;

[0109] determining the gradient direction of the element value in the potential stain area;

[0110] The gradient direction concentration of the potential stain area is calculated based on the gradient direction of the element value; if the gradient direction concentration is greater than a preset concentration threshold, the potential stain area is a non-stained area; if the gradient direction concentration is less than or equal to the preset concentration threshold, the potential stain area is a stained area, and the position of the corresponding stained area in the ground image is determined based on the position of the potential stain area in the luminance residual matrix.

[0111] Preferably, this embodiment provides a calculation formula for gradient direction concentration, as follows:

[0112]

[0113] Where C represents the gradient direction concentration of the potential stain area; N1 represents the number of element values ​​of the gradient direction in the potential stain area; θ i Indicates the gradient direction of the i-th element value where there is a gradient direction, and records the gradient direction in the form of angle; represents the average gradient direction of N1 element values ​​with gradient directions; cos(·) represents the cosine value.

[0114] Since the cosine value ranges from [0, 1], the gradient direction concentration C also ranges from [0, 1], as can be seen from the formula. Larger values ​​of C indicate more regular and consistent gradient directions for the elements in the submatrix corresponding to the potential stain area; smaller values ​​of C indicate more scattered and random gradient directions for the different element values. When C is greater than the preset concentration threshold, the large variance in the corresponding submatrix element values ​​is caused by overall brightness deviations due to ground reflections, over-correction, or under-correction of shadows, rather than brightness deviations caused by stains. When C is less than or equal to the preset concentration threshold, the large variance in the corresponding submatrix element values ​​is caused by irregular local brightness changes caused by stains absorbing and scattering ambient light, resulting in stains in the corresponding submatrix (i.e., the potential stain area).

[0115] The stain detection method proposed in this embodiment, which is based on the analysis of the brightness deviation rules between the ground image and the reference image, can distinguish the characteristics of regular deviations (such as reflections and shadow correction errors) and local mutation deviations (stains), reduce the misjudgment of stain areas, and improve detection reliability.

[0116] This application provides an example diagram of aligning a ground image with a reference image based on feature point matching, such as Figure 2 As shown in FIG. 1 , there are complex patterns on the ground of a certain material, which significantly interfere with the identification of the stained area. Figure 2In the image, two square marks, two triangle marks, or two solid circles in each image represent a set of successfully matched feature points. Using these three sets of successfully matched feature points as a reference system, the two images can be overlapped and the redundant portions of the reference image (i.e., portions where the reference image is larger than the ground image) can be trimmed. If necessary, the coordinate systems of the ground image and the reference image can be aligned using methods such as affine transformation (this is useful for extreme cases like camera jitter due to unexpected interference). Once the two are aligned through feature point matching, it is possible to determine which pixels in the ground image correspond to patterns and which to stains, thereby reducing misjudgments.

[0117] Different floor materials (such as tiles, marble, cement, etc.) adsorb and present stains in different ways. When the existing visual recognition system switches between multiple floor materials, the accuracy of cleanliness monitoring will decrease. In addition, floors with complex patterns or textures will also cause significant interference to visual recognition. This application sets a reference image of each floor material and aligns the reference image with the real-time collected floor image, thereby detecting the stained area by point-by-point comparison, making the cleanliness assessment more accurate and reliable.

[0118] Preferably, the stain recognition module further comprises a stain recognition unit; the stain recognition unit is used to recognize the type of stains in the ground image;

[0119] The stain recognition unit is configured with an image segmentation algorithm and a trained target detection model; the stain recognition unit uses the image segmentation algorithm to segment the stain area from the ground image; and then uses the target detection algorithm to identify the stain type.

[0120] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0121] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the purpose and scope of protection of the present invention, which are all protected by the present invention.

Claims

1. A stain recognition and detection system based on image processing, characterized by: It includes image acquisition module, environment perception module, optimization and adjustment module, and stain recognition module; among which: The image acquisition module is used to collect ground images; The environmental perception module is used to obtain the ambient lighting conditions; The optimization and adjustment module performs illumination compensation control on the acquisition of the ground image based on the ambient illumination conditions; the optimization and adjustment module also performs shadow recognition and correction on the ground image based on the ambient illumination conditions; The stain recognition module stores reference images of floors of different materials; the stain recognition module performs stain recognition detection based on the floor image and the reference images of floors of corresponding materials.

2. The stain recognition and detection system based on image processing according to claim 1, characterized in that: The image acquisition module includes a camera unit, a fill light unit, and a control unit; wherein: The camera unit is equipped with a camera; the camera collects ground images based on set camera parameters; the camera parameters include exposure time, gain coefficient, and aperture size; The fill light unit is equipped with a fill light; the fill light is used for fill light when collecting ground images; The control unit is used to control the camera parameters of the camera unit and the fill light intensity of the fill light.

3. The stain recognition and detection system based on image processing according to claim 2, characterized in that: The ambient light conditions include the camera light intensity and the dominant light direction; the environment perception module obtains the ambient light intensity at the camera through a light intensity sensor installed at the camera as the camera light intensity; the environment perception module also includes light intensity sensors installed at different positions on the ride-on floor scrubber, and the light intensity sensor installed at any position only collects the ambient light intensity in one direction; The environmental perception module is also equipped with a light intensity difference threshold; the method for obtaining the dominant lighting direction is as follows: collecting the light intensity in each direction; if the difference in light intensity between the two directions with the largest light intensity is greater than the light intensity difference threshold, the direction corresponding to the larger light intensity is the dominant lighting direction; otherwise, there is no dominant lighting direction.

4. The stain recognition and detection system based on image processing according to claim 3, characterized in that: The optimization and adjustment module includes an illumination compensation unit; The illumination compensation unit performs illumination compensation control on the acquisition of the ground image based on the camera light intensity, specifically including: Establish a mapping relationship between camera light intensity, camera parameters, fill light intensity and average brightness of ground images; Set the target brightness of the ground image; Collecting camera light intensity; calculating the optimal solution of camera parameters and fill light intensity based on the mapping relationship and the camera light intensity; The optimal solutions of the camera parameters and the fill light illumination intensity are sent to the control unit; and the control unit controls the camera parameters of the camera unit and the fill light illumination intensity of the fill light unit based on the optimal solutions.

5. The stain recognition and detection system based on image processing according to claim 4, characterized in that: The optimization and adjustment module further includes a shadow correction unit; the shadow correction unit performs shadow recognition and correction on the ground image based on the dominant illumination direction, specifically including: If there is a dominant lighting direction, shadow recognition and correction are performed on the ground image; Obtaining the angle between the dominant light direction and the forward direction of the ride-on floor scrubber; pre-marking a shadow area in the ground image based on the direction angle; Convert the ground image from RGB space to HSV space and obtain the brightness of each pixel; Refer to the pre-marked shadow area and perform shadow detection based on the brightness of the pixels; segment the ground image into shadow and non-shadow areas; Performing compensation correction on the shadow area based on the average brightness of the non-shadow area; Convert the compensated ground image back to RGB space.

6. The stain recognition and detection system based on image processing according to claim 5, characterized in that: The shadow detection based on the brightness of the pixels specifically includes: selecting M pixels from a pre-marked shadow area and calculating the average brightness of the M pixels; setting a brightness difference threshold range and sequentially calculating the difference between the brightness of each pixel and the average brightness of the M pixels; if the difference between the brightness of any pixel and the average brightness of the M pixels is within the brightness difference threshold range, marking the pixel as a shadow point; and connecting the shadow points into a shadow area; The compensation correction of the shadow area based on the average brightness of the non-shadow area specifically includes: calculating the average brightness of all pixels in the non-shadow area as the correction brightness; the compensation correction of the shadow area based on the correction brightness specifically includes: adjusting the pixel value of each pixel in the shadow area to the correction brightness.

7. The stain recognition and detection system based on image processing according to claim 6, characterized in that: The stain recognition module includes a comparison unit; the comparison unit is used to perform stain recognition detection; The comparison unit stores reference images of floors of different materials; the reference image of any floor material is a floor image captured by the camera unit under conditions of no stains, no shadows, and uniform lighting, and the average brightness of the reference image is the target brightness; The stain identification detection specifically includes: Identify the ground material corresponding to the ground image collected in real time; Get a reference image of the ground with the corresponding material; Based on a feature point matching algorithm, the ground image collected in real time is aligned with the reference image; Identify the soiled area in the ground image based on the aligned ground image and the reference image.

8. The stain recognition and detection system based on image processing according to claim 7, characterized in that: The method of identifying the stained area in the ground image based on the aligned ground image and the reference image specifically includes: calculating the pixel difference between the pixel points at corresponding positions in the ground image and the reference image point by point; the comparison unit is also configured with a pixel difference threshold; in the ground image, the pixels whose pixel difference is greater than the pixel difference threshold are marked as stained points; and the stained points in the ground image are connected to form a stained area.

9. The stain recognition and detection system based on image processing according to claim 8, characterized in that: The method of identifying a stain area in the ground image based on the aligned ground image and the reference image further includes: Convert the aligned ground image and reference image into HSV space and record the brightness of each pixel in the ground image and reference image; Comparing the brightness of pixels at corresponding positions in the ground image and the reference image point by point, and constructing a brightness residual matrix; the brightness residual matrix has the same dimension as the pixel matrix of the ground image; each element in the brightness residual matrix corresponds to a pixel at the same position in the pixel matrix of the ground image, and the element value is the brightness difference between the pixel at the corresponding position in the ground image and the reference image; A stain area in the ground image is identified based on the element values ​​in the brightness residual matrix.

10. The stain recognition and detection system based on image processing according to claim 9, characterized in that: Identifying a stained area in the ground image based on element values ​​in the brightness residual matrix specifically includes: Slidingly intercepting at least N sub-matrices from the luma residual matrix through a sliding window; Calculate the variance of all element values ​​in each submatrix; If the variance of the element values ​​of all submatrices is less than the preset variance threshold, there is no stain area in the ground image; if the variance of the element values ​​of at least one submatrix is ​​greater than the preset variance threshold, the corresponding submatrix is ​​marked as a potential stain area; determining the gradient direction of the element value in the potential stain area; The gradient direction concentration of the potential stain area is calculated based on the gradient direction of the element value; if the gradient direction concentration is greater than a preset concentration threshold, the potential stain area is a non-stained area; if the gradient direction concentration is less than or equal to the preset concentration threshold, the potential stain area is a stained area.

Images