Cleanness detection method and device, electronic equipment and storage medium

Abstract

The invention discloses a cleanliness detection method and device, electronic equipment and a storage medium. The method comprises the steps that first image data of a to-be-detected object is acquired, and the first image data comprises a stain area; processing the first image data through a pre-trained image segmentation model to obtain an image segmentation result of the dirty region, and determining first feature data corresponding to the dirty region based on the image segmentation result; performing classification processing on the image segmentation result based on a pre-trained color grade determination model to obtain second feature data corresponding to the stain region; and determining a cleanliness detection result of the dirty area based on the first feature data and the second feature data. According to the scheme, the multi-dimensional feature extraction is performed on the image data of the to-be-detected object, and the cleanliness detection result of the dirty area is determined according to the multiple feature data corresponding to the dirty area, so that human intervention is avoided, the dirty area can be intelligently detected, the detection result can be intelligently determined, and the detection efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of automated detection, and particularly to a cleanliness detection method, device, electronic device, and storage medium. Background Art

[0002] Cooking fume ducts are widely used in various catering places, such as restaurants, kitchens, hotels, etc., for discharging cooking fumes and waste gases. However, during long-term use, a large amount of grease and dirt will accumulate in the fume ducts, forming oil stain accumulations, which affect the smoke exhaust effect of the ducts, and may have a negative impact on the environment and air quality, and even pose a fire risk.

[0003] Traditional methods for cleaning and inspecting fume ducts mainly rely on manual operations, which are not only inefficient but also prone to missed inspections, increasing potential safety hazards. It is also possible to detect stains through sensor detection technology. However, the cleanliness detection technology of sensors mainly relies on sensors such as cooking fume concentration and fan operating status to indirectly detect the operating status of fume ducts. However, since the sensors cannot directly detect the accumulation of stains inside the fume ducts and can only provide indirect environmental parameters, they cannot accurately evaluate the cleanliness of the fume ducts. Therefore, although the sensor detection technology can capture some basic information during the operation of fume ducts, it cannot directly reflect the accumulation of stains inside the ducts, resulting in certain limitations in practical applications.

[0004] Based on the above existing technical solutions, there are problems such as relying on manual analysis or simple algorithms for detection, lacking intelligent evaluation and judgment capabilities, and being unable to efficiently and accurately identify the degree and distribution of stains inside fume ducts, resulting in inaccurate cleanliness detection. Summary of the Invention

[0005] The present invention provides a cleanliness detection method, device, electronic device, and storage medium to solve the problems of low efficiency and poor accuracy in detecting the cleanliness of stain areas in the prior art.

[0006] According to one aspect of the present invention, a cleanliness detection method is provided, including:

[0007] Obtain first image data of an object to be detected, where the first image data includes a stain area;

[0008] Process the first image data through a pre-trained image segmentation model to obtain an image segmentation result of the stain area, and determine first feature data corresponding to the stain area based on the image segmentation result;

[0009] Perform classification processing on the image segmentation result based on a pre-trained color level determination model to obtain second feature data corresponding to the stain area;

[0010] Determine the cleanliness detection result of the stain area based on the first feature data and the second feature data.

[0011] Optionally, the pre-trained image segmentation model includes a bounding box extraction sub-model and an image segmentation sub-model; processing the first image data through the pre-trained image segmentation model to obtain the image segmentation result of the stain area, including: performing bounding box extraction processing on the first image data through the bounding box extraction sub-model to obtain the second image data corresponding to the stain area in the first image data; performing image segmentation processing on the second image data through the image segmentation sub-model to obtain the image segmentation result corresponding to the stain area.

[0012] Optionally, determining the first feature data corresponding to the stain area based on the image segmentation result includes: determining the stain area data corresponding to the image segmentation result and the area data of the first image data, determining the stain area ratio data based on the stain area data and the area data of the first image data, and determining the stain area ratio data as the first feature data.

[0013] Optionally, the pre-trained color level determination model includes at least two color feature extraction sub-models and a color classification sub-model; classifying the image segmentation result based on the pre-trained color level determination model to obtain the second feature data corresponding to the stain area, including: processing the image segmentation result through at least two color feature extraction sub-models respectively to obtain the feature extraction results corresponding to each color feature extraction sub-model; performing classification processing on the feature extraction results corresponding to each color feature extraction sub-model through the color classification sub-model to obtain the color classification result, and determining the color classification result as the second feature data.

[0014] Optionally, the color feature extraction sub-model includes an HSV color feature extraction sub-model and an RGB color feature extraction sub-model, and the feature extraction result corresponding to the HSV color feature extraction sub-model includes HSV histogram feature data; the feature extraction result corresponding to the RGB color feature extraction sub-model includes RGB color multi-order moment feature data.

[0015] Optionally, determining the cleanliness detection result of the stain area based on the first feature data and the second feature data includes: obtaining the first mapping relationship between the first feature data and the cleanliness quantization data, and determining the first quantization data of the first feature data based on the first mapping relationship; and obtaining the second mapping relationship between the second feature data and the cleanliness quantization data, and determining the second quantization data of the second feature data based on the second mapping relationship; obtaining the weight data, and performing weighted summation processing on the first quantization data and the second quantization data based on the weight data to obtain the cleanliness detection result of the stain area.

[0016] Optionally, obtain weight data, including: obtaining at least two area thresholds corresponding to the first feature data, determining the weight data of the first quantization data based on the first quantization data corresponding to the first feature data and the at least two area thresholds; determining the weight data of the second quantization data based on the weight data of the first quantization data.

[0017] According to another aspect of the present invention, there is provided a cleanliness detection device, including:

[0018] A first image data acquisition module, configured to acquire first image data of an object to be detected, where the first image data includes a stain area;

[0019] A first feature data determination module, configured to process the first image data through a pre-trained image segmentation model to obtain an image segmentation result of the stain area, and determine first feature data corresponding to the stain area based on the image segmentation result;

[0020] A second feature data determination module, configured to perform classification processing on the image segmentation result based on a pre-trained color level determination model to obtain second feature data corresponding to the stain area;

[0021] A detection result determination module, configured to determine a cleanliness detection result of the stain area based on the first feature data and the second feature data.

[0022] According to another aspect of the present invention, there is provided an electronic device, the electronic device includes:

[0023] At least one processor; and

[0024] A memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the cleanliness detection method of any embodiment of the present invention.

[0026] According to another aspect of the present invention, there is provided a computer-readable storage medium, and the computer-readable storage medium stores computer instructions for causing a processor to implement the cleanliness detection method of any embodiment of the present invention when executed.

[0027] In the technical solution of the embodiment of the present invention, first image data of an object to be detected is obtained, where the first image data includes a stain area; the first image data is processed by a pre-trained image segmentation model to obtain an image segmentation result of the stain area, and first feature data corresponding to the stain area is determined based on the image segmentation result; the image segmentation result is classified by a pre-trained color level determination model to obtain second feature data corresponding to the stain area; and a cleanliness detection result of the stain area is determined based on the first feature data and the second feature data. In this solution, multi-dimensional feature extraction is performed on the image data corresponding to the object to be detected through different feature extraction methods, and the cleanliness detection result of the stain area is determined according to various feature data corresponding to the stain area, solving the problems of lack of intelligent evaluation and judgment capabilities and inaccurate cleanliness detection caused by the inability to efficiently and accurately identify the degree and distribution of stains inside the oil fume duct, avoiding manual intervention, and being able to intelligently detect the stain area and determine the detection result, improving the detection efficiency and accuracy.

[0028] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0030] Figure 1 is a flowchart of a cleanliness detection method provided in Embodiment 1 of the present invention;

[0031] Figure 2 is a schematic diagram of the distribution of weight coefficients applicable to the embodiments of the present invention;

[0032] Figure 3 is a flowchart of a cleanliness detection method provided in Embodiment 2 of the present invention;

[0033] Figure 4 is a flowchart of a cleanliness detection method provided in Embodiment 3 of the present invention;

[0034] Figure 5 is a schematic structural diagram of a cleanliness detection device provided in Embodiment 4 of the present invention;

[0035] Figure 6 is a schematic structural diagram of an electronic device for implementing the cleanliness detection method of the embodiments of the present invention. Detailed implementation manners

[0036] In order to enable those skilled in the art of the present technology to better understand the solution of the present invention, the following will clearly and completely describe the technical solution in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0037] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0038] Embodiment 1

[0039] Figure 1 is a flowchart of a cleanliness detection method provided in Embodiment 1 of the present invention. This embodiment is applicable to the situation of detecting the cleanliness of an object with stains. This method can be executed by a cleanliness detection device, which can be implemented in the form of hardware and / or software, and the cleanliness detection device can be configured in electronic devices such as computers, servers, and stain detection devices. As Figure 1 shown, this method includes:

[0040] S110. Obtain first image data of the object to be detected, where the first image data includes a stain area.

[0041] Among them, the object to be detected can be specifically understood as an object that needs to be subjected to cleanliness detection, which can be any object with stains. In this embodiment, the object to be detected can be an oil fume duct, especially the oil fume ducts in various catering places. During long-term use, a large amount of grease and dirt will accumulate in the oil fume duct, forming stain accumulation, affecting the smoke exhaust effect of the duct, and posing a safety risk. In addition, the oil fume duct covers different duct structures such as straight pipes and elbow pipes, and in different lighting environments, the stain forms corresponding to different degrees of stains will also be different, which brings certain difficulties to the detection of stains. Accurately detecting the cleanliness of the oil fume duct can better provide an economical and effective duct maintenance plan, improve the use safety and hygiene conditions of the duct, and avoid potential safety hazards caused by stain problems.

[0042] In this embodiment, the first image data can be specifically understood as an image used by a computer vision system to identify and locate stain areas. An image acquisition device can be used to acquire images of the object to be detected. For the acquired image data, preliminary image recognition processing can be performed to identify whether there are stain areas in the image data. If the recognition result of the acquired image data is that there are stain areas, the corresponding image data is determined as the first image data. If the recognition result of the acquired image data is that there are no stain areas, image acquisition continues, and image data containing stain areas is identified from it and determined as the first image data. This can avoid using image data without stain areas in the subsequent cleanliness detection process, reduce unnecessary image processing operations, reduce resource waste, and help improve the cleanliness detection efficiency. The stain areas in the first image data include but are not limited to areas with accumulated oil stains and areas with accumulated dust on the oil fume duct.

[0043] Preferably, in the scenario where the object to be detected is an oil fume duct, in order to ensure the clarity of the acquired image data, the image acquisition device needs to be inserted into the interior of the oil fume duct to obtain clear image data. In this case, the time for acquiring images needs to be set during the non-use period of the oil fume duct to avoid the oil fume passing through the oil fume duct from contaminating the lens of the image acquisition device, resulting in the problem of blurred images being acquired. A timed image acquisition task can be set according to the non-use period of the oil fume duct to control the image acquisition device to acquire images according to the timed image acquisition task.

[0044] Specifically, an image acquisition device can be installed at the object to be detected according to actual stain detection requirements, and the image acquisition device can be controlled to acquire images of the object to be detected through a timed task. Among them, the information set in the timed image acquisition task includes but is not limited to acquisition time and acquisition position data, enabling flexible acquisition of image data at different positions of the object to be detected, so as to obtain image data at different positions of the object to be detected, and thus enabling a comprehensive cleanliness detection of the object to be detected.

[0045] In this embodiment, by obtaining the first image data of the object to be detected, a reliable data basis is provided for intelligently determining the cleanliness detection result of the object to be detected, so as to realize intelligent cleanliness detection, which helps to improve the efficiency of cleanliness detection.

[0046] S120. Process the first image data through a pre-trained image segmentation model to obtain an image segmentation result of the stain area, and determine first feature data corresponding to the stain area based on the image segmentation result.

[0047] Among them, the pre-trained image segmentation model can be specifically understood as an identification model pre-constructed according to image processing requirements. Exemplarily, the pre-trained image segmentation model includes but is not limited to a bounding box extraction sub-model and an image segmentation sub-model. First, the first image data can be processed through the bounding box extraction sub-model to obtain the bounding box extraction result of the region of interest in the first image data, and then the image segmentation sub-model can be used to perform image segmentation processing on the bounding box extraction result to obtain the image segmentation result. Among them, the region of interest is the stain area. Exemplarily, the image segmentation result can be a stain area mask. The first feature data can be specifically understood as the feature data characterizing the stain area in the first image data, including but not limited to the stain area area feature data. The stain area area feature data includes one or more of the stain area area data and the stain area area ratio data. The stain area area data represents the pixel area data of the stain area, and the stain area area ratio data represents the ratio data of the pixel area data of the stain area relative to the pixel area of the first image data. Exemplarily, corresponding feature data determination algorithms can be respectively called to process the image segmentation result to obtain the corresponding first feature data.

[0048] Specifically, when the first image data of the object to be detected is obtained, the pre-trained image segmentation model is called, and the first image data is processed through the pre-trained image segmentation model to segment the region of interest in the first image data, that is, the image segmentation result of the stain area. The feature data determination algorithm is called to process the image segmentation result to determine the first feature data corresponding to the stain area. Optionally, the stain area area data of the image segmentation result can also be determined as the first feature data, and the pollution degree of the object to be detected can be characterized by the area data of the stain area, which can be set according to the actual situation.

[0049] In some embodiments, when obtaining the area data and / or the area ratio data of the stain region in the image segmentation result, the stain level can be determined according to the area data and / or the area ratio data of the stain region, which is used to more intuitively characterize the pollution level of the object to be detected. Correspondingly, the cleanliness level of the object to be detected can also be more intuitively characterized. When obtaining the area data of the stain region in the image segmentation result, multiple stain area ranges can be determined according to different detection requirements, and a corresponding stain level is set for each stain area range. Based on each stain level and the corresponding stain area range, a corresponding mapping relationship is constructed and the obtained corresponding relationship is stored. When determining the area data of the stain region in the image segmentation result, this mapping relationship can be called to obtain the stain level corresponding to the area data of the stain region. When obtaining the area ratio data of the stain region, a corresponding relationship curve can be constructed according to the area ratio data of the stain region and the corresponding stain level in the historical detection results, and a mapping relationship between the continuous area ratio data of the stain region and the corresponding stain level can be obtained, so as to more precisely determine the stain level. Then, when determining the area ratio data of the stain region in the image segmentation result, the corresponding relationship curve is called to obtain the stain level corresponding to the area ratio data of the stain region. Preferably, when obtaining the area data of the stain region and the area ratio data of the stain region in the image segmentation result, the stain levels corresponding to the area data of the stain region and the area ratio data of the stain region in the image segmentation result are determined respectively, and through weighted summation processing of each stain level, the target stain level is determined according to the result of the weighted summation processing. Preferably, the stain level corresponding to the area data of the stain region or the area ratio data of the stain region in the image segmentation result can be determined as the first feature data, and the result of weighted summation of the stain levels corresponding to the area data of the stain region or the area ratio data of the stain region in the image segmentation result can also be determined as the first feature data. Comprehensive analysis of the stain levels corresponding to the area data of the stain region and the area ratio data of the stain region is beneficial to improving the accuracy of judging the pollution level from the area factor.

[0050] Optionally, determining the first feature data corresponding to the stain region based on the image segmentation result includes: determining the area data of the stain region corresponding to the image segmentation result and the area data of the first image data, determining the area ratio data of the stain region based on the area data of the stain region and the area data of the first image data, and determining the area ratio data of the stain region as the first feature data.

[0051] In this embodiment, the image segmentation result is a stain region mask. After obtaining the stain region mask, the pixels in the stain region mask can be accumulated to obtain the area data of the stain region. The calculation formula for the area data of the stain region is as follows:

[0052]

[0053] Among them, A represents the pixel area of the stain region, H and W respectively represent the height and width of the first image data; M(i, j) represents that the mask pixel is the foreground region of the stain.

[0054] In some embodiments, the mask of the first image data can be determined, the pixels in the mask of the first image data are summed to obtain the area data of the first image data, the ratio of the stain region area data to the area data of the first image data is calculated, the obtained ratio is determined as the stain region area occupancy data, and the stain region area occupancy data is determined as the first feature data.

[0055] In this embodiment, by determining the stain region area occupancy data as the first feature data, the pollution degree of the stain region can be characterized more intuitively, and at the same time, the fine-grained characterization of the pollution degree of the stain region is improved, which helps to achieve high-precision pipeline cleanliness detection.

[0056] In this embodiment, the image segmentation result of the stain region is determined by a pre-trained image segmentation model, and the first feature data corresponding to the stain region is determined according to the image segmentation result. The first feature data is the area feature data representing the stain, and the detection result of the stain region can be determined from the area dimension, providing a reliable data basis for the subsequent comprehensive analysis of the cleanliness of the object to be detected.

[0057] S130. Classify the image segmentation result based on the pre-trained color level determination model to obtain the second feature data corresponding to the stain region.

[0058] Among them, the pre-trained color level determination model can be specifically understood as a model pre-trained according to image processing requirements, which is used to determine the color feature data of the stain area. The pre-trained color level determination model includes, but is not limited to, at least two color feature extraction sub-models and a color classification sub-model. The at least two color feature extraction sub-models are used to extract various color feature data corresponding to the image segmentation result, that is, to determine various color feature data corresponding to the stain area. Each color feature extraction sub-model in the at least two color feature extraction sub-models is a different model. The color feature extraction sub-model includes, but is not limited to, the HSV color feature extraction sub-model and the RGB color feature extraction sub-model. The HSV color feature extraction sub-model is used to extract HSV color features, and the RGB color feature extraction sub-model is used to extract RGB color features. By extracting color feature data through at least two color feature extraction sub-models, various color feature data can be obtained, thereby providing various color features for the color classification sub-model, which helps to improve the accuracy of the color classification result. The color classification sub-model is a machine learning model used to classify colors according to various color feature data, and the color classification sub-model can be constructed based on the random forest algorithm. The second feature data can be specifically understood as the color feature classification result, and the color feature classification result can be represented by classification levels. Exemplarily, the second feature data can be represented in ways such as first level, second level, and third level. It should be noted that other representation methods can also be used to represent the second feature data, which is not limited here.

[0059] Specifically, call the pre-trained color level determination model, use the image segmentation result as the input data of the pre-trained color level determination model, and perform classification processing on the image segmentation result through the pre-trained color level determination model to determine the color feature classification result corresponding to the stain area, that is, obtain the representation result of the pollution degree of the stain area in the color dimension. In this embodiment, using the image segmentation result as the input data of the pre-trained color level determination model to determine the color feature classification result corresponding to the stain area effectively utilizes the image segmentation result determined in the above steps. Only using the image segmentation result as the input data instead of the entire first image data can improve the efficiency of determining the color feature classification result and reduce the calculation amount.

[0060] In this embodiment, perform classification processing on the image segmentation result through the pre-trained color level determination model to obtain the second feature data corresponding to the stain area, thereby determining the color classification result of the stain area, and the detection result of the stain area can be determined from the color dimension, providing a reliable data basis for the subsequent comprehensive analysis of the cleanliness of the object to be detected.

[0061] S140. Determine the cleanliness detection result of the stain area based on the first feature data and the second feature data.

[0062] It should be noted that for the same object to be detected, under different degrees of contamination, the proportions of different characteristic data in the cleanliness detection result will also be different. That is, the proportions of the first characteristic data and the second characteristic data in the cleanliness detection result will be different. The corresponding weight data can be dynamically adjusted according to the actually detected characteristic data, so as to optimize the cleanliness detection result of the object to be detected to meet the cleanliness detection requirements of different objects to be detected. The cleanliness detection result can be specifically understood as the result representing the degree of stain accumulation, and can be characterized by different quantization specification data. Exemplarily, the cleanliness detection result can be characterized by quantization specification data such as cleanliness value, cleanliness level, etc.

[0063] Exemplarily, if it is detected that the area of the stain region is greater than the preset area threshold or the proportion of the stain region area is greater than the preset proportion threshold, it means that the proportion of the first characteristic data in the cleanliness detection result is greater than the proportion of the second characteristic data in the cleanliness detection result, and the weight data of the first characteristic data can be adjusted to be greater than the weight data of the second characteristic data; if it is detected that the color characteristic classification result meets the preset color classification level, it means that the influence proportion of the second characteristic data in the cleanliness detection result is greater than the influence proportion of the first characteristic data in the cleanliness detection result, and the weight data of the second characteristic data can be adjusted to be greater than the weight data of the first characteristic data, so as to realize the flexible adjustment of the weight data of each characteristic data according to the actual detection situation to obtain a more accurate cleanliness detection result of the stain region.

[0064] In some embodiments, the corresponding weight data is matched according to the requirements of the object to be detected and / or the cleanliness detection accuracy, and the first characteristic data and the second characteristic data are weighted and summed according to the corresponding weight data to obtain the cleanliness detection result of the stain region.

[0065] In some embodiments, the first characteristic data and the second characteristic data can also be processed by a pre-trained cleanliness prediction model to obtain the cleanliness detection result of the stain region. Among them, the pre-trained cleanliness prediction model is a machine learning model trained according to the characteristic data extracted from the historical cleanliness detection data corresponding to the object to be detected.

[0066] In this embodiment, the cleanliness detection result of the stain region is determined according to the first characteristic data and the second characteristic data, realizing the comprehensive processing of different characteristic data through mutual cooperation to obtain the cleanliness detection result, effectively avoiding the deviation problem caused by a single characteristic data, making the obtained cleanliness detection result more objective and accurate, and effectively improving the cleanliness detection efficiency.

[0067] In some embodiments, determining the cleanliness detection result of the stain area based on the first feature data and the second feature data includes: obtaining a first mapping relationship between the first feature data and the cleanliness quantization data, and determining the first quantization data of the first feature data based on the first mapping relationship; and obtaining a second mapping relationship between the second feature data and the cleanliness quantization data, and determining the second quantization data of the second feature data based on the second mapping relationship; obtaining weight data, and performing a weighted summation process on the first quantization data and the second quantization data based on the weight data to obtain the cleanliness detection result of the stain area.

[0068] It can be understood that the first feature data and the second feature data are feature data in different dimensions, so there is a problem of inconsistency in the corresponding quantization data. To improve the consistency and robustness of the cleanliness detection, the first feature data and the second feature data can be respectively subjected to unified quantization processing based on the cleanliness quantization data, and quantization data corresponding to the first feature data and the second feature data that meet the consistency can be obtained. The cleanliness quantization data can be specifically understood as data used to quantify the pollution degree of the stain area. In this embodiment, it refers to the quantization data set for the unified quantization processing of the first feature data and the second feature data. The cleanliness quantization data includes but is not limited to score data and grade data. Exemplarily, the score data can be represented by scores such as 0.1, 0.2, or 0.3, and the grade data can also be represented by numbers such as 1, 2, 3, etc., which are not limited here. The first quantization data represents the quantization data of the first feature data and is used to quantify and reflect the pollution degree of the stain area in the area feature dimension. Exemplarily, the first quantization data can be represented by grade data or score data. The second quantization data represents the quantization data corresponding to the second feature data and is used to quantify and reflect the pollution degree of the stain area in the color feature dimension. Exemplarily, the second quantization data can be represented by grade data or score data. It should be noted that the cleanliness quantization data corresponding to the first feature data and the second feature data is consistent. Any kind of cleanliness quantization data can be selected in advance. Exemplarily, the grade data is selected as the cleanliness quantization data, and the corresponding grade data is set for each first feature data to obtain the cleanliness quantization data corresponding to each feature data, and then the first mapping relationship between the first feature data and the corresponding cleanliness quantization data is constructed. The corresponding grade data is set for each second feature data to obtain the cleanliness quantization data corresponding to each feature data, and then the second mapping relationship between the second feature data and the corresponding cleanliness quantization data is constructed. After the first mapping relationship and the second mapping relationship are constructed, the first mapping relationship and the second mapping relationship are stored in a preset storage space for direct invocation when subsequent feature data quantization processing is performed.

[0069] It should be noted that the cleanliness detection requirements for different objects to be detected may vary. Therefore, corresponding mapping relationships can be constructed in advance for different types of objects to be detected, and during application, matching can be performed according to the type of the object to be detected or the cleanliness detection accuracy to obtain a mapping relationship that matches the object to be detected.

[0070] Specifically, retrieval can be performed in a preset storage space according to the type of the object to be detected or the cleanliness detection accuracy to obtain a first mapping relationship that matches the object to be detected, and a second mapping relationship that matches the object to be detected can be obtained from the preset storage space. Then, the first feature data is matched with the first mapping relationship, and the cleanliness quantization data that matches the first feature data is determined as the first quantization data corresponding to the first feature data. The second feature data is matched with the second mapping relationship, and the cleanliness quantization data that matches the second feature data is determined as the second quantization data corresponding to the second feature data. Further, weight data corresponding to each feature data can be obtained in the preset storage space, and the first quantization data and the second quantization data are weighted and summed according to the weight data to obtain the cleanliness detection result of the stain area.

[0071] Preferably, the corresponding weight data can be dynamically adjusted according to the actually detected first quantization data and second quantization data to improve the cleanliness detection result. Exemplarily, it can be determined by comparing the first quantization data with a preset first quantization threshold. If the first quantization data is greater than the first quantization threshold, the weight data corresponding to the first quantization data is increased, and the weight data corresponding to the second quantization data is decreased; correspondingly, it can also be determined by comparing the second quantization data with a preset second quantization threshold. If the second quantization data is greater than the second quantization threshold, the weight data corresponding to the second quantization data is increased, and the weight data corresponding to the first quantization data is decreased; if the first quantization data is greater than the first quantization threshold and the second quantization data is greater than the second quantization threshold, the weight data corresponding to the first quantization data and the second quantization data may not be adjusted.

[0072] Optionally, obtaining the weight data includes: obtaining at least two area thresholds corresponding to the first feature data, determining the weight data of the first quantization data based on the first quantization data corresponding to the first feature data and the at least two area thresholds; and determining the weight data of the second quantization data based on the weight data of the first quantization data.

[0073] It should be noted that when comprehensively considering the area feature and the color feature, if the area of the stain region is within different area ranges, the degree of influence on the cleanliness is different. Multiple corresponding area threshold data can be set according to the classification levels of the first feature data. Exemplarily, if the classification level of the first feature data is three levels, two area thresholds can be set, where the first area threshold is less than the second area threshold. Then, the first quantization data corresponding to the first feature data is compared with the two area thresholds. When the obtained first quantization data is less than the preset first area threshold, it indicates that the stain region in this case is not large. At this time, if the color is very deep, it is also likely to cause the second quantization data to be very large and greater than the preset color threshold. At this time, the traditional cleanliness monitoring method is likely to output a cleanliness level result of medium or even severe. However, in actual judgment, it is more desirable to define it as a mild cleanliness level. In addition, when the obtained first quantization data is greater than the preset second area threshold, it indicates that the stain region in this case is very large. At this time, if the color is very light, it is likely to cause the second quantization data to be very small. At this time, the traditional cleanliness monitoring method is likely to output a cleanliness level result of mild or medium. However, in actual judgment, it is more desirable to define it as a severe cleanliness level.

[0074] As described above, in actual applications, whether the area of the stain region is too small or too large, it is more desirable to ignore the influence of the color feature on the cleanliness level determination result, and it is more inclined to determine a mild cleanliness level when the area of the stain region is too small and a severe cleanliness level when the area of the stain region is too large. Therefore, it is necessary to further design a more reasonable weight allocation method.

[0075] Specifically, according to the classification method corresponding to the first feature data, at least two area thresholds corresponding to the first feature data are determined. Exemplarily, if the classification method is a three-stage classification, the first area threshold and the second area threshold corresponding to the first feature data can be set, where the first area threshold is less than the second area threshold. The weight calculation method is called to calculate the first quantization data corresponding to the first feature data and the at least two area thresholds, and the weight data of the first quantization data is obtained. The weight data of the second quantization data is adjusted according to the weight data of the first quantization data. The dynamic adjustment of the weight data corresponding to the first quantization data and the second quantization data is realized, which helps to improve the flexibility and adaptability of the cleanliness detection.

[0076] In a specific embodiment, the first quantization data is defined as A, the second quantization data is defined as B. If the first quantization data is divided into three levels, the corresponding preset first area threshold C and preset second area threshold D are defined, where D > C; the cleanliness level S calculation formula can be expressed as:

[0077] S = (1 - y)·A + y·B, 0 ≤ y ≤ 1;

[0078] Among them, 1 - y represents the weight data of the first quantization data, y represents the weight data of the second quantization data, and the calculation formula of y is as follows:

[0079] Among them,

[0080] Among them, k is a positive number; since the weight coefficient y is a normal distribution function of A, for example, as Figure 2 shown in a schematic diagram of a weight coefficient distribution, when A is less than C, y tends to 0, and thus the influence of B on S can be ignored. Even if the color is very deep during cleanliness detection, it will not be defined as a severe cleanliness level; similarly, when A is greater than D, y also tends to 0, and thus the influence of B on S can also be ignored. Even if the color is very light during cleanliness detection, it will not be defined as a medium or mild cleanliness level, but is easily defined as a severe cleanliness level. Moreover, through the dynamic non - polar adjustment method of the continuous function of the weight coefficient y, the weight can be dynamically and non - polarly adjusted to improve the output result of the cleanliness level. Specifically, refer to Figure 2 , Figure 2 which is a schematic diagram of a weight coefficient distribution applicable to the embodiments of the present invention.

[0081] It should be noted that the values of C, D, and k can be reasonably set in combination with the obtained value of A and the corresponding cleanliness situation. For example, C is set to 0.2, D is set to 0.6, and k is set to 3.

[0082] On the basis of the above - mentioned embodiments, after obtaining the cleanliness detection result of the stain area, a corresponding cleanliness evaluation rule can be set according to the cleanliness detection result. Preferably, the cleanliness can be divided into mild, medium, and severe levels according to the cleanliness detection result. The cleanliness evaluation rule is set as follows:

[0083]

[0084] Among them, S represents the cleanliness detection result, and S1 and S2 are preset pipeline cleanliness score thresholds, representing the division criteria for mild, medium, and severe levels respectively.

[0085] Based on the above embodiments, the cleanliness detection result includes one or more of a cleanliness value and a cleanliness level; the first characteristic data includes stain area data; the second characteristic data includes color characteristic data; the cleanliness detection method further includes: generating a corresponding detection report based on the cleanliness value and / or cleanliness level, stain area data, and color characteristic data according to a preset cleanliness detection report template, and displaying the detection report; and / or, if the cleanliness value and / or cleanliness level is greater than or equal to the corresponding preset warning threshold, generating a warning message and displaying the warning message.

[0086] In this embodiment, the cleanliness detection result can be preset. The cleanliness detection result includes one or more of a cleanliness value and a cleanliness level. Exemplarily, it can be default set to characterize the cleanliness detection result by the cleanliness level.

[0087] Specifically, after obtaining the first characteristic data, the second characteristic data, and the cleanliness detection result, retrieve the preset cleanliness detection report template, and process the first characteristic data, the second characteristic data, and the cleanliness detection result according to the index operation method corresponding to the index data in the detection report template to obtain the corresponding index data, and fill the corresponding index data into the detection report template to generate the corresponding detection report, and display the detection report on the corresponding display platform. Preferably, the corresponding chart result can also be generated based on the currently obtained first characteristic data, second characteristic data, and cleanliness detection result in combination with the corresponding historical inspection results, and information such as detection frequency, detection efficiency, characteristic change trend, and detection result change trend can be displayed in the form of a chart, which is convenient for users to comprehensively understand the pollution degree and distribution of the stain area of the object to be detected.

[0088] In the case of obtaining the cleanliness value and / or cleanliness level, compare one or more of the cleanliness value and cleanliness level with the corresponding preset warning threshold respectively. If one or more of the cleanliness value and cleanliness level is greater than or equal to the corresponding preset warning threshold, add the cleanliness value and cleanliness level to the warning message template, thereby generating a warning message, and transmit the warning message to the display platform, and the display platform displays the warning message. The warning message can also be sent to the user terminal device, which can remind the user to clean the object to be detected in time to remove the safety hazard caused by the stain problem in time.

[0089] The technical solution of this embodiment is to obtain the first image data of the object to be detected, where the first image data includes a stain area; process the first image data through a pre-trained image segmentation model to obtain the image segmentation result of the stain area, and determine the first feature data corresponding to the stain area based on the image segmentation result; classify and process the image segmentation result based on the pre-trained color level determination model to obtain the second feature data corresponding to the stain area; determine the cleanliness detection result of the stain area based on the first feature data and the second feature data. This solution performs multi-dimensional feature extraction on the image data corresponding to the object to be detected through different feature extraction methods, and determines the cleanliness detection result of the stain area according to various feature data corresponding to the stain area, solving the problem of inaccurate cleanliness detection caused by the lack of intelligent evaluation and judgment ability and the inability to efficiently and accurately identify the degree and distribution of stains inside the oil fume duct, avoiding human intervention, and being able to intelligently detect the stain area and determine the detection result, improving the detection efficiency and accuracy.

[0090] Embodiment 2

[0091] Figure 3 FIG. is a flowchart of a cleanliness detection method provided by the second embodiment of the present invention. The method of this embodiment is a further optimization of the method of the above embodiment. Optionally, the pre-trained image segmentation model includes a bounding box extraction sub-model and an image segmentation sub-model; the first image data is processed by the bounding box extraction sub-model for bounding box extraction to obtain the second image data corresponding to the stain area in the first image data; the second image data is processed by the image segmentation sub-model for image segmentation to obtain the image segmentation result corresponding to the stain area. As Figure 3 shown, the method includes:

[0092] S310. Obtain the first image data of the object to be detected, where the first image data includes a stain area.

[0093] S320. Process the first image data by the bounding box extraction sub-model for bounding box extraction to obtain the second image data corresponding to the stain area in the first image data.

[0094] Among them, the bounding box extraction sub-model can be specifically understood as an object detection model for identifying and extracting the bounding box information of the stain area in the first image data. The second image data can be specifically understood as the bounding box data corresponding to the stain area.

[0095] Specifically, in the case of obtaining the first image data of the object to be detected, the bounding box extraction sub-model is called, and the first image data is processed by the bounding box extraction sub-model for bounding box extraction to obtain the bounding box data corresponding to the stain area in the first image data, that is, the second image data corresponding to the stain area is obtained.

[0096] In a specific embodiment, the bounding box extraction sub-model is an object detection model for identifying oil stains in an oil pipeline. In order to obtain a model with good reliability and strong generality, an image generation model is used to synthesize images according to preset conditions to obtain synthetic image data that meets the preset conditions. Among them, the preset conditions are conditions set according to the oil stain accumulation situation, including but not limited to the shape, thickness, color, and distribution of the oil stains. A high-quality image data set containing various oil stain distributions and accumulation situations is constructed from the actually collected image data of the oil pipeline and the synthetic image data, providing image samples for the subsequent recognition model. The oil stain image data set is annotated with bounding boxes and region masks using an annotation tool, and a detection data set and a segmentation data set are established respectively. The complete data set is divided into a training set, a validation set, and a test set, which are used for model training, parameter tuning, and performance verification respectively to ensure the reliability and generality of the model. The bounding box extraction sub-model is trained using the determined training set, and this model accurately locates the stain area in the image and obtains the oil stain bounding box information. The output bounding box B of the detection model is:

[0097] B = {(x min , y min , x max , y max )};

[0098] Among them, x min , x max respectively represent the minimum and maximum values of the abscissa of the bounding box, and y min , y max respectively represent the minimum and maximum values of the ordinate of the bounding box. After the model is trained based on the training set, the trained bounding box extraction sub-model is further tuned for parameters and verified for performance through the validation set and the test set, so as to obtain a trained bounding box extraction sub-model.

[0099] In this embodiment, the bounding box extraction sub-model is used to perform bounding box extraction processing on the first image data to obtain the bounding box corresponding to the stain area in the first image data, that is, the second image data. This not only reduces the interference information in the image segmentation process but also avoids the omission of the effective information corresponding to the stain area, ensuring the integrity of the information of the stain area, providing accurate bounding box data for subsequent image segmentation, and helping to improve the efficiency and accuracy of determining the image segmentation result.

[0100] S330. Perform image segmentation processing on the second image data through the image segmentation sub-model to obtain the image segmentation result corresponding to the stain area.

[0101] Among them, the image segmentation sub-model can be specifically understood as an image semantic segmentation model, which is a model for segmenting image data. In this embodiment, the image segmentation sub-model can be specifically used to segment the stain area in the bounding box corresponding to the second image data to obtain a region mask corresponding to the stain area. Preferably, the image segmentation result can be a stain area mask.

[0102] It should be noted that the second image data specifically represents the bounding box information containing the stain area, rather than the contour information of the stain area. Therefore, in order to obtain accurate first feature data, the second image data needs to be subjected to image segmentation processing to obtain the segmentation result of the stain area. Specifically, in the case of obtaining the second image data of the object to be detected, the image segmentation sub-model is called, and the second image data is subjected to image segmentation processing by the image segmentation sub-model to obtain the image segmentation result corresponding to the stain area, that is, the oil stain area mask within the stain area.

[0103] In a specific embodiment, the data set processed by the bounding box extraction sub-model can be used as the training set of the image segmentation sub-model for training the image segmentation sub-model. Through per-pixel mask annotation, the model identifies the stain area within the bounding box and generates a high-precision oil stain mask. The binary output mask M of the image segmentation sub-model is:

[0104] M = {p ij |p ij ∈[0,1], (i,j)∈B};

[0105] Among them, P ij represents the pixel value of the pixel block at the i-th row and j-th column in the bounding box B. After the model is trained based on the training set, the validation set and test set processed by the bounding box extraction sub-model are used to optimize the parameters and verify the performance of the trained image segmentation sub-model, so as to obtain a trained image segmentation sub-model. The image segmentation sub-model realizes the fine segmentation of the stain area.

[0106] In this embodiment, the first image data is processed by combining the bounding box extraction sub-model and the image segmentation sub-model, that is, the bounding box extraction sub-model extracts the stain bounding box information in the first feature data, and further the image segmentation sub-model performs image segmentation processing on the stain bounding box information to achieve fine segmentation of the stain area, obtain the stain area mask within the accurate bounding box, reduce the misdetection problem in the stain detection and segmentation process, and perform integrated analysis on the results of the two sub-models to ensure the accuracy and integrity of the recognition result of the stain area.

[0107] S340. Determine the first feature data corresponding to the stain area based on the image segmentation result.

[0108] S350. Classify the image segmentation result based on the pre-trained color level determination model to obtain the second feature data corresponding to the stain area.

[0109] S360. Determine the cleanliness detection result of the stain area based on the first feature data and the second feature data.

[0110] In the technical solution of this embodiment, by obtaining the first image data of the object to be detected, where the first image data includes the stain area; performing bounding box extraction processing on the first image data through the bounding box extraction sub-model to obtain the second image data corresponding to the stain area in the first image data; performing image segmentation processing on the second image data through the image segmentation sub-model to obtain the image segmentation result corresponding to the stain area; determining the first feature data corresponding to the stain area based on the image segmentation result; classifying the image segmentation result based on the pre-trained color level determination model to obtain the second feature data corresponding to the stain area; determining the cleanliness detection result of the stain area based on the first feature data and the second feature data. This solution performs multi-dimensional feature extraction on the image data corresponding to the object to be detected through different feature extraction methods, and determines the cleanliness detection result of the stain area according to the multiple feature data corresponding to the stain area, solving the problem of inaccurate cleanliness detection caused by the lack of intelligent evaluation and judgment ability and the inability to efficiently and accurately identify the degree and distribution of stains inside the oil fume duct, avoiding human intervention, and being able to intelligently detect the stain area and determine the detection result, improving the detection efficiency and accuracy.

[0111] Embodiment III

[0112] Figure 4 is a flowchart of a cleanliness detection method provided in Embodiment III of the present invention. The method of this embodiment is a further optimization of the method of the above embodiment. Optionally, the pre-trained color level determination model includes at least two color feature extraction sub-models and a color classification sub-model; respectively process the image segmentation result through at least two color feature extraction sub-models to obtain the feature extraction results corresponding to each color feature extraction sub-model; classify the feature extraction results corresponding to each color feature extraction sub-model through the color classification sub-model to obtain the color classification result, and determine the color classification result as the second feature data. As Figure 4 shown, the method includes:

[0113] S410. Obtain the first image data of the object to be detected, where the first image data includes the stain area.

[0114] S420. Process the first image data through the pre-trained image segmentation model to obtain the image segmentation result of the stain area, and determine the first feature data corresponding to the stain area based on the image segmentation result.

[0115] S430. Process the image segmentation result through at least two color feature extraction sub - models respectively to obtain the feature extraction results corresponding to each color feature extraction sub - model.

[0116] Among them, the color feature extraction sub - model can be specifically understood as a model used to extract color features in image data. The color feature extraction sub - model includes but is not limited to the HSV color feature extraction sub - model and the RGB color feature extraction sub - model. The feature extraction results corresponding to different color feature extraction sub - models are different. Optionally, the color feature extraction sub - model includes the HSV color feature extraction sub - model and the RGB color feature extraction sub - model. The feature extraction result corresponding to the HSV color feature extraction sub - model includes HSV histogram feature data; the feature extraction result corresponding to the RGB color feature extraction sub - model includes RGB color multi - order moment feature data.

[0117] Specifically, call at least two color feature extraction sub - models according to the actual detection requirements. The color feature extraction sub - model can be the HSV color feature extraction sub - model or the RGB color feature extraction sub - model. At least two color feature extraction sub - models can be set as different models to obtain a variety of color feature data, improve the diversity of color feature extraction, and contribute to improving the accuracy of subsequent classification results. Process the image segmentation result through at least two color feature extraction sub - models respectively to obtain the feature extraction results corresponding to each color feature extraction sub - model.

[0118] Exemplarily, obtain the original sample data set, which includes the image data of the actual collected oil - stained pipeline and the corresponding synthetic image data. Classify and label according to the color features of the stains to establish a color - graded data set. In order to improve the diversity of the data set and the generalization ability of the model, an offline data augmentation method is used for data augmentation processing. The offline data augmentation method includes: Random contrast adjustment: Simulate the color changes of images under different lighting conditions; Random brightness adjustment: Expand the performance characteristics of images containing stained areas in high - light and low - light environments.

[0119] Apply the offline data augmentation method to the color - graded data set to generate a rich data set containing various color distributions and lighting conditions, providing high - quality samples for model training.

[0120] Before performing color feature extraction, image segmentation processing is carried out on the image data in the sample dataset. It can be processed through a pre-trained image segmentation model to obtain the corresponding stain area mask. Then, color feature extraction is performed on the stain area mask, and the color features are analyzed and a feature vector is constructed. Extracting the HSV color histogram specifically characterizes the extraction of the hue H, saturation S, and value V histogram features of the stain area to quantify the color distribution. The processing process of determining the HSV color histogram through the HSV color feature extraction sub-model is as follows:

[0121] Convert the image data in the sample dataset from the RGB color space to the HSV color space. The formula is as follows:

[0122] V = max(R, G, B);

[0123]

[0124]

[0125] Among them, H is the hue, representing the type of color; S is the saturation, representing the intensity or purity of the color; V is the value, representing the brightness of the color; R, G, and B are the red, green, and blue components of each pixel, and R, G, B ∈ [0, 1]. Extract the pixels of the stain area in the image according to the mask. The formula is as follows:

[0126] P H,S,V = {(H i,j , S i,j , V i,j ) | M i,j = 1};

[0127] Among them, M is the binary image of the segmentation mask, and M i,j = 1 means obtaining the foreground stain area corresponding to the pixel value of 1 at the i-th row and j-th column in the mask image.

[0128] Histogram partitioning: Divide the HSV components into n H , n S , n V intervals.

[0129] Count the number of HSV components of the regional pixels, calculate the number of pixels in each histogram interval, and the formula for the number of pixels H hist [k] in the k-th histogram interval of the H component is as follows:

[0130]

[0131] Among them, H i,j represents the hue H component of the pixel (i, j) in the image, is the width of the hue H interval, and M i,j= 1 represents the foreground stain area corresponding to the pixel value of 1 at the i-th row and j-th column in the mask image. represents mapping the hue H i,j to the corresponding interval index, that is, determining the interval to which the current pixel belongs. The δ function is used to count the pixels that meet the conditions. This formula is used to calculate M i,j = 1 area of the hue H histogram. By traversing all pixels (i, j), if the hue H of the pixel i,j is in the interval k and satisfies M i,j = 1, then it is included in H hist [k].

[0132] Finally, a hue H histogram H H with a length of n hist is obtained, and each element corresponds to the number of pixels in a histogram interval. Similarly, the saturation S histogram S hist and the brightness V histogram V hist are counted.

[0133] Histogram normalization: Normalize each histogram into a probability density distribution. The formula is as follows:

[0134]

[0135] Similarly, normalize the histogram probability density distributions of the saturation and brightness components S norm [k] and V norm [k]. Finally, output the normalized HSV histogram features H norm , S norm , V norm .

[0136] The RGB color feature extraction sub-model is used to calculate features such as the mean, standard deviation, and skewness of the stain area, representing the distribution concentration of colors. According to the obtained mean, standard deviation, and skewness data, the corresponding RGB color moment feature data is formed. The processing process for determining the RGB color multi-order moment feature data by the RGB color feature extraction sub-model is as follows:

[0137] For the RGB components of the stain area in the mask image, calculate the effective pixel set P c respectively:

[0138] P c = {p c,(i,j) | M i,j = 1}, c ∈ {R, G, B};

[0139] Among them, M i,j = 1 represents the foreground stain area corresponding to the pixel value of 1 at the pixel coordinates (i, j) in the mask image, c represents the RGB component of the stain image, and pc,(i,j) Represents the pixel value of the pixel coordinates (i, j) corresponding to the c-th color component of the image. For each RGB component, the first-order, second-order, and third-order color moments are calculated respectively.

[0140] The first-order moment represents the average value of the color component (R, G, B) over the entire valid region and is used to describe the overall intensity of the color. The formula is:

[0141]

[0142] Where, is the mean value of the c-th color component in the image, N is the total number of valid pixels in the image, and p c,i is the value of the i-th pixel on the color component c (i represents the i-th pixel, not the i in the pixel coordinates (i, j)).

[0143] The second-order moment represents the standard deviation of the color component and is used to describe the distribution range or dispersion degree of the color values. The formula is:

[0144]

[0145] Where, is the standard deviation of the c-th color component in the image, is the mean value of the c-th color component in the image.

[0146] The third-order moment represents the skewness of the color component and is used to describe the symmetry of the color distribution. The formula is:

[0147]

[0148] Where, is the skewness of the c-th color component in the image, is the standard deviation of the c-th color component in the image, which is used to normalize the deviation of the data, is the mean value of the c-th color component in the image. From the RGB color moment feature data is formed.

[0149] S440. Through the color classification sub-model, classify the feature extraction results corresponding to each color feature extraction sub-model to obtain the color classification result, and determine the color classification result as the second feature data.

[0150] Among them, the color classification sub-model is a random forest model constructed based on the random forest algorithm, which is used to determine the color classification result according to the extracted color feature data. By extracting the color features, the random forest model is trained to classify the stain colors. The random forest model is a parallel ensemble learning algorithm composed of multiple decision trees. When constructing each tree, features are randomly selected. By combining multiple weak classifiers, it can effectively handle high-dimensional features and noise interference. The final result is obtained through voting or taking the average, making the result of the overall model have high accuracy and generalization performance, and also having good stability, thus improving the classification accuracy.

[0151] Specifically, the feature extraction results corresponding to each color feature extraction sub-model are determined as the color feature data of the corresponding image data by the color classification sub-model. For example, the obtained HSV histogram feature H norm , S norm , V norm and the RGB color multi-moment feature data form the color feature vector of the corresponding image data. The obtained color feature vector is used as the input data of the color classification sub-model. The color classification sub-model processes the input data and outputs the color classification result, which is determined as the second feature data.

[0152] For training the color classification sub-model, the specific process is as follows:

[0153] (1) Construct a sample data set: The original data set set is a set containing N samples. Each sample consists of a pair (x i , y i ), where: x i is the input feature vector, representing the input data of the i-th sample, and y i is the label, representing the color classification label of the i-th sample. The sub-data set D t is obtained from the original data set D through sampling with replacement (Bootstrap sampling). For each sub-data set:

[0154]

[0155] where, D t represents the t-th sub-data set, and S t is an index set obtained through sampling with replacement (Bootstrap sampling), which contains the indices of N samples in the original data set.

[0156] (2) By repeatedly executing the Bootstrap sampling process, T sub-data sets can be generated Each sub-data set D tThey are all obtained by sampling with replacement from the original dataset D and have a size of N. Each sub-dataset D t is used to train a decision tree. During the training of each decision tree, when splitting at each node, a feature subset F is randomly selected from the entire feature set F t , and this subset is used to find the best split point:

[0157]

[0158] where the size m of the feature subset is generally set |F| is the size of the feature set.

[0159] (3) Each decision tree is trained based on the sub-dataset D t and the feature subset F t , and the classification prediction formula of the tree is:

[0160] h t (x) = Tree t (x);

[0161] where h t (x) represents the prediction result of the t-th tree for the input sample x.

[0162] (4) The random forest integrates the prediction results of all decision trees and uses the majority voting method to obtain the final classification result.

[0163] H(x) = majority_vote(h1(x), h2(x), …, h t (x));

[0164] where H(x) is the final prediction result of the random forest.

[0165] (5) The classification performance of the random forest is evaluated through the validation set and the test set, and metrics such as classification accuracy and recall are calculated.

[0166] The color feature data of the stain area is used as the input, and the stain color grade is used as the classification label output. By adjusting model parameters such as the number of trees and the maximum depth, the classification performance is optimized, and the final color classification sub-model is generated to determine the color grading result of the stain area.

[0167] S450. Determine the cleanliness detection result of the stain area based on the first feature data and the second feature data.

[0168] The technical solution of this embodiment obtains the first image data of the object to be detected, where the first image data includes a stain area; processes the first image data through a pre-trained image segmentation model to obtain the image segmentation result of the stain area, and determines the first feature data corresponding to the stain area based on the image segmentation result; processes the image segmentation result through at least two color feature extraction sub-models respectively to obtain the feature extraction results corresponding to the respective color feature extraction sub-models; classifies the feature extraction results corresponding to the respective color feature extraction sub-models through a color classification sub-model to obtain a color classification result, and determines the color classification result as the second feature data; determines the cleanliness detection result of the stain area based on the first feature data and the second feature data. This solution performs multi-dimensional feature extraction on the image data corresponding to the object to be detected through different feature extraction methods, and determines the cleanliness detection result of the stain area according to various feature data corresponding to the stain area, solving the problems of lack of intelligent evaluation and judgment capabilities and inaccurate cleanliness detection caused by the inability to efficiently and accurately identify the degree and distribution of stains inside the oil fume duct, and can intelligently detect the stain area and determine the detection result, improving the detection efficiency and accuracy.

[0169] Embodiment 4

[0170] Figure 5 is a schematic structural diagram of a cleanliness detection device provided in Embodiment 4 of the present invention. As Figure 5 shown, the device includes:

[0171] A first image data acquisition module 510, configured to acquire the first image data of the object to be detected, where the first image data includes a stain area;

[0172] A first feature data determination module 520, configured to process the first image data through a pre-trained image segmentation model to obtain the image segmentation result of the stain area, and determine the first feature data corresponding to the stain area based on the image segmentation result;

[0173] A second feature data determination module 530, configured to classify and process the image segmentation result based on a pre-trained color level determination model to obtain the second feature data corresponding to the stain area;

[0174] A detection result determination module 540, configured to determine the cleanliness detection result of the stain area based on the first feature data and the second feature data.

[0175] In the technical solution of this embodiment, the first image data acquisition module acquires the first image data of the object to be detected, where the first image data includes a stain area; the first feature data determination module processes the first image data through a pre-trained image segmentation model to obtain the image segmentation result of the stain area, and determines the first feature data corresponding to the stain area based on the image segmentation result; the second feature data determination module classifies and processes the image segmentation result based on a pre-trained color level determination model to obtain the second feature data corresponding to the stain area; the detection result determination module determines the cleanliness detection result of the stain area based on the first feature data and the second feature data. This solution performs multi-dimensional feature extraction on the image data corresponding to the object to be detected through different feature extraction methods, and determines the cleanliness detection result of the stain area according to various feature data corresponding to the stain area, solving the problems of lack of intelligent evaluation and judgment capabilities and inaccurate cleanliness detection caused by the inability to efficiently and accurately identify the degree and distribution of stains inside the oil fume duct. It can intelligently detect the stain area and determine the detection result, improving the detection efficiency and accuracy.

[0176] Based on the above embodiment, optionally, the pre-trained image segmentation model includes a bounding box extraction sub-model and an image segmentation sub-model; the first feature data determination module 520 includes a second image data determination unit and an image segmentation result determination unit. The second image data determination unit is configured to perform bounding box extraction processing on the first image data through the bounding box extraction sub-model to obtain the second image data corresponding to the stain area in the first image data; the image segmentation result determination unit is configured to perform image segmentation processing on the second image data through the image segmentation sub-model to obtain the image segmentation result corresponding to the stain area.

[0177] Optionally, the first feature data determination module 520 further includes a first feature data determination unit, which is configured to determine the stain area area data corresponding to the image segmentation result and the area data of the first image data, determine the stain area area ratio data based on the stain area area data and the first image data area data, and determine the stain area area ratio data as the first feature data.

[0178] Optionally, the pre-trained color level determination model includes at least two color feature extraction sub-models and a color classification sub-model; the second feature data determination module 530 includes a feature extraction result determination unit and a second feature data determination unit. The feature extraction result determination unit is configured to process the image segmentation result through at least two color feature extraction sub-models respectively to obtain the feature extraction results corresponding to the respective color feature extraction sub-models; the second feature data determination unit is configured to perform a classification process on the feature extraction results corresponding to the respective color feature extraction sub-models through the color classification sub-model to obtain a color classification result, and determine the color classification result as the second feature data. Optionally, the color feature extraction sub-model includes an HSV color feature extraction sub-model and an RGB color feature extraction sub-model. The feature extraction result corresponding to the HSV color feature extraction sub-model includes HSV histogram feature data; the feature extraction result corresponding to the RGB color feature extraction sub-model includes RGB color multi-order moment feature data.

[0179] Optionally, the detection result determination module 540 is specifically configured to obtain a first mapping relationship between the first feature data and the cleanliness quantization data, and determine first quantization data of the first feature data based on the first mapping relationship; and obtain a second mapping relationship between the second feature data and the cleanliness quantization data, and determine second quantization data of the second feature data based on the second mapping relationship; obtain weight data, and perform a weighted summation process on the first quantization data and the second quantization data based on the weight data to obtain a cleanliness detection result of the stain area.

[0180] Optionally, the detection result determination module 540 is further specifically configured to obtain at least two area thresholds corresponding to the first feature data, and determine weight data of the first quantization data based on the first quantization data corresponding to the first feature data and the at least two area thresholds; determine weight data of the second quantization data based on the weight data of the first quantization data.

[0181] The cleanliness detection device provided by the embodiments of the present invention can execute the cleanliness detection method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method.

[0182] Embodiment Five

[0183] Figure 6FIG. 0 is a schematic structural diagram of an electronic device provided in Embodiment 5 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as, personal digital processors, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described herein and / or claimed.

[0184] As Figure 6 shown, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. The memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0185] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0186] The processor 11 may be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the cleanliness detection method.

[0187] In some embodiments, the cleanliness detection method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the cleanliness detection method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the cleanliness detection method by any other suitable means (e.g., by means of firmware).

[0188] The various embodiments of the systems and techniques described above in this document may be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), systems-on-chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a special-purpose or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0189] The computer program for implementing the cleanliness detection method of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowchart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0190] Embodiment Six

[0191] Embodiment Six of the present invention further provides a computer-readable storage medium storing computer instructions for causing a processor to execute a cleanliness detection method, the method including:

[0192] Obtain first image data of an object to be detected, where the first image data includes a stain area;

[0193] Process the first image data through a pre-trained image segmentation model to obtain the image segmentation result of the stain area, and determine the first feature data corresponding to the stain area based on the image segmentation result;

[0194] Classify the image segmentation result based on a pre-trained color level determination model to obtain the second feature data corresponding to the stain area;

[0195] Determine the cleanliness detection result of the stain area based on the first feature data and the second feature data.

[0196] In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium may be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0197] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0198] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0199] A computing system can include a client and a server. The client and the server are generally remote from each other and typically interact via a communication network. The client-server relationship is created by computer programs running on respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0200] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is imposed herein.

[0201] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A cleanliness detection method, characterized in that, Including: Obtain first image data of an object to be detected, where the first image data includes a stain area; Process the first image data through a pre-trained image segmentation model to obtain an image segmentation result of the stain area, and determine first feature data corresponding to the stain area based on the image segmentation result; Perform classification processing on the image segmentation result based on a pre-trained color level determination model to obtain second feature data corresponding to the stain area; Determine a cleanliness detection result of the stain area based on the first feature data and the second feature data.

2. The method according to claim 1, characterized in that, The pre-trained image segmentation model includes a bounding box extraction sub-model and an image segmentation sub-model; The process of obtaining the image segmentation result of the stain area by processing the first image data through the pre-trained image segmentation model includes: Perform bounding box extraction processing on the first image data through the bounding box extraction sub-model to obtain second image data corresponding to the stain area in the first image data; Perform image segmentation processing on the second image data through the image segmentation sub-model to obtain an image segmentation result corresponding to the stain area.

3. The method according to claim 1, characterized in that, The determination of the first feature data corresponding to the stain area based on the image segmentation result includes: Determine the area data of the stain area corresponding to the image segmentation result and the area data of the first image data, determine the proportion data of the stain area based on the area data of the stain area and the area data of the first image data, and determine the proportion data of the stain area as the first feature data.

4. The method according to claim 1, characterized in that, The pre-trained color level determination model includes at least two color feature extraction sub-models and a color classification sub-model; The process of performing classification processing on the image segmentation result based on the pre-trained color level determination model to obtain the second feature data corresponding to the stain area includes Process the image segmentation result through the at least two color feature extraction sub-models respectively to obtain feature extraction results corresponding to the respective color feature extraction sub-models; Perform classification processing on the feature extraction results corresponding to the respective color feature extraction sub-models through the color classification sub-model to obtain a color classification result, and determine the color classification result as the second feature data.

5. The method according to claim 4, characterized in that The color feature extraction sub-model includes an HSV color feature extraction sub-model and an RGB color feature extraction sub-model. The feature extraction result corresponding to the HSV color feature extraction sub-model includes HSV histogram feature data, and the feature extraction result corresponding to the RGB color feature extraction sub-model includes RGB color multi-order moment feature data.

6. The method according to claim 1, wherein The determination of the cleanliness detection result of the stain area based on the first feature data and the second feature data includes: Obtain a first mapping relationship between the first feature data and cleanliness quantization data, and determine first quantization data of the first feature data based on the first mapping relationship; and obtain a second mapping relationship between the second feature data and cleanliness quantization data, and determine second quantization data of the second feature data based on the second mapping relationship; Obtain weight data, and perform weighted summation processing on the first quantization data and the second quantization data based on the weight data to obtain the cleanliness detection result of the stain area.

7. The method according to claim 6, characterized in that The obtaining of the weight data includes: Obtain at least two area thresholds corresponding to the first feature data, and determine the weight data of the first quantization data based on the first quantization data corresponding to the first feature data and the at least two area thresholds; Determine the weight data of the second quantization data based on the weight data of the first quantization data.

8. A cleanliness detection device, characterized in that, It includes: A first image data acquisition module, configured to acquire first image data of an object to be detected, where the first image data includes a stain area; A first feature data determination module, configured to process the first image data through a pre-trained image segmentation model to obtain an image segmentation result of the stain area, and determine first feature data corresponding to the stain area based on the image segmentation result; A second feature data determination module, configured to perform classification processing on the image segmentation result based on a pre-trained color level determination model to obtain second feature data corresponding to the stain area; A detection result determination module, configured to determine the cleanliness detection result of the stain area based on the first feature data and the second feature data.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the cleanliness detection method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the processor to implement the cleanliness detection method according to any one of claims 1-7 when executed.

Images