Cleaning personnel work supervision method based on machine vision analysis

Abstract

The invention provides a cleaning worker work supervision method based on machine vision analysis, and relates to the field of machine vision application, and the method comprises the steps: carrying out the mining of similarity threshold distribution characteristics through employing a clustering analysis algorithm according to historical cleaning effect data in different scenes, and obtaining an initial adaptive threshold setting; aiming at key cleaning steps, a standard action library containing multi-dimensional information is established, the standardization of cleaning actions is evaluated in real time by adopting a computer vision technology and a dynamic time warping algorithm, and differentiated evaluation and guidance are realized by combining individual characteristics of cleaning personnel and adopting personalized threshold setting and feedback strategies; by continuously tracking action performance changes, dynamically adjusting an evaluation strategy and updating standard action library parameters to form a closed-loop optimization mechanism, the accuracy and pertinence of cleaning quality evaluation can be effectively improved, continuous improvement of cleaning quality is promoted, and the method adapts to complex and changeable actual cleaning scene requirements.

Description

Technical Field

[0001] The invention relates to the field of machine vision applications, and in particular to a method for supervising the work of cleaning personnel based on machine vision analysis. Background Art

[0002] In the process of cleaning quality assessment, we face the challenges of changing environmental and personnel conditions. In different scenarios, the evaluation criteria for cleaning effects are also different, and a single similarity threshold is difficult to adapt to the complex and changing actual situation. In addition, the operating habits and action details of the cleaning staff will also affect the cleaning effect. How to objectively evaluate the standardization and effectiveness of cleaning actions is also an urgent problem to be solved.

[0003] In order to meet these challenges, it is necessary to establish an adaptive threshold adjustment mechanism and a standard action library. By clustering historical data, we can find the distribution characteristics of similarity thresholds in different scenarios and set the initial threshold based on this. In the actual monitoring process, we also need to continuously collect feedback data, dynamically update the clustering results through incremental learning, and realize adaptive adjustment of the threshold.

[0004] For key cleaning steps, it is necessary to pre-establish a standard action library containing multi-dimensional information such as action trajectory, amplitude, and strength. During the monitoring process, the actual action features of the cleaning staff are extracted and compared with the standard action library through algorithms such as dynamic time warping, and the action similarity is calculated to quantitatively evaluate the standardization and effectiveness of the cleaning actions.

[0005] However, when building a standard action library, how to select appropriate action features and similarity calculation methods to balance calculation efficiency and evaluation accuracy requires further research and optimization. At the same time, in practical applications, the individual differences and habits of different cleaning staff must also be taken into account. How to balance personalized evaluation and guidance on the basis of unified standards is also a problem worth exploring. Summary of the invention

[0006] The present invention provides a cleaning staff work supervision method based on machine vision analysis, which mainly includes:

[0007] Based on the historical cleaning effect data in different scenarios, a cluster analysis algorithm is used to mine the similarity threshold distribution characteristics to obtain the initial adaptive threshold setting;

[0008] In the actual cleaning quality monitoring process, we continuously collect cleaning effect feedback data and dynamically update the clustering results through incremental learning.

[0009] For key cleaning steps, a standard action library containing multi-dimensional information on action trajectory, amplitude, and strength is pre-established;

[0010] During the monitoring process, computer vision technology is used to extract the action features of cleaning staff in real time, and the action similarity score is calculated by comparing them with the standard action library through the dynamic time warping algorithm;

[0011] If the action similarity score is lower than the preset threshold, the cleaning action is judged to be irregular or invalid, and a corresponding feedback prompt message is generated;

[0012] On the basis of unified action evaluation standards, combined with the historical performance and individual characteristics of cleaning staff, personalized threshold setting and feedback strategies are adopted to achieve differentiated evaluation and guidance;

[0013] Continuously track the changes in the cleaning staff's performance, and determine the degree of improvement in their operating habits through time series analysis of the action similarity scores; the time series analysis uses the moving average method to calculate the average score of the last N cleaning tasks, and compares it with the historical average score to dynamically adjust the personalized evaluation strategy;

[0014] Update relevant parameters in the standard action library according to the dynamically adjusted personalized evaluation strategy;

[0015] Apply the updated standard action library to the next round of cleaning quality monitoring process to form a closed-loop optimization mechanism.

[0016] Furthermore, according to the historical cleaning effect data in different scenarios, a clustering analysis algorithm is used to mine the distribution characteristics of the similarity threshold to obtain an initial adaptive threshold setting; the initial threshold setting is used as a basic reference value for the subsequent monitoring process, including: obtaining historical cleaning effect data in at least one scenario, the historical cleaning effect data including images before and after cleaning and cleaning time; for each of the scenarios, a clustering algorithm is used to cluster the cleaning effect data to obtain the distribution characteristics of the adaptive threshold in the scenario; in the cleaning monitoring process, the images before and after cleaning in the current scenario are obtained, the cleaning effect similarity values ​​of the images before and after cleaning are calculated, and the cleaning effect similarity value is compared with a preset adaptive threshold; if the cleaning effect similarity value is less than the adaptive threshold, it is determined that the cleaning effect does not meet the standard.

[0017] Furthermore, in the actual cleaning quality monitoring process, cleaning effect feedback data is continuously collected, and clustering results are dynamically updated through incremental learning; incremental learning is specifically implemented through an online K-means algorithm, and the cluster center is updated each time a new data point is collected, thereby achieving adaptive adjustment of the similarity threshold to adapt to complex and changeable actual scenarios, including: obtaining images before and after cleaning, using median filtering to denoise and histogram equalization to enhance the contrast, and normalizing the image size to obtain a standardized image; inputting the standardized image into a pre-trained ResNet-50 network to extract a 512-dimensional feature vector; using cosine similarity to calculate the 512-dimensional feature vector to obtain the similarity of the images before and after cleaning; obtaining the start and end time of cleaning, and calculating the cleaning duration; if the cleaning duration exceeds a preset range, it is determined to be an abnormal data point.

[0018] Furthermore, for the key cleaning steps, a standard action library containing multi-dimensional information such as action trajectory, amplitude, and strength is pre-established as a reference for evaluating the standardization and effectiveness of cleaning actions, including: obtaining video stream data, extracting key frame images and corresponding timestamps in the cleaning process; for cleaning scenes, a standard action library containing multi-dimensional parameters of key actions is pre-established; a human posture estimation algorithm is used to obtain the key action sequence of the cleaner in the video stream; through a dynamic time warping algorithm, the extracted key action sequence is matched with the standard action library for similarity, and if the matching distance exceeds a preset threshold, the action sequence is determined to be non-standard; using an image similarity algorithm, the key frame images before and after cleaning are compared, and a quantitative score for the cleaning effect is given in combination with a preset scoring standard.

[0019] Furthermore, during the monitoring process, computer vision technology is used to extract the action features of the cleaning staff in real time, and the dynamic time warping algorithm is used to compare them with the standard action library to calculate the action similarity score; the dynamic time warping algorithm solves the problem of inconsistent action speed by aligning the time series and improves the accuracy of the comparison, including: obtaining video image data, using the OpenPose algorithm to extract the coordinates of the cleaning staff's skeleton key points, and obtaining action sequence data; using the YOLOv3 algorithm to identify and locate cleaning tools to determine whether the tools are used correctly; obtaining images before and after cleaning, using the FCN algorithm for semantic segmentation, extracting pixel information from different areas, and generating a warning message if improper use of the tool is detected or the tool is not detected; by comparing the segmentation results, calculating the percentage of cleanliness improvement in each area, and quantitatively evaluating the cleaning quality.

[0020] Furthermore, if the action similarity score is lower than a preset threshold, the cleaning action is judged to be irregular or invalid, and corresponding feedback prompt information is generated to guide the cleaning staff to adjust and optimize the action, including: obtaining a sequence of working video frames of the cleaning staff; for the video frames, using a human body posture estimation algorithm to obtain the coordinate information of the cleaning staff's skeletal joints; based on the skeletal joint coordinate information, constructing a standard action feature vector library describing the cleaning action; comparing the action feature vector with a preset standard action feature vector library, and calculating the cosine similarity; if the similarity is lower than a preset threshold, marking the action as unqualified, and recording the timestamp and action type in the task record file; obtaining images before and after cleaning, using image subtraction to calculate the difference before and after cleaning, obtaining a cleaning quality score, and entering the score result into the task record file.

[0021] Furthermore, on the basis of the unified action evaluation standard, combined with the historical performance and individual characteristics of the cleaning staff, personalized threshold settings and feedback strategies are adopted to achieve differentiated evaluation and guidance; the personalized threshold is obtained by weighted average historical performance scores and standard thresholds, including: obtaining the historical performance data of the cleaning staff, and adopting a weighted average algorithm to obtain the personalized threshold; obtaining the real-time work video and image data of the cleaning staff, and analyzing the video and image data through a computer vision library to determine whether the cleaning action meets the standard; if the cleaning action does not meet the standard, the real-time work data is compared with the personalized threshold, and the degree of deviation is determined by a similarity calculation method; according to the degree of deviation, a corresponding feedback strategy is adopted to generate personalized guidance information; the cleaning task record data is obtained, pre-processed through a data analysis library, and a time series prediction model is used to predict the work performance trend of the cleaning staff, and the personalized threshold and the feedback strategy are dynamically adjusted according to the prediction results.

[0022] Furthermore, the continuous tracking of the changes in the action performance of the cleaning staff is carried out, and the degree of improvement of their operating habits is judged by time series analysis of the action similarity scores; the time series analysis adopts the moving average method to calculate the average score of the most recent N cleaning tasks, and compares it with the historical average score, and dynamically adjusts the personalized evaluation strategy to promote the continuous improvement of the cleaning quality, including: continuously recording the work process of the cleaning staff through the camera, obtaining the image data and corresponding timestamp information before and after cleaning, and storing them in the task record file; using the image comparison algorithm, similarity calculation is performed on the images before and after cleaning to obtain the cleaning effect evaluation score, and the result and the timestamp are written into the task record file together; according to the standard operation time of the cleaning task, the timestamps of the start and end of the cleaning are extracted from the task record file, the actual duration of the cleaning task is calculated, and it is judged whether it meets the standard, and the judgment result is written into the task record file; the OpenPose open source human posture estimation model is used to analyze the image data in the task record file, and the key point coordinate sequence of the cleaning staff is extracted as the action feature; the extracted action feature is dynamically time-aligned with the pre-recorded standard action template, and the action similarity score is calculated to obtain the action performance evaluation result of the cleaning staff.

[0023] Furthermore, the updated standard action library is applied to the next round of cleaning quality monitoring process to form a closed-loop optimization mechanism to continuously improve the accuracy and pertinence of cleaning quality assessment, including: acquiring images before and after cleaning, analyzing the images using machine vision technology to obtain environmental difference information before and after cleaning, and quantifying the environmental difference information into numerical indicators; analyzing historical cleaning quality assessment records using an association rule mining algorithm to obtain the association law between cleaning quality and cleaning actions, and establishing a mapping model between cleaning actions and quality; using a gradient descent method to dynamically adjust the weight parameters in the mapping model, applying the adjusted mapping model to cleaning quality assessment, and realizing dynamically optimized closed-loop monitoring.

[0024] The technical solution provided by the embodiment of the present invention may have the following beneficial effects:

[0025] The present invention discloses a method for supervising the work of cleaning personnel based on machine vision analysis. The method obtains the initial adaptive threshold setting by clustering analysis of historical cleaning effect data, and dynamically updates it through incremental learning in actual monitoring. For key cleaning steps, a standard action library containing multi-dimensional information is established, and computer vision technology and dynamic time warping algorithm are used to evaluate the standardization of cleaning actions in real time. Combined with the individual characteristics of cleaning personnel, personalized threshold settings and feedback strategies are adopted to achieve differentiated evaluation and guidance. By continuously tracking changes in action performance, dynamically adjusting the evaluation strategy, and updating the parameters of the standard action library, a closed-loop optimization mechanism is formed. The present invention can effectively improve the accuracy and pertinence of cleaning quality assessment, promote the continuous improvement of cleaning quality, and adapt to the complex and changeable actual cleaning scene requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 The present invention is a flow chart of a cleaning staff work supervision method based on machine vision analysis.

[0027] Figure 2 It is a schematic diagram of a cleaning staff work supervision method based on machine vision analysis according to the present invention.

[0028] Figure 3 This is another schematic diagram of a cleaning staff work supervision method based on machine vision analysis according to the present invention. DETAILED DESCRIPTION

[0029] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] like Figure 1-Figure 3 In this embodiment, a cleaning staff work supervision method based on machine vision analysis may specifically include:

[0031] S101, based on the historical cleaning effect data in different scenarios, a cluster analysis algorithm is used to mine the similarity threshold distribution characteristics to obtain an initial adaptive threshold setting, which serves as a basic reference value for subsequent monitoring processes.

[0032] Acquire historical cleaning effect data in at least one scenario, wherein the historical cleaning effect data includes pre-cleaning images, post-cleaning images and cleaning duration; for each of the scenarios, cluster the cleaning effect data using a clustering algorithm to obtain the distribution characteristics of the cleaning effect similarity threshold in the scenario; in the cleaning monitoring process, acquire the pre- and post-cleaning images in the current scenario, calculate the cleaning effect similarity values ​​of the pre- and post-cleaning images, compare the cleaning effect similarity value with a preset adaptive threshold, and if the cleaning effect similarity value is less than the adaptive threshold, determine that the cleaning effect does not meet the standard.

[0033] Specifically, historical cleaning effect data under different scenarios are obtained, including information such as images before cleaning, images after cleaning, cleaning time, etc., to construct a cleaning effect data set. The cleaning effect data set is preprocessed, and image segmentation is performed using the Canny edge detection algorithm to extract the SIFT features of the images before and after cleaning. Based on the SIFT features of the images before and after cleaning, the Euclidean distance is used to calculate the similarity value of the cleaning effect. The cleaning effect data set is classified according to the scene to obtain subsets of cleaning effect data under different scenes. The K-means clustering algorithm is used to cluster the subsets of cleaning effect data under each scene, and the distribution characteristics of the cleaning effect similarity threshold under the scene are determined based on the clustering results. By analyzing the distribution characteristics of the cleaning effect similarity threshold under different scenes, the initial adaptive threshold setting is obtained, and it is configured in the cleaning effect monitoring system as the basic reference value for the subsequent monitoring process. In the actual monitoring process, the images before and after cleaning under the current scene are obtained through the camera, the cleaning effect similarity value is calculated, and compared with the adaptive threshold of the scene. If the cleaning effect similarity value is lower than the adaptive threshold, the cleaning effect is judged to be substandard, and an early warning message is generated and associated with the corresponding cleaning task and recorded in the task record file. Based on the monitoring results of multiple consecutive cleaning effects, the adaptive threshold setting is dynamically adjusted using the least squares method to adapt to changes in the scene and improve monitoring accuracy. By regularly collecting new cleaning effect data, updating the cleaning effect data set, and re-performing cluster analysis, the adaptive threshold setting mechanism is continuously improved.

[0034] S102. During the actual cleaning quality monitoring process, the cleaning effect feedback data is continuously collected, and the clustering results are dynamically updated through incremental learning. Incremental learning is specifically implemented through the online K-means algorithm. The cluster center is updated every time a new data point is collected, thereby achieving adaptive adjustment of the similarity threshold to adapt to complex and changing actual scenarios.

[0035] Get the images before and after cleaning, use median filtering to remove noise and histogram equalization to enhance contrast, and normalize the image size to obtain a standardized image. Input the standardized image into the pre-trained ResNet-50 network to extract a 512-dimensional feature vector. Use cosine similarity to calculate the feature vector and obtain the similarity of the images before and after cleaning. Get the start and end time of cleaning and calculate the cleaning duration. If the cleaning duration exceeds the preset range, it is determined as an abnormal data point.

[0036] Specifically, the image data before and after cleaning is collected by the camera and transmitted to the image processing module for preprocessing, including using the median filtering algorithm to remove image noise, using the histogram equalization algorithm to enhance the image contrast, and normalizing the image size to a fixed size to obtain standardized image data. The preprocessed image data is input into the feature extraction module, and the high-dimensional features of the image are extracted by the pre-trained ResNet-50 convolutional neural network to obtain a 512-dimensional feature vector representing the image content. According to the extracted feature vector, the cosine similarity is used to calculate the similarity between the images before and after cleaning. At the same time, according to the historical data with good cleaning effects manually annotated, the threshold range of similarity is obtained by statistics. If the similarity of the current image pair is lower than the lower threshold, it is judged that the cleaning effect is not good this time, and it is marked as an abnormal data point. The start time and end time of the cleaning task are obtained, and the cleaning time is calculated. Similarly, the range of normal cleaning time is obtained according to the historical task records. If the cleaning time exceeds the normal range, it is also marked as an abnormal data point. The image similarity, cleaning duration, and abnormal marking information are entered into a JSON-formatted task record file named after the current time, and saved according to the date directory structure for subsequent query and analysis. For the newly added task record files every day, the Pandas library is used to read into the memory and parse the JSON data to extract the similarity and duration of the abnormal data points. The KMeans algorithm in the scikit-learn library is used to perform cluster analysis on the abnormal data points, and the number of clusters is set to 2, corresponding to the two situations of poor cleaning effect and abnormal cleaning duration. According to the clustering results, the average values ​​of the similarity and duration of the two categories are calculated respectively as the updated monitoring threshold. The updated monitoring threshold is written to the configuration file for subsequent real-time monitoring tasks. When the new cleaning task is completed, the latest threshold configuration is read to judge the collected image data and cleaning duration information. If an abnormal situation is found, an alarm message is automatically generated by email or SMS and sent to the relevant person in charge to remind him to deal with it in time. Summarize and analyze recent task record files every week, use the data grouping and aggregation functions of the Pandas library to calculate the average similarity and cleaning time of each cleaning area, and generate a weekly report on the cleaning quality and efficiency of each area. Send the analysis results to the management staff in the form of emails, and attach visual statistical charts so that the management staff can understand the overall situation of the cleaning work in a timely manner, identify problem areas and formulate targeted improvement measures.

[0037] S103. For key cleaning steps, a standard action library containing multi-dimensional information such as action trajectory, amplitude, and strength is pre-established as a reference for evaluating the standardization and effectiveness of cleaning actions.

[0038] The video stream data is obtained to extract the key frame images and corresponding timestamps in the cleaning process. For the cleaning scene, a standard action library containing multi-dimensional parameters of key actions is pre-established. The human posture estimation algorithm is used to obtain the key action sequence of the cleaner in the video stream. The extracted key action sequence is matched with the standard action library through the dynamic time warping algorithm. If the matching distance exceeds the preset threshold, the action sequence is judged to be non-standard. The image similarity algorithm is used to compare the key frame images before and after cleaning, and a quantitative score of the cleaning effect is given in combination with the preset scoring criteria.

[0039] Specifically, a video surveillance system is used to monitor and record the work process of the cleaning staff in real time, obtain video stream data, and store it in the task record file. The key frame images before and after cleaning are intercepted from the video stream, and the corresponding timestamp information is recorded. According to the cleaning scene, a targeted cleaning action standard library is pre-established, including multi-dimensional parameters such as standard action trajectory, amplitude, and strength of key actions such as wiping and sweeping. Human posture estimation algorithms such as OpenPose are used to detect key points of the cleaners in the video stream frame by frame, extract bone information, and obtain the key action sequence in the cleaning process. The extracted key action sequence is matched with the samples in the standard action library by the dynamic time warping (DTW) algorithm, and the matching distance is calculated. If the matching distance exceeds the preset threshold, the action sequence is judged to be non-standard, marked as unqualified in the task record file, and the specific reasons for the unqualified are recorded, such as insufficient action amplitude, improper action strength, etc. The key frame images before and after cleaning are compared and analyzed using image similarity algorithms such as SSIM to calculate the image similarity score. Based on the image similarity score and the preset scoring criteria, a quantitative score of the cleaning effect is given, and the scoring result is written into the task record file. The timestamps of the start and end of cleaning are extracted from the task record file, and the difference between the two is calculated to obtain the cleaning time. The cleaning time is compared with the preset standard time interval. If it exceeds the upper limit of the interval, it is judged as timeout, and the cleaning is marked as timeout, and the timeout duration is recorded. Using text parsing technologies such as regular expression matching and keyword extraction, the task record file is structured and parsed to extract key information such as the standardization of cleaning actions, cleaning quality scores, and cleaning time, and convert them into structured evaluation dimension data. Based on the evaluation dimension data, data visualization libraries such as Matplotlib and Echarts are used to generate a work supervision report for cleaning personnel. The report intuitively presents the standardization of actions, cleaning quality, and reasonable time of each cleaning task in the form of charts, color markings, etc., which is convenient for managers to quickly review and analyze.

[0040] S104. During the monitoring process, computer vision technology is used to extract the action features of the cleaning staff in real time, and the action features are compared with the standard action library through the dynamic time warping algorithm to calculate the action similarity score. The dynamic time warping algorithm solves the problem of inconsistent action speed by aligning the time series and improves the comparison accuracy.

[0041] The video image data is obtained, and the OpenPose algorithm is used to extract the coordinates of the key points of the cleaning staff's skeleton to obtain the action sequence data; the YOLOv3 algorithm is used to identify and locate the cleaning tools to determine whether the tools are used correctly; the images before and after cleaning are obtained, and the FCN algorithm is used for semantic segmentation to extract pixel information in different areas. If improper use of the tool is detected or the tool is not detected, an early warning message is generated; by comparing the segmentation results, the percentage of cleanliness improvement in each area is calculated, and the cleaning quality is quantitatively evaluated.

[0042] Specifically, the video image data of the cleaning staff's work process is collected in real time by the camera, and the video image is preprocessed, including using the median filter to denoise the image and using the histogram equalization algorithm to enhance the image to improve the image quality. The OpenPose human posture estimation algorithm is used to analyze the preprocessed video image, extract the coordinate information of the key points of the skeleton of the cleaning staff, and obtain the action sequence data of the cleaning staff. According to the acquired action sequence data of the cleaning staff, the dynamic time warping (DTW) algorithm is used to compare with the action template in the pre-established standard action library, and the similarity score between the two action sequences is calculated. If the similarity score exceeds the preset threshold (such as 8), it is judged that the current cleaning staff's action matches the action in the standard action library and meets the standard requirements; otherwise, it is judged as an irregular action and a warning message is generated. The video image is analyzed by the YOLOv3 target detection algorithm to identify and locate cleaning tools such as rags and mops, and determine whether the cleaning staff uses the tools correctly. If the tool is not detected or the use method is improper (such as the rag has not been replaced for a long time, the mop is not soaked, etc.), a warning message is generated. Before and after the cleaning operation, images are captured. The semantic segmentation algorithm based on the fully convolutional neural network (FCN) is used to analyze the images before and after cleaning, and pixel information of different areas such as the ground, wall, and furniture is extracted. By comparing the segmentation results, the percentage of cleanliness improvement in each area is calculated, the cleaning effect is quantitatively evaluated, and the cleaning quality score is obtained. According to the timestamp information in the video image, the start and end time of cleaning is automatically recorded, and the total duration of the cleaning operation is calculated. According to the size and cleaning difficulty of different places, a reasonable operation time range is preset (such as 30 minutes in the office, 20 minutes in the conference room, etc.). If the actual operation time is less than 80% of the shortest time limit or exceeds 120% of the longest time limit, it is judged as abnormal duration and an early warning message is generated. The above analysis results, including action standardization scores, tool usage, comparison before and after cleaning, operation time and other information, are automatically generated into a cleaning task record file, named with a timestamp and archived. Statistical analysis is performed on the record files to evaluate the work performance of each cleaning staff, and key performance indicators (KPIs) such as action standardization rate, average operation time, and cleaning quality pass rate are calculated. Use data visualization tools such as Echarts to generate intuitive visualization charts such as bar charts, line charts, and pie charts to clearly show the changing trends of various indicators over time, making it easier for managers to review and supervise.

[0043] S105. If the action similarity score is lower than a preset threshold, the cleaning action is judged to be irregular or invalid, and corresponding feedback prompt information is generated to guide the cleaning staff to adjust and optimize the action.

[0044] A sequence of video frames of cleaning staff working is obtained; for the video frames, a human posture estimation algorithm is used to obtain the coordinate information of the cleaning staff's skeleton joints; based on the joint coordinate information, a feature vector describing the cleaning action is constructed; similarity is calculated between the cleaning action feature vector and samples in a pre-established standard action feature vector library to obtain a similarity score; if the similarity score is lower than a preset threshold, the action is judged to be non-standard and a feedback prompt message is generated; if the similarity score is higher than the preset threshold, the duration and completion quality of the action are recorded.

[0045] Specifically, the working video of the cleaning staff is collected in real time through the camera, and the video frame sequence is transmitted to the image processing module. The key points of the cleaning staff in the video frame are detected by using open source human posture estimation algorithms such as OpenPose to obtain the coordinate information of the skeletal joint points of the cleaning staff. According to the detected joint point coordinate information, the geometric features such as the angle and Euclidean distance between the body parts of the cleaning staff are calculated to construct a feature vector describing the cleaning action. The standard cleaning action is recorded and analyzed in advance, and its joint point coordinate information and geometric features are extracted to construct a standard action feature vector library. The cosine similarity calculation is performed on the currently extracted cleaning action feature vector and the samples in the standard action feature vector library to obtain the similarity score between the current action and each standard action. The similarity score threshold is set, and the threshold is adjusted according to business needs and experience judgment. If the highest similarity score of the current action is lower than the threshold, the action is judged to be irregular or invalid, and the corresponding feedback prompt information is generated, such as a red light or voice reminder. If the highest similarity score is higher than the threshold, the action is marked as a valid action, and its duration and completion quality are recorded. According to the results of continuous multi-frame motion analysis, the number of effective actions, invalid actions and average similarity scores of cleaning staff in a work cycle are counted as the action standardization score. The statistical results of the current work cycle are compared with the historical work data of the cleaning staff. By drawing visual methods such as line graphs or bar graphs, the improvement or decline trend of their work performance is judged, and a corresponding evaluation report is generated. The work supervision results of cleaning staff are displayed in real time through the visual interface of the Web or mobile terminal, including real-time video images, action standardization scores, statistics of effective and invalid action times, historical performance trend charts, etc., which is convenient for managers to intuitively understand the quality of cleaning work and make management decisions in time. At the same time, the system can also automatically generate daily and weekly quality assessment reports based on the analysis results, and regularly feedback to the relevant person in charge for assessment and improvement.

[0046] S106. Based on the unified action evaluation standard, combined with the historical performance and individual characteristics of cleaning staff, personalized threshold setting and feedback strategies are adopted to achieve differentiated evaluation and guidance. The personalized threshold is obtained by weighted average historical performance score and standard threshold.

[0047] Acquire historical performance data of cleaning staff, and use a weighted average algorithm to obtain the personalized threshold; acquire real-time work video and image data of cleaning staff, analyze the video and image data through a computer vision library, and determine whether the cleaning action is up to standard; if the cleaning action is not up to standard, compare the real-time work data with the personalized threshold, and determine the degree of deviation through a similarity calculation method; according to the degree of deviation, adopt a corresponding feedback strategy to generate personalized guidance information; acquire cleaning task record data, pre-process it through a data analysis library, use a time series prediction model to predict the work performance trend of cleaning staff, and dynamically adjust the personalized threshold and the feedback strategy according to the prediction result.

[0048] Specifically, based on the historical performance data of the cleaning staff, a weighted average algorithm is used to obtain a personalized threshold. The real-time work data of the cleaning staff, including video, image and other data, is obtained. The video and image are analyzed through computer vision libraries such as OpenCV to determine whether the cleaning action meets the standard. If the cleaning action does not meet the standard, the real-time work data is compared with the personalized threshold, and the degree of deviation is determined by similarity calculation methods such as Euclidean distance. According to the degree of deviation, a corresponding feedback strategy is adopted to generate personalized text format guidance information. Through speech synthesis libraries such as Python's pyttsx3, the guidance information in text format is converted into voice prompts and fed back to the cleaning staff in real time. The image data before and after cleaning is obtained, and the image subtraction method such as OpenCV is used to determine whether the cleaning effect meets the standard. If the cleaning effect does not meet the standard, the actual effect is compared with the personalized threshold, and the degree of deviation is determined by similarity calculation methods such as Euclidean distance. According to the degree of deviation, a corresponding feedback strategy is adopted to generate personalized text format guidance information and record it in a task record file in CSV format. Through data analysis libraries such as pandas, we preprocess the task record files in CSV format by data cleaning and feature engineering, and then use time series prediction models such as LSTM to predict the performance trend of cleaning staff, and dynamically adjust the personalized threshold and feedback strategy based on the prediction results. The personalized threshold is regularly updated using a weighted average algorithm based on the historical performance data of cleaning staff.

[0049] S107. Keep track of changes in the performance of cleaning staff and determine the degree of improvement in their operating habits through time series analysis of action similarity scores. Time series analysis uses the moving average method to calculate the average score of the last N cleaning tasks and compare it with the historical average score to dynamically adjust the personalized evaluation strategy to promote the continuous improvement of cleaning quality.

[0050] The camera continuously records the working process of the cleaning staff, obtains the image data before and after cleaning and the corresponding timestamp information, and stores them in the task record file. Use image comparison algorithms, such as perceptual hashing algorithms, to calculate the similarity of the images before and after cleaning, obtain the cleaning effect evaluation score, and write the result and timestamp into the task record file together. According to the standard operation time of the cleaning task, extract the timestamps of the start and end of cleaning from the task record file, calculate the actual duration of the cleaning task, judge whether it meets the standard, and write the judgment result into the task record file. Use open source human posture estimation models such as OpenPose to analyze the image data in the task record file, extract the key point coordinate sequence of the cleaning staff as action features. Perform dynamic time alignment on the extracted action features and the pre-recorded standard action template, calculate the action similarity score, and obtain the action performance evaluation result of the cleaning staff. Obtain the action similarity score of the cleaning staff's most recent 20 cleaning tasks, and use the moving average method to calculate the average score as the current action performance level. From the historical task record file, obtain the previous average score of the cleaning staff's action performance and compare it with the current average score. If the current average score is higher than the historical average score by more than 2%, it is judged that the cleaning staff's operating habits have improved, and the personalized evaluation strategy is dynamically adjusted to increase the evaluation standard threshold by 1%. If the current average score is lower than the historical average score by more than 2%, it is judged that the cleaning staff's operating habits have declined, and the personalized evaluation strategy is dynamically adjusted to reduce the evaluation standard threshold by 1%, and the cleaning staff is marked in the work management system, and supervisors are arranged for on-site guidance and training.

[0051] Specifically, the working process of the cleaning staff is continuously recorded by the camera, and the image data before and after cleaning and the corresponding timestamp information are obtained and stored in the task record file. The image comparison algorithm, such as the perceptual hash algorithm, is used to calculate the similarity of the images before and after cleaning to obtain the cleaning effect evaluation score, and the result is written into the task record file together with the timestamp. According to the standard operation time of the cleaning task, the timestamps of the start and end of the cleaning are extracted from the task record file, the actual duration of the cleaning task is calculated, and it is judged whether it meets the standard, and the judgment result is written into the task record file. The image data in the task record file is analyzed by using open source human posture estimation models such as OpenPose, and the key point coordinate sequence of the cleaning staff is extracted as the action feature. The extracted action features are dynamically time-warped and aligned with the pre-recorded standard action template, and the action similarity score is calculated to obtain the action performance evaluation result of the cleaning staff. The action similarity scores of the cleaning staff's most recent 20 cleaning tasks are obtained, and the average score is calculated by the moving average method as the current action performance level. From the historical task record file, the average score of the cleaning staff's previous action performance is obtained and compared with the current average score. If the current average score is higher than the historical average score by more than 2%, it is judged that the cleaning staff's operating habits have improved, and the personalized evaluation strategy is dynamically adjusted to increase the evaluation standard threshold by 1%. If the current average score is lower than the historical average score by more than 2%, it is judged that the cleaning staff's operating habits have declined, and the personalized evaluation strategy is dynamically adjusted to reduce the evaluation standard threshold by 1%, and the cleaning staff is marked in the work management system, and supervisors are arranged for on-site guidance and training.

[0052] S108. According to the dynamically adjusted personalized evaluation strategy, the relevant parameters in the standard action library are updated to adapt to the characteristics and scene requirements of different cleaning personnel. The update process adopts a weighted average method, comprehensively considering the original parameters and the new personalized parameters.

[0053] A human posture estimation algorithm is used to analyze the video frame sequence, and the body posture and hand movement features of the cleaning staff are extracted to obtain a motion feature vector; the motion feature vector is compared with a preset standard motion feature vector library, and the cosine similarity is calculated; if the similarity is lower than a preset threshold, the action is marked as unqualified, and the timestamp and action type are recorded in the task record file; the images before and after cleaning are obtained, and the difference before and after cleaning is calculated by image subtraction method to obtain a cleaning quality score, and the score result is entered into the task record file.

[0054] Specifically, the video of the cleaning staff's work process is collected in real time through the camera, and the video frame sequence is transmitted to the background server for analysis and processing. The start time of video collection is recorded as the initial time. The human posture estimation algorithm in OpenCV, such as OpenPose, is used to analyze the video frame sequence, extract the body posture and hand movement of the cleaning staff, and obtain the action feature vector. The extracted action feature vector is compared with the pre-established standard action feature vector library to calculate the cosine similarity. The initial similarity threshold is set to 8. If the similarity is lower than the threshold, the action is marked as unqualified, and the timestamp and action type are recorded and stored in the task record file. According to the historical performance and personal characteristics of the cleaning staff, the weighted average method is used to dynamically adjust the personalized evaluation strategy. The original threshold weight is set to 7, the new personalized threshold weight is set to 3, and the relevant parameters in the standard action feature vector library are updated to adapt to the characteristics and scene requirements of different cleaning staff. The images before and after cleaning are obtained, and the image subtraction method in OpenCV is used to calculate the difference before and after cleaning to obtain the cleaning quality score, and the score result is entered into the task record file. Extract the end time of video acquisition from the video frame sequence as the end time, calculate the cleaning time, determine whether it exceeds the preset range, and enter the time information into the task record file. Name and save the task record file according to the naming rule of "date_cleaning staff ID_task number" for subsequent query and analysis. Use association rule mining algorithms, such as the Apriori algorithm, to analyze the working patterns and habits of cleaning staff, and provide data support for optimizing the cleaning process and improving management efficiency.

[0055] S109. Apply the updated standard action library to the next round of cleaning quality monitoring process to form a closed-loop optimization mechanism and continuously improve the accuracy and pertinence of cleaning quality assessment.

[0056] The images before and after cleaning are acquired, and the images are analyzed by machine vision technology to obtain the environmental difference information before and after cleaning, and the environmental difference information is quantified into numerical indicators; the historical cleaning quality assessment records are analyzed by association rule mining algorithm to obtain the association law between cleaning quality and cleaning actions, and a mapping model between cleaning actions and quality is established; the gradient descent method is used to dynamically adjust the weight parameters in the mapping model, and the adjusted mapping model is applied to cleaning quality assessment to achieve dynamic optimization closed-loop monitoring.

[0057] Specifically, the images before and after cleaning are analyzed by machine vision technology to obtain the environmental difference information before and after cleaning, and the difference information is quantified into numerical indicators. According to the time information before and after cleaning, the cleaning time is calculated, and the time information is associated with the environmental difference index to form a cleaning quality evaluation record. The historical cleaning quality evaluation records are analyzed by association rule mining algorithm to find the association law between cleaning quality and cleaning actions, and form a mapping model between cleaning actions and quality. By performing action recognition on the cleaning process video, the action sequence of the cleaning staff is extracted, and compared with the standard action library to calculate the action similarity. According to historical data and expert experience, the action similarity threshold is set. If the action similarity is lower than the threshold, the action is judged as an abnormal action, and the time point and action type of the abnormal action are recorded. According to the time point of the abnormal action, the corresponding environmental difference index is found in the cleaning quality evaluation record, and the impact of the abnormal action on the cleaning quality is analyzed. The abnormal actions are classified by the K-means clustering algorithm, high-frequency abnormal actions are identified, and the standard action library is optimized in a targeted manner. In the next round of cleaning, the action sequence of the cleaning staff is compared with the updated standard action library in real time, the action similarity is calculated, and real-time cleaning action monitoring is realized. According to the frequency of abnormal actions and environmental difference indicators in the real-time monitoring results, the weight parameters in the mapping model of cleaning actions and quality are dynamically adjusted using the gradient descent method, and the adjusted model is applied to the cleaning quality assessment to form a dynamically optimized closed-loop monitoring mechanism. Through continuous data accumulation and model optimization, the accuracy and real-time performance of cleaning quality assessment are continuously improved.

[0058] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the concept of the present application. For example, the above features are replaced with the technical features with similar functions disclosed in the present application (but not limited to) to form a technical solution.

Claims

1. A cleaning staff work supervision method based on machine vision analysis, characterized in that: The method comprises: Based on the historical cleaning effect data in different scenarios, a cluster analysis algorithm is used to mine the similarity threshold distribution characteristics to obtain the initial adaptive threshold setting; In the actual cleaning quality monitoring process, we continuously collect cleaning effect feedback data and dynamically update the clustering results through incremental learning. For key cleaning steps, a standard action library containing multi-dimensional information on action trajectory, amplitude, and strength is pre-established; During the monitoring process, computer vision technology is used to extract the action features of cleaning staff in real time, and the action similarity score is calculated by comparing them with the standard action library through the dynamic time warping algorithm; If the action similarity score is lower than the preset threshold, the cleaning action is judged to be irregular or invalid, and a corresponding feedback prompt message is generated; On the basis of unified action evaluation standards, combined with the historical performance and individual characteristics of cleaning staff, personalized threshold setting and feedback strategies are adopted to achieve differentiated evaluation and guidance; Continuously track the changes in the cleaning staff's performance, and determine the degree of improvement in their operating habits through time series analysis of the action similarity scores. The time series analysis uses the moving average method to calculate the average score of the last N cleaning tasks, and compare it with the historical average score to dynamically adjust the personalized evaluation strategy; Update relevant parameters in the standard action library according to the dynamically adjusted personalized evaluation strategy; Apply the updated standard action library to the next round of cleaning quality monitoring process to form a closed-loop optimization mechanism.

2. The cleaning staff work supervision method based on machine vision analysis according to claim 1 is characterized in that: According to the historical cleaning effect data in different scenarios, a cluster analysis algorithm is used to mine the similarity threshold distribution characteristics to obtain the initial adaptive threshold setting, including: Acquire historical cleaning effect data in at least one scenario, wherein the historical cleaning effect data includes images before and after cleaning and cleaning duration; For each of the scenarios, clustering algorithm is used to perform clustering processing on the cleaning effect data to obtain the distribution characteristics of the adaptive threshold in the scenario; During the cleaning monitoring process, the images before and after cleaning in the current scene are obtained, the cleaning effect similarity values ​​of the images before and after cleaning are calculated, and the cleaning effect similarity values ​​are compared with a preset adaptive threshold. If the cleaning effect similarity value is less than the adaptive threshold, it is determined that the cleaning effect does not meet the standard.

3. The cleaning staff work supervision method based on machine vision analysis according to claim 1 is characterized in that: In the actual cleaning quality monitoring process, the cleaning effect feedback data is continuously collected, and the clustering results are dynamically updated by incremental learning, including: Obtain images before and after cleaning, use median filtering to remove noise and histogram equalization to enhance contrast, and normalize the image size to obtain a standardized image; Input the standardized image into a pre-trained ResNet-50 network to extract a 512-dimensional feature vector; The cosine similarity was used to calculate the 512-dimensional feature vector and obtain the similarity of the images before and after cleaning; Get the cleaning start and end time and calculate the cleaning duration; If the cleaning time exceeds the preset range, it is determined to be an abnormal data point.

4. The cleaning staff work supervision method based on machine vision analysis according to claim 1 is characterized in that: For the key cleaning steps, a standard action library containing multi-dimensional information of action trajectory, amplitude, and force is pre-established, including: Obtain video stream data and extract key frame images and corresponding timestamps during the cleaning process; For cleaning scenarios, a standard action library containing multi-dimensional parameters of key actions is pre-established; The human posture estimation algorithm is used to obtain the key action sequence of the cleaner in the video stream; Through the dynamic time warping algorithm, the extracted key action sequence is matched with the standard action library for similarity. If the matching distance exceeds the preset threshold, the action sequence is judged to be non-standard; The image similarity algorithm is used to compare the key frame images before and after cleaning, and a quantitative score of the cleaning effect is given based on the preset scoring criteria.

5. The cleaning staff work supervision method based on machine vision analysis according to claim 4 is characterized in that: In the monitoring process, computer vision technology is used to extract the action features of the cleaning staff in real time, and the action similarity score is calculated by comparing it with the standard action library through the dynamic time warping algorithm, including: Obtain video image data, use the OpenPose algorithm to extract the coordinates of the cleaning staff's skeleton key points, and obtain action sequence data; Use the YOLOv3 algorithm to identify and locate cleaning tools and determine whether the tools are used correctly; Obtain images before and after cleaning, use the FCN algorithm for semantic segmentation, extract pixel information in different areas, and generate warning information if improper use of tools is detected or no tools are detected; By comparing the segmentation results, the percentage of cleanliness improvement in each area is calculated and the cleaning quality is quantitatively evaluated.

6. The cleaning staff work supervision method based on machine vision analysis according to claim 1 is characterized in that: If the action similarity score is lower than the preset threshold, the cleaning action is judged to be irregular or invalid, and corresponding feedback prompt information is generated, including: Obtain the video frame sequence of the cleaning staff's work; For the video frames, the human body posture estimation algorithm is used to obtain the coordinate information of the cleaning staff's skeleton joint points; According to the skeletal joint point coordinate information, a standard action feature vector library describing cleaning actions is constructed; Compare the action feature vector with a preset standard action feature vector library to calculate the cosine similarity; If the similarity is lower than a preset threshold, the action is marked as unqualified, and the timestamp and action type are recorded in the task record file; The images before and after cleaning are obtained, and the difference between the images before and after cleaning is calculated by image subtraction method to obtain a cleaning quality score, and the score result is entered into the task record file.

7. The cleaning staff work supervision method based on machine vision analysis according to claim 1 is characterized in that: On the basis of the unified action evaluation standard, combined with the historical performance and individual characteristics of the cleaning staff, personalized threshold settings and feedback strategies are adopted to achieve differentiated evaluation and guidance, including: Obtaining historical performance data of cleaning personnel, and using a weighted average algorithm to obtain the personalized threshold; Acquire real-time working video and image data of cleaning personnel, analyze the video and image data through a computer vision library, and determine whether the cleaning actions meet the standards; If the cleaning action does not meet the standard, the real-time work data is compared with the personalized threshold, and the degree of deviation is determined by a similarity calculation method; According to the degree of deviation, a corresponding feedback strategy is adopted to generate personalized guidance information; The cleaning task record data is obtained, preprocessed through a data analysis library, and a time series prediction model is used to predict the work performance trend of the cleaning staff, and the personalized threshold and the feedback strategy are dynamically adjusted according to the prediction results.

8. The cleaning staff work supervision method based on machine vision analysis according to claim 1 is characterized in that: The continuous tracking of the changes in the cleaning staff's performance, and the determination of the degree of improvement of their operating habits through time series analysis of the action similarity scores; Time series analysis uses the moving average method to calculate the average score of the last N cleaning tasks and compare it with the historical average score to dynamically adjust the personalized evaluation strategy, including: The camera continuously records the work process of the cleaning staff, obtains the image data and corresponding timestamp information before and after cleaning, and stores them in the task record file; Use the image comparison algorithm to calculate the similarity of the images before and after cleaning, obtain the cleaning effect evaluation score, and write the result and timestamp into the task record file; According to the standard operation time of the cleaning task, extract the timestamps of the start and end of the cleaning from the task record file, calculate the actual duration of the cleaning task, determine whether it meets the standard, and write the judgment result into the task record file; The OpenPose open-source human posture estimation model is used to analyze the image data in the task record file and extract the key point coordinate sequence of the cleaning staff as the action feature; The extracted motion features are dynamically time-aligned with the pre-recorded standard motion templates, and the motion similarity scores are calculated to obtain the motion performance evaluation results of the cleaning staff.

9. The cleaning staff work supervision method based on machine vision analysis according to claim 1 is characterized in that: The updated standard action library is applied to the next round of cleaning quality monitoring process to form a closed-loop optimization mechanism, including: Acquire images before and after cleaning, analyze the images using machine vision technology, obtain environmental difference information before and after cleaning, and quantify the environmental difference information into numerical indicators; Based on the historical cleaning quality assessment records, the association rule mining algorithm is used to analyze the association rules between cleaning quality and cleaning actions, and a mapping model between cleaning actions and quality is established; The gradient descent method is used to dynamically adjust the weight parameters in the mapping model, and the adjusted mapping model is applied to the cleaning quality assessment to achieve dynamic optimization closed-loop monitoring.

Images