Image recognition-based automatic evaluation of hygiene condition of shared space and cleaning scheduling method

Abstract

The application discloses a shared space health condition automatic evaluation and cleaning scheduling method and system based on image recognition, a computer device and a storage medium. The method comprises the following steps: acquiring real-time space images in a shared space; performing multi-task identification and cross-task collaborative calibration on the real-time space images to generate health state data; calculating a dynamic health index based on historical health state sequences and spatial correlation between adjacent areas according to the health state data; when the dynamic health index is lower than a threshold, marking a region to be cleaned and encoding the health state data of the region to a cleaning demand feature vector; calculating the similarity between the vector and each candidate strategy feature vector in a strategy library, determining a target cleaning strategy according to the matching result and sending the target cleaning strategy to a cleaning execution end to perform cleaning work; and acquiring cleaning result data to update the dynamic health index. The application realizes real-time perception, scientific evaluation and accurate scheduling of the health condition of the shared space, and improves the utilization efficiency of cleaning resources.

Description

Technical Field

[0001] This application relates to the field of intelligent cleaning management technology, specifically a method, system, computer equipment, and storage medium for automatic assessment and cleaning scheduling of the hygiene status of shared spaces based on image recognition. Background Technology

[0002] With the popularization of the sharing economy model, shared spaces such as shared chess and card rooms, shared tea rooms, and shared meeting rooms are increasingly used in the commercial and leisure sectors. These spaces typically operate without staff or are self-service, with users booking online and entering independently. Due to the lack of on-site management personnel for real-time supervision, hygiene issues such as litter, cigarette ash, water stains, and tabletop stains generated during use are difficult to detect and address promptly. Currently, hygiene maintenance in these shared spaces relies mainly on automatic reminders after user use or regular inspections by cleaning staff. However, the former is often neglected by users, while the latter suffers from significant delays, making it difficult to guarantee the hygiene of the spaces during high-frequency usage periods. Furthermore, existing video surveillance solutions can only perform simple anomaly detection, failing to provide a comprehensive quantitative assessment of hygiene conditions and lacking effective coordination with cleaning operations. Summary of the Invention

[0003] Therefore, it is necessary to provide a method, system, computer equipment, and storage medium for automatic assessment of hygiene conditions and dynamic scheduling of cleaning in shared spaces based on image recognition, which can realize automatic assessment of hygiene conditions and dynamic scheduling of cleaning.

[0004] Firstly, a method for automatically assessing the hygiene status and scheduling cleaning of shared spaces based on image recognition is provided, the method comprising: Acquire real-time spatial images within the shared space; Multi-task recognition and cross-task collaborative calibration are performed on the real-time spatial image to generate sanitary status data. The multi-task recognition includes at least parallel recognition of regional boundaries, garbage objects, and stained areas. Based on the aforementioned health status data, and considering the historical health status sequence and the spatial correlation between adjacent areas, a dynamic health index is calculated through spatiotemporal joint assessment. When the dynamic hygiene index is lower than the cleaning trigger threshold, the target area is marked as a cleaning area and the hygiene status data of the cleaning area is encoded into a cleaning demand feature vector; wherein the target area is the area corresponding to the dynamic hygiene index that is lower than the cleaning trigger threshold; Calculate the similarity between the cleaning demand feature vector and the strategy feature vector of each candidate strategy in the preset cleaning strategy library, and determine the target cleaning strategy based on the matching result of the similarity in the cleaning strategy library; A cleaning instruction is generated based on the target cleaning strategy, and the cleaning instruction is sent to the cleaning execution terminal to instruct the cleaning execution terminal to perform cleaning operations on the area to be cleaned in accordance with the target cleaning strategy. During or after the cleaning operation, the cleaning result data fed back by the cleaning execution terminal is obtained, and the dynamic hygiene index is updated based on the cleaning result data.

[0005] In one embodiment, the step of performing multi-task recognition and cross-task collaborative calibration on the real-time spatial image to generate health status data includes: The real-time spatial image is input into a multi-task recognition model, which includes: a shared feature extraction network, a semantic segmentation branch, an object detection branch, and a texture analysis branch connected to the shared feature extraction network, and a cross-task feature interaction layer connected to the semantic segmentation branch, the object detection branch, and the texture analysis branch. The shared feature map of the real-time spatial image is extracted by the shared feature extraction network, and the shared feature map is processed by the semantic segmentation branch, the object detection branch and the texture analysis branch respectively to generate an initial region segmentation map, an initial object detection box and an initial stain texture map; The initial region segmentation map, the initial target detection box, and the initial stain texture map are input into the cross-task feature interaction layer. The cross-task feature interaction layer performs spatial position calibration on the identified objects in the initial target detection box and the initial stain texture map based on the initial region segmentation map. Based on the semantic correlation between the calibrated identified objects, the boundary of the initial region segmentation map is refined and corrected, and the fused region segmentation map, target detection box, and stain texture map are output. Based on the region segmentation map, the target detection box, and the stain texture map, the number of garbage objects, the type of garbage objects, and the coverage area of ​​the stain area in each region are counted to generate the hygiene status data.

[0006] In one embodiment, the calculation of the dynamic health index based on historical health status sequences and spatial relationships between adjacent areas through spatiotemporal joint assessment includes: Obtain the historical health status sequence of each region within a preset historical time period; Calculate the time decay coefficient of the health status of each region at the current moment based on the historical health status sequence; A pollution diffusion impact matrix is ​​constructed based on a preset regional adjacency relationship. The pollution diffusion impact matrix is ​​used to characterize the mutual influence weights of the sanitary conditions between adjacent regions. The current basic value of the sanitation index is obtained by weighting and summing the number of garbage objects, the pollution weight corresponding to the garbage object category, and the percentage of the area covered by stains in the current sanitation status data. Multiply the current baseline value of the health index by the time decay coefficient to obtain the time-corrected health index. The dynamic health index is obtained by convolving the time-corrected health index with the health status data of adjacent areas in the pollution diffusion impact matrix.

[0007] In one embodiment, encoding the hygiene status data of the area to be cleaned into a cleaning demand feature vector includes: The hygiene status data of the area to be cleaned is input into the trained autoencoder; The autoencoder's encoding layer performs dimensionality reduction encoding on the hygiene status data, outputting a fixed-dimensional feature vector of the cleaning requirements.

[0008] In one embodiment, calculating the similarity between the cleaning demand feature vector and the strategy feature vectors of each candidate strategy in a preset cleaning strategy library, and determining the target cleaning strategy based on the matching result of the similarity in the cleaning strategy library, includes: Calculate the cosine similarity between the cleaning demand feature vector and the strategy feature vector corresponding to each candidate strategy in the cleaning strategy library; The candidate strategy with the highest cosine similarity that exceeds a preset similarity threshold is determined as the target cleaning strategy; The strategy feature vector is obtained by encoding the cleaning execution information associated with each candidate strategy. The cleaning execution information includes the execution terminal type, tool parameters, and operation mode.

[0009] In one embodiment, The cleaning execution end includes intelligent cleaning equipment or cleaning personnel equipped with mobile terminals; When the cleaning execution end is the intelligent cleaning device, the cleaning instruction includes automatic execution parameters, and the cleaning result data includes operation status data fed back by sensors mounted on the intelligent cleaning device; When the cleaning execution terminal is the cleaning personnel, the cleaning instruction includes work guidance information pushed to the mobile terminal, and the cleaning result data includes post-cleaning images or confirmation information submitted through the mobile terminal.

[0010] In one embodiment, updating the dynamic hygiene index based on cleaning result data fed back from the cleaning execution terminal includes: During the cleaning operation, images of the cleaning process fed back by the cleaning execution terminal are acquired in real time. The cleaning process images are processed by image recognition to identify the garbage objects and stained areas after cleaning, and to generate real-time hygiene status data during the cleaning process. Calculate the real-time hygiene index based on the real-time hygiene status data, and calculate the index deviation between the real-time hygiene index and the expected target index; When the index deviation exceeds the preset deviation range, the tool parameters or operation mode in the target cleaning strategy are dynamically adjusted according to the value of the index deviation and the auxiliary information corresponding to the type of the cleaning execution terminal. Wherein, when the cleaning execution end is the intelligent cleaning equipment, the auxiliary information includes the operation status data fed back by the sensor; when the cleaning execution end is the cleaning personnel, the auxiliary information includes the operation remarks information submitted through the mobile terminal.

[0011] Secondly, an image recognition-based automatic assessment and cleaning scheduling system for the hygiene of shared spaces is provided, the system comprising: The image acquisition module is used to acquire real-time spatial images within the shared space; The recognition and processing module is used to perform multi-task recognition and cross-task collaborative calibration on the real-time spatial image to generate sanitary status data. The multi-task recognition includes at least parallel recognition of regional boundaries, garbage objects and stained areas. The index assessment module is used to calculate a dynamic health index based on the health status data, historical health status sequences, and spatial relationships between adjacent areas through spatiotemporal joint assessment. The area marking module is used to mark the target area as an area to be cleaned when the dynamic hygiene index is lower than the cleaning trigger threshold. The target area is the area corresponding to the dynamic hygiene index that is lower than the cleaning trigger threshold. The feature encoding module is used to encode the hygiene status data of the area to be cleaned into a cleaning requirement feature vector; The strategy matching module is used to calculate the similarity between the cleaning demand feature vector and the strategy feature vector of each candidate strategy in the preset cleaning strategy library, and to determine the target cleaning strategy based on the matching result of the similarity in the cleaning strategy library. The cleaning scheduling module is used to generate cleaning instructions based on the target cleaning strategy and send the cleaning instructions to the cleaning execution terminal to instruct the cleaning execution terminal to perform cleaning operations on the area to be cleaned in accordance with the target cleaning strategy. The feedback update module is used to obtain cleaning result data fed back by the cleaning execution terminal during or after the cleaning operation, and update the dynamic hygiene index based on the cleaning result data.

[0012] Thirdly, a computer device is provided, including a memory and a processor, wherein the memory is communicatively connected to the processor, and the memory stores a computer program that can run on the processor, wherein when the processor executes the computer program, it implements the above-described method for automatic assessment and cleaning scheduling of the hygiene status of shared spaces based on image recognition.

[0013] Fourthly, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the above-described method for automatic assessment and cleaning scheduling of the hygiene status of a shared space based on image recognition.

[0014] The aforementioned image recognition-based automatic assessment and cleaning scheduling method, system, computer equipment, and storage medium for shared space hygiene conditions can accurately and comprehensively generate hygiene status data reflecting the true hygiene conditions of each area by acquiring real-time spatial images and performing multi-task recognition and cross-task collaborative calibration, avoiding the bias of a single recognition task. By combining historical hygiene status sequences and spatial correlations between adjacent areas to perform spatiotemporal joint assessment and calculate a dynamic hygiene index, the quantification of hygiene conditions becomes more scientific and reasonable, effectively reflecting the hygiene change patterns of shared spaces at different usage times and in different areas. By encoding hygiene status data into cleaning demand feature vectors and matching them with a strategy library based on similarity, precise adaptation of cleaning strategies is achieved. By updating the dynamic hygiene index based on cleaning result data, a closed-loop mechanism is formed, enabling dynamic tracking of hygiene conditions and continuous optimization of cleaning effects, thereby improving the utilization efficiency of cleaning resources and ensuring the continuous availability of shared spaces. Attached Figure Description

[0015] Figure 1 This is an application environment diagram of an image recognition-based shared space hygiene status automatic assessment and cleaning scheduling method in one embodiment; Figure 2 This is a flowchart illustrating an image recognition-based method for automatically assessing the hygiene status of shared spaces and scheduling cleaning services in one embodiment. Figure 3 This is a schematic diagram of the structure of an image recognition-based automatic assessment and cleaning scheduling system for the hygiene of shared spaces in one embodiment; Figure 4 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0016] To facilitate understanding of the technical solutions provided in the embodiments of this application, the background technology involved in the embodiments of this application will be described below.

[0017] With the deepening development of the sharing economy, various unattended shared spaces, such as shared chess and card rooms, shared tea rooms, shared meeting rooms, and shared study rooms, are increasingly widely used in the commercial and leisure sectors. The typical operating model for these spaces is as follows: users make online reservations via mobile applications, scan a code to open the door, and use the space independently; the system automatically settles the payment after use. This model greatly improves the utilization efficiency of space resources and reduces labor operating costs.

[0018] However, this unattended operation model also brings new management challenges, the most prominent of which is ensuring environmental hygiene. In traditional manned venues, managers or service personnel can observe the hygiene conditions of the space in real time and intervene promptly to clean when dirt is found during breaks or during use. But in unattended shared spaces, due to the lack of real-time supervision from on-site managers, various hygiene problems generated during use are difficult to detect and handle in a timely manner. Taking shared chess and card rooms as an example, users may generate various types of pollution during use, such as sunflower seed shells, cigarette ash, cigarette butts, beverage stains, food scraps, and table stains. These sources of pollution not only affect the experience of subsequent users, but long-term accumulation may also damage the facilities.

[0019] Currently, the following methods are mainly used for the hygiene maintenance of such shared spaces: The first is a scheduled inspection method, where cleaning staff inspect and clean all rooms according to a fixed schedule (e.g., every hour or every two hours). This method has significant delays and inconsistencies. During peak hours, rooms may have been used multiple times between inspections and become dirty without timely cleaning; conversely, during off-peak hours, rooms may have been unused for extended periods but are still repeatedly cleaned, resulting in a waste of cleaning manpower and materials. The second method is a user-triggered reminder method, where the system sends a "Please take your trash with you" message to users after use or sends an inspection notification to cleaning staff. This method relies on user self-discipline, has limited effectiveness, and still cannot solve the problem of hygiene deterioration during use. The third method is a simple video surveillance solution, which involves installing cameras in the space for post-event tracking or simple anomaly detection (e.g., detecting smoke). These types of solutions can usually only identify a single type of abnormal event, and cannot comprehensively and quantitatively assess the hygiene status of various areas in the space (such as desktops, floors, sofas, ashtrays, etc.), nor can they effectively link the assessment results with cleaning operations.

[0020] In summary, existing technologies lack a closed-loop management mechanism capable of real-time sensing of the hygiene status of various areas within a shared space, scientifically quantifying and assessing the hygiene level, and precisely scheduling cleaning operations based on the assessment results. This leads to technical problems such as delayed response, resource waste, and uncontrollable cleaning quality in the hygiene maintenance of shared spaces, making it difficult to meet the hygiene requirements of high-frequency turnover spaces under unattended operation. This embodiment addresses this issue by proposing an image recognition-based method, system, computer equipment, and storage medium for automatic assessment and cleaning scheduling of the hygiene status of shared spaces.

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0022] The image recognition-based automatic assessment and cleaning scheduling method for shared spaces provided in this application can be applied to, for example... Figure 1 In the application environment shown, the image acquisition terminal 102 is deployed in a shared space to acquire real-time spatial images and send them to the server 104 via the network. The server 104 performs multi-task recognition and cross-task collaborative calibration on the received real-time spatial images to generate hygiene status data. Based on the hygiene status data, and considering historical hygiene status sequences and spatial relationships between adjacent areas, it calculates a dynamic hygiene index through spatiotemporal joint evaluation. When the dynamic hygiene index is lower than the cleaning trigger threshold, the corresponding area is marked as a cleaning area, and the hygiene status data of the cleaning area is encoded into a cleaning demand feature vector. The similarity between the cleaning demand feature vector and the strategy feature vectors of each candidate strategy in the preset cleaning strategy library is calculated, and the target cleaning strategy is determined based on the matching results of the similarity in the cleaning strategy library. A cleaning instruction is generated based on the target cleaning strategy, and the cleaning instruction is sent to the cleaning execution terminal via the network to instruct the cleaning execution terminal to perform cleaning operations on the cleaning area according to the target cleaning strategy. During or after the cleaning operation, server 104 also acquires cleaning result data through the cleaning execution terminal and updates the dynamic hygiene index based on the cleaning result data. The image acquisition terminal 102 can be, but is not limited to, various network cameras, smart cameras, IoT devices with image acquisition capabilities, etc.; the cleaning execution terminal can be, but is not limited to, intelligent cleaning equipment (such as sweeping robots, floor scrubbing robots) or portable devices such as smartphones and tablets used by cleaning personnel equipped with mobile terminals; server 104 can be implemented using a standalone server or a server cluster composed of multiple servers.

[0023] Furthermore, it should be noted that in the technical solutions described in the embodiments of this application, the collection, processing, and use of all image data are carried out only with the explicit authorization of the space owner, operator, and relevant personnel entering the space. Any user personal information and image data involved in the embodiments of this application will be de-identified and strictly protected in accordance with relevant laws and regulations to ensure that personal privacy is not infringed.

[0024] Firstly, in the hygiene management of shared spaces, how to accurately identify the hygiene conditions within the space from raw monitoring images and automatically trigger reasonable cleaning schedules based on the identification results is a pressing technical problem that needs to be solved. To address this, in one embodiment, an image recognition-based method for automatically assessing the hygiene conditions and scheduling cleaning in shared spaces is provided, which is then applied to… Figure 1 Taking server 104 as an example, the explanation is as follows: Figure 2 As shown, the method includes the following steps: Step S1: Obtain real-time spatial images within the shared space.

[0025] Specifically, shared spaces can be any unattended or semi-unattended space that users can book, such as a chess and card room, tea room, meeting room, study room, or gym. These spaces are pre-deployed with image acquisition terminals, such as webcams, to collect image data in real time. Real-time spatial images refer to spatial images captured at the current moment or within the most recent time window; these can be single frames or sequences of multiple frames.

[0026] Step S2: Perform multi-task recognition and cross-task collaborative calibration on the real-time spatial image to generate sanitary status data. Multi-task recognition includes at least parallel recognition of regional boundaries, garbage objects, and stained areas.

[0027] Specifically, multi-task recognition and cross-task collaborative calibration of real-time spatial images can be achieved through deep learning models that simultaneously execute at least three recognition tasks: identifying region boundaries (e.g., segmenting the image into desktop, floor, sofa, ashtray, etc.), identifying litter objects (e.g., identifying specific litter items such as sunflower seed shells, cigarette butts, and tissues), and identifying stained areas (e.g., identifying non-solid contaminant areas such as water stains, oil stains, and dust). Crucially, these three tasks are not performed independently but rather through a cross-task collaborative calibration mechanism. The recognition results of one task are used to correct the recognition results of another. For example, region boundary information is used to calibrate the spatial location of litter objects, ensuring that the identified litter objects are indeed located in reasonable areas such as desktops or floors. Simultaneously, the identified litter objects and stained areas are used to correct the accuracy of region boundary segmentation, thereby generating more accurate and consistent hygiene status data. Hygiene status data is a quantitative description of the hygiene conditions of each region within a space, including at least the number of litter objects, the type of litter objects, and the coverage area of ​​stained areas within each region.

[0028] Step S3: Based on the health status data, and considering the historical health status sequence and the spatial correlation between adjacent areas, calculate the dynamic health index through spatiotemporal joint assessment.

[0029] Specifically, after obtaining the current sanitation status data, a comprehensive dynamic sanitation index needs to be calculated. This index is not based solely on static data at the current moment, but rather on historical sanitation status sequences (i.e., the trend of sanitation changes in the region over a past period) and spatial relationships between adjacent areas (such as the potential spread of contamination from one area to neighboring areas), calculated through a spatiotemporal joint assessment. This allows the dynamic sanitation index to reflect the dynamic changes in sanitation conditions in both time and space, making it more scientific and reasonable than purely static indicators.

[0030] Step S4: When the dynamic hygiene index is lower than the cleaning trigger threshold, the target area is marked as the area to be cleaned, and the hygiene status data of the area to be cleaned is encoded into a cleaning demand feature vector; where the target area is the area corresponding to the dynamic hygiene index that is lower than the cleaning trigger threshold.

[0031] Specifically, when the dynamic sanitation index of a certain area falls below a pre-set cleaning trigger threshold, it indicates that the sanitation condition of that area has deteriorated to the point where intervention is required, and the area is marked as an area awaiting cleaning. To match the most suitable cleaning plan to the area awaiting cleaning, the sanitation status data of the area (such as garbage type, quantity, stain type, area, etc.) needs to be encoded and transformed into a fixed-dimensional cleaning demand feature vector. This feature vector is a mathematical representation of the sanitation needs of that area.

[0032] Step S5: Calculate the similarity between the cleaning demand feature vector and the strategy feature vector of each candidate strategy in the preset cleaning strategy library, and determine the target cleaning strategy based on the matching result of the similarity in the cleaning strategy library.

[0033] Specifically, the cleaning strategy library is a pre-stored database containing multiple candidate cleaning strategies, each corresponding to a strategy feature vector. The strategy feature vector is obtained by encoding the execution information of the cleaning strategy (such as the type of cleaning execution terminal it applies to, the tools used, and the work mode employed). By calculating the similarity between the cleaning requirement feature vector of the area to be cleaned and the strategy feature vectors of each candidate strategy, the strategy with the highest similarity is selected as the target cleaning strategy. This strategy selection method based on feature vector matching enables precise matching of hygiene needs with cleaning solutions.

[0034] Step S6: Generate cleaning instructions based on the target cleaning strategy and send the cleaning instructions to the cleaning execution terminal to instruct the cleaning execution terminal to perform cleaning operations on the area to be cleaned in accordance with the target cleaning strategy.

[0035] Specifically, the cleaning instruction includes various parameters required to perform the cleaning operation, such as the type of cleaning execution device, the specific task to be performed, the work path, and tool parameters. The cleaning instruction is then sent to the designated cleaning execution device. The cleaning execution device can be a smart cleaning device (such as a robotic vacuum cleaner or a robotic floor scrubber) or a cleaning staff member equipped with a mobile terminal. Upon receiving the instruction, the cleaning execution device performs the cleaning operation on the target cleaning area according to the target cleaning strategy.

[0036] Step S7: During or after the cleaning operation, obtain the cleaning result data fed back from the cleaning execution terminal, and update the dynamic hygiene index based on the cleaning result data.

[0037] Specifically, during or after the cleaning operation, the cleaning execution unit will feed back the cleaning result data. This data can be operational status data recorded by sensors on smart devices, or post-cleaning images or confirmation information submitted by cleaning staff via mobile devices. Based on this feedback data, the system updates the previously calculated dynamic hygiene index. For example, if the feedback data shows good cleaning results, the dynamic hygiene index for that area can be improved; if the feedback data shows poor cleaning results, rescheduling or strategy adjustments may be necessary. This forms a complete closed loop of perception-assessment-scheduling-execution-feedback, enabling dynamic tracking of hygiene conditions and continuous optimization of cleaning effectiveness.

[0038] Based on the above, this method starts with real-time image perception, and goes through multi-dimensional accurate identification, spatiotemporal dynamic evaluation, demand feature encoding, intelligent strategy matching, precise scheduling and execution, and result feedback closed loop. It realizes real-time perception, scientific evaluation and precise scheduling of the hygiene status of shared spaces, improves the utilization efficiency of cleaning resources and ensures the continuous availability of shared spaces.

[0039] In the multi-task recognition process, ensuring that the results of different recognition tasks can coordinate and correct each other to improve the accuracy of health status data generation is a technical problem that needs to be further solved. Therefore, in one embodiment, multi-task recognition and cross-task collaborative calibration are performed on real-time spatial images to generate health status data, specifically including: Step S21: Input the real-time spatial image into the multi-task recognition model. The multi-task recognition model includes: a shared feature extraction network, a semantic segmentation branch, an object detection branch, and a texture analysis branch connected to the shared feature extraction network, and a cross-task feature interaction layer connected to the semantic segmentation branch, the object detection branch, and the texture analysis branch.

[0040] Specifically, the multi-task recognition model is an end-to-end neural network model. The shared feature extraction network is located at the front end of the model and typically consists of multiple convolutional and pooling layers, such as existing convolutional neural network architectures like ResNet and VGG. Its function is to extract general, high-dimensional visual features from the input real-time spatial image and generate a shared feature map. This shared feature map contains low-level information such as edges, textures, and shapes in the image and can be shared by multiple subsequent task branches, thereby reducing redundant computation and improving model efficiency.

[0041] Step S22: Extract shared feature maps from real-time spatial images through a shared feature extraction network, and process the shared feature maps through semantic segmentation branch, object detection branch and texture analysis branch respectively to generate initial region segmentation map, initial object detection box and initial stain texture map.

[0042] Specifically, after the shared feature extraction network, three parallel task branches are connected: a semantic segmentation branch, an object detection branch, and a texture analysis branch. The semantic segmentation branch performs pixel-level classification of the image, determining which region each pixel belongs to, such as desktop, floor, wall, sofa, etc. This branch typically uses a fully convolutional network structure, outputting an initial region segmentation map of the same size as the input image, where the value of each pixel represents the region category it belongs to. The object detection branch identifies discrete trash objects in the image, such as cigarette butts, tissues, bottles, etc. This branch can use object detection algorithms such as YOLO, Faster R-CNN, etc., outputting a series of initial object detection boxes, each corresponding to an identified trash object, along with the object's category label and confidence score. The texture analysis branch identifies non-solid stains in the image, such as water stains, oil stains, dust, etc. This branch analyzes the image's local texture features and color features to output an initial stain texture map, marking the pixel regions identified as stains.

[0043] Step S23: Input the initial region segmentation map, initial target detection box, and initial stain texture map into the cross-task feature interaction layer. The cross-task feature interaction layer performs spatial position calibration on the identified objects in the initial target detection box and initial stain texture map based on the initial region segmentation map. Based on the semantic correlation between the calibrated identified objects, the boundary of the initial region segmentation map is refined and corrected. The fused region segmentation map, target detection box, and stain texture map are then output.

[0044] Specifically, while the initial results output by these three branches already contain rich information, inconsistencies or even contradictions may exist between them. For example, the object detection branch might detect a "cigarette butt" in a non-desktop area, but according to the semantic segmentation branch, this area belongs to the ground, where a cigarette butt is reasonable. However, if the area belongs to a wall, it might be a false detection. The cross-task feature interaction layer is designed to address these issues. It takes the initial results from the three branches as input and performs the following collaborative calibration operation: First, spatial location calibration is performed. The cross-task feature interaction layer utilizes the initial region segmentation map output by the semantic segmentation branch to provide a "spatial context" for each object detection box and each stain region. It checks whether the center point or boundary of each object detection box falls within a reasonable region category. If an object detection box falls into an unreasonable region (e.g., detecting a water glass on a wall), the confidence of the detection box is reduced, or it is corrected according to the region boundary to pull it back to a reasonable region boundary. For stain regions, calibration is also performed based on the region segmentation map to ensure that the stain region label matches the actual physical surface.

[0045] Secondly, semantic correlation correction is performed. The cross-task feature interaction layer also analyzes the semantic correlation between the calibrated object detection boxes and the stain regions. For example, if a water glass and a water stain are detected on a table at the same time, there may be a causal relationship between the two, and the model may increase the recognition confidence of the water stain. Conversely, if a stain region exists in isolation without any related trash objects, the model may need to re-evaluate whether the region is a real stain. More importantly, the semantic correlation between these identified objects can be used to refine the boundaries of the region segmentation map. For example, if the object detection branch detects a series of consecutive trash objects near the edge of a region, and these trash objects should belong to the same operational plane according to semantic correlation, the cross-task feature interaction layer may adjust the boundaries of the region segmentation map outward to include the regions containing these objects.

[0046] After the aforementioned collaborative calibration, the cross-task feature interaction layer outputs a fused region segmentation map, object detection bounding boxes, and stain texture map. These fused results are more consistent and accurate in terms of spatial location and semantic relationships.

[0047] Step S24: Based on the region segmentation map, target detection box, and stain texture map, count the number of garbage objects, the type of garbage objects, and the coverage area of ​​the stain area in each region to generate hygiene status data.

[0048] Specifically, based on the fused identification results, sanitation status data is statistically generated. Specifically, the boundary range of each region can be determined according to the fused region segmentation map. Then, the fused object detection boxes are traversed to count the number of waste objects falling within each region and record the category of each object. Simultaneously, based on the fused stain texture map, the total number of stain pixels in each region is calculated, thus obtaining the coverage area of ​​the stain region within that region (usually expressed as pixel area or actual physical area). These statistical data constitute the sanitation status data of that space at the current moment.

[0049] Based on the above, by utilizing multi-task learning and cross-task feature interaction mechanisms, the limitations that may exist in a single identification task are solved. Through mutual calibration and correction between tasks, the accuracy and robustness of hygiene status data generation are significantly improved, providing a reliable data foundation for subsequent hygiene assessment and cleaning scheduling.

[0050] After obtaining health status data for each region, a key technical problem to be solved is how to comprehensively consider historical trends and inter-regional interactions to calculate a dynamic health index that scientifically reflects the current urgency of public health. To this end, in one embodiment, a dynamic health index is calculated through spatiotemporal joint assessment based on historical health status sequences and spatial relationships between adjacent regions. Specifically, this includes: Step S31: Obtain the historical health status sequence of each region within a preset historical time period.

[0051] Specifically, the preset historical time period can be set according to actual needs, such as the past 2 hours, the past 24 hours, etc. The historical sanitation status sequence records the sanitation status data of the area at various points in the past. This sequence reflects the trend of changes in the sanitation status of the area over time.

[0052] Step S32: Calculate the time decay coefficient of the health status of each region at the current moment based on the historical health status sequence.

[0053] Specifically, the time decay coefficient is based on the physical fact that the longer garbage and stains exist, the greater their negative impact on overall sanitation, or in other words, the higher their age. Therefore, for an area, if its sanitation data has continuously deteriorated over a period of time, or if certain pollutants have remained for an excessively long period, then even if the current sanitation data is the same as another area, its actual sanitation urgency may be higher. The time decay coefficient can be calculated using various mathematical models. In a feasible embodiment, an exponential decay model can be used. Assuming the historical sanitation sequence is... ,in Indicates the past number At a certain point in time, This indicates the current moment. So, the region... Time decay coefficient at the current moment It can be calculated in the following ways: First, calculate the gradient of historical health status changes, or more simply, calculate the duration of historical health status above a certain threshold. An exemplary calculation method is as follows: in, The number of historical time points. It is an indicator function, when time point The key indicators in the sanitation data (such as the amount of garbage) are set to 1 when they exceed a certain level, and 0 otherwise. It is a time decay rate control parameter. It's an adjustment coefficient. The physical meaning of this formula is: the more recent and more numerous historically unsanitary periods there are, the higher the time decay coefficient. A higher value (greater than 1) will amplify the current sanitation index in subsequent calculations, indicating a greater need for attention. Conversely, if historical sanitation conditions have been consistently good, then... Approaching 1.

[0054] Step S33: Construct a pollution diffusion impact matrix based on the preset regional adjacency relationship. The pollution diffusion impact matrix is ​​used to characterize the mutual influence weight of the sanitary conditions between adjacent regions.

[0055] Specifically, the pollution diffusion impact matrix is ​​based on predefined regional adjacency relationships. First, the shared space is divided into multiple regions, and the adjacency relationships between these regions are defined, such as adjacent, relative, etc. Assuming shared... If there is a region, then construct a matrix , of which elements Indicates the region For the region The weight of pollution diffusion impact. Typically, if the region... and region If they are not adjacent, then If adjacent, then This is a non-zero value, which can be preset based on factors such as the actual spatial layout, air circulation, and pedestrian movement paths. For example, two areas directly connected by a doorway can have a higher weighting for their mutual influence; areas connected only by a small window can have a lower weighting. This value can typically be set to 1, representing the region's impact on itself. (Pollution diffusion impact matrix) The purpose is to incorporate the sanitation status of adjacent areas into the assessment of the current area. If an area itself is very clean, but its adjacent areas are very dirty, then that area also faces the risk of contamination spreading, and its dynamic sanitation index should be appropriately increased.

[0056] Step S34: Weight and sum the number of garbage objects, the pollution weight corresponding to the garbage object category, and the percentage of the area covered by stains in the current sanitation status data to obtain the current basic value of the sanitation index.

[0057] Specifically, the current baseline value of the sanitation index quantifies the current static sanitation status of a region. Let the region be... Current sanitation status data includes: the number of waste objects. The collection of garbage object categories and the percentage of the area covered by stains. (Percentage of stained area to total area). Different categories of waste have different pollution weights; for example, cigarette butts may have a higher weight than paper towels, and liquids may have a higher weight than solids. Assume there are a total of [number missing] waste objects. Types, each type The corresponding pollution weight is .area The first one identified inside The categories of the garbage objects are: The corresponding pollution weight is So, the region Current baseline health index We can obtain the result by weighted summation: in, and These are the balanced weighting coefficients for the "garbage object" and "stain area" items, which can be adjusted according to the actual scenario. For example, in a card room, cigarette ash and sunflower seed shells are the main pollutants, so the weighting coefficients can be appropriately increased. In the tea room, water stains are a major problem; improvements could be made by increasing the water level appropriately. . The larger the value, the worse the current sanitation conditions.

[0058] Step S35: Multiply the current baseline health index value by the time decay coefficient to obtain the time-corrected health index.

[0059] Specifically, the time-adjusted health index The calculation formula is: This introduces the influence of the historical time dimension.

[0060] Step S36: Convolve the time-corrected sanitation index with the sanitation status data of adjacent areas in the pollution diffusion impact matrix to obtain the dynamic sanitation index.

[0061] Specifically, the convolution calculation here refers to a weighted summation that takes into account the influence of adjacent areas, i.e., the dynamic health index. The calculation formula is: Alternatively, a normalized form can be used: in, It is the region in the pollution diffusion impact matrix For the region The influence weight. This formula means that the region... The dynamic health index depends not only on its own time-adjusted health index It also depends on all the adjacent areas that it is spatially associated with. Time-corrected health index The data is then aggregated according to preset weights. If an area is surrounded by multiple dirty areas, its final dynamic hygiene index will be calculated. This will increase accordingly, potentially triggering cleaning and achieving a preventative cleaning effect.

[0062] Based on the above, the dynamic hygiene index calculated by this spatiotemporal joint assessment method takes into account not only the current static level of dirt and grime, but also historical trends and the mutual influence between adjacent areas. Therefore, it can more comprehensively and scientifically reflect the true hygiene urgency of each area in the shared space, providing an accurate basis for subsequent cleaning decisions.

[0063] After identifying the area to be cleaned, the key step in connecting hygiene assessment and strategy selection is to transform its complex, multi-dimensional hygiene status data into a mathematical representation that facilitates matching calculations. Therefore, in one embodiment, the hygiene status data of the area to be cleaned is encoded into a cleaning demand feature vector, specifically including: Step S41: Input the hygiene status data of the area to be cleaned into the trained autoencoder.

[0064] Specifically, sanitation status data is typically a combination of multiple pieces of information, such as the quantity of waste, waste type, and area of ​​stains. This information can be highly dimensional, and complex nonlinear relationships may exist between different dimensions. Directly matching raw data is not only computationally inefficient but also fails to capture deep-seated sanitation demand patterns. Therefore, this embodiment introduces an autoencoder for feature encoding. An autoencoder is an unsupervised learning neural network model whose goal is to learn a compressed representation (encoding) of the input data and then reconstruct the original input as accurately as possible from this compressed representation (decoding). An autoencoder typically consists of two parts: an encoder and a decoder. The encoder maps high-dimensional input data to a low-dimensional latent space (i.e., the encoding layer), obtaining a compressed feature vector; the decoder then reconstructs the original input data from this low-dimensional feature vector. Through training, by minimizing the reconstruction error, the encoder learns how to extract the most core and representative features from the input data.

[0065] Step S42: The hygiene status data is dimensionality reduced by the encoding layer of the autoencoder to output a fixed-dimensional cleaning requirement feature vector.

[0066] Specifically, the training data for the autoencoder can be a large number of hygiene status data samples collected from historical cleaning records. After training, the hygiene status data of the area to be cleaned is input into the trained autoencoder. The encoding layer of the autoencoder processes the input hygiene status data, performing nonlinear transformations and dimensionality reduction through a series of neural network layers (such as fully connected layers), ultimately outputting a fixed-dimensional vector, which is the cleaning requirement feature vector. The dimension of this vector is much smaller than the dimension of the original hygiene status data; for example, the original dozens of dimensions can be compressed to 8 or 16 dimensions. This low-dimensional vector condenses the core features of the hygiene requirements of the area to be cleaned, such as potential patterns like "heavy soot pollution requirement" and "light water stain cleaning requirement," facilitating efficient matching with subsequently encoded cleaning strategies.

[0067] Based on the above, the complex and unstructured health status data is transformed into a structured, fixed-dimensional feature vector, laying the foundation for subsequent precise strategy matching.

[0068] After obtaining the cleaning demand feature vector, the crucial decision-making problem is how to accurately select the most suitable cleaning strategy from a pre-defined cleaning strategy library. To address this, in one embodiment, the similarity between the cleaning demand feature vector and the strategy feature vectors of each candidate strategy in the pre-defined cleaning strategy library is calculated. The target cleaning strategy is then determined based on the matching results of the similarity scores in the cleaning strategy library. Specifically, this includes: Step S51: Calculate the cosine similarity between the cleaning demand feature vector and the strategy feature vector corresponding to each candidate strategy in the cleaning strategy library. The strategy feature vector is obtained by encoding the cleaning execution information associated with each candidate strategy. The cleaning execution information includes the execution terminal type, tool parameters, and operation mode.

[0069] Specifically, the cleaning strategy library stores multiple candidate cleaning strategies. Each candidate strategy is associated with specific cleaning execution information, which consists of the specific parameters required to execute the strategy. The cleaning execution information includes: Execution type: Specifies which type of cleaning execution device this strategy applies to, such as a robot vacuum cleaner, a floor scrubber, or manual cleaning. If it is manual cleaning, you can further specify whether it is a cleaner equipped with ordinary tools or a professional cleaner equipped with a high-pressure washer; Tool parameters: Specifies the tools and their operating parameters required to execute this strategy. For smart devices, tool parameters may include suction power (e.g., 500Pa, 1000Pa), water volume (e.g., low water volume, high water volume), brush speed, etc. For manual cleaning, tool parameters may specify the type of cleaning tool, such as microfiber cloth, sponge mop, and the recommended type of cleaning agent, etc. Job mode: Specifies the job path and method when executing this strategy. For example, it can be a one-time full-coverage cleaning mode or a focused area repeated cleaning mode; it can be a zigzag path or an edge path; for manual cleaning, it can be a suggested cleaning sequence, such as tabletops first and then floors.

[0070] Furthermore, to transform this cleaning execution information into a strategy feature vector that can be used for similarity calculation, it needs to be encoded. The encoding methods can be varied. For example, the execution type can be one-hot encoded, tool parameters can be normalized and quantified, and the work mode can also be vectorized. More preferably, another encoding model (such as an autoencoder similar to those described above, or a simple fully connected network) can be used to map all this information into a fixed-dimensional vector space, generating a strategy feature vector. This strategy feature vector represents the position of the cleaning strategy in the capability space.

[0071] Step S52: The candidate strategy with the highest cosine similarity that exceeds the preset similarity threshold is determined as the target cleaning strategy.

[0072] Specifically, after obtaining the cleaning demand feature vector of the area to be cleaned and the strategy feature vector of each candidate strategy in the cleaning strategy library, the similarity between them can be calculated. This embodiment uses cosine similarity, a technique already in use, to measure this similarity. The calculated cosine similarity value ranges from -1 to 1. The closer the value is to 1, the more consistent the directions of the two vectors are, meaning the better the hygiene needs of the area to be cleaned match the capabilities of the cleaning strategy.

[0073] After calculating the cosine similarity between all candidate strategies and the cleaning requirement feature vectors, the candidate strategy with the highest similarity is selected. However, simply having the highest similarity may not be enough, because if the highest similarity is still low, it indicates that the strategy library may not contain a strategy truly suitable for the current needs. Therefore, a preset similarity threshold is set, for example, 0.8. Only when the highest cosine similarity is greater than this threshold is the candidate strategy determined as the target cleaning strategy. If the similarity of all candidate strategies is below the threshold, the default strategy can be used, or an alarm can be triggered to prompt the manager to add a new cleaning strategy.

[0074] Based on the above, a precise match between hygiene needs and cleaning solutions was achieved, ensuring that the scheduled cleaning operations were most suitable for the current type of dirt, thereby improving cleaning efficiency and quality.

[0075] In practical applications, the cleaning execution end can be different types of entities. How to generate appropriate cleaning instructions and process the cleaning result data fed back by these different types of execution ends requires further consideration. Therefore, in one embodiment, the cleaning execution end includes intelligent cleaning equipment or cleaning personnel equipped with mobile terminals. When the cleaning execution end is intelligent cleaning equipment, the cleaning instructions include automatic execution parameters, and the cleaning result data includes work status data fed back by sensors mounted on the intelligent cleaning equipment. When the cleaning execution end is a cleaning personnel, the cleaning instructions include work guidance information pushed to the mobile terminal, and the cleaning result data includes post-cleaning images or confirmation information submitted through the mobile terminal.

[0076] Specifically, intelligent cleaning equipment can include robotic vacuum cleaners, robotic floor scrubbers, and robotic window cleaners. These devices can receive instructions via a network and execute them automatically. When the cleaning execution end is an intelligent cleaning device, the generated cleaning instructions need to include automatic execution parameters. These parameters are machine instructions that the device can understand and execute, such as: navigation path parameters: such as the coordinates of the target area and the planning data of the cleaning route; operation parameters: such as suction setting (e.g., set to 800Pa), water volume setting (e.g., set to medium), brush speed setting, whether to turn on UV sterilization, etc.; task sequence parameters: such as performing dry sweeping first, followed by wet mopping. During the cleaning operation, various sensors on the intelligent cleaning device will generate real-time operation status data, which is fed back as part of the cleaning result data. These sensors can include: position sensors (such as lidar and gyroscopes) to provide feedback on the device's current location and covered area; status sensors (such as dustbin overflow sensors and water tank level sensors) to provide feedback on the device's own status; environmental sensors (such as dust sensors) to provide feedback on the amount of dust sucked in during cleaning, indirectly reflecting the degree of dirtiness in the area; and vision sensors (such as the device's built-in camera to capture images during the operation). This operational status data is fed back to the server in real time or periodically to monitor the progress and quality of the operation.

[0077] Mobile terminals can be devices such as smartphones and tablets carried by cleaning staff. When the cleaning execution is performed by the cleaning staff, the generated cleaning instructions need to be translated into work guidance information that the staff can understand and pushed to the cleaning staff's mobile terminal via the network. Work guidance information may include: task description: such as "Please clean the tabletops and floors in area A of the card room"; location guidance: such as a floor plan or specific location description of the area to be cleaned; cleaning requirements: such as "There are stubborn water stains on the tabletops, it is recommended to use a special cleaning agent"; "There are a lot of sunflower seed shells on the floor, please sweep first and then mop"; tool suggestions: such as "It is recommended to use a vacuum cleaner to clean the floor"; standard procedures: such as the standard operating steps for cleaning this area. After receiving the work guidance information, the cleaning staff will perform manual cleaning according to the instructions. After completing the cleaning work, the cleaning staff needs to submit the cleaning result data through the mobile terminal. This typically takes two forms: post-cleaning images: Cleaning staff use mobile devices to take photos of the cleaned area and upload them; these images will serve as the basis for subsequent acceptance and evaluation. Confirmation information: After completing the work, cleaning staff click the "Confirm Completion" button on their mobile devices or fill out a simple feedback form, such as "Completed, stubborn stains require special treatment," etc. These images or confirmation information submitted by cleaning staff constitute the cleaning result data in manual cleaning scenarios.

[0078] Based on the above, it can flexibly adapt to different types of cleaning execution terminals. Whether it is fully automatic intelligent equipment or cleaning personnel that require manual intervention, it can obtain effective instructions and result feedback, thus expanding the scope of application of this method.

[0079] During cleaning operations, how to monitor the results in real time and dynamically adjust cleaning strategies based on actual conditions to achieve higher quality cleaning needs further clarification. Therefore, in one embodiment, a dynamic hygiene index is updated based on cleaning result data fed back from the cleaning execution end, specifically including: Step S71: During the cleaning operation, acquire images of the cleaning process fed back from the cleaning execution terminal in real time.

[0080] Specifically, this step focuses on real-time adjustments during the cleaning operation, rather than final acceptance upon completion. For smart cleaning equipment, images of the cleaning process can be captured and uploaded in real-time by the equipment's built-in camera. For cleaning personnel, images of the cleaning process can be captured and uploaded by them using a mobile device during the operation, as required (e.g., after cleaning each area).

[0081] Step S72: Perform image recognition processing on the cleaning process images to identify the garbage objects and stained areas after cleaning, and generate real-time hygiene status data during the cleaning process.

[0082] Specifically, image recognition processing can be similar to the method used to generate hygiene status data, but the goal is to identify the post-cleaning hygiene status from the image, i.e., the degree of cleaning achieved. The identified results include residual trash and stained areas in the current image, thereby generating real-time hygiene status data during the cleaning process. This data reflects the effectiveness of the cleaned areas and the residue in the uncleaned areas.

[0083] Step S73: Calculate the real-time hygiene index based on the real-time hygiene status data, and calculate the index deviation between the real-time hygiene index and the expected target index.

[0084] Specifically, the calculation method for the real-time hygiene index can be similar to the method described above for calculating the current baseline hygiene index, such as weighted summation of residual garbage and stains. Simultaneously, a pre-set expected compliance index is established, representing the hygiene requirements for the area after cleaning. The index deviation is the difference between the real-time hygiene index and the expected compliance index. A positive index deviation indicates that the current hygiene condition is still below the standard, and cleaning progress is insufficient; a negative index deviation or close to zero indicates that the standard has been approached or met.

[0085] Step S74: When the index deviation exceeds the preset deviation range, dynamically adjust the tool parameters or operation mode in the target cleaning strategy based on the value of the index deviation and the auxiliary information corresponding to the type of cleaning execution terminal. Specifically, when the cleaning execution terminal is an intelligent cleaning device, the auxiliary information includes operation status data fed back by sensors; when the cleaning execution terminal is a cleaning worker, the auxiliary information includes operation remarks submitted via a mobile terminal.

[0086] Specifically, when the index deviation falls within the preset deviation range, the work progress is considered normal and no intervention is needed. When the index deviation exceeds this range, it indicates that the cleaning effect is far from expected, and the strategy needs to be adjusted. At this time, based on the value of the index deviation and the auxiliary information corresponding to the cleaning execution type, the tool parameters or operation mode in the target cleaning strategy are dynamically adjusted. The value of the index deviation determines the magnitude of the adjustment; for example, if the index deviation is large, it may be necessary to increase the suction power by one level or increase the number of cleaning cycles.

[0087] In addition, auxiliary information provides a basis for adjustments. For intelligent cleaning equipment, sensor feedback on the operating status might show that the dustbin is full, causing a decrease in suction power. This explains why the cleaning effect is poor. In this case, the adjustment strategy might be to pause the operation and prompt the user to empty the dustbin, rather than simply increasing the suction power. Alternatively, data might show that the equipment is repeatedly slipping in a certain area, possibly due to oil stains on the floor. In this case, the adjustment strategy might be to switch to a stronger cleaning mode. For cleaning personnel, auxiliary information includes job notes submitted via mobile devices. For example, a cleaning personnel might note "adhesive residue on the desktop is difficult to remove." After receiving this information, the server can adjust the strategy, pushing more specific instructions to the personnel, such as "Please use adhesive remover," or adjusting the subsequent cleaning strategy, marking the cleaning mode for that area as "stubborn stain mode."

[0088] Adjustments can be specifically applied to tool parameters or operating modes within the target cleaning strategy. For example, the suction power of a smart device can be adjusted from 500Pa to 800Pa, or the operating mode can be changed from "single cleaning" to "two cleanings + targeted mopping." For manual cleaning, the pushed operation instructions can be updated, suggesting the use of different tools or cleaning agents.

[0089] Based on the above, deviations can be detected and corrected in a timely manner during the cleaning operation, avoiding the problem of incomplete cleaning that may be caused by simple cleaning, thereby ensuring the final cleaning quality.

[0090] In some embodiments, in order to enable the hygiene assessment model and cleaning strategy library to continuously optimize themselves as actual operational data accumulates and achieve continuous performance improvement, the model parameters and strategy feature vectors can also be optimized using existing technologies, which will not be elaborated in this embodiment.

[0091] It should be understood that, although Figure 2 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 2 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0092] Secondly, this embodiment provides an image recognition-based automatic assessment and cleaning scheduling system for the hygiene status of shared spaces, such as... Figure 3As shown, it includes an image acquisition module, a recognition and processing module, an index evaluation module, a region marking module, a feature encoding module, a strategy matching module, a cleaning scheduling module, and a feedback update module.

[0093] An image acquisition module is used to acquire real-time spatial images within a shared space. This module is deployed within the shared space, and its physical entity can be various types of image acquisition devices, such as network cameras, smart cameras, or IoT devices with image acquisition capabilities. The module can continuously acquire images at a preset frame rate (e.g., one frame per second) and transmit the acquired real-time spatial images to the subsequent processing module via wired or wireless networks. The image acquisition module and the recognition processing module are connected via a communication interface, and data transmission can be performed using existing network communication protocols, such as HTTP or RTSP. Using a camera with night vision capabilities ensures normal image acquisition even in low-light environments. This application does not limit the specific model of the image acquisition module, as long as it meets the basic requirements for image acquisition.

[0094] The recognition and processing module is used for multi-task recognition and cross-task collaborative calibration of real-time spatial images to generate sanitation status data. Multi-task recognition includes at least parallel recognition of region boundaries, litter objects, and soiled areas. This module can be a software module deployed on a server, utilizing the server's GPU resources for efficient image inference computation.

[0095] The index assessment module is used to calculate a dynamic health index based on health status data, historical health status sequences, and spatial relationships between adjacent areas, through a spatiotemporal joint assessment. The dynamic health index output by this module is either a numerical value or a vector, used to quantify the health urgency of each area.

[0096] The area marking module marks target areas as areas requiring cleaning when the dynamic hygiene index falls below the cleaning trigger threshold, and encodes the hygiene status data of these areas into a cleaning requirement feature vector. The target area is defined as the area corresponding to the dynamic hygiene index falling below the cleaning trigger threshold. This module internally stores a preset cleaning trigger threshold, which can be set and adjusted according to the operational strategies of different shared spaces. For example, the threshold can be set lower (i.e., higher hygiene requirements) for high-end business conference rooms, while it can be appropriately relaxed for ordinary card and board game rooms. The module outputs a list of areas requiring cleaning, which is then passed to the feature encoding module for further processing.

[0097] The feature encoding module encodes the hygiene status data of the area to be cleaned into a cleaning demand feature vector. This module is responsible for transforming the hygiene status data of the area to be cleaned into a mathematical representation that facilitates matching calculations.

[0098] The strategy matching module calculates the similarity between the cleaning requirement feature vector and the strategy feature vectors of each candidate strategy in the preset cleaning strategy library, and determines the target cleaning strategy based on the matching results of the similarity in the cleaning strategy library. The output of this module is the identifier of the selected target cleaning strategy and its associated cleaning execution information.

[0099] The cleaning scheduling module generates cleaning instructions based on the target cleaning strategy and sends these instructions to the cleaning execution terminal, instructing the terminal to perform cleaning operations according to the target strategy. This module sends the generated cleaning instructions to the designated cleaning execution terminal via the network and initiates the cleaning operation.

[0100] The feedback and update module is used to acquire cleaning result data from the cleaning execution end during or after the cleaning operation, and update the dynamic hygiene index based on the cleaning result data. This module is responsible for collecting the execution results of the cleaning operation and forming a closed-loop optimization.

[0101] The collaboration between the above modules constitutes a complete data flow and processing chain: the image acquisition module collects raw data → the recognition and processing module extracts hygiene features → the index assessment module quantifies hygiene conditions → the area marking module identifies cleaning needs → the feature encoding module converts needs into vectors → the strategy matching module selects the best strategy → the cleaning scheduling module executes scheduling instructions → the feedback update module collects results and optimizes the model. The modules can communicate and collaborate through system bus, message queues, API calls, and other means.

[0102] In one embodiment, the recognition processing module includes an image recognition processing unit.

[0103] The image recognition processing unit is used to input real-time spatial images into a multi-task recognition model. The multi-task recognition model includes: a shared feature extraction network, semantic segmentation branches, object detection branches, and texture analysis branches connected to the shared feature extraction network, and a cross-task feature interaction layer connected to the semantic segmentation, object detection, and texture analysis branches. It is used to extract shared feature maps from the real-time spatial images through the shared feature extraction network, and process these shared feature maps through the semantic segmentation, object detection, and texture analysis branches to generate initial region segmentation maps, initial object detection boxes, and initial stain texture maps. It is also used to... The initial region segmentation map, initial object detection boxes, and initial stain texture map are input to the cross-task feature interaction layer. The cross-task feature interaction layer performs spatial location calibration on the identified objects in the initial object detection boxes and initial stain texture map based on the initial region segmentation map, and refines the boundaries of the initial region segmentation map based on the semantic correlation between the calibrated identified objects. The output is a fused region segmentation map, object detection boxes, and stain texture map. It is also used to count the number of garbage objects, the type of garbage objects, and the coverage area of ​​the stain area in each region based on the region segmentation map, object detection boxes, and stain texture map, and generate sanitary status data.

[0104] In one embodiment, the index assessment module includes a dynamic health index calculation unit.

[0105] The dynamic sanitation index calculation unit is used to obtain the historical sanitation status sequence of each region within a preset historical period; to calculate the sanitation status time decay coefficient of each region at the current moment based on the historical sanitation status sequence; to construct a pollution diffusion influence matrix based on the preset regional adjacency relationship, which is used to characterize the mutual influence weight of sanitation status between adjacent regions; to perform a weighted summation of the number of garbage objects, the pollution weight corresponding to the garbage object category, and the proportion of the area covered by stains in the sanitation status data at the current moment to obtain the current basic value of the sanitation index; to multiply the current basic value of the sanitation index by the time decay coefficient to obtain the time-corrected sanitation index; and to perform convolution calculation between the time-corrected sanitation index and the sanitation status data of adjacent regions in the pollution diffusion influence matrix to obtain the dynamic sanitation index.

[0106] In one embodiment, the feature encoding module includes a feature encoding unit.

[0107] The feature encoding unit is used to input the hygiene status data of the area to be cleaned into the trained autoencoder; and to perform dimensionality reduction encoding on the hygiene status data through the encoding layer of the autoencoder, outputting a fixed-dimensional cleaning requirement feature vector.

[0108] In one embodiment, the policy matching module includes a target policy matching unit.

[0109] The target strategy matching unit is used to calculate the cosine similarity between the cleaning demand feature vector and the strategy feature vector corresponding to each candidate strategy in the cleaning strategy library. The strategy feature vector is obtained by encoding the cleaning execution information associated with each candidate strategy. The cleaning execution information includes the execution terminal type, tool parameters, and operation mode. The unit is also used to determine the candidate strategy with the highest cosine similarity that exceeds the preset similarity threshold as the target cleaning strategy.

[0110] In one embodiment, the cleaning scheduling module includes an instruction generation unit.

[0111] The instruction generation unit is used to, when the cleaning execution end is an intelligent cleaning device, provide cleaning instructions including automatic execution parameters and cleaning result data including operation status data fed back by sensors mounted on the intelligent cleaning device; and when the cleaning execution end is a cleaning staff member equipped with a mobile terminal, provide cleaning instructions including operation guidance information pushed to the mobile terminal and cleaning result data including post-cleaning images or confirmation information submitted through the mobile terminal.

[0112] In one embodiment, the feedback update module includes an exponential update unit.

[0113] The index update unit is used to acquire cleaning process images fed back by the cleaning execution terminal in real time during the cleaning operation; to perform image recognition processing on the cleaning process images to identify garbage objects and stained areas after cleaning, and generate real-time hygiene status data during the cleaning process; to calculate the real-time hygiene index based on the real-time hygiene status data, and to calculate the index deviation between the real-time hygiene index and the expected target index; and to dynamically adjust the tool parameters or operation mode in the target cleaning strategy according to the value of the index deviation and the auxiliary information corresponding to the type of cleaning execution terminal when the index deviation exceeds the preset deviation range. When the cleaning execution terminal is an intelligent cleaning device, the auxiliary information includes operation status data fed back by sensors; when the cleaning execution terminal is a cleaning person, the auxiliary information includes operation remarks information submitted through a mobile terminal.

[0114] Specific limitations regarding the image recognition-based automatic assessment and cleaning scheduling system for shared space hygiene can be found in the limitations of the image recognition-based automatic assessment and cleaning scheduling method for shared space hygiene above, and will not be repeated here. Each module in the aforementioned image recognition-based automatic assessment and cleaning scheduling system for shared space hygiene can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0115] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides the environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores all data required for system operation. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements an image recognition-based method for automatically assessing and scheduling the sanitation status of a shared space.

[0116] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0117] Thirdly, a computer device is provided, including a memory and a processor. The memory and the processor are communicatively connected, and the memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the image recognition-based automatic assessment and cleaning scheduling method for the hygiene status of shared spaces as described above. Furthermore, the specific limitations of the computer device in implementing this image recognition-based automatic assessment and cleaning scheduling method for the hygiene status of shared spaces can be found in the limitations of the image recognition-based automatic assessment and cleaning scheduling method for shared spaces described above, and will not be repeated here.

[0118] Fourthly, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program implements the image recognition-based automatic assessment and cleaning scheduling method for the hygiene of shared spaces as described above. Furthermore, the specific limitations of the computer-readable storage medium in implementing the image recognition-based automatic assessment and cleaning scheduling method for the hygiene of shared spaces can be found in the limitations of the image recognition-based automatic assessment and cleaning scheduling method for shared spaces described above, and will not be repeated here.

[0119] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0120] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for automatic assessment and cleaning scheduling of the hygiene status of shared spaces based on image recognition, characterized in that, The method includes: Acquire real-time spatial images within the shared space; Multi-task recognition and cross-task collaborative calibration are performed on the real-time spatial image to generate sanitary status data. The multi-task recognition includes at least parallel recognition of regional boundaries, garbage objects, and stained areas. Based on the aforementioned health status data, and considering the historical health status sequence and the spatial correlation between adjacent areas, a dynamic health index is calculated through spatiotemporal joint assessment. When the dynamic hygiene index is lower than the cleaning trigger threshold, the target area is marked as a cleaning area and the hygiene status data of the cleaning area is encoded into a cleaning demand feature vector; wherein the target area is the area corresponding to the dynamic hygiene index that is lower than the cleaning trigger threshold; Calculate the similarity between the cleaning demand feature vector and the strategy feature vector of each candidate strategy in the preset cleaning strategy library, and determine the target cleaning strategy based on the matching result of the similarity in the cleaning strategy library; A cleaning instruction is generated based on the target cleaning strategy, and the cleaning instruction is sent to the cleaning execution terminal to instruct the cleaning execution terminal to perform cleaning operations on the area to be cleaned in accordance with the target cleaning strategy. During or after the cleaning operation, the cleaning result data fed back by the cleaning execution terminal is obtained, and the dynamic hygiene index is updated based on the cleaning result data.

2. The method according to claim 1, characterized in that, The step of performing multi-task recognition and cross-task collaborative calibration on the real-time spatial image to generate health status data includes: The real-time spatial image is input into a multi-task recognition model, which includes: a shared feature extraction network, a semantic segmentation branch, an object detection branch, and a texture analysis branch connected to the shared feature extraction network, and a cross-task feature interaction layer connected to the semantic segmentation branch, the object detection branch, and the texture analysis branch. The shared feature map of the real-time spatial image is extracted by the shared feature extraction network, and the shared feature map is processed by the semantic segmentation branch, the object detection branch and the texture analysis branch respectively to generate an initial region segmentation map, an initial object detection box and an initial stain texture map; The initial region segmentation map, the initial target detection box, and the initial stain texture map are input into the cross-task feature interaction layer. The cross-task feature interaction layer performs spatial position calibration on the identified objects in the initial target detection box and the initial stain texture map based on the initial region segmentation map. Based on the semantic correlation between the calibrated identified objects, the boundary of the initial region segmentation map is refined and corrected, and the fused region segmentation map, target detection box, and stain texture map are output. Based on the region segmentation map, the target detection box, and the stain texture map, the number of garbage objects, the type of garbage objects, and the coverage area of ​​the stain area in each region are counted to generate the hygiene status data.

3. The method according to claim 1, characterized in that, The dynamic health index, calculated through spatiotemporal joint assessment based on historical health status sequences and spatial correlations between adjacent areas, includes: Obtain the historical health status sequence of each region within a preset historical time period; Calculate the time decay coefficient of the health status of each region at the current moment based on the historical health status sequence; A pollution diffusion impact matrix is ​​constructed based on a preset regional adjacency relationship. The pollution diffusion impact matrix is ​​used to characterize the mutual influence weights of the sanitary conditions between adjacent regions. The current basic value of the sanitation index is obtained by weighting and summing the number of garbage objects, the pollution weight corresponding to the garbage object category, and the percentage of the area covered by stains in the current sanitation status data. Multiply the current baseline value of the health index by the time decay coefficient to obtain the time-corrected health index. The dynamic health index is obtained by convolving the time-corrected health index with the health status data of adjacent areas in the pollution diffusion impact matrix.

4. The method according to claim 1, characterized in that, Encoding the hygiene status data of the area to be cleaned into a cleaning demand feature vector includes: The hygiene status data of the area to be cleaned is input into the trained autoencoder; The autoencoder's encoding layer performs dimensionality reduction encoding on the hygiene status data, outputting a fixed-dimensional feature vector of the cleaning requirements.

5. The method according to claim 4, characterized in that, The step of calculating the similarity between the cleaning demand feature vector and the strategy feature vectors of each candidate strategy in the preset cleaning strategy library, and determining the target cleaning strategy based on the matching result of the similarity in the cleaning strategy library, includes: Calculate the cosine similarity between the cleaning demand feature vector and the strategy feature vector corresponding to each candidate strategy in the cleaning strategy library; The candidate strategy with the highest cosine similarity that exceeds a preset similarity threshold is determined as the target cleaning strategy; The strategy feature vector is obtained by encoding the cleaning execution information associated with each candidate strategy. The cleaning execution information includes the execution terminal type, tool parameters, and operation mode.

6. The method according to any one of claims 1 to 5, characterized in that, The cleaning execution end includes intelligent cleaning equipment or cleaning personnel equipped with mobile terminals; When the cleaning execution end is the intelligent cleaning device, the cleaning instruction includes automatic execution parameters, and the cleaning result data includes operation status data fed back by sensors mounted on the intelligent cleaning device; When the cleaning execution terminal is the cleaning personnel, the cleaning instruction includes work guidance information pushed to the mobile terminal, and the cleaning result data includes post-cleaning images or confirmation information submitted through the mobile terminal.

7. The method according to claim 6, characterized in that, The step of updating the dynamic hygiene index based on the cleaning result data fed back from the cleaning execution terminal includes: During the cleaning operation, images of the cleaning process fed back by the cleaning execution terminal are acquired in real time. The cleaning process images are processed by image recognition to identify the garbage objects and stained areas after cleaning, and to generate real-time hygiene status data during the cleaning process. Calculate the real-time hygiene index based on the real-time hygiene status data, and calculate the index deviation between the real-time hygiene index and the expected target index; When the index deviation exceeds the preset deviation range, the tool parameters or operation mode in the target cleaning strategy are dynamically adjusted according to the value of the index deviation and the auxiliary information corresponding to the type of the cleaning execution terminal. Wherein, when the cleaning execution end is the intelligent cleaning equipment, the auxiliary information includes the operation status data fed back by the sensor; when the cleaning execution end is the cleaning personnel, the auxiliary information includes the operation remarks information submitted through the mobile terminal.

8. An automatic assessment and cleaning scheduling system for the hygiene status of shared spaces based on image recognition, characterized in that, The system includes: The image acquisition module is used to acquire real-time spatial images within the shared space; The recognition and processing module is used to perform multi-task recognition and cross-task collaborative calibration on the real-time spatial image to generate sanitary status data. The multi-task recognition includes at least parallel recognition of regional boundaries, garbage objects and stained areas. The index assessment module is used to calculate a dynamic health index based on the health status data, historical health status sequences, and spatial relationships between adjacent areas through spatiotemporal joint assessment. The area marking module is used to mark the target area as an area to be cleaned when the dynamic hygiene index is lower than the cleaning trigger threshold. The target area is the area corresponding to the dynamic hygiene index that is lower than the cleaning trigger threshold. The feature encoding module is used to encode the hygiene status data of the area to be cleaned into a cleaning requirement feature vector; The strategy matching module is used to calculate the similarity between the cleaning demand feature vector and the strategy feature vector of each candidate strategy in the preset cleaning strategy library, and to determine the target cleaning strategy based on the matching result of the similarity in the cleaning strategy library. The cleaning scheduling module is used to generate cleaning instructions based on the target cleaning strategy and send the cleaning instructions to the cleaning execution terminal to instruct the cleaning execution terminal to perform cleaning operations on the area to be cleaned in accordance with the target cleaning strategy. The feedback update module is used to obtain cleaning result data fed back by the cleaning execution terminal during or after the cleaning operation, and update the dynamic hygiene index based on the cleaning result data.

9. A computer device comprising a memory and a processor, the memory being communicatively connected to the processor, and the memory storing a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the image recognition-based automatic assessment and cleaning scheduling method for the hygiene status of shared spaces as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the image recognition-based automatic assessment and cleaning scheduling method for the hygiene status of shared spaces as described in any one of claims 1 to 7.

Images