Cleaning quality assessment methods and related equipment

Abstract

The embodiment of the present invention provides a cleaning quality assessment method, which obtains a monitoring image of a target area, wherein the target area is a campus cleaning area; extracts an image sequence of a target person in the monitoring image, and performs motion detection on the image sequence to obtain a motion detection result of the target person; and based on the motion detection result of the target person, when the cleaning of the target area is completed, assesses the cleaning quality of the target area to obtain a cleaning quality assessment result of the target area. By monitoring the campus cleaning area, obtaining a monitoring image of the campus cleaning area for motion detection, obtaining a motion detection result of the target person who is cleaning, and assessing the cleaning quality of the campus cleaning area based on the motion detection result, the cleaning quality can be assessed after the cleaning of the campus cleaning area is completed, the assessment speed is fast, and the motion factor of the target person who is cleaning is taken into account, so that the accuracy of the cleaning quality assessment can be improved.

Description

Technical Field

[0001] The present invention relates to the field of smart campus, and in particular to a cleaning quality assessment method and related equipment. Background Art

[0002] The campus cleaning area is a means of school hygiene management. By dividing the school into different areas and subcontracting the cleaning to students from each class, it not only cultivates students' love of labor, but also ensures the hygiene and cleanliness of the school. In the management process of the campus cleaning area, the students of the school student union evaluate and score the cleaning situation of the cleaning area. The existing cleaning quality assessment is usually conducted by students of the student union, which is not only delayed but also has low accuracy. Summary of the invention

[0003] The embodiment of the present invention provides a cleaning quality assessment method and related equipment, which monitors the campus cleaning area, obtains monitoring images of the campus cleaning area for motion detection, obtains motion detection results of target cleaning personnel, and assesses the cleaning quality of the campus cleaning area based on the motion detection results. The cleaning quality can be assessed after the cleaning of the campus cleaning area is completed. The assessment speed is fast and the motion factors of the target cleaning personnel are taken into account, so the accuracy of the cleaning quality assessment can be improved.

[0004] In a first aspect, an embodiment of the present invention provides a cleaning quality assessment method, the method comprising:

[0005] Acquire a surveillance image of a target area, where the target area is a clean area of ​​the campus;

[0006] Extracting an image sequence of a target person from the surveillance image, and performing motion detection on the image sequence to obtain a motion detection result of the target person;

[0007] According to the motion detection result of the target person, when the cleaning of the target area is completed, the cleaning quality of the target area is evaluated to obtain the cleaning quality evaluation result of the target area.

[0008] Optionally, extracting the image sequence of the target person in the surveillance image includes:

[0009] Performing target detection on the monitoring image to obtain candidate personnel information and cleaning item information;

[0010] Determine target personnel information according to the candidate personnel information and the cleaning item information;

[0011] According to the target person information, an image sequence of the target person is extracted from the monitoring image.

[0012] Optionally, the candidate person information includes a human body detection frame, and the cleaning object information includes an object detection frame, and determining the target person information according to the candidate person information and the cleaning object information includes:

[0013] Calculating the intersection-and-union ratio of the human detection frame and the object detection frame, and selecting the human detection frame and the object detection frame whose intersection-and-union ratio is greater than a preset value to combine, to obtain a candidate combination frame, wherein the candidate combination frame includes the human detection frame and the object detection frame;

[0014] Relation detection is performed on the images in the candidate combination box to obtain a relationship detection result of the images in the candidate combination box, and the behavior detection result includes whether there is a holding relationship between the cleaning article information and the candidate personnel information.

[0015] Optionally, the selecting and combining the human body detection frame and the object detection frame whose intersection-over-union ratio is greater than a preset value to obtain a candidate combination frame includes:

[0016] Calculate a minimum enclosing frame between the human detection frame and the object detection frame whose intersection-and-union ratio is greater than a preset value, wherein the minimum enclosing frame simultaneously covers the human detection frame and the object detection frame whose intersection-and-union ratio is greater than the preset value and has the smallest area;

[0017] The minimum bounding box is determined as the candidate combination box.

[0018] Optionally, the target person information includes a human body detection frame and an object detection frame of the target person, and extracting an image sequence of the target person from the surveillance image according to the target person information includes:

[0019] According to the human body detection frame and the object detection frame of the target person, an image sequence of the target person is cropped out from the monitoring image.

[0020] Optionally, performing motion detection on the image sequence to obtain a motion detection result of the target person includes:

[0021] The image sequence is subjected to motion detection using a preset motion detection model to obtain a motion detection result of the target person.

[0022] Optionally, the evaluating the cleaning quality of the target area according to the action detection result of the target person to obtain the cleaning quality evaluation result of the target area includes:

[0023] Acquire a completed image of the target area after cleaning;

[0024] The completed image and the motion detection result of the target person are input into a preset cleaning quality assessment model for assessment to obtain a cleaning quality assessment result of the target area.

[0025] In a second aspect, an embodiment of the present invention provides a cleaning quality assessment device, the device comprising:

[0026] An acquisition module is used to acquire a monitoring image of a target area, where the target area is a clean area of ​​the campus;

[0027] An extraction module is used to extract an image sequence of a target person in the surveillance image, and perform motion detection on the image sequence to obtain a motion detection result of the target person;

[0028] The evaluation module is used to evaluate the cleaning quality of the target area according to the action detection result of the target person when the target area is cleaned, so as to obtain the cleaning quality evaluation result of the target area.

[0029] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the cleaning quality assessment method provided in the embodiment of the present invention when executing the computer program.

[0030] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the cleaning quality assessment method provided in the embodiment of the invention are implemented.

[0031] In an embodiment of the present invention, a monitoring image of a target area is obtained, and the target area is a campus cleaning area; an image sequence of a target person in the monitoring image is extracted, and motion detection is performed on the image sequence to obtain a motion detection result of the target person; based on the motion detection result of the target person, when the cleaning of the target area is completed, the cleaning quality of the target area is evaluated to obtain a cleaning quality evaluation result of the target area. By monitoring the campus cleaning area, obtaining a monitoring image of the campus cleaning area for motion detection, obtaining a motion detection result of the target person who is cleaning, and evaluating the cleaning quality of the campus cleaning area based on the motion detection result, the cleaning quality can be evaluated after the cleaning of the campus cleaning area is completed, the evaluation speed is fast, and the motion factor of the target person who is cleaning is taken into account, so that the accuracy of the cleaning quality evaluation can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 is a flow chart of a cleaning quality assessment method provided by an embodiment of the present invention;

[0034] Figure 2 is a schematic structural diagram of a cleaning quality assessment device provided by an embodiment of the present invention;

[0035] Figure 3 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] See also Figure 1 , Figure 1 is a flow chart of a cleaning quality assessment method provided by an embodiment of the present invention. Figure 1 As shown, the cleaning quality assessment method includes the following steps:

[0038] 101. Obtain a surveillance image of the target area.

[0039] In the embodiment of the present invention, the target area can be monitored by an imaging device arranged near the target area, so that an image of the target area is captured in real time as a monitoring image.

[0040] Furthermore, the imaging device can rotate during the non-cleaning period, so as to monitor the target area and the surrounding area of ​​the target area. The imaging device only monitors the target area during the cleaning period, so as to obtain the monitoring image of the target area during the cleaning period.

[0041] The above-mentioned cleaning quality assessment method can be embedded in a server or in an image device. When the cleaning quality assessment method is embedded in a server, a large number of image devices can be provided for access, and each image device can access the server through a user identifier to obtain the cleaning quality assessment service provided by the server. When the cleaning quality assessment method is embedded in an image device, each image device can provide an offline cleaning quality assessment service.

[0042] The target area is a clean area on campus, and the target area can be determined by drawing lines in an image device. Specifically, when the school clearly defines the boundaries of each clean area, the target area can be obtained by drawing lines in a corresponding image device according to the boundaries of each clean area.

[0043] 102. Extract an image sequence of a target person from the surveillance image, and perform motion detection on the image sequence to obtain a motion detection result of the target person.

[0044] In the embodiment of the present invention, the target person is a person who cleans the target area. The image sequence of the target person can be understood as one target person corresponding to one image sequence. The position of the target person can be determined by target detection, and the image sequence of the target person can be cut out from the monitoring image according to the position of the target person. In this way, it can be avoided that non-cleaning personnel occupy the computing resources of subsequent action detection.

[0045] The monitoring images mentioned above include a plurality of continuous frames of images to be processed, each frame of the images to be processed can be used for target detection of cleaning objects and human bodies, and the sequence of images to be detected can be used for target tracking of non-motor vehicles and human bodies.

[0046] The cleaning objects may be brooms, trash cans, garbage shovels, and other objects used for outdoor cleaning.

[0047] Optionally, in the step of extracting the image sequence of the target person in the surveillance image, target detection can be performed on the surveillance image to obtain candidate person information and cleaning item information; the target person information is determined based on the candidate person information and the cleaning item information; and the image sequence of the target person is extracted from the surveillance image based on the target person information.

[0048] In the embodiment of the present invention, target detection can be performed on each frame of the image to be processed in the image sequence to be processed in time sequence through a target detection model. The target detection can be object detection and human body detection. The object detection can be clean object detection.

[0049] Furthermore, by performing object detection on the image to be processed, first object information can be obtained, and the above-mentioned object information may include an object image and an object detection frame, wherein the object image may be a small image captured by the object detection frame in the image to be processed.

[0050] By performing human body detection on the image to be processed, human body information can be obtained, and the human body information may include a human body image and a human body detection frame, wherein the human body image may be a small image captured by the human body detection frame in the image to be processed.

[0051] It should be noted that object detection and human detection can be simultaneously detected by a multi-target detection model. The multi-target detection model includes two detection targets, one detection target can be an object, and the other detection target can be a human.

[0052] The image to be processed is input into the multi-target detection model, and the object detection result and the human body detection result are output. The first object information can be obtained through the object detection result, and the human body information can be obtained through the human body detection result. It should be noted that the embodiment of the present invention does not limit the multi-target detection model, and the existing multi-target detection model can realize the detection of objects and humans. For example, the above multi-target detection model can be based on the YOLO series detection model.

[0053] Optionally, the candidate personnel information includes a human body detection frame, and the cleaning item information includes an item detection frame. In the step of determining the target personnel information based on the candidate personnel information and the cleaning item information, the intersection and union ratio of the human body detection frame and the item detection frame can be calculated, and the human body detection frame and the item detection frame whose intersection and union ratio is greater than a preset value are selected to combine to obtain a candidate combination frame, which includes the human body detection frame and the item detection frame; relationship detection is performed on the image in the candidate combination frame to obtain a relationship detection result of the image in the candidate combination frame, and the behavior detection result includes whether there is a holding relationship between the cleaning item information and the candidate personnel information.

[0054] Furthermore, in the embodiment of the present invention, it is possible to detect whether there is a holding relationship between the human body information and the first object information through the human body information and the first object information. The above holding relationship can be understood as the holder driving the object.

[0055] Furthermore, the human body information may include posture information, and it is determined whether the posture information matches the first item information. When the posture information matches the first item information, it can be considered that there is a holding relationship between the human body information and the first item information.

[0056] In a possible embodiment, the human body information includes a human body image, and the object includes an object image. The human body image and the object image can be input into a pre-trained prediction model to predict whether the relationship between the human body and the object is a holding relationship.

[0057] Optionally, the first item information includes an item detection frame, and the human body information includes a human body detection frame. If it is detected that the human body information has a holding relationship with the first item, the first item is associated with the human body information to obtain the second item information with a holding attribute, before the step of calculating the intersection and union ratio of the item detection frame and the human body detection frame, and selecting the item detection frame and the human body detection frame whose intersection and union ratio is greater than a preset value to combine to obtain a candidate combination frame, the candidate combination frame includes the item detection frame and the human body detection frame; and performing behavior detection on the image in the candidate combination frame to obtain a behavior detection result of the image in the candidate combination frame, the behavior detection result including whether there is a holding relationship between the human body information and the first item information.

[0058] In the embodiment of the present invention, the above-mentioned intersection over union (IoU) refers to the ratio of the intersection of the object detection frame and the human body detection frame to the union of the object detection frame and the human body detection frame. When the intersection over union is equal to 1, it means that the object detection frame and the human body detection frame are of the same size and overlap in position. Therefore, it can be seen that the larger the intersection over union of the object detection frame and the human body detection frame, the closer the object detection frame and the human body detection frame are and the more similar their shapes are, and the greater the probability that the person corresponding to the human body detection frame is the holder of the object corresponding to the object detection frame.

[0059] The object detection frame and the human body detection frame whose intersection and ratio are greater than the preset value are combined to obtain a candidate combination frame, which includes the object detection frame and the human body detection frame. Then, the corresponding image can be intercepted from the image to be processed through the candidate combination frame as a candidate image, and the candidate image includes the object and the human body. The candidate image is subjected to behavior detection to detect whether there is a holding behavior line in the candidate image. If there is a holding behavior, it means that there is a holding relationship between the first object information and the human body information. If there is no holding behavior, the candidate combination frame is discarded, and the next candidate combination frame is subjected to behavior detection until each object corresponds to a holding person.

[0060] Optionally, in the step of selecting an object detection frame and a human detection frame whose intersection-and-union ratio is greater than a preset value to combine and obtain a candidate combination frame, the minimum enclosing frame between the object detection frame and the human detection frame whose intersection-and-union ratio is greater than the preset value can be calculated, and the minimum enclosing frame simultaneously covers the object detection frame and the human detection frame whose intersection-and-union ratio is greater than the preset value and has the smallest area; and the minimum enclosing frame is determined as the candidate combination frame. By taking the minimum enclosing frame of the object detection frame and the human detection frame as the candidate combination frame, the image size within the candidate combination frame can be reduced, thereby improving the detection accuracy of the holding relationship.

[0061] In a possible embodiment, the image to be processed may be a depth image, so that the image captured by the candidate combination box is also a depth image, and behavior detection can be performed by including objects, human bodies and depth images, thereby making the accuracy of behavior detection higher.

[0062] In an embodiment of the present invention, a pre-trained prediction model can be used to perform behavior detection on the image in the candidate combination box, thereby obtaining whether the image in the candidate combination box has a holding behavior. The above prediction model can be a prediction model constructed based on a resnet18 network.

[0063] Furthermore, positive and negative samples are collected.

[0064] In the embodiment of the present invention, the collection of positive and negative samples is to construct a data set, wherein the positive and negative samples are divided into positive samples and negative samples, the positive samples are pictures of objects with holders, and the negative samples include pictures of objects without holders, pedestrians and other backgrounds. The sample label corresponding to the positive sample can be "yes", and the sample label corresponding to the negative sample can be "no".

[0065] Image data normalization. In an embodiment of the present invention, the images corresponding to the positive and negative samples in the above data set are normalized, and the above image data normalization refers to scaling the image pixel values ​​to between -1 and 1. Before normalizing the image data, the images corresponding to the positive and negative samples may be resized, for example, resizing the images corresponding to the positive and negative samples to 224×224 pixels.

[0066] Model training. In the embodiment of the present invention, specifically, the positive and negative samples in the data set can be input into the prediction model, and the model parameters are iterated according to the error between the output result of the prediction model and the sample label to update the model parameters of the prediction model.

[0067] Determine whether Loss < threshold value. In the embodiment of the present invention, when the error loss Loss of the prediction model is less than the preset threshold value, it means that the prediction model training is completed, and the process goes to step 205; when the predicted error loss Loss is greater than the preset threshold value, the process goes to step 206.

[0068] The output model is used for prediction. In the embodiment of the present invention, when the error loss Loss of the prediction model is less than a preset threshold, the prediction model is output for prediction.

[0069] Update model parameters. In an embodiment of the present invention, when the error loss Loss of the prediction model is greater than a preset threshold, the model parameter iteration is continued to update the model parameters of the prediction model.

[0070] By predicting the relationship between item information and human body information through the prediction model, the relationship between item information and human body information can be quickly and accurately obtained, thereby determining the target person based on the relationship between item information and human body information.

[0071] Optionally, the target person information includes a human body detection frame and an object detection frame of the target person. In the step of extracting an image sequence of the target person in the surveillance image according to the target person information, the image sequence of the target person can be cropped out in the surveillance image according to the human body detection frame and the object detection frame of the target person.

[0072] The image sequence of the target person can be obtained by cropping the corresponding frame to be processed in the monitoring image according to the position of the human body detection frame and the object detection frame of the target person in the image to be processed of the corresponding frame.

[0073] After obtaining the image sequence of the target person, the image sequence of the target person can be input into a preset action detection model, and the image sequence can be subjected to action detection by the preset action detection model. The above-mentioned action detection model can be an existing action detection model, except that the sample image sequence used in the data set is an image sequence containing a cleaning action. The above-mentioned action detection model can be a temporal action detection model based on Temporal Action Localization, a temporal action detection model based on Temporal Action Detection, etc.

[0074] 103. According to the motion detection result of the target person, when the target area is cleaned, the cleaning quality of the target area is evaluated to obtain the cleaning quality evaluation result of the target area.

[0075] In the embodiment of the present invention, after the target person's motion detection result is obtained, the cleaning quality of the target area can be evaluated when the target area is cleaned, and the cleaning quality evaluation result of the target area can be obtained. In this way, the cleaning quality of the target area can be evaluated as soon as the cleaning of the target area is completed, without hysteresis.

[0076] Optionally, a completed image of the target area after cleaning is completed may be obtained; the completed image and the target person's motion detection result are input into a preset cleaning quality assessment model for evaluation to obtain a cleaning quality assessment result of the target area.

[0077] In an embodiment of the present invention, the above-mentioned cleaning quality assessment model can be a classification model based on a convolutional neural network. The completed image and the motion detection results of the target person are input into the convolutional neural network model, dynamic assessment semantic features are extracted from the motion detection results of the target person, and static assessment semantic features are extracted from the completed image. The dynamic assessment semantic features are fused with the static assessment semantic features to output the quality estimation results.

[0078] The cleaning quality assessment model can be trained using sample action detection results and sample completed images as input. The error is calculated between the true scores corresponding to the sample action detection results and the sample completed images and the output results of the cleaning quality assessment model. The parameters of the cleaning quality assessment model are adjusted through the error back propagation algorithm. When the error is less than a preset threshold, a trained cleaning quality assessment model is obtained.

[0079] In an embodiment of the present invention, a monitoring image of a target area is obtained, and the target area is a campus cleaning area; an image sequence of a target person in the monitoring image is extracted, and motion detection is performed on the image sequence to obtain a motion detection result of the target person; based on the motion detection result of the target person, when the cleaning of the target area is completed, the cleaning quality of the target area is evaluated to obtain a cleaning quality evaluation result of the target area. By monitoring the campus cleaning area, obtaining a monitoring image of the campus cleaning area for motion detection, obtaining a motion detection result of the target person who is cleaning, and evaluating the cleaning quality of the campus cleaning area based on the motion detection result, the cleaning quality can be evaluated after the cleaning of the campus cleaning area is completed, the evaluation speed is fast, and the motion factor of the target person who is cleaning is taken into account, so that the accuracy of the cleaning quality evaluation can be improved.

[0080] It should be noted that the cleaning quality assessment method provided in the embodiment of the present invention can be applied to devices such as smart phones, computers, servers, etc. that can perform cleaning quality assessment.

[0081] Optional, see Figure 2 , Figure 2 is a schematic diagram of the structure of a cleaning quality assessment device provided by an embodiment of the present invention, such as Figure 2 As shown, the device comprises:

[0082] An acquisition module 201 is used to acquire a monitoring image of a target area, where the target area is a clean area of ​​the campus;

[0083] An extraction module 202 is used to extract an image sequence of a target person in the surveillance image, and perform motion detection on the image sequence to obtain a motion detection result of the target person;

[0084] The evaluation module 203 is used to evaluate the cleaning quality of the target area according to the action detection result of the target person when the target area is cleaned, so as to obtain the cleaning quality evaluation result of the target area.

[0085] Optionally, the extraction module 202 includes:

[0086] A first detection submodule is used to perform target detection on the monitoring image to obtain candidate personnel information and cleaning item information;

[0087] A determination submodule, used to determine target personnel information according to the candidate personnel information and the cleaning item information;

[0088] The extraction submodule is used to extract the image sequence of the target person in the monitoring image according to the target person information.

[0089] Optionally, the candidate person information includes a human body detection frame, the cleaning object information includes an object detection frame, and the determination submodule includes:

[0090] a combining unit, configured to calculate an intersection-and-union ratio of the human detection frame and the object detection frame, and select the human detection frame and the object detection frame whose intersection-and-union ratio is greater than a preset value for combination to obtain a candidate combination frame, wherein the candidate combination frame includes the human detection frame and the object detection frame;

[0091] The detection unit is used to perform relationship detection on the images in the candidate combination box to obtain a relationship detection result of the images in the candidate combination box, wherein the behavior detection result includes whether there is a holding relationship between the cleaning product information and the candidate personnel information.

[0092] Optionally, the combination unit includes:

[0093] a calculation subunit, configured to calculate a minimum enclosing frame between the human detection frame and the object detection frame whose intersection-and-union ratio is greater than a preset value, wherein the minimum enclosing frame simultaneously covers the human detection frame and the object detection frame whose intersection-and-union ratio is greater than the preset value and has a minimum area;

[0094] A determination subunit is used to determine the minimum bounding box as the candidate combination box.

[0095] Optionally, the target person information includes a human body detection frame and an object detection frame of the target person, and the extraction submodule includes:

[0096] The cropping unit is used to crop the image sequence of the target person in the monitoring image according to the human body detection frame and the object detection frame of the target person.

[0097] Optionally, the extraction module 202 includes:

[0098] The second detection submodule is used to perform motion detection on the image sequence through a preset motion detection model to obtain a motion detection result of the target person.

[0099] Optionally, the evaluation module 203 includes:

[0100] An acquisition submodule, used for acquiring a completed image of the target area after cleaning;

[0101] The evaluation submodule is used to input the completed image and the motion detection result of the target person into a preset cleaning quality evaluation model for evaluation to obtain the cleaning quality evaluation result of the target area.

[0102] It should be noted that the cleaning quality assessment device provided in the embodiment of the present invention can be applied to devices such as smart phones, computers, servers, etc. that can perform cleaning quality assessment.

[0103] The cleaning quality assessment device provided in the embodiment of the present invention can implement each process implemented by the cleaning quality assessment method in the above method embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0104] See also Figure 3 , Figure 3 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 3 As shown, it includes: a memory 302, a processor 301, and a computer program of a cleaning quality assessment method stored in the memory 302 and executable on the processor 301, wherein:

[0105] The processor 301 is used to call the computer program stored in the memory 302 and execute the following steps:

[0106] Acquire a surveillance image of a target area, where the target area is a clean area of ​​the campus;

[0107] Extracting an image sequence of a target person from the surveillance image, and performing motion detection on the image sequence to obtain a motion detection result of the target person;

[0108] According to the motion detection result of the target person, when the cleaning of the target area is completed, the cleaning quality of the target area is evaluated to obtain the cleaning quality evaluation result of the target area.

[0109] Optionally, the extracting of the image sequence of the target person in the surveillance image performed by the processor 301 includes:

[0110] Performing target detection on the monitoring image to obtain candidate personnel information and cleaning item information;

[0111] Determine target personnel information according to the candidate personnel information and the cleaning item information;

[0112] According to the target person information, an image sequence of the target person is extracted from the monitoring image.

[0113] Optionally, the candidate person information includes a human body detection frame, and the cleaning object information includes an object detection frame, and the processor 301 determines the target person information according to the candidate person information and the cleaning object information, including:

[0114] Calculating the intersection-and-union ratio of the human detection frame and the object detection frame, and selecting the human detection frame and the object detection frame whose intersection-and-union ratio is greater than a preset value to combine, to obtain a candidate combination frame, wherein the candidate combination frame includes the human detection frame and the object detection frame;

[0115] Relation detection is performed on the images in the candidate combination box to obtain a relationship detection result of the images in the candidate combination box, and the behavior detection result includes whether there is a holding relationship between the cleaning article information and the candidate personnel information.

[0116] Optionally, the processor 301 performs the step of selecting and combining the human body detection frame and the object detection frame whose intersection-over-union ratio is greater than a preset value to obtain a candidate combination frame, including:

[0117] Calculate a minimum enclosing frame between the human detection frame and the object detection frame whose intersection-and-union ratio is greater than a preset value, wherein the minimum enclosing frame simultaneously covers the human detection frame and the object detection frame whose intersection-and-union ratio is greater than the preset value and has the smallest area;

[0118] The minimum bounding box is determined as the candidate combination box.

[0119] Optionally, the target person information includes a human body detection frame and an object detection frame of the target person, and the processor 301 extracts an image sequence of the target person from the surveillance image according to the target person information, including:

[0120] According to the human body detection frame and the object detection frame of the target person, an image sequence of the target person is cropped out from the monitoring image.

[0121] Optionally, the processor 301 performs motion detection on the image sequence to obtain a motion detection result of the target person, including:

[0122] The image sequence is subjected to motion detection using a preset motion detection model to obtain a motion detection result of the target person.

[0123] Optionally, the processor 301 performs the step of evaluating the cleaning quality of the target area according to the action detection result of the target person to obtain the cleaning quality evaluation result of the target area, including:

[0124] Acquire a completed image of the target area after cleaning;

[0125] The completed image and the motion detection result of the target person are input into a preset cleaning quality assessment model for assessment to obtain a cleaning quality assessment result of the target area.

[0126] The electronic device provided in the embodiment of the present invention can implement each process implemented by the cleaning quality assessment method in the above method embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0127] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the cleaning quality assessment method or each process of the application-end cleaning quality assessment method provided in the embodiment of the present invention is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0128] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0129] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A cleaning quality assessment method, characterized in that: The following steps are involved: During the cleaning period, a monitoring image of a target area is obtained, where the target area is a cleaning area of ​​the campus, and the boundaries of each cleaning area are drawn in a corresponding image device to obtain the target area; Performing target detection on the monitoring image to obtain candidate person information and cleaning item information; determining target person information based on the candidate person information and the cleaning item information, wherein the target person information includes a human body detection frame and an item detection frame of the target person; cropping an image sequence of the target person from the monitoring image based on the human body detection frame and the item detection frame of the target person, wherein one target person corresponds to one image sequence, and the target person is a person cleaning the target area; and performing action detection on the image sequence to obtain an action detection result of the target person; Acquire a completed image of the target area after cleaning; Inputting the completed image and the target person's motion detection result into a preset cleaning quality assessment model for assessment to obtain the target area cleaning quality assessment result; The cleaning quality assessment model is a classification model based on a convolutional neural network. The completed image and the action detection results of the target person are input into the cleaning quality assessment model, dynamic assessment semantic features are extracted from the action detection results of the target person, static assessment semantic features are extracted from the completed image, the dynamic assessment semantic features are fused with the static assessment semantic features, and the cleaning quality assessment result is output; The cleaning quality assessment model is trained by using sample action detection results and sample completed images as inputs, and the error is calculated between the real scores corresponding to the sample action detection results and the sample completed images and the output results of the cleaning quality assessment model. The parameters of the cleaning quality assessment model are adjusted by an error back propagation algorithm. When the error is less than a preset threshold, a trained cleaning quality assessment model is obtained.

2. The method according to claim 1, characterized in that The candidate person information includes a human body detection frame, the cleaning object information includes an object detection frame, and determining the target person information according to the candidate person information and the cleaning object information includes: Calculating the intersection-and-union ratio of the human detection frame and the object detection frame, and selecting the human detection frame and the object detection frame whose intersection-and-union ratio is greater than a preset value to combine, to obtain a candidate combination frame, wherein the candidate combination frame includes the human detection frame and the object detection frame; Relation detection is performed on the images in the candidate combination box to obtain a relationship detection result of the images in the candidate combination box, wherein the relationship detection result includes whether there is a holding relationship between the cleaning article information and the candidate personnel information.

3. The method according to claim 2, characterized in that The selecting and combining the human body detection frame and the object detection frame whose intersection-over-combination ratio is greater than a preset value to obtain a candidate combination frame includes: Calculate a minimum enclosing frame between the human detection frame and the object detection frame whose intersection-and-union ratio is greater than a preset value, wherein the minimum enclosing frame simultaneously covers the human detection frame and the object detection frame whose intersection-and-union ratio is greater than the preset value and has the smallest area; The minimum bounding box is determined as the candidate combination box.

4. The method according to claim 3, characterized in that The performing action detection on the image sequence to obtain the action detection result of the target person includes: The image sequence is subjected to motion detection using a preset motion detection model to obtain a motion detection result of the target person.

5. A cleaning quality assessment device, characterized in that: The device comprises: An acquisition module is used to acquire monitoring images of a target area during a cleaning period, wherein the target area is a cleaning area of ​​the campus, and the target area is obtained by drawing lines on the corresponding image device through the boundaries of each cleaning area; The extraction module is used to perform target detection on the monitoring image to obtain candidate person information and cleaning item information; determine the target person information based on the candidate person information and the cleaning item information, wherein the target person information includes a human body detection frame and an item detection frame of the target person; cut out an image sequence of the target person from the monitoring image based on the human body detection frame and the item detection frame of the target person, wherein one target person corresponds to one image sequence, and the target person is a person cleaning the target area; and perform action detection on the image sequence to obtain an action detection result of the target person; An evaluation module is used to obtain a completed image of the target area after cleaning; the completed image and the motion detection result of the target person are input into a preset cleaning quality evaluation model for evaluation to obtain a cleaning quality evaluation result of the target area; the cleaning quality evaluation model is a classification model based on a convolutional neural network, the completed image and the motion detection result of the target person are input into the cleaning quality evaluation model, dynamic evaluation semantic features are extracted from the motion detection result of the target person, static evaluation semantic features are extracted from the completed image, the dynamic evaluation semantic features are merged with the static evaluation semantic features, and the quality estimation result is output; the cleaning quality evaluation model is trained by using sample motion detection results and sample completed images as inputs, the error calculation is performed between the true scores corresponding to the sample motion detection results and the sample completed images and the output results of the cleaning quality evaluation model, the parameters of the cleaning quality evaluation model are adjusted by an error back propagation algorithm, and when the error is less than a preset threshold, a trained cleaning quality evaluation model is obtained.

6. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the cleaning quality assessment method according to any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the cleaning quality assessment method according to any one of claims 1 to 4 are implemented.

Images