A method for identifying work clothes wearing behavior, terminal device and storage medium

Abstract

The present invention provides a method for identifying work uniform wearing behavior, a terminal device and a storage medium, the method comprising: obtaining a preset activity area of ​​employees in a store during working hours; obtaining a monitoring video stream; the area monitored by the monitoring video stream includes the preset activity area of ​​the monitored store; and finding out the illegal employees who are not wearing work uniforms in the preset activity area according to the monitoring video stream. The present invention improves the accuracy and reliability of identifying the situation of employees wearing work uniforms.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method for identifying work clothes wearing behavior, a terminal device and a storage medium. Background Art

[0002] With the continuous development of the economy, people have a higher demand for food safety supervision and quality control in the catering industry, and are extremely concerned about the safety of stores. Among them, whether employees wear work clothes in accordance with corporate regulations is an important assessment point for food safety supervision in the catering industry.

[0003] Traditional on-site supervision is time-consuming and laborious. Currently, there are some uniform recognition methods, such as VGGNet, GoogleNet, YOLO and other object recognition algorithms. These algorithms can be trained with specific data to identify whether a specific doorman is wearing uniforms. However, since there are customers in the camera's field of view, customers who do not wear store uniforms will be identified as illegal employees, resulting in a large number of misjudgments.

[0004] To solve the problem of traditional image recognition algorithms failing due to the presence of customers in store work areas, we must first correctly distinguish between employees and customers. The current commonly used method is to enter the facial features of employees, and then compare the data collected in real time by the camera with the pre-collected facial feature data of employees, so as to achieve the goal of distinguishing between employees and customers. However, this aspect faces a feasibility problem in application areas such as street-side stores, because there are temporary employees among the employees, and the personnel structure is unstable, making it difficult to collect facial feature data from them. Summary of the invention

[0005] The purpose of the present invention is to provide a method, terminal device and storage medium for identifying work clothes wearing behavior, so as to solve the problem that the prior art can only be used for non-street-facing stores or only for situations where employees are relatively stable, and the identification results of employees wearing work clothes are easily inaccurate or misjudged.

[0006] The technical solution provided by the present invention is as follows:

[0007] The present invention provides a method for identifying work clothes wearing behavior, comprising the steps of:

[0008] Obtain the preset activity areas of employees in the store during working hours;

[0009] Obtaining a surveillance video stream; the area monitored by the surveillance video stream includes a preset activity area of ​​the monitored store;

[0010] According to the monitoring video stream, find out the illegal employees who are not wearing work clothes in the preset activity area.

[0011] Furthermore, the step of obtaining the preset activity area of ​​the employees in the store during the working period includes the following steps:

[0012] Acquire a historical surveillance video stream before the current moment, and extract an image sequence from the historical surveillance video stream; the image sequence includes a plurality of image frames arranged in chronological order;

[0013] Input the image frame into a pre-trained target recognition model, and output a bounding box and position information corresponding to the work clothes; the work clothes are contained in the bounding box;

[0014] According to the bounding box and position information corresponding to the work clothes, the occurrence frequency of employees wearing the work clothes at different positions is counted to generate an employee distribution heat map;

[0015] The service areas corresponding to the location information with the highest frequency of occurrence are summarized to obtain the preset activity area.

[0016] Furthermore, the step of counting the appearance frequency of employees wearing work clothes at different locations according to the bounding box and position information corresponding to the work clothes to generate an employee distribution heat map includes the following steps:

[0017] Count the occurrence frequency of the center point coordinates of the boundary box corresponding to each work uniform, and determine the weight of the employee at the location of each work uniform; the occurrence frequency is proportional to the weight;

[0018] A color gradient employee heat map is generated based on the center point coordinates of the bounding box corresponding to each work uniform, the length and width information of the bounding box, and the weight; the weight is proportional to the depth of the color.

[0019] Furthermore, the step of finding out the illegal employees who are not wearing work clothes in the preset activity area according to the monitoring video stream includes the following steps:

[0020] Performing image processing on the surveillance video stream to obtain an image to be identified;

[0021] Inputting the image to be identified into a pre-trained target recognition model to identify and output the bounding box and position information corresponding to the target object; the target object includes a human body and the work clothes;

[0022] The illegal employee is found according to the boundary box and position information of the target object.

[0023] Furthermore, the step of performing image processing on the surveillance video stream to obtain the image to be identified includes the following steps:

[0024] Performing frame processing on the surveillance video stream to obtain a first type of image to be identified;

[0025] The step of inputting the image to be identified into a pre-trained target recognition model and identifying and outputting the bounding box and position information corresponding to the target object comprises the following steps:

[0026] Inputting the first type of image to be recognized into the target recognition model, and outputting the bounding boxes and position information corresponding to the human body and the work clothes respectively;

[0027] The step of finding the illegal employee according to the boundary box and location information of the target object comprises the following steps:

[0028] According to the bounding boxes and position information corresponding to the employee and the work clothes respectively, determining whether the human body bounding box corresponding to the employee includes the work clothes bounding box;

[0029] If the number of the same work clothes boundary boxes existing in the human body boundary box is greater than one, the determination of the human body boundary box is abandoned;

[0030] If there is no work clothes boundary box in the human body boundary box, the person corresponding to the human body boundary box is determined to be the target person; the target person is a person who is not wearing work clothes;

[0031] Determine whether the location information corresponding to the target person is within the preset activity area;

[0032] If the location information corresponding to the target person is within the preset activity area, the target person is output as the illegal employee.

[0033] Furthermore, the step of performing image processing on the surveillance video stream to obtain the image to be identified includes the following steps:

[0034] Performing frame processing on the surveillance video stream to obtain a first type of image to be identified;

[0035] The step of inputting the image to be identified into a pre-trained target recognition model and identifying and outputting the bounding box and position information corresponding to the target object comprises the following steps:

[0036] Inputting the first type of image to be recognized into the target recognition model, and outputting the bounding boxes and position information corresponding to the human body and the work clothes respectively;

[0037] The step of finding the illegal employee according to the boundary box and location information of the target object comprises the following steps:

[0038] Find out the employees in the preset activity area according to the position information corresponding to the human body;

[0039] According to the bounding boxes and position information corresponding to the employee and the work clothes respectively, determining whether the human body bounding box corresponding to the employee includes the work clothes bounding box;

[0040] If the number of the same work clothes boundary boxes existing in the human body boundary box is greater than one, the determination of the human body boundary box is abandoned;

[0041] If any work clothes boundary box does not exist in the human body boundary box, it is determined that the person corresponding to the human body boundary box is the illegal employee.

[0042] Furthermore, the step of performing image processing on the surveillance video stream to obtain the image to be identified includes the following steps:

[0043] Performing frame processing on the surveillance video stream to obtain a first type of image to be identified;

[0044] Finding target pixel points corresponding to the preset activity area from the first type of images to be identified, and performing image segmentation according to the target pixel points to obtain a second type of images to be identified;

[0045] The step of finding the illegal employee according to the boundary box and location information of the target object comprises the following steps:

[0046] Inputting the second type of to-be-recognized images into the target recognition model, and outputting the boundary boxes and position information corresponding to the employees and work clothes in the preset activity area respectively;

[0047] The step of finding the illegal employee according to the boundary box and location information of the target object comprises the following steps:

[0048] According to the bounding boxes and position information corresponding to the employee and the work clothes respectively, determining whether the human body bounding box corresponding to the employee includes the work clothes bounding box;

[0049] If the number of the same work clothes boundary boxes in the human body boundary box is greater than one, the judgment on the human body boundary box is abandoned;

[0050] If any work clothes boundary box does not exist in the human body boundary box, it is determined that the person corresponding to the human body boundary box is the illegal employee.

[0051] Furthermore, the step of obtaining the preset activity area of ​​the employees in the store during the working period includes the following steps:

[0052] Acquire a large number of sample images that have been annotated with marking frames; the marking frames include human marking frames and work clothes marking frames;

[0053] The sample images are randomly divided into a training sample set and a test sample set, and a target recognition model is generated according to the training sample set;

[0054] The trained target recognition model is tested and adjusted according to the test sample set until the training is completed when the recognition error rate of each target object is lower than a preset threshold.

[0055] The present invention also provides a terminal device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor is used to execute the computer program stored in the memory to implement the operations performed by the method for identifying work clothes wearing behavior.

[0056] The present invention also provides a storage medium, in which at least one instruction is stored, and the instruction is loaded and executed by a processor to implement the operations performed by the method for identifying work clothes wearing behavior.

[0057] The method for identifying work clothes wearing behavior, terminal equipment and storage medium provided by the present invention can improve the accuracy and reliability of identifying employees wearing work clothes. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The following will explain the preferred implementation mode in a clear and understandable manner with reference to the accompanying drawings, and further explain the above-mentioned characteristics, technical features, advantages and implementation methods of a method for identifying work clothes wearing behavior, a terminal device and a storage medium.

[0059] Figure 1 It is a flow chart of an embodiment of a method for identifying work clothes wearing behavior of the present invention;

[0060] Figure 2 is a flow chart of another embodiment of a method for identifying work clothes wearing behavior of the present invention;

[0061] Figure 3 is a flow chart of another embodiment of a method for identifying work clothes wearing behavior of the present invention;

[0062] Figure 4 It is a flow chart of another embodiment of a method for identifying work clothes wearing behavior of the present invention. DETAILED DESCRIPTION

[0063] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0064] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections.

[0065] In order to simplify the drawings, only the parts related to the present invention are schematically shown in each figure, and they do not represent the actual structure of the product. In addition, in order to simplify the drawings and facilitate understanding, in some figures, only one of the parts with the same structure or function is schematically drawn or marked. In this article, "one" not only means "only one", but also means "more than one".

[0066] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0067] In addition, in the description of the present application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0068] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the specific implementation methods of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings and other implementation methods can be obtained based on these drawings without creative work.

[0069] [First embodiment] Figure 1 As shown, a method for identifying work clothes wearing behavior includes:

[0070] S100 obtains the preset activity areas of employees in the store during the working period;

[0071] Specifically, a store refers to an offline physical store that provides catering services. The store includes multiple employees, and some employees have irregular work locations. For example, the same waiter can provide a variety of services such as ordering, delivering meals, and settling bills. Therefore, it is necessary to obtain the service area where each employee in the store performs work during work hours, and then summarize the service areas of all employees in the store to obtain the preset activity area. It should be noted that physical stores that provide catering services include but are not limited to restaurant stores, beverage stores (such as milk tea shops), and supermarket stores (such as convenience stores that sell food).

[0072] S200 obtains a surveillance video stream; the surveillance video stream includes a preset activity area of ​​the monitored store;

[0073] Specifically, the camera may be a device having basic functions such as video recording / transmission and static image capture. A surveillance camera may be installed in a store, and the lens of the surveillance camera faces the preset activity area. Of course, surveillance cameras may also be deployed on the street, and the lens of the surveillance camera faces the preset activity area. Among them, the field of view of the surveillance camera deployed on the street may include multiple stores so as to monitor multiple stores at the same time, that is, the surveillance video stream in the video captured by the camera may include multiple stores.

[0074] The monitored store is photographed by a camera at a fixed position (a camera inside the store or a camera outside the store on the street), and a surveillance video stream of the store is obtained from the camera. Preferably, cameras with different viewing angles are installed at multiple angles and positions, and surveillance video streams of cameras with different viewing angles are obtained by summarizing.

[0075] S300 finds out the illegal employees who are not wearing work clothes in the preset activity area according to the monitoring video stream.

[0076] Specifically, the work clothes of employees engaged in catering services include work clothes, work hats, masks, etc. The monitoring video stream acquired from the camera is processed to obtain the image frame corresponding to each moment, and then, based on the image frame at the current moment, it is judged whether the employees in the preset activity area are wearing work clothes at the current moment. If any employee in the preset activity area is not wearing work clothes at the current moment, the illegal employee is found, and the corresponding recognition result is output and a reminder can be initiated. Of course, if all employees in the preset activity area are wearing work clothes at the current moment, the illegal employee in the preset activity area at the next moment will continue to be judged and found based on the image frame at the next moment.

[0077] The present invention pre-sets a preset activity area, and then performs image recognition processing based on the monitoring video stream including the preset activity area, monitors in real time whether the employees in the preset activity area are wearing work clothes, and realizes comprehensive real-time monitoring of the illegal wearing behavior of each employee in the store, so as to improve the supervision and efficiency of employees wearing work clothes. The present invention pre-constructs the work area, and only those who are not wearing work clothes in the work area are considered illegal store employees, thereby avoiding the defects of misjudging customers as illegal store employees or the algorithm being effective only in specific scenarios. It can effectively judge whether store employees have violated the rules of not wearing work clothes, and can be applied to various types of stores and employee teams, with a low misjudgment rate.

[0078] [Second embodiment] A method for identifying work clothes wearing behavior, comprising:

[0079] S110 obtains a historical surveillance video stream before the current moment, and extracts an image sequence from the historical surveillance video stream; the image sequence includes a plurality of image frames arranged in chronological order;

[0080] Specifically, this embodiment is an optimized embodiment of the above embodiment, and the parts of this embodiment that are the same as the above embodiment refer to the above embodiment. A surveillance video stream of a certain length before the current moment is obtained as a historical surveillance video stream for employee activity area identification. For example, a surveillance video stream with a resolution of 640*640 and a working period of a whole day before the current moment is generally collected. Several image frames are uniformly extracted from the historical surveillance video stream at certain time intervals. Since each image frame corresponds to timestamp information, several image frames can be arranged in chronological order to obtain an image sequence. Among them, since the information contained in adjacent frames of the historical surveillance video stream may be relatively similar, the present invention extracts image frames at certain time intervals (such as 1 minute intervals) to obtain an image sequence with a higher degree of difference.

[0081] S120: input the image frame into a pre-trained target recognition model, and output a bounding box and position information corresponding to the work clothes; the work clothes are contained in the bounding box;

[0082] Specifically, the bounding box is often a rectangular box. The process of identifying the target object includes classification and positioning. Target classification means that the category of the target object in the image can be identified by inputting the image, and target positioning means that the location of the category object in the image can be marked by inputting the image (rectangular box). The target recognition model obtained through training with a large number of sample images can effectively identify the work clothes in the image frame, so that the work clothes can be quickly marked in the form of bounding boxes, and the location information corresponding to each bounding box can be output. The work clothes are contained in the bounding box.

[0083] The position information can be expressed as the center point coordinates of the bounding box (b x , b y ) or (b x , b y , b z ), the length of the bounding box is b l , the width of the bounding box is b w Of course, the position information can also be expressed as the coordinates of the upper left corner of the bounding box (l x , l y ), the length of the bounding box is b l , the width of the bounding box is b w Of course, the position information can also be expressed as the coordinates of the upper left corner of the bounding box (l x , l y ) and the coordinates of the lower right corner (r x , r y). The present invention can use such marked bounding boxes and their position information to perform some data statistics. Therefore, the trained target recognition model will construct several different bounding boxes to mark the work clothes identified from the image frame, and output the position information of the bounding boxes corresponding to each work clothes.

[0084] S130, based on the bounding box and position information corresponding to the work clothes, counting the appearance frequency of employees wearing the work clothes at different locations to generate an employee distribution heat map;

[0085] Specifically, the employee distribution heat map can visualize the density of the activity track points of employees in the preset activity area during the working period, and use the historical monitoring video stream of the historical time period to perform image recognition to obtain the corresponding bounding box and position information of the work clothes, analyze and count the appearance frequency of the work clothes in the preset activity area in the historical time period (for example, the previous day, the previous week, etc.), and render the color of the store map including the preset activity area by the appearance frequency to distinguish the density or appearance frequency of employees in the preset activity area of ​​the monitored store during the working period. The darker the color, the more frequent the appearance, and the lighter the color, the less frequent the appearance. In this way, the employee distribution heat map can intuitively see the appearance frequency of employees in the preset activity area. Since the marked bounding box and its position information can be used to perform some data statistics, after obtaining the bounding box and position information corresponding to the work clothes in the preset activity area in the historical time period according to the above process, the appearance frequency of employees wearing work clothes in different positions in the preset activity area in the historical time period can be directly counted, and then the corresponding employee distribution heat map is generated according to the statistical appearance frequency information, the bounding box and position information corresponding to the work clothes.

[0086] S140 summarizes the service areas corresponding to the location information with the highest appearance frequency to obtain the preset activity area;

[0087] Specifically, when the image sequence extracted from the surveillance video stream is input into the target recognition model, the work clothes will be identified, thereby obtaining a series of bounding boxes and the position information of the bounding boxes (i.e., the center point coordinates, the length and width information of the bounding boxes). Through these bounding boxes and position information, the employee distribution heat map can be drawn with reference to the above process. Then, the service areas corresponding to the low-frequency position information in the employee distribution heat map can be filtered out to obtain the preset activity area, that is, the frequency of occurrence of the area where the bounding box of each work clothes is located in the employee distribution heat map is arranged in order of size, and the service areas corresponding to the low-frequency position information are discarded. The service areas corresponding to the high-frequency position information are retained to obtain the preset activity area. For example, the service areas corresponding to the first five position information arranged in the order of frequency of occurrence are merged to obtain the preset activity area.

[0088] S200 obtains a surveillance video stream; the surveillance video stream includes a preset activity area of ​​the monitored store;

[0089] S300 finds out the illegal employees who are not wearing work clothes in the preset activity area according to the monitoring video stream.

[0090] The present invention uses heat map technology and image recognition and positioning technology to obtain statistics of preset activity areas where employees appear more frequently in the store during work, and then performs image recognition processing based on the monitoring video stream including the preset activity area, and monitors in real time whether the employees in the preset activity area are wearing work clothes, so as to achieve comprehensive real-time monitoring of the illegal wearing behavior of each employee in the store, so as to improve the supervision and efficiency of employees wearing work clothes. The present invention limits the preset activity area and only judges whether employees in the preset activity area are wearing work clothes, so as to avoid identifying customers who do not wear store work clothes as illegal store employees, thereby avoiding a large number of misjudgments, and improving the accuracy and reliability of employee wearing work clothes recognition detection.

[0091] [Third embodiment] A method for identifying work clothes wearing behavior, comprising:

[0092] S110 obtains a historical surveillance video stream before the current moment, and extracts an image sequence from the historical surveillance video stream; the image sequence includes a plurality of image frames arranged in chronological order;

[0093] S120: input the image frame into a pre-trained target recognition model, and output a bounding box and position information corresponding to the work clothes; the work clothes are contained in the bounding box;

[0094] S131: Count the occurrence frequency of the center point coordinates of the boundary box corresponding to each work uniform, and determine the weight of the employee at the location of each work uniform; the occurrence frequency is proportional to the weight;

[0095] S132 generates a color gradient employee heat map according to the center point coordinates of the bounding box corresponding to each work uniform, the length and width information of the bounding box, and the weight; the weight is proportional to the depth of the color;

[0096] Specifically, this embodiment is an optimized embodiment of the above embodiment. For the parts of this embodiment that are the same as the above embodiment, refer to the above embodiment. First, referring to the above embodiment, if the position information is represented by the center point coordinates of the bounding box (b x , b y ) or (b x , b y , b z ), the length of the bounding box is b l , the width of the bounding box is b w, then first identify all the work clothes in the image frame (including work clothes, work hats, masks, etc.), and obtain the center point coordinates corresponding to the boundary boxes of different work clothes (b x , b y , b z ), the coordinates of the center points of all the uniform boundary boxes are put into a list type variable set data, that is, data = {(b x1 , b y1 , b z1 ), (b x2 , b y2 , b z2 ), ..., (b xn , b yn , b zn )}, where n is the serial number corresponding to a certain uniform. Then, the frequency of occurrence of the center point of the uniform boundary box is counted according to the variable set, and the corresponding weight is calculated according to the frequency of occurrence of the position corresponding to the coordinates of the center point of the uniform boundary box. Since the frequency of occurrence is proportional to the weight, and the weight is proportional to the depth of the color, the darker the color, the more frequent the occurrence. In this way, according to the frequency of occurrence of the position corresponding to the coordinates of the center point of the uniform boundary box, the corresponding center point and its boundary box can be colored with the corresponding color to generate an employee heat map.

[0097] Of course, referring to the above embodiment, if the position information is represented by the coordinates of the upper left corner of the bounding box (l x , l y , l z ), the length of the bounding box is b l , the width of the bounding box is b w , the coordinates of the center point of the bounding box can be converted and calculated (b x , b y ) or (b x , b y , b z ). Similarly, if the position information is represented by the coordinates of the upper left corner of the bounding box (l x , l y ) and the coordinates of the lower right corner (r x , r y ), the coordinates of the center point of the bounding box can also be converted and calculated (b x , b y ), and the length of the bounding box is b l =|l x -r x |, the width of the bounding box is b w =|l y -r y |.

[0098] S140 summarizes the service areas corresponding to the location information with the highest appearance frequency to obtain the preset activity area;

[0099] S200 obtains a surveillance video stream; the surveillance video stream includes a preset activity area of ​​the monitored store;

[0100] S300 finds out the illegal employees who are not wearing work clothes in the preset activity area according to the monitoring video stream.

[0101] The present invention uses heat map technology and image recognition and positioning technology to obtain statistics of preset activity areas where employees appear more frequently in the store during work, and then performs image recognition processing based on the monitoring video stream including the preset activity area, and monitors in real time whether the employees in the preset activity area are wearing work clothes, so as to achieve comprehensive real-time monitoring of the illegal wearing behavior of each employee in the store, so as to improve the supervision and efficiency of employees wearing work clothes. The present invention limits the preset activity area and only judges whether employees in the preset activity area are wearing work clothes, so as to avoid identifying customers who do not wear store work clothes as illegal clerks, thereby causing a large number of misjudgments, and improving the accuracy and reliability of employee wearing work clothes recognition detection.

[0102] [Fourth embodiment] A method for identifying work clothes wearing behavior, comprising:

[0103] S100 obtains the preset activity areas of employees in the store during the working period;

[0104] S200 obtains a surveillance video stream; the surveillance video stream includes a preset activity area of ​​the monitored store;

[0105] S310 performs image processing on the surveillance video stream to obtain an image to be identified;

[0106] S320: input the image to be recognized into a pre-trained target recognition model, recognize and output the bounding box and position information corresponding to the target object; the target object includes a human body and the work clothes;

[0107] S330: Find the offending employee according to the boundary box and position information of the target object.

[0108] Specifically, this embodiment is an optimized embodiment of the above embodiment, and the parts of this embodiment that are the same as the above embodiment refer to the above embodiment. After the monitoring video stream is processed to obtain the image to be identified, the image to be identified is input into the target recognition model. As in the above embodiment, the target recognition model identifies and outputs the bounding boxes and position information corresponding to all human bodies and all work clothes (including work clothes, work hats, masks, etc.) in the preset activity area. Since each target object (including human body and work clothes) corresponds to its own position information in the world coordinate system, according to the spatial position of the preset activity area in the world coordinate system, the position information of each target object can be matched with the preset activity area to obtain a judgment result of whether there is a spatial overlap, and then the work clothes wearing recognition results of all employees in the preset activity area are obtained according to the judgment result to find out the illegal employees in the preset activity area.

[0109] The present invention monitors in real time whether employees in a preset activity area are wearing work uniforms, and realizes comprehensive real-time monitoring of the illegal wearing behavior of each employee in the store, so as to improve the supervision intensity and efficiency of employees wearing work uniforms. The present invention limits the preset activity area and only judges whether employees in the preset activity area are wearing work uniforms, avoiding the identification of customers who do not wear store uniforms as illegal store employees, thereby avoiding a large number of misjudgments, and improving the accuracy and reliability of employee wearing uniforms identification detection.

[0110] [Fifth embodiment] Figure 2 As shown, a method for identifying work clothes wearing behavior includes:

[0111] S100 obtains the preset activity areas of employees in the store during the working period;

[0112] S200 obtains a surveillance video stream; the surveillance video stream includes a preset activity area of ​​the monitored store;

[0113] S311 performs frame processing on the surveillance video stream to obtain a first type of image to be identified;

[0114] Specifically, this embodiment is an optimized embodiment of the above embodiment, and the parts of this embodiment that are the same as the above embodiment refer to the above embodiment. Set an interception time interval, and perform frame processing on the monitoring video stream according to the interception time interval, that is, intercept the first frame image, the second frame image, and the mth frame image of the monitoring video stream as the first type of image to be identified according to the interception time interval, and save the intercepted first type of image to be identified to a local or cloud space or server.

[0115] S321 inputs the first type of image to be recognized into the target recognition model, and outputs the bounding boxes and position information corresponding to the human body and the work clothes respectively;

[0116] Specifically, after performing image processing on the surveillance video stream to obtain the first type of images to be identified, the first type of images to be identified are input into the target recognition model. As in the above embodiment, the target recognition model recognizes and outputs the bounding boxes and position information corresponding to all human bodies and all work clothes (including work clothes, work hats, masks, etc.) in the first type of images to be identified. Since each target object (including human bodies and work clothes) corresponds to its own position information in the world coordinate system, according to the spatial position of the preset activity area in the world coordinate system, the position information of each target object can be matched with the preset activity area to obtain a judgment result of whether there is spatial overlap, and then the work clothes wearing recognition result of all employees in the preset activity area is obtained according to the judgment result.

[0117] S331, judging whether the human body boundary box corresponding to the human body includes the work clothes boundary box according to the boundary boxes and position information corresponding to the human body and the work clothes respectively;

[0118] S332: if the number of the same work clothes boundary boxes existing in the human body boundary box is greater than one, abandoning the determination of the human body boundary box;

[0119] Specifically, since the bounding box and position information of each target object are known, it is possible to judge whether the person corresponding to each human body bounding box is wearing the work clothes correctly according to the bounding box and position information of the target object. If the human body bounding box includes various types of work clothes bounding boxes, and the number of the same type of work clothes bounding boxes in the human body bounding box is one, it means that the person corresponding to the human body bounding box is neatly wearing the work clothes (any one or more of work clothes, work hats and masks) required to be worn, then switch to the next human body bounding box to continue searching for the target person. Of course, if the number of the same type of work clothes bounding boxes in the human body bounding box is greater than one, it means that the human body bounding box has an overlapping area with at least two of the same type of work clothes bounding boxes, and the judgment of the human body bounding box is abandoned, and the next human body bounding box is switched to continue searching for the target person.

[0120] Preferably, for situations where people are next to each other or only part of the employee's body appears in the camera area, the recognition requirement based on the work clothes wearing behavior is high precision, while the recall rate is not required. In order to avoid false alarms caused by occlusion, in actual applications, only the human bounding boxes that have no intersection with other human bounding boxes and are not on the image boundary, and the number of work clothes bounding boxes of the same type is one, are used to determine whether the work clothes are worn.

[0121] Preferably, in order to distinguish the human body boundary box and the work clothes boundary box (including the work clothes boundary box, the work hat boundary box, the mask boundary box, etc.), and to avoid errors, the boundary boxes belonging to the same target object are assigned the same color. For example, the work clothes boundary box is red, the human body boundary box is green, the work hat boundary box is yellow, and the mask boundary box is blue. The present invention can be used only to determine whether a person is wearing any type of work clothes, or can also be used to determine whether a person is wearing all types of work clothes at the same time.

[0122] For example, the human body bounding box of an employee A in the image is C1, the work clothes bounding box corresponding to the work clothes L1 worn by employee A is CL1, and the work clothes bounding box corresponding to the work clothes L2 worn by employee B is CL2. If employee A is next to employee B, resulting in the human body bounding box C1 in the first type of image to be identified obtained by camera imaging, and the work clothes bounding box CL1 and the work clothes bounding box CL2 exist at the same time, then, in order to avoid false alarms, the judgment of whether employee A is wearing the work clothes correctly is abandoned, and the search for the target person is continued by directly switching to the next human body bounding box.

[0123] S333: if there is no work clothes boundary box in the human body boundary box, determining that the person corresponding to the human body boundary box is a target person; the target person is a person who is not wearing work clothes;

[0124] Specifically, if there is no work clothes boundary box in the human body boundary box, it means that all types of work clothes boundary boxes are outside the human body boundary box, that is, there is no overlapping area between the human body boundary box and all types of work clothes boundary boxes, then the person corresponding to this human body boundary box is one of the target persons in the current first category of images to be identified, and switch to the next human body boundary box to continue searching for the target person.

[0125] For example, the human body bounding box of a certain person C in the image is C1, the work clothes bounding box corresponding to the work clothes L1 worn by person C is CL1, the work hat bounding box corresponding to the work hat M1 worn by person C is CM1, and the mask bounding box corresponding to the mask N1 worn by person C is CN1. If the store employees' work clothes wearing detection requirements are only to detect the wearing of employees' work clothes and work hats, then if the human body bounding box C1 includes the work clothes bounding box CL1 and the work hat bounding box CM1, it means that person C is not the target person who is not wearing work clothes and work hats. Of course, if the human body bounding box C1 only includes the work clothes bounding box CL1 (or the work hat bounding box CM1), it means that person C is the target person who is not wearing work clothes and work hats.

[0126] S334 determines whether the location information corresponding to the target person is within the preset activity area;

[0127] S335: If the location information corresponding to the target person is within the preset activity area, output that the target person is the illegal employee.

[0128] Specifically, after finding all the target persons in the first category of images to be identified, i.e., persons who are not wearing work clothes, according to the above process, it is further determined whether the position information corresponding to the target person is located in the preset activity area according to the position information corresponding to the target person. If the position information corresponding to the target person is not located in the preset activity area, it means that the target person in the preset activity area is not wearing work clothes, but is not an employee in the preset activity area of ​​the store. However, if the position information corresponding to the target person is located in the preset activity area, it means that the target person is an employee in the preset activity area of ​​the store, and since the target person is a person who is not wearing work clothes, it means that the target person is an employee who is not wearing work clothes in the preset activity area.

[0129] The present invention identifies the position information of various types of target objects such as work clothes, work hats, human bodies, masks, etc. in the image by applying a pre-trained target recognition model. Whether the person is wearing work clothes is marked according to whether the position information of the human body contains the work clothes to be detected, and the position information of the target person who is not wearing work clothes is intersected with the preset activity area, so as to identify whether the target person is an illegal employee (i.e., an employee who is not wearing work clothes in the preset activity area). The present invention directly identifies the first type of image to be identified to find the target person who is not wearing work clothes, and then determines whether the target person is within the range of the preset activity area. If the target person is within the range of the preset activity area, it means that the target person is an illegal employee. The present invention can realize comprehensive real-time monitoring of the illegal wearing behavior of each employee in the store, so as to improve the supervision and efficiency of employees wearing work clothes. The present invention limits the preset activity area, and only determines whether the employees in the preset activity area are wearing work clothes, so as to avoid identifying customers who do not wear store work clothes as illegal clerks, thereby causing a large number of misjudgments, and improving the accuracy and reliability of employee wearing work clothes recognition detection.

[0130] [Sixth embodiment] Figure 3 As shown, a method for identifying work clothes wearing behavior includes:

[0131] S100 obtains the preset activity areas of employees in the store during the working period;

[0132] S200 obtains a surveillance video stream; the surveillance video stream includes a preset activity area of ​​the monitored store;

[0133] S311 performs frame processing on the surveillance video stream to obtain a first type of image to be identified;

[0134] S321 inputs the first type of image to be recognized into the target recognition model, and outputs the bounding boxes and position information corresponding to the human body and the work clothes respectively;

[0135] S341 searches for employees in the preset activity area according to the position information corresponding to the human body;

[0136] S342: judging whether the human body bounding box corresponding to the employee includes the work clothes bounding box according to the bounding boxes and position information corresponding to the employee and the work clothes respectively;

[0137] S343: if the number of the same work clothes boundary boxes existing in the human body boundary box is greater than one, abandoning the determination of the human body boundary box;

[0138] S344: If any work clothes boundary box does not exist in the human body boundary box, determine that the person corresponding to the human body boundary box is the illegal employee.

[0139] Specifically, this embodiment is an optimized embodiment of the above embodiment, and the parts of this embodiment that are the same as those of the above embodiment refer to the above embodiment. Figure 2 The difference between the corresponding embodiments shown is that in this embodiment, the employee is identified in the preset activity area, and then it is determined whether the employee is not wearing work clothes. If so, the employee is determined to be an illegal employee. That is, the corresponding position information of the human body is first used to find the employees in the preset activity area, that is, the position information corresponding to the human body is processed with the position coordinates of the preset activity area, that is, the human body boundary box in the preset activity area is found, and the person corresponding to the human body boundary box found is the employee in the preset activity area. Then, it is determined whether the human body boundary box corresponding to the employee includes the work clothes boundary box. If the human body boundary box includes various types of work clothes boundary boxes, and the number of the same type of work clothes boundary boxes in the human body boundary box is one, it means that the employee corresponding to the human body boundary box is neatly wearing the work clothes (any one or more of work clothes, work hats and masks) required to be worn, then switch to the next human body boundary box to continue the judgment. Of course, if the number of the same type of work clothes boundary boxes in the human body boundary box is greater than one, it means that the human body boundary box has an overlapping area with at least two of the same type of work clothes boundary boxes, and the judgment of the human body boundary box is abandoned, and the judgment is switched to the next human body boundary box to continue. In addition, if any work uniform boundary box does not exist in the human body boundary box, it means that all types of work uniform boundary boxes are outside the human body boundary box, that is, there is no overlapping area between the human body boundary box corresponding to the employee and all types of work uniform boundary boxes, then the employee corresponding to this human body boundary box is the violating employee.

[0140] The present invention identifies the position information of various types of target objects such as work clothes, work hats, human bodies, masks, etc. in the image by applying a pre-trained target recognition model. According to the intersection processing of the position information of the human body and the preset activity area, the employees in the preset activity area are identified. Then, according to the intersection processing of the position information of the employee and the position information of the work clothes, it is determined whether the employee is wearing the work clothes, and the employees who are not wearing the work clothes are identified as illegal employees (i.e., employees who are not wearing work clothes in the preset activity area). The present invention directly identifies the first type of images to be identified that are collected to find out the employees in the preset activity area, and then determines whether the employees in the preset activity area are wearing work clothes. If the employees in the preset activity area are not wearing work clothes, it means that the employees are illegal employees. Through the present invention, it is possible to realize comprehensive real-time monitoring of the illegal wearing behavior of each employee in the store, so as to improve the supervision and efficiency of employees wearing work clothes. The present invention limits the preset activity area, and only determines whether the employees in the preset activity area are wearing work clothes, so as to avoid identifying customers who do not wear store work clothes as illegal store employees, thereby causing a large number of misjudgments, and improving the accuracy and reliability of employee wearing work clothes recognition detection.

[0141] [Seventh embodiment] Figure 4 As shown, a method for identifying work clothes wearing behavior includes:

[0142] S100 obtains the preset activity areas of employees in the store during the working period;

[0143] S200 obtains a surveillance video stream; the surveillance video stream includes a preset activity area of ​​the monitored store;

[0144] S311 performs frame processing on the surveillance video stream to obtain a first type of image to be identified;

[0145] S312: finding target pixel points corresponding to the preset activity area from the first type of image to be identified, and performing image segmentation according to the target pixel points to obtain a second type of image to be identified;

[0146] Specifically, this embodiment is an optimized embodiment of the above embodiment, and the parts of this embodiment that are the same as the above embodiment refer to the above embodiment. After the first type of image to be identified is intercepted and obtained through the above embodiment, image recognition is performed on the first type of image to be identified to identify the preset active area in the first type of image to be identified, and the image segmentation technology can be used to extract and segment the preset active area in the first type of image to be identified to obtain the second type of image to be identified. Among them, the image segmentation processing technology is a prior art and will not be described in detail here. Any technology for segmenting and generating the second type of image to be identified is within the protection scope of the present invention.

[0147] Of course, the employee distribution heat map can also be intersected with the first type of images to be identified, that is, according to the pixel coordinates of each pixel point in the first type of images to be identified, the original image of the first type of images to be identified is divided into a grid of length*width, and the target pixel grid falling in the employee distribution heat map area is traversed to find the target pixel grid, and the area composed of the target pixel grid is extracted from the original image of the first type of images to be identified, and divided out to form a second type of images to be identified that only includes the preset activity area, that is, the second type of images to be identified are images after the image features of the first type of images to be identified except the preset activity area are eliminated.

[0148] S351: input the second type of image to be recognized into the target recognition model, and output the boundary boxes and position information corresponding to the employees and work clothes in the preset activity area respectively;

[0149] Specifically, as in the above embodiment, since the second type of images to be identified only include human bodies and work clothes in the preset activity area, and the human bodies in the preset activity area are actually employees, the target recognition model can identify and output the bounding boxes and position information corresponding to all employees in the second type of images to be identified, as well as the bounding boxes and position information corresponding to all work clothes (including work clothes, work hats, masks, etc.). Since the second type of images to be identified excludes image features other than the preset activity area, that is, the second type of images to be identified only include image features of the preset activity area, therefore, since each target object (including human bodies and work clothes) corresponds to its own position information in the world coordinate system, according to the spatial position of the preset activity area in the world coordinate system, the position information of each target object can be matched with the preset activity area to obtain a judgment result of whether there is spatial overlap, and then the work clothes wearing recognition result of all employees in the preset activity area is obtained according to the judgment result.

[0150] S352: judging whether the human body bounding box corresponding to the employee includes the work clothes bounding box according to the bounding boxes and position information corresponding to the employee and the work clothes respectively;

[0151] S353: if the number of the same work clothes boundary boxes existing in the human body boundary box is greater than one, abandoning the determination of the human body boundary box;

[0152] Specifically, since the bounding box and position information of each target object are known, it is possible to judge whether the employees corresponding to each human body bounding box are wearing work clothes correctly based on the bounding box and position information of the target object. Because the calculation of the bounding box and position information of the target object comes from the second image to be identified, if the human body bounding box includes various types of work clothes bounding boxes, and the number of the same type of work clothes bounding boxes in the human body bounding box is one, it means that the employee corresponding to the human body bounding box is neatly wearing the work clothes (any one or more of work clothes, work hats and masks) required to be worn, then switch to the next human body bounding box to continue the judgment. Of course, if the number of the same type of work clothes bounding boxes in the human body bounding box is greater than one, it means that there is an overlapping area between the human body bounding box and at least two of the same type of work clothes bounding boxes, and the judgment of the human body bounding box is abandoned, and the judgment is switched to the next human body bounding box to continue.

[0153] S354: If any work clothes boundary box does not exist in the human body boundary box, determine that the person corresponding to the human body boundary box is the illegal employee.

[0154] Specifically, since the calculation of the bounding box and position information of the target object is derived from the second image to be identified, if there is no work uniform bounding box within the human bounding box, it means that all types of work uniform bounding boxes are outside the human bounding box, that is, there is no overlapping area between the human bounding box and all types of work uniform bounding boxes, then the employee corresponding to this human bounding box is the illegal employee in the current second type of image to be identified. Figure 2 and Figure 3 The difference between the corresponding embodiment shown is that this embodiment identifies and determines whether there are people who are not wearing work clothes in the preset activity area, and if so, the person is determined to be an illegal employee. That is, the first image to be identified is cut to obtain a second image to be identified that only includes the preset activity area, and then the person who is an employee is found based on the second image to be identified, and then it is further determined whether the person who is an employee is wearing work clothes. This embodiment is relative to Figure 4 The advantages of the corresponding embodiment shown are that it can reduce the workload of image recognition and improve the recognition speed and efficiency of employees who violate regulations.

[0155] The present invention identifies the position information of various types of target objects such as work clothes, work hats, human bodies, masks, etc. in the image by applying a pre-trained target recognition model, and can directly find out the employees in the preset activity area based on the corresponding position information of the human body output by the second image to be recognized. The position information of the employee recognized and output by the second image to be recognized and the position information of the work clothes are processed by intersection to determine whether the employee is wearing the work clothes, and the employees who are not wearing the work clothes are identified as illegal employees (i.e., employees who are not wearing work clothes in the preset activity area). The present invention directly identifies the collected second type of images to be recognized to find out the employees in the preset activity area, and then determines whether the employees in the preset activity area are wearing work clothes. If the employees in the preset activity area are not wearing work clothes, it means that the employees are illegal employees. The present invention can realize comprehensive real-time monitoring of the illegal wearing behavior of each employee in the store, so as to improve the supervision and efficiency of employees wearing work clothes. The present invention limits the preset activity area, and only determines whether the employees in the preset activity area are wearing work clothes, so as to avoid identifying customers who do not wear store work clothes as illegal store employees, thereby causing a large number of misjudgments, and improving the accuracy and reliability of employee wearing work clothes recognition detection.

[0156] Based on any of the second to seventh embodiments described above, the present invention further includes a target recognition model training step before S100, that is, before S100 obtains the preset activity area of ​​the employees in the store during the working period, the steps include:

[0157] S010 obtains a large number of sample images that have been annotated with marking frames; the marking frames include human body marking frames and work clothes marking frames;

[0158] Specifically, a sample image marked manually is obtained, and the marking frame in the sample image encompasses the boundary of the target object, that is, the human body marking frame encompasses the boundary contour of the human body, and the work clothes (including but not limited to work clothes, work hats, and masks) marking frame encompasses the boundary contour of the work clothes. Since the marking is done manually, the marked marking frame can often achieve a very high match with the boundary contour of the target object. Among them, the marking frame is often a rectangular frame. Since various work clothes need to be identified in the field of catering services, it is necessary to obtain sample images of various work clothes.

[0159] After obtaining a small amount of manually labeled sample images, a sample image enhancement operation is performed on the sample images. The sample image enhancement operation can use image enhancement methods in the prior art, such as rotating, flipping, scaling, translating, and other changes to the sample images to increase the number of expanded sample images, wherein the enhanced sample images are also labeled images.

[0160] S020 randomly divides the sample images into a training sample set and a test sample set, and generates a target recognition model according to the training sample set;

[0161] Specifically, sample images are randomly extracted, and a large number of sample images are divided into training sample sets and test sample sets. Generally, the ratio of the number of sample images in the training sample set to the number of sample images in the test sample set is 7:3. Of course, other ratios can also be set. The present invention can use network structures such as VGGNet and VGGNet to establish a candidate model, and then input the sample images in the training sample set obtained by the above steps into the candidate model constructed above for training to obtain a trained target recognition model.

[0162] S030 Testing and adjusting the trained target recognition model according to the test sample set until the training is completed when the recognition error rate of each target object is lower than a preset threshold;

[0163] Specifically, after obtaining a trained target recognition model through training with the training sample set, the sample images in the test sample set are input into the trained target recognition model to count the recognition error rate of each target object. The misjudgment rate is obtained by comparing whether the category and position of the identified target object are consistent with the category and position of the marking box. Only when the category and position are consistent with the category and position of the marking box, it is considered to be a correct recognition, otherwise it is a misjudgment. After counting the recognition error rate of each target object, the parameters of the previously trained target recognition model are adjusted until the recognition error rate of the target recognition model for each target object is lower than the preset threshold, and the training is terminated to obtain the final target recognition model.

[0164] S100 obtains the preset activity areas of employees in the store during the working period;

[0165] S200 obtains a surveillance video stream; the surveillance video stream includes a preset activity area of ​​the monitored store;

[0166] S300 finds out the illegal employees who are not wearing work clothes in the preset activity area according to the monitoring video stream.

[0167] The present invention enhances the number of sample images, which can reduce the workload of manually collecting original images and marking sample images, and at the same time, makes targeted parameter adjustments according to the recognition error rate to improve the recognition rate and positioning accuracy of the trained target recognition model. The present invention pre-sets a preset activity area, and then classifies, identifies and locates the monitoring video stream including the preset activity area according to the trained target recognition model, and determines whether the employees in the preset activity area are wearing work clothes according to the classification and positioning results, so as to realize comprehensive real-time monitoring of the illegal wearing behavior of each employee in the store, so as to improve the supervision and efficiency of employees wearing work clothes.

[0168] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned program modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program units or modules to complete all or part of the functions described above. The program modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a processing unit, and the above-mentioned integrated unit can be implemented in the form of hardware or in the form of software program units. In addition, the specific names of the program modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application.

[0169] One embodiment of the present invention is a terminal device, including a processor and a memory, wherein the memory is used to store a computer program; the processor is used to execute the computer program stored in the memory to implement the method for identifying work clothes wearing behavior in the above-mentioned corresponding method embodiment.

[0170] The terminal device may be a desktop computer, a notebook, a PDA, a tablet computer, a mobile phone, a human-computer interaction screen, and other devices. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the above are merely examples of terminal devices and do not constitute a limitation on the terminal device. The terminal device may include more or fewer components than the above, or a combination of certain components, or different components. For example, the terminal device may also include an input / output interface, a display device, a network access device, a communication bus, a communication interface, and the like. The communication interface and the communication bus may also include an input / output interface, wherein the processor, the memory, the input / output interface, and the communication interface communicate with each other via the communication bus. The memory stores a computer program, and the processor is used to execute the computer program stored in the memory to implement the method for identifying the work clothes wearing behavior in the above corresponding method embodiment.

[0171] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0172] The memory may be an internal storage unit of the terminal device, such as a hard disk or memory of the terminal device. The memory may also be an external storage device of the terminal device, such as a plug-in hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), etc. equipped on the terminal device. Furthermore, the memory may include both an internal storage unit of the terminal device and an external storage device. The memory is used to store the computer program and other programs and data required by the terminal device. The memory may also be used to temporarily store data that has been output or is to be output.

[0173] The communication bus is a circuit that connects the elements described and implements transmission between these elements. For example, the processor receives commands from other elements through the communication bus, decrypts the received commands, and performs calculations or data processing according to the decrypted commands. The memory may include program modules, such as a kernel, middleware, an application programming interface (API), and an application. The program module may be composed of software, firmware, or hardware, or at least two of them. The input / output interface forwards commands or data entered by the user through the input / output interface (such as a sensor, keyboard, or touch screen). The communication interface connects the terminal device to other network devices, user devices, and networks. For example, the communication interface may be connected to the network via wired or wireless connection to connect to other external network devices or user devices. Wireless communication may include at least one of the following: wireless fidelity (WiFi), Bluetooth (BT), near field communication technology (NFC), global satellite positioning system (GPS), and cellular communication, etc. Wired communication may include at least one of the following: universal serial bus (USB), high-definition multimedia interface (HDMI), asynchronous transmission standard interface (RS-232), etc. The network may be a telecommunication network and a communication network. The communication network may be a computer network, the Internet, the Internet of Things, or a telephone network. The terminal device may be connected to the network via a communication interface, and the protocol used by the terminal device and other network devices to communicate may be supported by at least one of an application, an application programming interface (API), a middleware, a kernel, and a communication interface.

[0174] One embodiment of the present invention is a storage medium, in which at least one instruction is stored, and the instruction is loaded and executed by a processor to implement the operations performed by the corresponding embodiment of the above-mentioned method for identifying work clothes wearing behavior. For example, the storage medium can be a read-only memory (ROM), a random access memory (RAM), a read-only compact disk (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device.

[0175] They can be implemented with program codes executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0176] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0177] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed with hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0178] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0179] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0180] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0181] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by sending instructions to related hardware through a computer program. The computer program can be stored in a storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program can be in source code form, object code form, executable file or some intermediate form. The storage medium may include: any entity or device capable of carrying the computer program, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example: in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0182] It should be noted that the above embodiments can be freely combined as needed. The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention.

Claims

1. A method for identifying work clothes wearing behavior, characterized in that: Includes steps: Obtain the preset activity areas of employees in the store during working hours; Obtaining a surveillance video stream; the area monitored by the surveillance video stream includes a preset activity area of ​​the monitored store; Find out the illegal employees who are not wearing work clothes in the preset activity area according to the monitoring video stream; The step of obtaining the preset activity area of ​​employees in the store during working hours includes the following steps: Obtain a historical surveillance video stream before the current moment, and extract an image sequence from the historical surveillance video stream; the image sequence includes a plurality of image frames arranged in chronological order; Input the image frame into a pre-trained target recognition model, and output a bounding box and position information corresponding to the work clothes; the work clothes are contained in the bounding box; According to the bounding box and position information corresponding to the work clothes, the occurrence frequency of employees wearing the work clothes at different positions is counted to generate an employee distribution heat map; The service areas corresponding to the location information with the highest frequency of occurrence are summarized to obtain the preset activity area.

2. The method for identifying work clothes wearing behavior according to claim 1, characterized in that: The method of counting the occurrence frequency of employees wearing work clothes at different locations according to the bounding box and position information corresponding to the work clothes to generate an employee distribution heat map comprises the following steps: Count the occurrence frequency of the center point coordinates of the boundary box corresponding to each work uniform, and determine the weight of the employee at the location of each work uniform; the occurrence frequency is proportional to the weight; Generate a color gradient employee heat map based on the center point coordinates, length and width information, and weight of the bounding box corresponding to each work uniform; The weight is proportional to the depth of the color.

3. The method for identifying work clothes wearing behavior according to claim 1, characterized in that: The step of finding out the illegal employees who are not wearing work clothes in the preset activity area according to the monitoring video stream includes the following steps: Performing image processing on the surveillance video stream to obtain an image to be identified; Input the image to be recognized into a pre-trained target recognition model to recognize and output the bounding box and position information corresponding to the target object; The target object includes a human body and the work clothes; The illegal employee is found according to the boundary box and position information of the target object.

4. The method for identifying work clothes wearing behavior according to claim 3, characterized in that: The image processing of the monitoring video stream to obtain the image to be identified comprises the following steps: Performing frame processing on the surveillance video stream to obtain a first type of image to be identified; The step of inputting the image to be identified into a pre-trained target recognition model and identifying and outputting the bounding box and position information corresponding to the target object comprises the following steps: Inputting the first type of image to be recognized into the target recognition model, and outputting the bounding boxes and position information corresponding to the human body and the work clothes respectively; The step of finding the illegal employee according to the boundary box and location information of the target object comprises the following steps: According to the bounding boxes and position information corresponding to the human body and the work clothes, respectively, determining whether the human body bounding box corresponding to the human body includes the work clothes bounding box; If the number of the same work clothes boundary boxes existing in the human body boundary box is greater than one, the determination of the human body boundary box is abandoned; If there is no work clothes boundary box in the human body boundary box, the person corresponding to the human body boundary box is determined to be the target person; the target person is a person who is not wearing work clothes; Determine whether the location information corresponding to the target person is within the preset activity area; If the location information corresponding to the target person is within the preset activity area, the target person is output as the illegal employee.

5. The method for identifying work clothes wearing behavior according to claim 3, characterized in that: The image processing of the monitoring video stream to obtain the image to be identified comprises the following steps: Performing frame processing on the surveillance video stream to obtain a first type of image to be identified; The step of inputting the image to be identified into a pre-trained target recognition model and identifying and outputting the bounding box and position information corresponding to the target object comprises the following steps: Inputting the first type of image to be recognized into the target recognition model, and outputting the bounding boxes and position information corresponding to the human body and the work clothes respectively; The step of finding the illegal employee according to the boundary box and location information of the target object comprises the following steps: Find out the employees in the preset activity area according to the position information corresponding to the human body; According to the bounding boxes and position information corresponding to the employee and the work clothes respectively, determining whether the human body bounding box corresponding to the employee includes the work clothes bounding box; If the number of the same work clothes boundary boxes existing in the human body boundary box is greater than one, the determination of the human body boundary box is abandoned; If any work clothes boundary box does not exist in the human body boundary box, it is determined that the person corresponding to the human body boundary box is the illegal employee.

6. The method for identifying work clothes wearing behavior according to claim 3, characterized in that: The image processing of the surveillance video to obtain the image to be identified comprises the following steps: Performing frame processing on the surveillance video to obtain a first type of image to be identified; Finding target pixel points corresponding to the preset activity area from the first type of images to be identified, and performing image segmentation according to the target pixel points to obtain a second type of images to be identified; The step of finding the illegal employee according to the boundary box and location information of the target object comprises the following steps: Inputting the second type of to-be-recognized images into the target recognition model, and outputting the boundary boxes and position information corresponding to the employees and work clothes in the preset activity area respectively; The step of finding the illegal employee according to the boundary box and location information of the target object comprises the following steps: According to the bounding boxes and position information corresponding to the employee and the work clothes respectively, determining whether the human body bounding box corresponding to the employee includes the work clothes bounding box; If the number of the same work clothes boundary boxes in the human body boundary box is greater than one, the judgment on the human body boundary box is abandoned; If any work clothes boundary box does not exist in the human body boundary box, it is determined that the person corresponding to the human body boundary box is the illegal employee.

7. The method for identifying work clothes wearing behavior according to any one of claims 1 to 6, characterized in that: The method of obtaining the preset activity area of ​​the employees in the store during the working period includes the following steps: Acquire a large number of sample images that have been annotated with marking frames; the marking frames include human marking frames and work clothes marking frames; The sample images are randomly divided into a training sample set and a test sample set, and a target recognition model is generated according to the training sample set; The trained target recognition model is tested and adjusted according to the test sample set until the training is completed when the recognition error rate of each target object is lower than a preset threshold.

8. A terminal device, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor is used to execute the computer program stored in the memory to implement the operations performed by the method for identifying work clothes wearing behavior as described in any one of claims 1 to 7.

9. A storage medium, characterized in that: The storage medium stores at least one instruction, which is loaded and executed by the processor to implement the operations performed by the method for identifying work clothes wearing behavior as described in any one of claims 1 to 7.

Images