Gray box model-driven gas station employee dressing standard identification system and method

Abstract

The invention relates to a grey box model-driven gas station employee dressing standard identification system. The system is composed of five modules including a dressing nonstandard behavior module, a video inspection strategy module, a target detection model module, a physical priori knowledge module and a physical information-driven dressing nonstandard identification algorithm module. The non-standard dressing behaviors comprise non-standard behaviors of not wearing black insulating shoes and not wearing work cards. According to the method, intelligent identification can be carried out on the dressing normativity of employees in the video monitoring service industry under the conditions of poor network environment, poor hardware facilities, poor monitoring environment and complex scenes, and on one hand, the method is assisted to be copied and popularized in the video monitoring industries of supermarkets and retail industries; meanwhile, by means of the scheme with low economic cost and high accuracy, standard video inspection of gas station staff can be completed instead of manpower to a certain extent, gas station service supervision is assisted, and enterprise management cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and particularly to a gray-box model-driven identification system and method for the dressing norms of gas station employees. Background Art

[0002] At present, with the rapid development of artificial intelligence technology, especially the wide application of deep learning algorithms in the field of video intelligent analysis, all walks of life are accelerating towards the direction of intelligence and automation. This trend is particularly significant in the gas station service field. The traditional manual video inspection method has many challenges in supervising employees' service norms, including dressing norms: high labor costs and time costs, long inspection cycles, and reduced accuracy due to the influence of human factors (such as subjective judgment and fatigue). To solve these problems, deep learning computer vision technology has been introduced, providing strong support for the real-time video analysis and automatic early warning of employees' service norms, especially dressing norms.

[0003] However, despite technological progress, the development of an identification system for non-compliant dressing of gas station employees still faces many technical problems. As an important standard in the service industry, the compliance of dressing norms is crucial for enhancing the corporate image and ensuring service quality. Therefore, we propose a gray-box model-driven method based on the fusion of physical information and deep learning. Summary of the Invention

[0004] The purpose of the present invention is to provide a gray-box model-driven identification system and method for the dressing norms of gas station employees, which solves the problems mentioned in the background art.

[0005] To achieve the above purpose, the present invention provides the following technical solutions:

[0006] A gray-box model-driven identification system for the dressing norms of gas station employees consists of 5 modules: non-compliant dressing behaviors, video inspection strategies, target detection models, physical prior knowledge, and a physical information-driven non-compliant dressing identification algorithm; non-compliant dressing behaviors include non-compliant behaviors such as not wearing black insulating shoes and not wearing work badges. For such inspection tasks, the video inspection strategy module is responsible for accessing multiple on-site cameras, implementing a timed inspection task mechanism, and realizing the function of associating cameras and tasks. Then, after the inspection strategy is issued, the target detection model is requested to obtain the target category and location information, where the detection ability is trained by the detection category data designed in the scheme. Then, the non-compliant dressing identification algorithm is implemented through the physical information knowledge module proposed in this scheme, and after analysis, it is asynchronously returned to the requester and a pop-up warning is given through the monitoring large screen.

[0007] Preferably, for the non - standard dressing behavior, this module is a requirement conversion matrix that abstracts the requirements for the dressing norms of gas station employees into an AI problem that can be realized. The non - standard dressing behaviors include: not wearing black insulating shoes and not wearing a work ID.

[0008] Preferably, for the video inspection strategy, this module is responsible for converting the gas station monitoring cameras into the inspection strategies of this solution;

[0009] 1) Connect the cameras to the Visual Internet platform, analyze the inspection tasks suitable for each camera, and map the correlation mechanism between the cameras and the non - standard dressing behavior tasks to the parameter configuration management module of this platform through software development for easy later maintenance;

[0010] 2) According to the inspection time requirements, map the camera's regular inspection task mechanism to the parameter configuration management module of this platform through software development. Relevant personnel can set the regular inspection tasks for each non - standard dressing behavior automatic inspection through the parameter management module of this platform. For example, the regular inspection time set for the inspection task of employees not wearing a work ID corresponding to camera No. 1 is every 30 minutes, and the regular inspection time set for the inspection task of not wearing black insulating shoes corresponding to camera No. 2 is every 20 minutes;

[0011] 3) Since the backend requests the AI model service in a concurrent form and some AI inspection tasks require continuous multi - frame analysis, there is a practical problem of long AI inference time. Therefore, an asynchronous mechanism is used to realize the message docking between the backend and the AI model to ensure that the service does not time out and the data is transmitted consistently.

[0012] Preferably, for target detection, this method uses the lightweight target detection model YOLOv5 and innovatively designs the detection categories, and combines interpretable physical information as prior knowledge to effectively complete the identification of non - standard dressing behaviors.

[0013] Preferably, for physical prior knowledge, this method innovatively regards the human behaviors characterized by employees' non - standard dressing behaviors and the conditions on which their interaction with the environment depends as a kind of prior knowledge with low uncertainty and high interpretability. The gray - box model that fuses this knowledge with the target detection module can effectively identify complex employees' non - standard dressing behaviors and greatly reduce the GPU resource requirements;

[0014] According to the non - standard dressing behavior requirement matrix,

[0015] 1) The prior knowledge existing in the service process is: the condition that there must be employees present, that is, the target category "employees wearing blue work uniforms" exists;

[0016] 2) The prior knowledge existing during the movement of the employee is that the employee is walking towards the camera face-on, that is, the target category "employee in blue work clothes" exists, and the coordinate positions of the "employee" in consecutive frames of images are continuously moving downward in the frame.

[0017] Preferably, for the dressing non-standard recognition algorithm, this algorithm does not require data to train model parameters, so it has interpretability and verifiability. It should be noted that when the vertical displacement variable threshold of the employee is greater than Δy and the horizontal displacement change threshold is less than Δx, it is determined that the employee is moving face-on towards the camera. At this time, for the detection of the work permit and logo targets, there is a more accurate effect, effectively improving the accuracy of the algorithm for recognizing employees without a work permit.

[0018] 1) The pseudo-algorithm design for recognizing employees without a work permit is as follows:

[0019] Input: Camera and task association mechanism, time information, space information, target detection model, annotation rules related to this task, prior physical knowledge that the employee is moving face-on towards the camera (vertical displacement of the employee is greater than the threshold Δy, horizontal displacement is less than the threshold Δx)

[0020] Process:

[0021] 1: Start a timed task to inspect the camera list of the inspection event (gas station area) and extract frames from the video address.

[0022] 2: Send the extracted image into the target detection model to obtain the detection result det_reco.

[0023] 3: If the employee (blue_cloth) exists in det_reco # Start quality inspection, meeting the prerequisite conditions.

[0024] 4: Record the coordinate position of the employee # For employee judgment.

[0025] 5: If only one work permit / logo (badge) is detected in det_reco.

[0026] 6: Extract frames at intervals and execute steps 2 - 5. If the coordinate displacement of the employee in the first and last frames of the inspection meets the specified thresholds Δy, Δx # The employee does not wear a work permit and is moving face-on towards the camera. 7: Then append [“event_behavior”: not wearing a work permit (xxx),”event_time”:xxxx,”event_camera”:xxx,”event_image”:xxxx] to response_result.

[0029] 8: Return response_result # Initialized to empty.

[0030] Output: response_result#json, exception information (event timestamp, any image of the event frame extraction, camera number); 2) The pseudo-algorithm for identifying that employees do not wear black insulating shoes is as follows:

[0031] Input: camera and task association mechanism, time information, space information, target detection model, annotation rules related to this task

[0032] Process:

[0033] 1: Start a scheduled task, inspect the camera list of the patrol event (gas station area), and extract frames from the video address

[0034] 2: Send the extracted frame image into the target detection model to obtain the detection result det_reco

[0035] 3: If an employee (blue_cloth) exists in det_reco # start quality inspection, meet the prerequisite conditions

[0036] 4: If it is detected that an employee is not wearing insulating shoes (other_shoes) in det_reco

[0037] 5: Extract frames at intervals and execute steps 2-4. If the number of times an employee not wearing insulating shoes (other_shoes) is detected is greater than the specified threshold z

[0038] # The employee does not wear insulating shoes

[0039] 6: Then append ["event_behavior": not wearing black insulating shoes (xxx), "event_time": xxxx, "event_camera": xxx, "event_image": xxxx] to response_result

[0040] 7: Return response_result # initialized to empty

[0041] Output: response_result#json, exception information (event timestamp, any image of the event frame extraction, camera number).

[0042] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0043] The present invention can intelligently identify the dress code compliance of employees in the video surveillance service industry under the conditions of poor network environment, poor hardware facilities, poor monitoring environment, and complex scenarios. On the one hand, it helps to replicate and promote this method in the video surveillance industries such as supermarkets and retail. At the same time, through the proposed low economic cost and high accuracy solution, it can replace manual labor to a certain extent to complete the standardized video inspection of gas station employees, helping to supervise gas station services and reduce the enterprise management cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is the system framework diagram of the present invention;

[0045] Figure 2 is the example diagram of the target detection category annotation of the present invention;

[0046] Figure 3 is the physical prior knowledge representation diagram for the present invention to determine that an employee does not wear an ID card;

[0047] Figure 4 is the recognition effect diagram of the present invention applied to the dress code non-compliance of employees in a certain gas station. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0049] Please refer to Figures 1-4 , a system for identifying the dress code compliance of gas station employees driven by a gray box model, which consists of 5 modules: non-compliant dress behavior, video inspection strategy, target detection model, physical prior knowledge, and a dress code non-compliance recognition algorithm driven by physical information; non-compliant dress behavior includes non-compliant behaviors such as not wearing black insulating shoes and not wearing an ID card. For such inspection tasks, the video inspection strategy module is responsible for accessing multiple on-site cameras, implementing a timed inspection task mechanism, and the function of associating cameras and tasks. Then, after the inspection strategy is issued, the target detection model is requested to obtain the target category and location information, where the detection ability is trained by the detection category data designed in the solution. Then, the dress code non-compliance recognition algorithm is implemented through the physical information knowledge module proposed in this solution, and after analysis, it is asynchronously returned to the requester and a pop-up warning is issued through the monitoring large screen.

[0050] Among them, the non-compliant dress behavior module is a requirement conversion matrix that abstracts the dress code requirements of gas station employees into an AI problem that can be realized. The non-compliant dress behavior includes: not wearing black insulating shoes and not wearing an ID card.

[0051] Among them, for the video patrol inspection strategy, this module is responsible for converting the gas station monitoring cameras into the patrol inspection strategy of this solution;

[0052] 1) Connect the cameras to the Visual Internet platform, analyze the suitable patrol inspection tasks for each camera, and map the association mechanism between the cameras and the tasks of unregulated dressing behaviors to the parameter configuration management module of this platform through software development for easy later maintenance;

[0053] 4) According to the requirements of the patrol inspection time, map the camera's regular patrol inspection task mechanism to the parameter configuration management module of this platform through software development. Relevant personnel can set the regular patrol inspection tasks for each unregulated dressing behavior automatically through the parameter management module of this platform. For example, the regular patrol inspection time set for the task of the employee corresponding to camera No. 1 not wearing a work permit is every 30 minutes, and the regular patrol inspection time set for the task of not wearing black insulating shoes corresponding to camera No. 2 is every 20 minutes;

[0054] 5) Since the backend requests the AI model service in a concurrent form and some AI patrol inspection tasks require continuous multi-frame analysis, there is a practical problem of long AI inference time. Therefore, an asynchronous mechanism is used to implement the message docking between the backend and the AI model to ensure that the service does not time out and the data is transmitted consistently.

[0055] Among them, for object detection, this method uses the lightweight object detection model YOLOv5 and innovatively designs the detection categories, and combines interpretable physical information as prior knowledge, which can effectively complete the identification of unregulated dressing behaviors.

[0056] Specifically, the Chinese and English labels of the object detection categories are as follows, and the annotation examples are as Figures 2-4 shown.

[0057] Chinese label English label Employee in blue work clothes blue_cloth Work ID / Logo badge Shoes of other colors other_shoes Black insulating shoes black_shoes

[0058] Among them, for physical prior knowledge, this method innovatively regards the human behaviors characterized by employees' unregulated dressing behaviors and the conditions on which their interaction with the environment depends as a kind of prior knowledge with low uncertainty and high interpretability. The gray-box model that integrates this knowledge with the object detection module can effectively identify complex employees' unregulated dressing behaviors and greatly reduce the GPU resource requirements;

[0059] According to the unregulated dressing behavior requirement matrix,

[0060] 1) The prior knowledge existing in the service process is: the condition that there must be employees present, that is, the target category "employees wearing blue work uniforms" exists;

[0061] 2) The prior knowledge existing during the movement of the employee is as follows: The employee is walking towards the camera with the front facing the camera, that is, the target category "employee wearing blue work clothes" exists, and the coordinate position of the "employee" in consecutive frames of images is continuously moving downward in the picture.

[0062] Among them, for the non-standard dressing recognition algorithm, this algorithm does not require data to train model parameters, so it has interpretability and verifiability. It should be noted that when the vertical displacement variable threshold of the employee is greater than Δy and the horizontal displacement change threshold is less than Δx, it is judged that the employee is moving towards the camera face-on. At this time, for the detection of the work permit and logo targets, there is a more accurate effect, effectively improving the accuracy of the algorithm for identifying employees without wearing work permits.

[0063] 1) The design of the pseudo-algorithm for identifying employees without wearing work permits is as follows:

[0064] Input: Camera and task association mechanism, time information, space information, target detection model, annotation rules related to this task, prior physical knowledge that the employee is moving towards the camera face-on (the vertical displacement of the employee is greater than the threshold Δy and the horizontal displacement is less than the threshold Δx)

[0065] Process:

[0066] 1: Start a timed task, inspect the camera list of the patrol event (gas station area), and extract frames from the video address

[0067] 2: Send the extracted picture into the target detection model to obtain the detection result det_reco

[0068] 3: If the employee (blue_cloth) exists in det_reco # Start quality inspection, meeting the prerequisite conditions

[0069] 4: Record the coordinate position of the employee # For employee judgment

[0070] 5: If only one work permit / logo (badge) is detected in det_reco

[0071] 6: Extract frames at intervals and execute steps 2-5. If the coordinate displacement of the employee in the first and last frames of the patrol meets the specified thresholds Δy and Δx # The employee is not wearing a work permit and is moving towards the camera face-on 7: Then append [“event_behavior”: not wearing a work permit (xxx),”event_time”:xxxx,”event_camera”:xxx,”event_image”:xxxx] to response_result

[0074] 8: Return response_result # Initialized to empty

[0075] Output: response_result#json, exception information (event timestamp, any image of the event frame extraction, camera number); 2) The pseudo-algorithm for identifying that an employee is not wearing black insulating shoes is as follows:

[0076] Input: camera and task association mechanism, time information, space information, target detection model, annotation rules related to this task

[0077] Process:

[0078] 1: Start a timed task, inspect the camera list of the patrol event (gas station area), and extract frames from the video address

[0079] 2: Send the extracted frame image into the target detection model to obtain the detection result det_reco

[0080] 3: If an employee (blue_cloth) exists in det_reco # start quality inspection and meet the preconditions

[0081] 4: If it is detected that an employee is not wearing insulating shoes (other_shoes) in det_reco

[0082] 5: Extract frames at intervals and execute steps 2 - 4. If the number of times an employee is detected not wearing insulating shoes (other_shoes) is greater than the specified threshold z

[0083] # The employee is not wearing insulating shoes

[0084] 6: Then append ["event_behavior": not wearing black insulating shoes (xxx), "event_time": xxxx, "event_camera": xxx, "event_image": xxxx] to response_result

[0085] 7: Return response_result # initialized to empty

[0086] Output: response_result#json, exception information (event timestamp, any image of the event frame extraction, camera number).

[0087] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A gray box model driven gas station staff dress code recognition system, characterized by: The system consists of five modules, including irregular dress behavior, video inspection strategy, target detection model, physical prior knowledge, and physical information-driven irregular dress recognition algorithm. Irregular dress behavior includes not wearing black insulated shoes and not wearing work badges. For such inspection tasks, the video inspection strategy module is responsible for the access of multiple cameras on site, the scheduled inspection task mechanism, and the camera and task association function. Then, after the inspection strategy is issued, the target detection model is requested to obtain the target category and location information. The detection capability is trained by the detection category data designed by the scheme. Then, the irregular dress recognition algorithm is implemented through the physical information knowledge module proposed in this scheme. After analysis, it is asynchronously returned to the requester and an early warning is issued through a pop-up window on the monitoring screen.

2. The method of the gray box model driven gas station employee dress code recognition system according to claim 1 is characterized by: Inappropriate dress behavior, this module is a demand conversion matrix, which abstracts the dress code requirements of gas station employees into an achievable AI problem. Inappropriate dress behavior includes: not wearing black insulated shoes and not wearing a work badge.

3. The method of the gray box model driven gas station employee dress code recognition system according to claim 1, characterized in that: Video inspection strategy: This module is responsible for converting the gas station surveillance camera into the inspection strategy of this solution; 1) Connect the cameras to the visual networking platform, analyze the inspection tasks that each camera is suitable for, and map the camera and irregular dressing behavior task association mechanism to the parameter configuration management module of this platform through software development to facilitate later maintenance; 2) According to the inspection time requirements, the camera scheduled inspection task mechanism is mapped to the parameter configuration management module of this platform through software development. Relevant personnel can set the scheduled inspection task of each irregular dress behavior through the parameter management module of this platform. For example, the scheduled inspection time set for the inspection task of employees not wearing work badges corresponding to camera No. 1 is set at an interval of 30 minutes, and the scheduled inspection time set for the inspection task of not wearing black insulating shoes corresponding to camera No. 2 is set at an interval of 20 minutes; 3) Since the backend requests AI model services in a concurrent manner and some AI inspection tasks require continuous multi-frame analysis, there is a practical problem of long AI reasoning time. Therefore, an asynchronous mechanism is used to realize message docking between the backend and the AI ​​model to ensure that the service does not time out and data is transmitted consistently.

4. The method of the gray box model driven gas station employee dress code recognition system according to claim 1, characterized in that: Target detection,This method adopts the lightweight target detection model YOLOv5 and innovatively designs the detection categories, and combines interpretable physical information as prior knowledge, which can effectively complete the identification of improper dressing behavior.

5. The method of the gray box model driven gas station employee dress code recognition system according to claim 1, characterized in that: Physical prior knowledge: This method innovatively regards the human behavior represented by employees' irregular dressing behavior and the conditions on which it interacts with the environment as an interpretable prior knowledge with low uncertainty. The gray-box model that integrates this knowledge with the target detection module can effectively identify complex irregular dressing behaviors of employees and significantly reduce GPU resource requirements. According to the irregular dressing behavior demand matrix, 1) The prior knowledge that exists in the service process is: there is and must be a condition for the employee to be present, that is, the target category "employees in blue work clothes" exists; 2) The prior knowledge that exists during the employee's movement is that the employee is walking towards the camera head-on, that is, the target category "employee in blue work clothes" exists, and the coordinate position of "employee" in multiple consecutive frames of images continues to move toward the bottom of the screen.

6. The method of the gray box model driven gas station employee dress code recognition system according to claim 1, characterized in that: The algorithm for identifying irregular dress does not require data training model parameters, so it is explainable and verifiable. It should be noted that when the employee's vertical displacement variable threshold is greater than Δy and the horizontal displacement change threshold is less than Δx, it is judged that the employee is moving towards the camera. At this time, the detection of the work badge and logo target has a more accurate effect, which effectively improves the accuracy of the algorithm for identifying people who are not wearing work badges; 1) The pseudo algorithm for identifying employees who do not wear work badges is designed as follows: Input: camera and task association mechanism, time information, spatial information, target detection model, annotation rules related to this task, physical prior knowledge of the employee moving towards the camera (the employee's vertical displacement is greater than the threshold Δy, and the horizontal displacement is less than the threshold Δx) process: Output: response_result#json, exception information (event timestamp, any picture of the event frame, camera number); 2) The pseudo-algorithm for identifying employees who are not wearing black insulating shoes is as follows: Input: camera and task association mechanism, time information, spatial information, target detection model, and annotation rules related to this task. Process: Output: response_result#json, exception information (event timestamp, any picture of the event frame, camera number).

Images