Employee status identification method, device, electronic device and medium

Abstract

The present application discloses a method, device, electronic device and medium for identifying employee status. The method includes: obtaining a surveillance video of a workstation to be identified; inputting the surveillance video into a pedestrian detection model to obtain pedestrian feature information; obtaining a target position corresponding to the pedestrian feature information and a reference position corresponding to the workstation to be identified, and determining the positional relationship between the target position and the reference position; if the target position is within the reference position, determining that the workstation to be identified is in an employee on-duty state, and being able to automatically identify the on-duty status based on the video surveillance data.

Description

Technical Field

[0001] The present disclosure generally relates to the field of video recognition technology, and more particularly to an employee status recognition method, device, electronic device, and medium. Background Art

[0002] Some workplaces require staff to be on duty continuously according to production needs. In related technologies, security operators usually monitor by "staring at the screen", which seriously wastes manpower and material resources. Summary of the Invention

[0003] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide an employee status recognition method, device, electronic device and medium that can automatically recognize the on-the-job status based on video surveillance data.

[0004] In a first aspect, an embodiment of the present application provides a method for identifying employee status, including:

[0005] Obtain surveillance video of the workstation to be identified;

[0006] Inputting the surveillance video into a pedestrian detection model to obtain pedestrian feature information;

[0007] Acquire a target position corresponding to the pedestrian feature information and a reference position corresponding to the workstation to be identified, and determine a positional relationship between the target position and the reference position;

[0008] If the target position is within the reference position, it is determined that the workstation to be identified is in the employee on-duty state.

[0009] In some embodiments, the pedestrian feature information includes a bounding box for framing the pedestrian in the surveillance video, and the target position is two lower corner points of the bounding box in the pedestrian feature information. The method further includes:

[0010] If the two lower corner points are both within the reference position, it is determined that the workstation to be identified is in the employee on-duty state.

[0011] In some embodiments, after determining that the target position is within the reference position corresponding to the workstation to be identified, the method further includes:

[0012] Tracking and identifying a pedestrian based on the pedestrian feature information, and obtaining a plurality of pieces of pedestrian feature information corresponding to the pedestrian at a plurality of different moments;

[0013] When, among the plurality of pedestrian feature information corresponding to the pedestrians at a plurality of different moments, the number of pedestrian feature information within the reference position is greater than or equal to a first preset number, it is determined that the workstation to be identified is in the employee on-duty state.

[0014] In some embodiments, if the target position is not within the reference position, obtaining the working status corresponding to the workstation to be identified;

[0015] When the workstation to be identified is in a working state, an alarm message is sent according to a preset rule.

[0016] In some embodiments, the reference position is a spatial range where the employee is located when maintaining a working state within the workstation area covered by the workstation to be identified.

[0017] In some embodiments, after identifying that the workstation to be identified is in the employee on-duty state, the process further includes:

[0018] Continue to obtain the surveillance video of the workstation to be identified, and crop the surveillance video to obtain a cropped video;

[0019] Inputting the cropped video into a video behavior analysis model to obtain employee status analysis results;

[0020] When the employee status analysis result reaches the alarm condition, an alarm message is sent according to the preset rules.

[0021] In some embodiments, inputting the cropped video into a video behavior analysis model to obtain an employee status analysis result includes:

[0022] Inputting the cropped video frame by frame into the video behavior analysis model to obtain employee status information corresponding to each frame of the cropped video;

[0023] When the employee status analysis result reaches the alarm condition, sending an alarm message according to the preset rules includes:

[0024] Performing abnormality identification on the employee status information corresponding to each frame of the cropped video;

[0025] The abnormal number of the cropped video frames in which the employee status is abnormal is counted, and when the abnormal number reaches a second preset number, an alarm message is sent according to a preset rule.

[0026] In some embodiments, the alarm time is counted, and when the alarm condition is met next time, it is determined whether the alarm time reaches a preset time threshold;

[0027] If the alarm time does not reach the preset time threshold, cancel the current alarm;

[0028] The alarm time is the cumulative time since the alarm was issued.

[0029] In a second aspect, an embodiment of the present application provides an employee status identification device, comprising:

[0030] An acquisition module is used to acquire surveillance video of the workstation to be identified;

[0031] A recognition module is used to input the monitoring video into a pedestrian detection model to obtain pedestrian feature information;

[0032] A determination module is used to obtain the target position corresponding to the pedestrian feature information and the reference position corresponding to the workstation to be identified, and to judge the positional relationship between the target position and the reference position. If the target position is within the reference position, it is determined that the workstation to be identified is in the employee on-duty state.

[0033] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the embodiment of the present application when executing the program.

[0034] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, which, when executed by a processor, implements the method described in the embodiment of the present application.

[0035] The employee status recognition method proposed in the embodiment of the present application obtains pedestrian feature information by analyzing the surveillance video of the workstation to be identified, and then determines that the workstation to be identified is in the on-the-job status of an employee when the target position is within the reference position based on the target position corresponding to the pedestrian feature information and the reference position corresponding to the workstation to be identified. This realizes automated monitoring of the on-the-job status without the need for additional supervisory personnel, effectively improving the efficiency and reliability of workstation status recognition.

[0036] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0038] Figure 1 The following is a diagram showing an implementation environment architecture of the employee status identification method provided in an embodiment of the present application;

[0039] Figure 2 A flow chart of an employee status identification method provided by an embodiment of the present application is shown;

[0040] Figure 3 A schematic diagram showing a flow chart of an employee status identification method provided by another embodiment of the present application is shown;

[0041] Figure 4A diagram showing the relationship between pedestrian feature information and reference positions provided by an embodiment of the present application is shown;

[0042] Figure 5 A flowchart of an employee status identification method provided by another embodiment of the present application is shown;

[0043] Figure 6 A schematic diagram showing a flow chart of an employee status identification method provided in yet another embodiment of the present application is shown;

[0044] Figure 7 A schematic diagram of the structure of cropping a video frame according to an embodiment of the present application is shown;

[0045] Figure 8 A flowchart of an employee status identification method provided by a specific embodiment of the present application is shown;

[0046] Figure 9 An exemplary structural block diagram of an employee status identification device provided in an embodiment of the present application is shown;

[0047] Figure 10 A schematic diagram of the structure of a computer system of an electronic device or server suitable for implementing an embodiment of the present application is shown. DETAILED DESCRIPTION

[0048] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the invention are shown in the accompanying drawings.

[0049] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0050] The specific implementation environment of the employee status recognition method proposed in this application can be found in Figure 1 . Figure 1 The following diagram shows an implementation environment architecture diagram of the employee status identification method provided in an embodiment of the present application.

[0051] like Figure 1 As shown, the implementation environment architecture includes: video acquisition equipment 1, server 2 and monitoring terminal 3.

[0052] Among them, the implementation environment may include at least one video capture device 1, which is used to capture video of the workstation to be identified to obtain surveillance video. Optionally, one video capture device 1 can correspond to at least one workstation to be identified, that is, a one-to-one corresponding video capture device 1 can be set for each workstation to be identified, or one video capture device 1 can simultaneously capture surveillance videos of multiple workstations to be identified. Among them, when one video capture device 1 simultaneously captures surveillance videos of multiple workstations to be identified, multiple workstations to be identified should be within the visual range of the video capture device 1.

[0053] The server 2 is used to analyze the surveillance video collected by the video acquisition device 1 to obtain the employee status recognition result and feed it back to the supervision terminal 3. Figure 1 As shown, it may further include an algorithm server 21, a data server 22 and a business server 23, wherein the algorithm server 21 may be connected to at least one video acquisition device 1 to receive surveillance videos and perform algorithmic analysis on the surveillance videos to obtain, for example, pedestrian feature information and employee status information, etc. It should be understood that the algorithm server 21 may be provided with a trained or self-supervisory pedestrian detection model and a video behavior analysis model to analyze the surveillance videos through the pedestrian detection model to obtain pedestrian feature information and to analyze the surveillance videos through the video behavior analysis model to obtain a video behavior analysis model. Then the algorithm server 21 sends the pedestrian feature information and employee status information to the data server 22 connected to it. The data server 22 performs data statistical analysis on the feature information and employee status information respectively to obtain the attendance status and employee status analysis results of the workstation to be identified. The data server 22 sends the analyzed attendance status and employee status analysis results of the workstation to be identified to the business server 23. The business server 23 stores the received attendance status and employee status analysis results of the workstation to be identified and sends an alarm message to the supervision terminal 3 when the alarm condition is met, or sends the stored attendance status and employee status analysis results of the workstation to be identified according to the request instruction when the supervision terminal 3 sends a request instruction.

[0054] Among them, server 2 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.

[0055] The video capture device 1, algorithm server 21, data server 22, service server 23, and supervisory terminal 3 are directly or indirectly connected via wired or wireless communication. Optionally, the wireless or wired network utilizes standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network, or any combination of a virtual private network.

[0056] The employee status identification method proposed in this application can be implemented by an employee status identification device, and the employee status identification device can be installed on a server.

[0057] In order to further illustrate the technical solutions provided by the embodiments of the present application, this is described in detail below with reference to the accompanying drawings and specific embodiments. Although the embodiments of the present application provide the method operation instruction steps shown in the following embodiments or drawings, more or fewer operation instruction steps may be included in the method based on conventional or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiments of the present application. The method may be executed in the order of the methods shown in the embodiments or drawings or in parallel during the actual processing process or when the device is executed.

[0058] Please refer to Figure 2 , Figure 2 FIG. 1 shows a flow chart of an employee status identification method provided by an embodiment of the present application. Figure 2 As shown, the method includes:

[0059] Step 101: Obtain surveillance video of the workstation to be identified.

[0060] Among them, the monitoring video can be obtained by the video acquisition device through video acquisition of the workstation to be identified. The video acquisition device can correspond one-to-one to the workstation to be identified, that is, one video acquisition device corresponds to collecting the monitoring video of one workstation to be identified. The video acquisition device can also correspond to multiple workstations to be identified at the same time, that is, one video acquisition device corresponds to collecting the monitoring videos of multiple workstations to be identified.

[0061] Optionally, the surveillance video may be a segmented video or a spliced ​​video. For example, when a video acquisition device captures surveillance videos of multiple workstations to be identified, the original video captured by the video acquisition device may be segmented according to the position information or identification information of the workstations to be identified to obtain surveillance videos corresponding to each workstation to be identified. Alternatively, if the video acquisition device is a video acquisition system composed of multiple cameras, the original videos captured by the multiple cameras may be spliced ​​according to the spatial position relationship to obtain a spatial surveillance video, and then segmented based on the position information or identification information of each workstation to be identified to obtain surveillance videos corresponding to each workstation to be identified. Alternatively, the workstation to be identified may be captured by at least two cameras for partial areas, and then the videos of multiple partial areas may be spliced ​​according to the spatial position relationship to obtain surveillance videos corresponding to the workstation to be identified.

[0062] Step 102: Input the surveillance video into a pedestrian detection model to obtain pedestrian feature information.

[0063] Among them, the pedestrian detection model can be a target recognition model trained using pedestrian label information. The pedestrian detection model can identify people in video images and mark them with bounding boxes, and output the video images with bounding box marks as pedestrian feature information.

[0064] It should be understood that the embodiments of the present application do not improve the target recognition model structure used as a pedestrian detection model. That is, during the application process, the embodiments of the present application can directly select a relatively mature target recognition model that can analyze video data, and then use a training set with pedestrian labels for training until the expected effect is achieved.

[0065] Optionally, the pedestrian detection model is set in the server. Specifically, the pedestrian detection model is set in the algorithm server, wherein a pedestrian detection model may be set in the algorithm server, and surveillance videos corresponding to multiple workstations to be identified may be input into the pedestrian detection model at the same time. The pedestrian detection model outputs the pedestrian feature information obtained by analysis in sequence or simultaneously according to the analysis completion time. In order to improve the analysis efficiency, multiple pedestrian detection models may also be set in the algorithm server at the same time, and the surveillance videos may be input into multiple pedestrian detection models according to strategies such as grouping information or acquisition time of the workstations to be identified, so as to reduce the amount of surveillance videos analyzed by each pedestrian detection model and improve the efficiency of pedestrian feature analysis. Optionally, a pedestrian detection model corresponding to the number of surveillance videos may be set in the algorithm server, so that each surveillance video can be input into the corresponding pedestrian detection model in a timely manner to ensure the timeliness of pedestrian feature analysis.

[0066] Step 103 : obtaining a target position corresponding to the pedestrian feature information and a reference position corresponding to the workstation to be identified, and determining a positional relationship between the target position and the reference position.

[0067] It should be noted that since the pedestrian feature information includes a bounding box used to frame the pedestrian in the image, the target position of the pedestrian in the image can be determined based on the bounding box. Therefore, based on the target position of the pedestrian in the image and the reference position of the workstation to be identified set in advance, it can be determined whether there is someone at the workstation to be identified, thereby determining the arrival status of the workstation to be identified.

[0068] In one or more embodiments, the surveillance video of the workstation to be identified is a surveillance video collected from a fixed area, that is, the position of the video acquisition device remains unchanged or the position is the same when collecting the same workstation to be identified. Therefore, the reference position in the surveillance video can be set according to the attribute information such as the position and size of the workstation to be identified, and then the arrival status of the workstation to be identified is determined based on the positional relationship between the target position and the reference position.

[0069] In one or more embodiments, the surveillance video can be parsed to obtain the spatial coordinates of each position point in the surveillance video, and then the spatial coordinates corresponding to the workstation to be identified are determined based on the position of the attribute characteristics of the workstation to be identified in the surveillance video, and the spatial coordinates corresponding to the workstation to be identified are used as the reference position. After the pedestrian detection model analyzes the surveillance video to obtain pedestrian feature information, the spatial coordinates corresponding to the pedestrian feature information are parsed and the spatial coordinates corresponding to the pedestrian feature information are used as the target position, and then the arrival status of the workstation to be identified is determined based on the positional relationship between the target position and the reference position.

[0070] Step 104: If the target position is within the reference position, it is determined that the workstation to be identified is in the employee arrival status.

[0071] That is to say, when the position of the pedestrian marked by the pedestrian characteristic information is within the reference corresponding to the workstation to be identified, it means that the pedestrian is in the area corresponding to the workstation to be identified, that is, the pedestrian is an on-duty employee at the workstation to be identified. Therefore, it is determined that the workstation to be identified is in the employee on-duty state.

[0072] Therefore, the employee status recognition method proposed in the embodiment of the present application obtains pedestrian feature information by analyzing the surveillance video of the workstation to be identified, and then determines the employee's on-the-job status at the workstation to be identified when the target position corresponds to the pedestrian feature information and the reference position corresponding to the workstation to be identified when the target position is within the reference position, thereby realizing automated monitoring of the on-the-job status without the need for additional supervisory personnel, and effectively improving the efficiency and reliability of workstation status recognition.

[0073] In one or more embodiments, Figure 3 As shown, the method includes:

[0074] Step 1031: Determine the positional relationship between the target position and the reference position.

[0075] The target position and the reference position use the same coordinate system, such as pixel coordinates or spatial coordinates.

[0076] Step 104: If the target position is within the reference position, it is determined that the workstation to be identified is in the employee arrival status.

[0077] Step 1041: If the target position is not within the reference position, the working status corresponding to the workstation to be identified is obtained.

[0078] Step 1042: When the workstation to be identified is in working state, an alarm message is sent according to a preset rule.

[0079] That is to say, the present application may also include a process for identifying the working status of the workstation to be identified. It should be understood that if the starting condition of the employee status identification method proposed in the present application is the working status, for example, the company sets the working hours as the time period corresponding to the working status, then, by identifying the working status of the workstation to be identified, it is possible to first determine whether the current state is working time, thereby effectively avoiding the misjudgment of determining that the employee is off-duty during non-working hours. Optionally, the entire employee status identification system (including video acquisition equipment, server and monitoring terminal) can be controlled to be shut down based on the identification of the working status of the workstation to be identified, thereby effectively reducing costs, such as power loss.

[0080] Optionally, the positional relationship between the target position and the reference position can be determined by comparing their coordinate values ​​in the same coordinate system. The reference position is the reference position of the workstation area corresponding to the workstation to be identified. That is, the reference position includes at least two coordinates: the minimum coordinate value of the area range and the maximum coordinate value of the area range. Therefore, the target position needs to be compared with at least two reference positions to determine the positional relationship between the target position and the reference position.

[0081] For example, when both the target position and the reference position are expressed by pixel coordinates, the pixel coordinate value corresponding to the target position is compared with the minimum coordinate value of the area range and the maximum coordinate value of the area range corresponding to the reference position respectively. If the target position is greater than or equal to the minimum coordinate value of the area range and less than or equal to the maximum coordinate value of the area range, it is determined that the target position is within the reference position, indicating that there is an employee in the reference position, and the workstation to be identified is determined to be in the employee on-duty state. If the target position is less than the minimum coordinate value of the area range or the target position is greater than the maximum coordinate value of the area range, it means that there is no employee in the reference position, then it is determined that the target position is not within the reference position, and the working status corresponding to the workstation to be identified is obtained. If the workstation to be identified is in a non-working state, return to the step of obtaining the surveillance video of the workstation to be identified, or control to stop obtaining the surveillance video of the workstation to be identified to save resources. If the workstation to be identified is in a working state, then when the workstation to be identified is in a working state, an alarm message is sent according to the preset rules to remind the supervisory end that the employee at the workstation is absent.

[0082] Therefore, the embodiment of the present application can effectively determine whether there are employees in the workstation area by comparing the target position in the pedestrian feature information with the preset reference position corresponding to the workstation to be identified, thereby determining the workstation's attendance status. Moreover, if the workstation to be identified is in a working state, an effective alarm can be issued, thereby reducing production anomalies caused by employee absence.

[0083] Furthermore, the target position is the two lower corner points of the bounding box in the pedestrian feature information. According to the target position corresponding to the pedestrian feature information and the reference position corresponding to the workstation to be identified, the on-the-job status of the workstation to be identified is determined, including: if both lower corner points are within the reference position, it is determined that the workstation to be identified is in the on-the-job status of the employee.

[0084] It should be understood that the image of a person has a certain width. In order to avoid the problem that the workstation to be identified is close and part of the body area of ​​the employee at the adjacent workstation is mistakenly identified as the employee of the current workstation to be identified, this application proposes to use the two lower corner points of the bounding box in the pedestrian feature information as redundant judgment conditions, that is, the target coordinates corresponding to the two lower corner points expressing the width of the pedestrian must be within the reference position to determine that the workstation to be identified is in the employee on-duty state.

[0085] Specifically, if Figure 4As shown, the first and second lower corners of the bounding box in the pedestrian feature information are obtained respectively, and it is determined whether the first lower corner is within the reference position. If the first lower corner is within the reference position, it is further determined whether the second lower corner is within the reference position. If the second lower corner is also within the reference position, it indicates that the pedestrian is completely at the workstation, and the workstation to be identified is determined to be an employee on duty. If the second lower corner is not within the reference position, it indicates that the pedestrian is a pedestrian in the nearby area, which is a false positive. If the first lower corner is not within the reference position, it indicates that the pedestrian does not belong to the workstation. The first lower corner can be the lower left corner of the pedestrian feature information, and the second lower corner can be the lower right corner of the pedestrian feature information.

[0086] Optionally, the two upper corner points of the bounding box in the pedestrian feature information can also be used as redundant judgment conditions. However, since the heights of employees may vary greatly, it is not as accurate as the determined ground area range judgment. Therefore, it is preferred to use the two lower focal points as redundant judgment conditions.

[0087] Optional, such as Figure 4 As shown, Box 1 can represent the spatial range covered by the workstation to be identified. To further reduce judgment errors, the spatial range of the workstation to be identified can be reduced, with Box 2 serving as the reference position for the workstation to be identified. The size of Box 2 can be the spatial range within the spatial range covered by the workstation to be identified, as determined based on historical data, where an employee remains in a working state. Boxes 1 and 2 are located in the pedestrian's foot area, with Box 1 nested outside of Box 2.

[0088] In one or more embodiments, in order to further ensure the accuracy of employee status identification, for example, to avoid misjudgment caused by other employees passing by the workstation to be identified, the present application, after determining that the target position is within the reference position corresponding to the workstation to be identified, further includes: tracking and identifying pedestrians based on pedestrian feature information, obtaining multiple pedestrian feature information corresponding to pedestrians at multiple different times, and when the number of pedestrian feature information within the reference position among the multiple pedestrian feature information corresponding to pedestrians at multiple different times is greater than a preset first set number, it is determined that the workstation to be identified is in the employee on-duty status.

[0089] Optionally, the tracking time or the number of tracking image frames can be set. Taking the tracking time as an example, after the pedestrian feature information is first identified as being in the reference position, multiple pedestrian feature information of the pedestrian within the tracking time is obtained based on the pedestrian feature information, and then the positional relationship between the multiple pedestrian feature information and the reference position is judged respectively. If the number of multiple pedestrian feature information within the reference position is greater than or equal to a first preset number, it is determined that the workstation to be identified is in the employee on-duty state. If the number of multiple pedestrian feature information within the reference position is less than the first preset number, it is determined that the workstation to be identified is in the employee off-duty state or the working status corresponding to the workstation to be identified is further obtained, and when the workstation to be identified is in the working state, an alarm message is sent according to the preset rules.

[0090] In one or more embodiments, Figure 5 As shown, the employee status identification method proposed in the embodiment of the present application further includes:

[0091] Step 105 : Continue to obtain the surveillance video of the workstation to be identified, and crop the surveillance video to obtain a cropped video.

[0092] It should be understood that since surveillance videos are continuously captured image frames, it is difficult to identify the user's movements when the user's movements are small or there is a brief pause. For example, blinking may be misjudged as sleeping. Therefore, this application crops and splices the surveillance videos so that the cropped video after splicing contains image information over a larger time range.

[0093] Specifically, the process of cropping a surveillance video involves extracting image frames from the surveillance video according to a preset rule, and then stitching the extracted image frames together to produce a cropped video. The preset rule may be to extract one frame every predetermined number of image frames. The number of intervening image frames can be set based on actual conditions. In the embodiment of the present application, the number of intervening image frames is required to ensure that a dozen seconds of surveillance video can be cropped to produce 16 frames of images for stitching the cropped video.

[0094] Step 106: Input the cropped video into a video behavior analysis model to obtain employee status analysis results.

[0095] Step 107: When the employee status analysis result reaches the alarm condition, an alarm message is sent according to a preset rule.

[0096] Among them, the alarm condition is that the employee status analysis result is abnormal.

[0097] It should be noted that the video behavior analysis model can be a mature video analysis model. When training the video behavior analysis model, labels of abnormal employee status that the company needs to judge can be used for training, such as sleeping on the job, smoking, making phone calls, not wearing a safety helmet, etc. In other words, the video behavior analysis model can be trained on demand according to the needs of the company to improve the matching degree between the video behavior analysis model and the needs of the company.

[0098] Therefore, this application effectively improves the efficiency of judging employee status by using a video behavior analysis model to replace traditional image classification or logical judgments such as whether a face is seen or whether a person is moving. That is, the video behavior analysis model can output multiple employee statuses, reducing the amount of calculation required for sequential or simultaneous judgments between the logics of the image, thereby improving judgment efficiency and reducing calculation costs.

[0099] Specifically, if Figure 6 As shown, in step 106, the cropped video is input into the video behavior analysis model to obtain employee status analysis results, including:

[0100] In step 1061 , the cropped video is input frame by frame into the video behavior analysis model to obtain employee status information corresponding to each frame of the cropped video.

[0101] Step 107, when the employee status analysis result reaches the alarm condition, an alarm message is sent according to the preset rules, including:

[0102] Step 1071 , performing abnormality identification on the employee status information corresponding to each frame of cropped video.

[0103] Step 1072 , counting the number of abnormal cropped video frames in which the employee status is abnormal, and when the number of abnormalities reaches a second preset number, sending an alarm message according to a preset rule.

[0104] That is to say, after the cropped video is input into the video behavior analysis model frame by frame, the employee status is identified for each frame of the cropped video image input, and the recognition result of the video behavior analysis model for each frame of the cropped video image, that is, the employee status information, is recorded. Then, according to the preset mapping relationship of whether the employee status information is abnormal, the employee status information corresponding to each frame of the cropped video is identified as abnormal. For example, the label of the frame of cropped video is marked as abnormal or not abnormal, and the number of abnormal cropped states in which the employee status is abnormal is counted. When the number of abnormalities reaches a second preset number, an alarm message is sent according to the preset rules, indicating that the employee at the workstation is in an abnormal state.

[0105] It should be understood that if Figure 7As shown, when counting the abnormal number of cropped video frames in which the employee status is abnormal, statistics can be performed based on the video segments generated by the corresponding rules for splicing the cropped videos, that is, the abnormal number of cropped video frames in which the employee status is abnormal in 16 consecutive cropped video frames (1 to 16 frames, 2 to 17 frames, or 3 to 18 frames) is determined, thereby determining whether the employee has exhibited abnormal behavior within the time period, and avoiding misjudgment based on discontinuous and occasional anomalies.

[0106] In one or more embodiments, it also includes: counting the alarm time, and when the alarm condition is met next time, determining whether the alarm time reaches the preset time threshold, and canceling the current alarm if the alarm time does not reach the preset time threshold.

[0107] Among them, the alarm time is the cumulative length of time since the alarm was issued. That is to say, two consecutive alarms need to be issued at an interval of a preset time threshold, thereby effectively reducing the frequency of alarm issuance and avoiding repeated alarms.

[0108] As a specific embodiment, Figure 8 As shown, the employee status identification method includes the following steps:

[0109] Step 201: A video capture device obtains a surveillance video of a workstation to be identified.

[0110] Step 202: The video capture device sends the surveillance video to the server.

[0111] In step 203, the server inputs the surveillance video into the pedestrian detection model to obtain pedestrian feature information.

[0112] Step 204 : Determine the positional relationship between the target position in the pedestrian feature information and the reference position.

[0113] If the target position is within the reference position, then execute step 205;

[0114] If the target position is not within the reference position, execute step 208;

[0115] Step 205: Track and identify the pedestrian in the pedestrian feature information.

[0116] Step 206 : Determine the positional relationship between the two lower corner points corresponding to the multiple pedestrian feature information tracked and identified and the reference position corresponding to the workstation to be identified.

[0117] If the number of the plurality of pedestrian feature information within the reference position is greater than or equal to the first preset number, executing step 207;

[0118] If the number of the plurality of pedestrian feature information within the reference position is less than the first preset number, step 209 is executed.

[0119] Step 207: Determine whether the workstation to be identified is in the employee on-duty state.

[0120] Step 208: Count the number of times the target position is continuously outside the reference range and reaches a first preset number.

[0121] Step 209: Obtain the working status corresponding to the workstation to be identified.

[0122] If the workstation to be identified is in working state, an alarm message is sent according to the preset rules;

[0123] If the workstation to be identified is in a non-working state, the video acquisition device is controlled to stop acquiring surveillance video.

[0124] Step 210: Continue to obtain the surveillance video of the workstation to be identified, and crop the surveillance video to obtain a cropped video.

[0125] In step 211 , the cropped video is input into a video behavior analysis model to obtain employee status information corresponding to each frame of the cropped video.

[0126] Step 212: performing abnormality identification on the employee status information corresponding to each frame of the cropped video.

[0127] Step 213: Count the number of abnormalities in the cropped video frames in which the employee status is abnormal.

[0128] Step 214: Determine whether the number of abnormalities reaches a second preset number.

[0129] If the number of abnormalities reaches a second preset number, an alarm message is sent according to the preset rules;

[0130] If the number of abnormalities does not reach the second preset number, return to step 210.

[0131] To sum up, the employee status recognition method proposed in the embodiment of the present application obtains pedestrian feature information by analyzing the surveillance video of the workstation to be identified, and then determines that the workstation to be identified is in the employee on-the-job status when the target position is within the reference position based on the target position corresponding to the pedestrian feature information and the reference position corresponding to the workstation to be identified, thereby realizing automated monitoring of the on-the-job status without the need for additional supervisory personnel, and effectively improving the efficiency and reliability of workstation status recognition.

[0132] It should be noted that although the operations of the present method are described in a particular order in the drawings, this does not require or imply that the operations must be performed in this particular order, or that all illustrated operations must be performed to achieve desirable results.

[0133] Figure 9 An exemplary structural block diagram of an employee status identification device provided in an embodiment of the present application is shown.

[0134] like Figure 9 As shown, the employee status identification device 10 includes:

[0135] An acquisition module 11 is used to acquire surveillance video of the workstation to be identified;

[0136] The recognition module 12 is used to input the monitoring video into the pedestrian detection model to obtain pedestrian feature information;

[0137] The determination module 13 is used to obtain the target position corresponding to the pedestrian feature information and the reference position corresponding to the workstation to be identified, and to judge the positional relationship between the target position and the reference position. If the target position is within the reference position, it is determined that the workstation to be identified is in the employee on-duty state.

[0138] In some embodiments, the pedestrian feature information includes a bounding box for framing the pedestrian in the surveillance video, and the target position is the two lower corner points of the bounding box in the pedestrian feature information. The determination module 13 is further configured to:

[0139] If the two lower corner points are both within the reference position, it is determined that the workstation to be identified is in the on-duty state.

[0140] In some embodiments, the determination module 13 is further configured to:

[0141] Tracking and identifying a pedestrian based on the pedestrian feature information, and obtaining a plurality of pieces of pedestrian feature information corresponding to the pedestrian at a plurality of different moments;

[0142] When the number of the pedestrian feature information within the reference position in the plurality of pedestrian feature information corresponding to the pedestrian at multiple different moments is greater than or equal to a first preset number, it is determined that the workstation to be identified is in the employee on-duty state.

[0143] In some embodiments, the determination module 13 is further configured to:

[0144] If the target position is not within the reference position, obtaining the working status corresponding to the workstation to be identified;

[0145] When the workstation to be identified is in a working state, an alarm message is sent according to a preset rule.

[0146] In some embodiments, the reference position is a spatial range where the employee is located when maintaining a working state, determined within the spatial range covered by the workstation to be identified.

[0147] In some embodiments, the acquisition module 11 is further configured to: continue acquiring the surveillance video of the workstation to be identified, and crop the surveillance video to obtain a cropped video;

[0148] The recognition module 12 is further configured to: input the cropped video into a video behavior analysis model to obtain an employee status analysis result;

[0149] The determination module 13 is further configured to send an alarm message according to a preset rule when the employee status analysis result reaches an alarm condition.

[0150] In some embodiments, the recognition module 12 is further configured to: input the cropped video frame by frame into the video behavior analysis model to obtain employee status information corresponding to each frame of the cropped video;

[0151] The determination module 13 is further configured to: identify anomalies in the employee status information corresponding to each frame of the cropped video;

[0152] The abnormal number of the cropped video frames in which the employee status is abnormal is counted, and when the abnormal number reaches a second preset number, an alarm message is sent according to a preset rule.

[0153] In some embodiments, the determination module 13 is further configured to:

[0154] Counting the alarm time, and determining whether the alarm time reaches a preset time threshold when the alarm condition is met next time;

[0155] If the alarm time does not reach the preset time threshold, cancel the current alarm;

[0156] The alarm time is the cumulative time since the alarm was issued.

[0157] It should be understood that the units or modules described in the apparatus 10 are similar to those described in the reference Figure 2 The various steps in the described method correspond to each other. Therefore, the operations and features described above for the method are also applicable to the device 10 and the units contained therein, and will not be repeated here. The device 10 can be pre-implemented in the browser or other security application of the electronic device, or loaded into the browser or its security application of the electronic device by downloading or other means. The corresponding units in the device 10 can cooperate with the units in the electronic device to implement the solution of the embodiment of the present application.

[0158] The several modules or units mentioned in the detailed description above are not necessarily divided into one module or unit. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0159] To summarize, the employee status recognition device proposed in the embodiment of the present application obtains pedestrian feature information by analyzing the surveillance video of the workstation to be identified, and then determines that the workstation to be identified is in the employee on-the-job status when the target position is within the reference position based on the target position corresponding to the pedestrian feature information and the reference position corresponding to the workstation to be identified, thereby realizing automated monitoring of the on-the-job status without the need for additional supervisory personnel, and effectively improving the efficiency and reliability of workstation status recognition.

[0160] Reference below Figure 10 , Figure 10 A schematic diagram of the structure of a computer system of an electronic device or server suitable for implementing the embodiments of the present application is shown.

[0161] like Figure 10 As shown, the computer system includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1002 or the program loaded from the storage part 1008 into the random access memory (RAM) 1003. Various programs and data required for the operation instructions of the system are also stored in the RAM 1003. The CPU 1001, ROM 1002 and RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0162] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, and the like; an output section 1007 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 1008 including a hard disk; and a communication section 1009 including a network interface card such as a LAN card or a modem. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 1010 as needed, so that computer programs read therefrom can be installed into the storage section 1008 as needed.

[0163] In particular, according to the embodiment of the present application, the above reference flow chart Figure 2The described process can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 1009, and / or installed from a removable medium 1011. When the computer program is executed by the central processing unit (CPU) 1001, the above-mentioned functions defined in the system of the present application are executed.

[0164] It should be noted that the computer-readable medium shown in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical cable, RF, or any suitable combination thereof.

[0165] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operating instructions of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the aforementioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than the order marked in the accompanying drawings. For example, the boxes represented by two connections can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operating instruction, or can be implemented using a combination of dedicated hardware and computer instructions.

[0166] The units or modules described in the embodiments of the present application may be implemented in software or hardware. The units or modules described may also be provided in a processor. For example, they may be described as: a processor including an acquisition module, an identification module, and a determination module. The names of these units or modules do not, in certain circumstances, constitute limitations on the units or modules themselves. For example, the acquisition module may also be described as "acquiring surveillance video of the workstation to be identified."

[0167] As another aspect, the present application further provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments, or may exist independently and not be incorporated into the electronic device. The computer-readable storage medium stores one or more programs, which, when used by one or more processors, execute the employee status identification method described in the present application.

[0168] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the aforementioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for identifying employee status, characterized in that: include: Obtain surveillance video of the workstation to be identified; Inputting the surveillance video into a pedestrian detection model to obtain pedestrian feature information, wherein the pedestrian detection model may be a target recognition model trained using pedestrian label information, and the pedestrian detection model may identify people in the surveillance video and mark them with bounding boxes. The pedestrian feature information is a video image marked with the bounding box, and the pedestrian feature information includes a bounding box for framing the pedestrian in the surveillance video; Obtaining a target position corresponding to the pedestrian feature information and a reference position corresponding to the workstation to be identified, and determining a positional relationship between the target position and the reference position, wherein the target position is the two lower corner points of the bounding box; If the target position is within the reference position, it is determined that the workstation to be identified is in the employee on-duty state; the target position is within the reference position, including that both lower corner points of the bounding box are within the reference position.

2. The method according to claim 1, characterized in that After determining that the target position is within the reference position corresponding to the workstation to be identified, the method further includes: Tracking and identifying a pedestrian based on the pedestrian feature information, and obtaining a plurality of pieces of pedestrian feature information corresponding to the pedestrian at a plurality of different moments; When, among the plurality of pedestrian feature information corresponding to the pedestrians at a plurality of different moments, the number of pedestrian feature information within the reference position is greater than or equal to a first preset number, it is determined that the workstation to be identified is in the employee on-duty state.

3. The method according to claim 1, characterized in that Also includes: If the target position is not within the reference position, obtaining the working status corresponding to the workstation to be identified; When the workstation to be identified is in a working state, an alarm message is sent according to a preset rule.

4. The method according to claim 1, wherein The reference position is a spatial range where the employee is located when maintaining a working state, determined within the spatial range covered by the workstation to be identified.

5. The method according to claim 1, characterized in that After identifying that the workstation to be identified is in the employee on-duty state, the process further includes: Continue to obtain the surveillance video of the workstation to be identified, and crop the surveillance video to obtain a cropped video; Inputting the cropped video into a video behavior analysis model to obtain employee status analysis results; When the employee status analysis result reaches the alarm condition, an alarm message is sent according to the preset rules.

6. The method according to claim 5, characterized in that The cropped video is input into a video behavior analysis model to obtain employee status analysis results, including: Inputting the cropped video frame by frame into the video behavior analysis model to obtain employee status information corresponding to each frame of the cropped video; When the employee status analysis result reaches the alarm condition, sending an alarm message according to the preset rules includes: Performing abnormality identification on the employee status information corresponding to each frame of the cropped video; The abnormal number of the cropped video frames in which the employee status is abnormal is counted, and when the abnormal number reaches a second preset number, an alarm message is sent according to a preset rule.

7. The method according to any one of claims 3, 5 or 6, characterized in that: Also includes: Counting the alarm time, and determining whether the alarm time reaches a preset time threshold when the alarm condition is met next time; If the alarm time does not reach the preset time threshold, cancel the current alarm; The alarm time is the cumulative time since the alarm was issued.

8. An employee status recognition device, characterized in that: include: An acquisition module is used to acquire surveillance video of the workstation to be identified; a recognition module, configured to input the surveillance video into a pedestrian detection model to obtain pedestrian feature information. The pedestrian detection model may be a target recognition model trained using pedestrian label information. The pedestrian detection model may identify people in the surveillance video and identify them with bounding boxes. The pedestrian feature information is a video image with the bounding box identification, and the pedestrian feature information includes a bounding box for framing pedestrians in the surveillance video. A determination module is used to obtain the target position corresponding to the pedestrian feature information and the reference position corresponding to the workstation to be identified, and to determine the positional relationship between the target position and the reference position. If the target position is within the reference position, it is determined that the workstation to be identified is in the employee on-duty state. The target position is the two lower corner points of the bounding box. The target position is within the reference position, including that the two lower corner points of the bounding box are both within the reference position.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the employee status identification method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the employee status identification method according to any one of claims 1 to 7 is implemented.

Images