A method for identifying mobile phone usage behavior in low-resolution surveillance scenarios

Abstract

The present invention discloses a method for identifying mobile phone usage behavior in low-resolution monitoring scenarios, comprising the following steps: collecting low-resolution video images for frame-by-frame analysis, and identifying the wrist joint and head position in the analyzed low-resolution video images through an AlphaPose posture recognition model; obtaining a first probability that the mobile phone is being held by obtaining a mobile phone ROI area and its confidence based on a target detection model YOLOv7; based on the first probability, according to the wrist joint and head position, identifying whether there are several abnormal mobile phone usage situations in the current frame that indicate that the mobile phone is prohibited from being used in the current frame through an LSTM-based timing judgment model; the present invention realizes the identification of abnormal mobile phone usage behavior in low-resolution monitoring scenarios, and improves the applicability of intelligent recognition technology in low-resolution scenarios.

Description

Technical Field

[0001] The present invention relates to the field of intelligent recognition technology, and in particular to a method for identifying mobile phone usage behavior in low-resolution monitoring scenarios. Background Art

[0002] Conventional video footage has certain distance and clarity requirements; if these are exceeded, matching and recognition often becomes impossible. Furthermore, due to the small size of mobile phones, they can partially obstruct the view when held by a person, and are even smaller when facing the surveillance camera sideways. Therefore, recognition is even more difficult in low-resolution images.

[0003] Currently, the following methods are used to identify mobile phone usage in low-resolution images:

[0004] (1) Image super-resolution technology based on deep learning: using neural networks such as SRGAN and ClassSR to increase the image resolution, thereby increasing the pixel area of ​​the detection target and the probability of correct detection. This method has the disadvantages of large computational complexity, high hardware requirements, and long time consumption.

[0005] (2) Image super-resolution technology based on traditional methods: interpolation methods, sparse representation methods, and example-based methods are used for image super-resolution, which increases the area of ​​the target pixel and the probability of correct detection. This method has the disadvantage of poor results after super-resolution and is not as effective as image super-resolution technology based on deep learning.

[0006] (3) Image feature value classification based on SVM: Extract the features of each image block corresponding to the detection object, including color features, LBP features, Gabor filter features, and Schmid filter features, and use SVM classification to identify the category of each image block. This method requires the classification target to be determined in advance, which is difficult to detect in the case of low-resolution images.

[0007] The above-mentioned image super-resolution technologies based on deep learning and traditional methods all have the problem of high resolution of the super-resolution image, which will exponentially increase the time required for subsequent mobile phone detection and make it difficult to meet the requirements of real-time monitoring.

[0008] Considering the ineffectiveness of traditional image super-resolution techniques, the computational complexity of deep learning super-resolution methods, and the resulting increase in image resolution, which exponentially increases detection time and makes it difficult to meet the real-time requirements of video surveillance, SVM-based image feature value classification methods can determine whether a mobile phone is present in an image, but they struggle to detect and classify images from low-resolution images. Therefore, a method for identifying mobile phone use in low-resolution surveillance scenarios is urgently needed to address the challenges of existing technologies. Summary of the Invention

[0009] In order to improve the accuracy of identifying the use of mobile phones in low-resolution monitoring scenarios when the mobile phone may be blocked, the purpose of the present invention is to provide a method for identifying the use of mobile phones in low-resolution monitoring scenarios. Through a cascaded timing scheme, the precise matching and recognition of low-resolution video targets are greatly improved, which is conducive to improving the utilization rate of the video and improving the recognition accuracy.

[0010] To achieve the above technical objectives, the present invention provides a method for identifying mobile phone usage behavior in low-resolution monitoring scenarios, comprising the following steps:

[0011] Collect low-resolution video images and analyze them frame by frame. Use the AlphaPose posture recognition model to identify the wrist joints and head positions in the analyzed low-resolution video images.

[0012] Based on the object detection model YOLOv7, the first probability of the phone being held is obtained by obtaining the phone's ROI area and its confidence level;

[0013] Based on the first probability, according to the wrist joint and head position, a LSTM-based timing judgment model is used to identify whether there is abnormal use of the mobile phone in the current frame. Among them, abnormal use of the mobile phone is used to indicate several behaviors that are prohibited from using the mobile phone in the current frame.

[0014] Preferably, in the process of parsing the low-resolution video image frame by frame, the parsed color coding format is converted into RGB format, and the image size is scaled to 640*480. When the image is scaled, the aspect ratio is maintained until the length or width after scaling is exactly equal to the target size. When the image pixels are smaller than the target image pixels, the blank part of the target size RGB channels are filled with 0 respectively.

[0015] Preferably, in the process of obtaining the wrist joint and head position, the ROI area and key point data of the person in the parsed low-resolution video image are obtained based on the AlphaPose posture recognition model trained by the MSCOCO dataset, wherein the AlphaPose posture recognition model trained by the MSCOCO dataset is used to detect the key point data of the person;

[0016] The AlignedReID pedestrian re-identification method is adopted. A deep convolutional neural network is used to extract features from the detected human images. The Euclidean distance of the features is used as the measure of the similarity between the two images to identify the wrist joints and head positions.

[0017] Preferably, in the process of obtaining key point data, the key point data consists of nose, left eye, right eye, left shoulder, right shoulder, left elbow joint, right elbow joint, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle.

[0018] Preferably, in the process of obtaining the first probability, a first time for representing the head lowering action is obtained based on the coordinates of the key points of the head position;

[0019] Based on the coordinates of the hand key points of the wrist joint, obtaining a second time for representing the hand raising action;

[0020] Based on the continuity of each frame image corresponding to the first time and / or the second time, an image sequence corresponding to the second probability of using a mobile phone behavior is obtained, and the mobile phone ROI area and its confidence level of the image sequence are obtained through the target detection model YOLOv7 to obtain the first probability.

[0021] Preferably, in the process of obtaining the mobile phone ROI area and its confidence, the center point of each mobile phone ROI area is obtained;

[0022] Obtain the Euclidean distance from the left or right wrist closest to the center point to the center point. If the Euclidean distance is greater than twice the length of the phone's ROI rectangle, the phone is considered not to be held. Otherwise, a confidence score is generated to characterize the phone holding behavior.

[0023] Preferably, in the process of identifying whether there is abnormal use of the mobile phone in the current frame, a time series of confidences having time series characteristics is generated according to the time series characteristics of the image sequence corresponding to the confidence, and is used as the input of the time series judgment model;

[0024] According to the timing characteristics corresponding to the output image of the timing judgment module, the abnormal use of the mobile phone is obtained, wherein the time interval between adjacent output images is obtained and compared with the set threshold to determine whether the output image contains abnormal use of the mobile phone.

[0025] Preferably, in the process of generating the time series features, a time series is constructed according to the sum of the confidences of the mobile phone appearing in the current frame and the time series features, wherein the sum of the confidences is expressed as:

[0026]

[0027] Among them, S i is the sum of the confidence scores of all mobile phones appearing in the i-th frame, n is the total number of mobile phones in the current frame, C k is the confidence of the k-th mobile phone.

[0028] Preferably, in the process of using the time series as the input of the temporal judgment model, a hand and head key point coordinate matrix is ​​constructed according to the head key point coordinates and the hand key point coordinates of the image sequence;

[0029] Based on the time series, the input matrix as the input of the temporal judgment model is constructed by transposing the coordinate matrix of the hand and head key points.

[0030] Preferably, in the process of constructing the input matrix, the input matrix is ​​expressed as:

[0031]

[0032] Among them, P i T is the transpose of the coordinate matrix Pi of the key points of the hand and head in the i-th frame, S i is the sum of the confidence scores of all mobile phones appearing in the i-th frame, and Input represents the input matrix.

[0033] The present invention discloses the following technical effects:

[0034] The present invention realizes the recognition of abnormal mobile phone usage behavior in low-resolution monitoring scenarios, and improves the applicability of intelligent recognition technology in low-resolution scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0036] Figure 1 Schematic diagram of the method of the present invention. DETAILED DESCRIPTION

[0037] To make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0038] like Figure 1 As shown, the present invention provides a method for identifying mobile phone usage behavior in a low-resolution monitoring scenario, comprising the following steps:

[0039] Collect low-resolution video images and analyze them frame by frame. Use the AlphaPose posture recognition model to identify the wrist joints and head positions in the analyzed low-resolution video images.

[0040] Based on the object detection model YOLOv7, the first probability of the phone being held is obtained by obtaining the phone's ROI area and its confidence level;

[0041] Based on the first probability, according to the wrist joint and head position, a LSTM-based timing judgment model is used to identify whether there is abnormal use of the mobile phone in the current frame. Among them, abnormal use of the mobile phone is used to indicate several behaviors that are prohibited from using the mobile phone in the current frame.

[0042] The low resolution mentioned in the present invention is used to indicate that the display screen or displayed image is distorted or unclear relative to the human eye. The resolution below a certain pixel value can be summarized as low resolution; it can also be understood that the resolution corresponding to the image that cannot be recognized or accurately recognized by the human eye is low resolution. The purpose of low resolution is to indicate that a certain feature in the current screen or image cannot be recognized or accurately recognized by the human eye.

[0043] Further preferably, in the process of parsing the low-resolution video image frame by frame, the present invention converts the parsed color coding format into RGB format, and scales the image size to 640*480, wherein, when the image is scaled, the aspect ratio is maintained until the length or width after scaling is exactly equal to the target size, and when the image pixel is smaller than the target image pixel, the blank part of the target size RGB three channels are filled with 0 respectively.

[0044] Further preferably, in the process of obtaining the wrist joint and head position, the present invention is based on the AlphaPose posture recognition model trained by the MSCOCO dataset, and the present invention obtains the ROI area and key point data of the person in the parsed low-resolution video image, wherein the AlphaPose posture recognition model trained by the MSCOCO dataset is used to detect the key point data of the person;

[0045] The AlignedReID pedestrian re-identification method is adopted. A deep convolutional neural network is used to extract features from the detected human images. The Euclidean distance of the features is used as the measure of the similarity between the two images to identify the wrist joints and head positions.

[0046] Further preferably, in the process of obtaining key point data of the present invention, the key point data mentioned in the present invention consists of nose, left eye, right eye, left shoulder, right shoulder, left elbow joint, right elbow joint, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle.

[0047] Further preferably, in the process of obtaining the first probability, the present invention obtains a first time for representing the head-lowering action based on the coordinates of the key points of the head position;

[0048] Based on the coordinates of the hand key points of the wrist joint, the present invention obtains a second time for representing the hand-raising action;

[0049] Based on the continuity of each frame image corresponding to the first time and / or the second time, an image sequence corresponding to the second probability of using a mobile phone behavior is obtained, and the mobile phone ROI area and its confidence level of the image sequence are obtained through the target detection model YOLOv7 to obtain the first probability.

[0050] Further preferably, in the process of obtaining the mobile phone ROI area and its confidence, the present invention obtains the center point of each mobile phone ROI area;

[0051] The present invention obtains the Euclidean distance from the left or right wrist closest to the center point to the center point. If the Euclidean distance is greater than twice the length of the phone's ROI rectangular box, it is considered that the phone is not being held. Otherwise, a confidence level is generated to characterize the behavior of holding the phone.

[0052] Further preferably, in the process of identifying whether there is abnormal use of the mobile phone in the current frame, the present invention generates a time series of confidences having time series characteristics according to the time series characteristics of the image sequence corresponding to the confidence, as input to the time series judgment model;

[0053] According to the timing characteristics corresponding to the output image of the timing judgment module, the abnormal use of the mobile phone is obtained, wherein the time interval between adjacent output images is obtained and compared with the set threshold to determine whether the output image contains abnormal use of the mobile phone.

[0054] Further preferably, in the process of generating the time series features, the present invention constructs a time series according to the sum of the confidences of the mobile phone appearing in the current frame and the time series features, wherein the sum of the confidences is expressed as:

[0055]

[0056] Among them, S i is the sum of the confidence scores of all mobile phones appearing in the i-th frame, n is the total number of mobile phones in the current frame, C k is the confidence of the k-th mobile phone.

[0057] Further preferably, in the process of using the time series as the input of the temporal judgment model, the present invention constructs a hand and head key point coordinate matrix based on the head key point coordinates and the hand key point coordinates of the image sequence;

[0058] Based on the time series, the input matrix as the input of the temporal judgment model is constructed by transposing the coordinate matrix of the hand and head key points.

[0059] Further preferably, in the process of constructing the input matrix of the present invention, the input matrix mentioned in the present invention is expressed as:

[0060]

[0061] Among them, P i T is the transpose of the coordinate matrix Pi of the key points of the hand and head in the i-th frame, S i is the sum of the confidence scores of all mobile phones appearing in the i-th frame, and Input represents the input matrix.

[0062] The present invention also discloses a recognition system for implementing a recognition method using mobile phone behavior in a low-resolution monitoring scenario, comprising:

[0063] The data acquisition and processing module is used to collect low-resolution video images for frame-by-frame analysis, and identify the wrist joints and head positions in the analyzed low-resolution video images using the AlphaPose posture recognition model;

[0064] The first recognition module is used to obtain a first probability that the mobile phone is being held by obtaining the mobile phone ROI area and its confidence based on the object detection model YOLOv7;

[0065] The second recognition module is used to identify whether there is abnormal use of the mobile phone in the current frame based on the first probability, according to the wrist joint and head position, through an LSTM-based timing judgment model, where the abnormal use of the mobile phone is used to indicate several behaviors that are prohibited from using the mobile phone in the current frame.

[0066] The present invention discloses a computer program and a removable storage device. The computer program implements a method for identifying the use of mobile phones in low-resolution monitoring scenarios, and forms executable software that is embedded in an intelligent device to assist in monitoring abnormal use of mobile phones in low-resolution monitoring scenarios. The removable storage device mentioned in the present invention is used to carry a system for identifying the use of mobile phones in low-resolution monitoring scenarios, and to interact with existing low-resolution monitoring scene devices to determine whether there is abnormal use of mobile phones in the scenario.

[0067] The present invention provides a method for identifying mobile phone usage behavior in low-resolution surveillance scenarios, which specifically includes the following technical processes:

[0068] (1) Obtain the video and perform parsing and preprocessing:

[0069] (11) Use OpenCV to obtain and parse the H264 encoded real-time video transmitted by the camera through the RTSP protocol.

[0070] (12) The parsed color encoding format is converted from BGR to RGB format, and the image size is scaled to 640*480. The aspect ratio is maintained when the image is scaled until the length or width after scaling is exactly equal to the target size. If the image pixels are smaller than the target image pixels, the blank part of the target size is filled with 0 in the three RGB channels.

[0071] (2) Use the posture recognition model to identify the wrist joint and head position of each frame in the video sequence:

[0072] (21) Recognition points of the posture recognition model: The AlphaPose posture recognition model is trained on the MSCOCO dataset and detects 17 key points of the human body, which are, in order, nose, left eye, right eye, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle. The processed RGB image is input into the AlphaPose model to obtain the ROI (region of interest) of each person in the image and the data of each person's 17 key points.

[0073] (22) The pedestrian re-identification method is used to determine the association between the previous and next postures: the AlignedReID pedestrian re-identification method is used to extract features from the detected human images using a deep convolutional neural network, and the Euclidean distance of the features is used as a measure of the similarity between the two images.

[0074] (23) Separate the key points of the hand and head: the key points of the head are numbered 0 to 4, and the key points of the hand are numbered 10 and 11. Take the coordinates of the key points of the head and the key points of the hand, that is, the coordinate matrix of the key points of the hand and the head in a single image is P = [kp_0_x, kp_0_y, kp_1_x, kp_1_y, kp_2_x, kp_2_y, kp_3_x, kp_3_y, kp_4_x, kp_4_y, kp_10_x, kp_10_y, kp_11_x, kp_11_y], where kp_i_x and kp_i_y are the length and width coordinates of the i-th key point respectively. If the key point position is not detected in the picture, the key point coordinates are set to (-1, -1).

[0075] (24) Divide the time series segments of the action:

[0076] Considering the correlation between the actions before and after, the hand and head movements of the same person in a time series are combined to determine whether there is any phone call or answering behavior. Assume that the time taken to raise the hand and lower the head is M seconds, and the camera has F frames per second, then the time series length N = M*F. Generally, M = 2, F = 25. For continuous video, detect the coordinates of the key points of the hand and head of the same person in each frame. If the current frame does not belong to any time series and the hand and head key points are detected in the current frame, set it as the starting frame Pi of the new time series, and then the time series of the action is obtained. T = [P i T , P i+1 T , P i+2 T ,…P i+N T ], and personnel ID, where P i T The coordinate matrix P of the key points of the hand and head in the i-th frame i The transpose of .

[0077] (3) Using the target detection model, detect the probability of holding a mobile phone in each frame:

[0078] (31) Use the object detection model to detect mobile phones: Use the official YOLOv7 model and set it to detect only mobile phones. Set the parameter class to 67, the confidence level conf-thres to 0.2, and the intersection-over-union threshold iou-thres to 0.4. The model inputs the RGB image preprocessed in step (12) and obtains the ROI area and confidence level of each mobile phone in the image.

[0079] (32) Excluding the case where the mobile phone is not held: Calculate the Euclidean distance between the center point of each mobile phone ROI area and the nearest left wrist or right wrist detected by AlphaPose. If the distance is greater than twice the length of the mobile phone ROI rectangle, it is considered that the mobile phone is not held. Otherwise, the confidence C of holding the mobile phone is obtained.

[0080] (33) Calculate the confidence of all mobile phones appearing in the current frame: S i is the sum of the confidence scores of all mobile phones appearing in the i-th frame, n is the total number of mobile phones in the current frame, C k is the confidence of the k-th mobile phone.

[0081] (4) The wrist joint, head position, and probability of holding a mobile phone are input into the LSTM-based temporal judgment model to determine whether there is illegal use of the mobile phone:

[0082] (41) The time series of actions and confidence of each person ID is used as the input of the LSTM model: Among them, P i T is the transpose of the coordinate matrix Pi of the key points of the hand and head in the i-th frame, S i is the sum of the confidence scores of all mobile phones appearing in the i-th frame. The input matrix size is (15,50);

[0083] (42) Use the trained LSTM model for inference: set the step size timestep = 30, and output the output to determine whether there is a call or answering action, with a size of (1, 21). If there is a call or answering action, output 1, otherwise output 0.

[0084] (43) Determine the number of phone calls and duplicates to reduce repeated alarms: According to the personnel ID classification, the output sequence of all the outputs of the ID is recorded in the order of the video frames as L. If the pth number L of the sequence L p =1, that is, L p If a mobile phone is used at a specific moment, and if a result of 1 is found in the sequence L for that ID within 3 seconds, it is considered a continuous phone call and no alert is issued. If not, it is considered a new violation and an alert is issued. Each person ID must be individually evaluated to avoid missing an alert for the current user due to someone else using their phone.

[0085] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0086] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0087] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for identifying mobile phone usage behavior in low-resolution surveillance scenarios, characterized in that: The following steps are involved: Collect low-resolution video images and analyze them frame by frame, and use the AlphaPose posture recognition model to identify the wrist joints and head positions in the analyzed low-resolution video images; Based on the object detection model YOLOv7, the first probability of the phone being held is obtained by obtaining the phone's ROI area and its confidence level; Based on the first probability, according to the wrist joint and the head position, a LSTM-based timing judgment model is used to identify whether there is abnormal use of the mobile phone in the current frame, wherein the abnormal use of the mobile phone is used to indicate several behaviors that prohibit the use of the mobile phone in the current frame.

2. The method for identifying mobile phone usage behavior in low-resolution surveillance scenarios according to claim 1, characterized in that: In the process of parsing low-resolution video images frame by frame, the parsed color coding format is converted into RGB format, and the image size is scaled to 640*480. When the image is scaled, the aspect ratio is maintained until the length or width after scaling is exactly equal to the target size. When the image pixels are smaller than the target image pixels, the blank part of the target size is filled with 0 in the three RGB channels.

3. The method for identifying mobile phone usage behavior in low-resolution surveillance scenarios according to claim 2, characterized in that: In the process of obtaining the wrist joint and head position, based on the AlphaPose posture recognition model trained by the MSCOCO dataset, obtaining the ROI area and key point data of the person in the parsed low-resolution video image, wherein the AlphaPose posture recognition model trained by the MSCOCO dataset is used to detect the key point data of the person; The AlignedReID pedestrian re-identification method is adopted to extract features from the detected human body image using a deep convolutional neural network. The Euclidean distance of the features is used as a measure of the similarity between the two images to identify the wrist joint and the head position.

4. The method for identifying mobile phone usage behavior in low-resolution surveillance scenarios according to claim 3, characterized in that: In the process of obtaining key point data, the key point data consists of nose, left eye, right eye, left shoulder, right shoulder, left elbow joint, right elbow joint, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle.

5. The method for identifying mobile phone usage behavior in low-resolution surveillance scenarios according to claim 4, characterized in that: In the process of obtaining the first probability, a first time for indicating the head lowering action is obtained based on the coordinates of the head key points of the head position; Based on the hand key point coordinates of the wrist joint, obtaining a second time for representing a hand-raising action; Based on the continuity of each frame image corresponding to the first time and / or the second time, an image sequence corresponding to the second probability of using a mobile phone behavior is obtained, and through the target detection model YOLOv7, the mobile phone ROI area of ​​the image sequence and its confidence are obtained to obtain the first probability.

6. The method for identifying mobile phone usage behavior in low-resolution surveillance scenarios according to claim 5, characterized in that: In the process of obtaining the mobile phone ROI area and its confidence, obtaining the center point of each mobile phone ROI area; Obtain the Euclidean distance from the left wrist or right wrist closest to the center point to the center point. If the Euclidean distance is greater than twice the length of the phone ROI rectangular box, it is considered that the phone is not being held. Otherwise, generate the confidence level for characterizing the behavior of holding the phone.

7. The method for identifying mobile phone usage behavior in low-resolution surveillance scenarios according to claim 6, characterized in that: In the process of identifying whether there is abnormal use of the mobile phone in the current frame, generating a time series of the confidence levels having the time series features according to the time series features of the image sequence corresponding to the confidence levels as input to the time series judgment model; The abnormal use of the mobile phone is obtained according to the timing characteristics corresponding to the output image of the timing judgment module, wherein the time interval between adjacent output images is obtained and compared with a set threshold to determine whether the output image contains the abnormal use of the mobile phone.

8. The method for identifying mobile phone usage behavior in low-resolution surveillance scenarios according to claim 7, characterized in that: In the process of generating the time series features, the time series is constructed according to the sum of the confidences of the mobile phone appearing in the current frame and the time series features, wherein the sum of the confidences is expressed as: Among them, S i is the sum of the confidence scores of all mobile phones appearing in the i-th frame, n is the total number of mobile phones in the current frame, C k is the confidence of the k-th mobile phone.

9. The method for identifying mobile phone usage behavior in low-resolution surveillance scenarios according to claim 8, characterized in that: In the process of using the time series as the input of the temporal judgment model, constructing a hand and head key point coordinate matrix according to the head key point coordinates and the hand key point coordinates of the image sequence; Based on the time series, an input matrix serving as the input of the timing judgment model is constructed by transposing the hand and head key point coordinate matrix.

10. The method for identifying mobile phone usage behavior in low-resolution surveillance scenarios according to claim 9, characterized in that: In the process of constructing the input matrix, the input matrix is ​​expressed as: Among them, P i T The coordinate matrix P of the key points of the hand and head in the i-th frame i The transpose of S i is the sum of the confidence scores of all mobile phones appearing in the i-th frame, and Input represents the input matrix.

Images