Driver passenger limb conflict behavior identification method and electronic equipment

Abstract

The invention discloses a driver passenger limb conflict behavior identification method and electronic equipment. The method comprises the following steps: S1, obtaining a monitoring picture of a cab; s2, detecting to obtain an arm detection frame in the monitoring picture; s3, detecting in all the arm detection frames to obtain an arm detection frame of the driver; and S4, when it is detected that the arm detection frame of a non-driver makes contact with the arm detection frame of the driver, it is judged that the passenger and the driver have limb conflicts. The method does not need multiple frames of pictures, can give an alarm once contact occurs, and has the performance of low delay. Compared with a video classification model, the detection model occupies less memory, is convenient to deploy in a vehicle-mounted machine, distinguishes the arms of a driver and a passenger when the driver and passenger limb conflict behaviors occur, and triggers an alarm only when the passenger makes contact with the arms of the driver, thereby being more accurate.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a method and electronic equipment for identifying physical conflict between a driver and a passenger. Background Art

[0002] The bus is one of the common means of transportation for people's daily travel, which brings a lot of convenience to life, but physical conflicts between drivers and passengers on the bus occasionally occur. When the bus is driving, physical conflicts can easily cause traffic accidents and casualties. Real-time identification of physical conflicts between drivers and passengers can assist staff to call the police and make emergency stops in the first time to prevent the situation from further deteriorating. For this reason, some people use visual detection methods such as invention patent CN201780002312.8 to provide an alarm method for physical conflict behavior, which is used to solve the problem of how to achieve safe and effective timely alarms in the event of a physical conflict with the driver. The alarm method for physical conflict behavior includes: real-time acquisition of a video image of the driver's position on the vehicle, the video image including the scene of the driver's upper body; converting the video image into a specified data format; using the converted video image as input to a pre-trained convolutional neural network to obtain the output result of the convolutional neural network, the output result is the presence of physical conflict behavior or the absence of physical conflict behavior; if the output result is the presence of physical conflict behavior, an alarm message is issued. Invention patent CN202110737698.5 discloses a method, device, equipment and medium for identifying conflict behaviors of people in a vehicle cabin. The method comprises: obtaining multiple video frame images in a video stream in the vehicle cabin; performing feature extraction on the multiple video frame images respectively to obtain multiple image features corresponding to the multiple video frame images, wherein the image features represent the action features of the multiple people in the vehicle cabin; obtaining the action feature sequences of the multiple people in the vehicle cabin corresponding to the multiple video frame images based on the multiple image features; identifying the conflict behaviors between the multiple people in the vehicle cabin based on the action feature sequences, and obtaining the identification results of the conflict behaviors between the multiple people in the vehicle cabin.

[0003] However, there are many difficulties in real-time identification of physical conflicts between drivers and passengers. First, physical conflicts between drivers and passengers usually require the extraction of spatiotemporal features based on the video, and then whether a physical conflict has occurred based on these features. However, the extraction of spatiotemporal features takes a long time, and it is difficult to complete the alarm in the first time. Secondly, the extraction of spatiotemporal features is easily affected by background interference, and it is difficult to achieve good results when the camera is installed at different angles. Passengers in the picture are often misjudged for scanning codes, greeting, crowding, etc. as conflicts between drivers and passengers. Moreover, when the driver's arm is grabbed or the steering wheel is grabbed during driving, in order to ensure safety, there will be no large movements, and it is difficult to extract the characteristics of physical conflict. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a driver-passenger physical conflict behavior recognition method and electronic device. The purpose of the present invention is achieved through the following technical solutions:

[0005] A method for identifying driver-passenger physical conflict behavior comprises the following steps:

[0006] S1. Obtain monitoring pictures of the cab;

[0007] S2, detect and obtain the arm detection frame in the monitoring image;

[0008] S3, detecting the driver's arm detection frame from all arm detection frames;

[0009] S4. When it is detected that the arm detection frame of a person other than the driver is in contact with the arm detection frame of the driver, it is determined that a physical conflict has occurred between the passenger and the driver.

[0010] Further improvement: In step S3, when the vehicle is initially started and there are only two arm detection frames in the initial monitoring picture, the two arms are identified as the driver's arms.

[0011] Further improvement: In step S3, the driver's arm detection frame is obtained by tracking the driver's arm detection frame during the driving of the carriage. The specific tracking method is as follows:

[0012] S31, predicting the position information of the driver's arm detection frame in the current frame monitoring picture based on the position and speed information of the driver's arm detection frame in the previous frame monitoring picture;

[0013] S32, determining whether the position information of each arm detection frame of the current frame matches the predicted position information of the driver's arm frame of the current frame. The arm detection frame that matches is the driver's arm detection frame, and the arm detection frame that does not match is the passenger's arm detection frame.

[0014] As a further improvement, in step S31, the position of the driver's arm frame in the next frame is predicted using the Kalman filtering method.

[0015] Further improvement, the Kalman filtering method is as follows:

[0016] Get [x i 1,x i 2]; among them and x i 2 respectively represent the position and speed information of the first arm frame and the second arm frame of the driver in the i-th frame image; c=1 or 2, cx, cy, w, h represent the x-axis coordinate, y-axis coordinate, width and height of the center point of the driver's arm frame, respectively; vx, vy, vw, vh represent the change speed of the x-axis coordinate, y-axis coordinate of the center point of the driver's arm frame, and the change speed of the width and height of the driver's arm frame, respectively; if it is the first appearance of the driver's arm frame, each change speed value is initialized to 0; then prediction:

[0017]

[0018]

[0019] F is called the state transfer matrix, which is specifically expressed as:

[0020]

[0021] dt represents the derivative based on time t; c=1 or 2 means based on Predicted position and velocity information of the driver’s arm frame in the next frame.

[0022] A further improvement is to determine whether the arm detection box of the current frame matches the predicted driver arm box of the current frame.

[0023] Further improvement is to obtain the cost matrix of the arm detection frame in the current frame image and the predicted driver arm frame in the current frame, and then update the position and speed information of the predicted driver arm frame in the current frame to obtain the updated predicted driver arm frame, and match the arm detection frame in the current frame image with the predicted driver arm frame. The matched one is the driver arm frame, otherwise it is the passenger arm frame.

[0024] A further improvement, the Hungarian algorithm is as follows:

[0025] Detect n arm detection frames from the i+1th frame image [detection i+1 1,...,detection i+1 n ], compare the position and speed information of n arm detection frames with the predicted and Calculate the distance between each pair to obtain the cost matrix:

[0026] Predict the driver's arm frame and obtain the cost matrix:

[0027]

[0028] in Indicates the arm detection frame Driver arm box with prediction distance;

[0029] In the i+1th frame image, Minimum two arm detection frame

[0030] detection i+1 a ,detection i+1 b They are used as the arm detection frames of two pending drivers respectively;

[0031] The determined driver's arm detection frame is updated based on the pending driver's arm detection frame:

[0032] First, get the covariance of all arm frames in the i+1th frame image

[0033] P i+1 =FP i F T +Q

[0034] Where P i is the covariance of all arm frames in the i-th frame image, Q is the noise matrix of the system, which is randomly initialized to a value between (0,1); T represents the matrix transpose;

[0035] Calculate the Kalman gain K:

[0036] S=HP i+1 H T +R

[0037] K=P i+1 H T S -1

[0038] H is the transfer matrix, which changes the predicted arm box from 8 dimensions to 4 dimensions.

[0039]

[0040] R is the noise matrix

[0041]

[0042] Update the predicted arm box and covariance P i+1 :

[0043]

[0044]

[0045] new_P i+1 =(I-KH)P i+1

[0046] y represents the arm detection frame of one of the pending drivers. Indicates update The position and speed information of the next driver arm frame, new_P i+1 Indicates update The new covariance is obtained; similarly, the position and speed information of the other driver's arm frame is obtained

[0047] Extract separately and The x-axis coordinate, y-axis coordinate, width and height of the center point of the driver's arm frame in the track are obtained. i+1 1 and track i+1 2; detect the arm frame in the i+1th frame image i+1 1,...,detection i+1 n ] and track i+1 1 and track i+1 2 are matched separately. The driver's arm frame may be matched, but the passenger's arm frame cannot be matched.

[0048] As a further improvement, in step S4, if the intersection area of ​​the non-driver's arm detection frame and the driver's arm detection frame is greater than 0, it is determined that a physical conflict has occurred; if the intersection area is equal to 0, it is determined that no physical conflict has occurred.

[0049] For further improvement, the area is calculated as follows:

[0050] Obtain the horizontal coordinate rx1 and the vertical coordinate ry1 of the upper left corner point of the driver arm detection frame, the horizontal coordinate lx1 and the vertical coordinate ly1 of the lower right corner point, and the horizontal coordinate rx2 and the vertical coordinate ry2 of the upper left corner point of the non-driver arm detection frame, and the horizontal coordinate lx2 and the vertical coordinate ly2 of the lower right corner point in the i+1th frame image;

[0051] Assume that there is an intersecting area, and the upper left corner of the intersecting area has a horizontal coordinate ax and a vertical coordinate ay, and the lower right corner has a horizontal coordinate bx and a vertical coordinate by;

[0052] Then we have:

[0053] ax=max(lx1,lx2),ay=max(ly1,ly2),bx=min(rx1,rx2),by=min(ry1,ry2)

[0054] max() means taking the maximum value, min means taking the minimum value;

[0055] Calculate the width w and height hw of the intersection area = bx-ax

[0056] h=by-ay

[0057] Then we have:

[0058] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.

[0059] The beneficial effects of the present invention are:

[0060] First, the present invention does not require multiple frames of images, can alarm once contact occurs, and has low-latency performance.

[0061] Second: The detection model takes up less memory than the video classification model and is easier to deploy on the vehicle.

[0062] Third: When there is a physical conflict between the driver and the passenger, the present invention distinguishes between the driver's and the passenger's arms. The alarm is triggered only when the passenger's arm comes into contact with the driver's arm, which is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The present invention is further described with reference to the accompanying drawings, but the contents in the accompanying drawings do not constitute any limitation to the present invention.

[0064] Figure 1 It is a schematic diagram of the process of the present invention;

[0065] Figure 2 Example of an arm detection box Figure 1 ;

[0066] Figure 3 Example of an arm detection box Figure 2 ;

[0067] Figure 4 This is a signal for the driver's arm to touch your arm. Figure 1 ;

[0068] Figure 5 This is a signal for the driver's arm to touch your arm. Figure 2 ;

[0069] Figure 6 A sign that no physical conflict occurred Figure 2 ;

[0070] Figure 7 A sign that no physical conflict occurred Figure 3 ;

[0071] Figure 8 A sign that no physical conflict occurred Figure 4 ;

[0072] Fig. 9 It is a schematic diagram of the intersection area being greater than 0;

[0073] Fig.10 This is a schematic diagram showing that the area of ​​the intersection region is equal to 0. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solutions and advantages of the invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and examples.

[0075] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0076] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first" and "second" are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0077] In this application, the word "exemplary" is used to mean "serving as an example, illustration, or description". Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any technician in the field to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0078] The present invention provides a method for identifying driver-passenger physical conflict behavior. The overall process of the present invention is as follows: Figure 1The specific principles of the present invention are as follows:

[0079] First: read the real-time video stream, obtain the picture frame, scale and normalize the picture frame, and obtain a monitoring image Img with a size of 640x640 i Since the palm may be blocked during physical conflict and cannot be detected, the image is sent to the yolov7 model for detecting the palm and forearm, and only the arm is detected, and n arm detection frames are detected [detection1,...,detection n ], where detection = [cx, cy, w, h], cx, cy, w, h represent the horizontal coordinate of the center point of the arm detection frame, the vertical coordinate of the center point of the arm detection frame, the width of the arm detection frame, and the height of the arm detection frame, respectively. Figure 2 , Figure 3 shown.

[0080] Second: Because the driver's arm may come into contact with your own arm, such as Figure 4 , Figure 5 Therefore, the Kalman filter and Hungarian algorithm are introduced to track the driver's arm to distinguish the driver's arm from the passenger's arm, thereby filtering out the misjudgment caused by the contact between the driver's arm and the passenger's arm.

[0081] Traditional filtering methods can only be implemented under the condition that the useful signal and noise have different frequency bands. In the 1940s, N. Wiener and AH Kolmogorov introduced the statistical properties of signals and noise into filtering theory. Under the assumption that both the signal and the noise are stationary processes, the true value of the signal is estimated using the optimization method to achieve the filtering purpose, which is conceptually linked to the traditional filtering method and is called Wiener filtering. This method requires that both the signal and the noise must be stationary processes. In the early 1960s, Kalman (RE Kalman) and Busey (RS Bucy) published an important paper "New Results in Linear Filtering and Prediction Theory", proposing a new linear filtering and prediction theory, known as Kalman filtering. The characteristic is that the noisy input and observation signals are processed on the basis of linear state space representation to obtain the system state or true signal.

[0082] This theory is expressed in the time domain. The basic concept is: based on the state space representation of the linear system, the optimal estimate of the system state is obtained from the output and input observation data. The system state mentioned here is the set of minimum parameters that summarizes the effects of all past inputs and disturbances on the system. Knowing the state of the system can determine the entire behavior of the system together with future inputs and disturbances to the system.

[0083] Kalman filtering does not require the assumption that both the signal and the noise are stationary processes. For the system disturbance and observation error (i.e., noise) at each moment, as long as some appropriate assumptions are made on their statistical properties, by processing the observed signal containing noise, the estimated value of the real signal with the smallest error can be obtained in the average sense. Therefore, since the advent of the Kalman filter theory, it has been applied in many departments such as communication systems, power systems, aerospace, environmental pollution control, industrial control, radar signal processing, etc., and many successful application results have been achieved. For example, in image processing, Kalman filtering is used to restore images blurred due to certain noise influences. After making certain assumptions on the statistical properties of the noise, Kalman's algorithm can be used to recursively obtain the real image with the smallest mean square error from the blurred image, so that the blurred image can be restored. The present invention uses Kalman filtering to track and predict the arm detection frame, so it is different from its general use.

[0084] The Hungarian algorithm is a combinatorial optimization algorithm for solving the task allocation problem in polynomial time. It was proposed by American mathematician Harold Kuhn in 1955. The algorithm is called the Hungarian algorithm because a large part of the algorithm is based on the previous work of Hungarian mathematicians Dénes Knig and Jen Egerváry. It is mainly used in the field of operations research to solve the task allocation problem.

[0085] If in a match, |V1|≤|V2| and the number of matches |M|=|V1|, then this match is called a complete match, also known as a perfect match. In particular, when |V1|=|V2|, it is called a perfect match.

[0086] Tracking the driver's arms means assuming that the yolov7 model detects the driver's two arm frames for the i-th frame. i 1, detection i 2], detect n arm frames for the i+1th frame [detection i +1 1,...,detection i+1 n ], n>=2, how to find the position of the driver's two arm frames in n arm frames, the specific situation is as follows: detection i 1 represents the position information of the first driver's arm in the i-th frame (cx, cy, w, h), detection i 2 represents the second driver arm (cx, cy, w, h) in the i-th frame.

[0087] 1. Use Kalman filtering to predict the positions of the two driver arm frames in the current frame [tracki+1 1,track i+1 2], divided into three stages: track i+1 1 means predicting the position of one of the driver's arm frames in the current frame, track i+1 2] indicates predicting the position of another driver's arm frame in the current frame.

[0088] (1) Based on the two driver arm frames in the previous frame of the target [detection i 1, detection i 2] position information to predict the position of the driver's arm frame in the next frame [x i+1 1,x i+1 2].

[0089] First, the position information of the driver's arm frame in the current frame (i.e., the i-th frame) is detected. i 1, detection i 2] expands to [x i 1,x i 2], where Contains the position and speed information of the first driver arm frame in the i-th frame, which is composed of an 8-dimensional vector and is expressed as cx, cy, w, h represent the driver arm frame detection i 1, the horizontal coordinate of the center point, the vertical coordinate of the center point, the width of the driver's arm frame and the height of the driver's arm frame, vx, vy, vw, vh represent the x-axis change speed of the center point coordinate of the driver's arm frame, the y-axis change speed of the center point coordinate of the driver's arm frame, the change speed of the width of the driver's arm frame and the change speed of the height of the driver's arm frame respectively. If it is the first time that the driver's arm frame appears, each speed value is initialized to 0.

[0090] The formula for Kalman filtering is:

[0091]

[0092]

[0093] Indicates the position and speed information of the first driver's arm in the i+1th frame, represents the position and speed information of the second driver's arm in the i+1th frame;

[0094] F is called the state transfer matrix, which is specifically expressed as:

[0095]

[0096] dt represents the derivative based on time t; Indicates based on The predicted position of the driver’s arm frame in the next frame. (2) Using the Hungarian algorithm, the n arm detection frames [detection i+1 1,...,detection i+1 n ] to find the position [x] that best matches the predicted driver arm frame i+1 1,x i+1 2] Paired two arm frames [detection i+1 1, detection i+1 2]:

[0097] Calculate the distance between the n detected arm frames and the two predicted driver arm frames to obtain the cost matrix:

[0098]

[0099] in Represents the detection arm frame Driver arm box with prediction distance;

[0100] Using the Hungarian algorithm to find [detection i+1 a ,detection i+1 b ],make minimum; a represents the n arm frames in the i+1th frame that can best Matching arm frame, b represents the best matching arm frame among the n arm frames in the i+1th frame. matching arm frame;

[0101] Arm box detection based on the current frame i+1 a ,detection i+1 b ]Update the predicted driver arm frame of frame i+1 i+1 1, track i+1 2] The previous one predicts the arm frame of the driver in the i+1 frame based on the i-th frame. Here, the arm frame of the driver in the i+1 frame is updated and predicted after the i+1 frame is obtained. [detection i+1 a ,detection i+1 b ] is likely to be the driver’s, so the predicted arm frame is updated based on the positions of these two frames;

[0102] First predict the covariance of all arm boxes in the next frame

[0103] P i+1 =FPi F T +Q

[0104] Where P i is the covariance of all arm frames in the previous frame, Q is the noise matrix of the system, which is randomly initialized to a value between (0,1); T represents the matrix transpose;

[0105] Calculate the Kalman gain K

[0106] S=HP i+1 H T +R

[0107] K=P i+1 H T S -1

[0108] S represents the gain coefficient;

[0109] H is the transfer matrix, which changes the predicted arm box from 8 dimensions to 4 dimensions.

[0110]

[0111] R is the noise matrix

[0112]

[0113] Update the predicted arm box and covariance P i+1 :

[0114]

[0115]

[0116] new_P i+1 =(I-KH)P i+1

[0117] y represents the arm frame combined with the i+1th frame information, Represents the new arm frame containing position information and speed information, new_P i+1 Denote the new covariance

[0118] Extract The first four elements of get the predicted arm box track i+1 1＝ [cx,cy,w,h], track can be obtained in the same way i+1 2.

[0119] 3. Use the Hungarian algorithm to determine the arm frame of the current frame [detection i+1 1,...,detection i+1n ], whether it is consistent with the current frame driver arm frame predicted by Kalman filter [track i+1 1, track i+1 2] Matching, the one that can be matched is the driver's arm, and the one that cannot be matched is the passenger's arm.

[0120] Third: Due to the different camera installation angles, the arms of ordinary passengers may appear in the picture, such as Figure 6 ,7,8. Therefore, the intersection area is introduced to distinguish the arms that are not in physical contact, such as Fig. 9 ,10 shows two arm frame detections i+1 1=[cx1,cy1,w1,h1], detection i+1 2=[cx2,cy2,w2,h2], the steps to calculate the intersection area of ​​the two:

[0121] 1. Calculate the horizontal and vertical coordinates of the upper left and lower right corners of the two arm boxes i+1 1=[lx1,ly1,rx1,ry1],box i +1 2=[lx2,ly2,rx2,ry2],

[0122] in,

[0123] 2. Calculate the horizontal and vertical coordinates of the upper left and lower right corners of the intersection area cross = [ax, ay.bx, by]

[0124] Where ax=max(lx1,lx2),ay=max(ly1,ly2),bx=min(rx1,rx2),by=min(ry1,ry2),max means taking the maximum value,min means taking the minimum value

[0125] 3. Calculate the width w and height h of the intersecting rectangle

[0126] w=bx-ax

[0127] h=by-ay

[0128] 4. Calculate the area of ​​the intersecting rectangle

[0129]

[0130] If the intersection area>0, it is judged as physical contact.

[0131] Fourth: If a non-driver's arm comes into contact with the driver's arm, an alarm is triggered.

[0132] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for identifying physical conflict between a driver and a passenger, characterized by: The steps include: S1. Obtain monitoring pictures of the cab; S2, detect and obtain the arm detection frame in the monitoring image; S3, detecting the driver's arm detection frame from all arm detection frames; S4. When it is detected that the arm detection frame of a person other than the driver is in contact with the arm detection frame of the driver, it is determined that a physical conflict has occurred between the passenger and the driver.

2. The method for identifying driver-passenger physical conflict behavior according to claim 1, characterized in that: In step S3, when the vehicle is initially started and there are only two arm detection frames in the initial monitoring image, the two arms are identified as the driver's arms.

3. The method for identifying driver-passenger physical conflict behavior as claimed in claim 2, characterized in that: In step S3, the driver's arm detection frame is obtained by tracking the driver's arm detection frame during the driving process of the carriage. The specific tracking method is as follows: S31, predicting the position information of the driver's arm detection frame in the current frame monitoring picture based on the position and speed information of the driver's arm detection frame in the previous frame monitoring picture; S32, determining whether the position information of each arm detection frame of the current frame matches the predicted position information of the driver's arm frame of the current frame. The arm detection frame that matches is the driver's arm detection frame, and the arm detection frame that does not match is the passenger's arm detection frame.

4. The method for identifying driver-passenger physical conflict behavior as claimed in claim 3, characterized in that: In step S31, the position of the driver's arm frame in the current frame is predicted using the Kalman filter method.

5. The method for identifying driver-passenger physical conflict behavior as claimed in claim 4, characterized in that: The Kalman filter method is used to obtain the position and speed information of the driver's arm frame, so as to predict the position of the driver's arm frame.

6. The method for identifying driver-passenger physical conflict behavior according to claim 5, characterized in that: The Hungarian algorithm is used to determine whether the arm detection box of the current frame matches the predicted driver arm box of the current frame.

7. The method for identifying driver-passenger physical conflict behavior according to claim 6, characterized in that: Get the cost matrix of the arm detection frame in the current frame image and the predicted driver arm frame of the current frame, then update the position and speed information of the predicted driver arm frame of the current frame, get the updated predicted driver arm frame, match the arm detection frame in the current frame image with the predicted driver arm frame, the one that matches is the driver arm frame, otherwise it is the passenger arm frame.

8. The method for identifying driver-passenger physical conflict behavior as claimed in claim 1, characterized in that: In step S4, if the intersection area of ​​the non-driver's arm detection frame and the driver's arm detection frame is greater than 0, it is determined that a physical conflict has occurred; if the intersection area is equal to 0, it is determined that no physical conflict has occurred.

9. The method for identifying driver-passenger physical conflict behavior as claimed in claim 8, characterized in that: The area is calculated as follows: Obtain the horizontal coordinate rx1 and the vertical coordinate ry1 of the upper left corner point of the driver arm detection frame, the horizontal coordinate lx1 and the vertical coordinate ly1 of the lower right corner point, and the horizontal coordinate rx2 and the vertical coordinate ry2 of the upper left corner point of the non-driver arm detection frame, and the horizontal coordinate lx2 and the vertical coordinate ly2 of the lower right corner point in the i+1th frame image; Assume that there is an intersecting area, and the upper left corner of the intersecting area has a horizontal coordinate ax and a vertical coordinate ay, and the lower right corner has a horizontal coordinate bx and a vertical coordinate by; Then we have: ax=max(lx1,lx2),ay=max(ly1,ly2),bx=min(rx1,rx2),by=min(ry1,ry2) max() means taking the maximum value, min means taking the minimum value; Calculate the width w and height h of the intersection area w=bx-ax h=by-ay Then we have:

10. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 9.

Images