Motion detection method, motion detection system, and computer-readable storage medium

Abstract

The present application discloses an action detection method, an action detection system and a computer-readable storage medium, wherein the detection method comprises: obtaining the first key point information of each target among the multiple targets contained in the current video frame; based on the first key point information of each target and the second key point information of each target, determining the target whose action performed in each target meets the requirements of similarity with the target action as a candidate target; wherein the second key point information of a target is determined based on the key point information of a target in a historical video frame; from the current video frame, extracting the image areas containing each determined candidate target respectively, and determining the target that performs the target action based on the image areas of each determined candidate target. First using the full-image key points for pre-inspection, and then using the local key points of the precise target for secondary analysis, can not only reduce the possibility of false alarms of actions, but also improve the accuracy of action detection.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to an action detection method, an action detection system, and a computer-readable storage medium. Background Art

[0002] In the field of educational examinations, in order to ensure the fairness of the examinations, a lot of manpower and material resources are currently needed to prevent candidates from cheating, especially in national examinations such as the college entrance examination. With the rapid development of artificial intelligence technology, target behavior analysis can effectively assist examination institutions in monitoring candidates' cheating behavior, and can promptly notify invigilators when cheating occurs, so as to stop cheating in time.

[0003] At present, the commonly used method is to simply perform an abnormal analysis on the target. When it is determined that the target's limbs are different from the previous state, no in-depth analysis is performed, and it is judged whether the target is performing a cheating action. For example, if someone raises his head and stretches, the algorithm will judge that the target is performing a cheating action, which can easily cause false alarms. Summary of the invention

[0004] The main technical problem solved by the present application is to provide a motion detection method, a motion detection system and a computer-readable storage medium, which can reduce the possibility of false motion alarms.

[0005] In order to solve the above technical problems, a technical solution adopted by the present application is: to provide a cheating detection method, including: obtaining the first key point information of each target among multiple targets contained in the current video frame; based on the first key point information of each target and the second key point information of each target, determining the target whose action executed in each target meets the requirements of similarity with the target action as a candidate target; wherein the second key point information of a target is determined based on the key point information of the target in the historical video frame; from the current video frame, respectively extracting the image areas containing the determined candidate targets, and based on the image areas of the determined candidate targets, determining the target that performs the target action.

[0006] The target includes a human body, and the target action includes an action of the head of the human body moving in a preset direction; or the target action includes an action of the hand of the human body contacting with the hand of another human body.

[0007] Wherein, the target action includes an action of the human body's head moving in a preset direction, and the step of determining, based on the first key point information of each target and the second key point information of each target, a target whose action performed in each target meets the requirement of similarity with the target action as a candidate target includes: obtaining, based on the first key point information, an actual value of the head-to-shoulder aspect ratio of each target in the current video frame, and obtaining, based on the second key point information, an average value of the head-to-shoulder aspect ratio of each target in multiple historical video frames before the current video frame; and determining, in response to a first deviation degree between the actual value of the head-to-shoulder aspect ratio and the average value of the head-to-shoulder aspect ratio being greater than a first threshold, the target as a candidate target.

[0008] The first offset degree is the square of the difference between the actual value of the head-to-shoulder aspect ratio and the average value of the head-to-shoulder aspect ratio, and the first threshold is the product of the first coefficient and the variance of the head-to-shoulder aspect ratio in the multiple historical video frames.

[0009] Among them, the target action includes the action of the human body's head moving in a preset direction, and the step of determining the target whose action performed in each target meets the requirement of similarity with the target action as a candidate target based on the first key point information of each target and the second key point information of each target also includes: obtaining the actual distance between the left and right shoulder key points of each target in the current video frame based on the first key point information, and obtaining the average distance between the left and right shoulder key points of each target in multiple historical video frames before the current video frame based on the second key point information; in response to the second offset degree between the actual distance between the left and right shoulder key points and the average distance between the left and right shoulder key points being greater than a second threshold, determining the target as a candidate target.

[0010] Among them, the second offset degree is the square of the difference between the actual distance between the left and right shoulder key points and the average distance between the left and right shoulder key points, and the second threshold is the product of the second coefficient and the variance of the distance between the left and right shoulder key points in the multiple historical video frames.

[0011] The steps of respectively extracting image areas containing each determined candidate target from the current video frame, and determining the target that performs the target action based on the image areas of each determined candidate target, include: in response to the candidate target performing the action of moving the human head in a preset direction, respectively extracting first image areas containing each determined candidate target from the current video frame; wherein the first image area contains the head and shoulders of the candidate target; and determining, based on the first image area, from the current video frame and a plurality of subsequent video frames after the current video frame, a target in which a first offset between an actual value of the head-to-shoulder aspect ratio and an average value of the head-to-shoulder aspect ratio is greater than a first threshold, or a second offset between an actual distance between the left and right shoulder key points and an average distance between the left and right shoulder key points is greater than a second threshold.

[0012] Among them, the step of respectively extracting the first image area containing each determined candidate target from the current video frame includes: respectively extracting the first detection frame containing each determined candidate target from the current video frame; wherein the first detection frame contains the head and shoulders of the candidate target; and expanding the first detection frame to obtain the first image area of ​​each determined candidate target.

[0013] Among them, the target action includes the action of the hand of the human body contacting the hand of other human bodies, and based on the first key point information of each target and the second key point information of each target, the target whose action performed in each target meets the requirement of similarity with the target action is determined as a candidate target. For each of the targets, the step of using the multiple key point position information in the current video frame and the historical video frame to judge whether the current target has a cheating action also includes: obtaining the actual value of the elbow angle of the current target in the current video frame, and the average value of the elbow angle of the current target in multiple historical video frames before the current video frame; judging that the third offset degree between the actual value of the elbow angle and the average value of the elbow angle is greater than a third threshold, and determining the target as a candidate target.

[0014] The third offset degree is the square of the difference between the actual value of the elbow angle and the average value of the elbow angle, and the third threshold is the product of the third coefficient and the variance of the elbow angles in the multiple historical video frames.

[0015] Among them, the steps of respectively extracting image areas containing each determined candidate target from the current video frame, and determining the target that performs the target action based on the image areas of each determined candidate target, include: in response to the candidate target performing the action of the human body's hand contacting the hand of other human bodies, respectively extracting from the current video frame a second image area containing each determined candidate target and a third image area containing neighboring targets located around each determined candidate target; wherein the second image area contains the wrist of the candidate target, the third image area contains the wrist of the neighboring target, the number of the neighboring targets and the third image area are four respectively, and the four neighboring targets are respectively located in four directions around the candidate target; based on the second image area and the third image area, respectively obtaining the distance between the wrist of each determined candidate target and the wrist key point of the corresponding neighboring target; and determining the target whose distance is less than or equal to a preset threshold according to the distance.

[0016] Among them, the step of respectively extracting from the current video frame a second image area containing each determined candidate target, and a third image area containing neighboring targets located around each determined candidate target, comprises: respectively extracting from the current video frame a second detection frame containing each determined candidate target, and a third detection frame containing neighboring targets located around each determined candidate target; wherein the second detection frame contains the wrist of the candidate target, and the third detection frame contains the wrist of the neighboring target; expanding the second detection frame to obtain the second image area, and expanding the third detection frame to obtain the third image area.

[0017] To solve the above technical problems, another technical solution adopted in the present application is: to provide an action detection system, comprising a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the action detection method mentioned in any of the above embodiments.

[0018] In order to solve the above technical problems, another technical solution adopted in the present application is: providing a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is used to implement the action detection method mentioned in any of the above embodiments.

[0019] Different from the prior art, the beneficial effects of the present application are as follows: the cheating detection method provided by the present application includes: obtaining the first key point information of each target among the multiple targets contained in the current video frame, and based on the first key point information of each target and the second key point information of each target, determining the target whose action performed in each target meets the requirements of similarity with the target action as a candidate target; wherein the second key point information of a target is determined based on the key point information of a target in the historical video frame; from the current video frame, respectively extracting the image areas containing the determined candidate targets, and based on the image areas of the determined candidate targets, determining the target that performs the target action. Through such a design method, the whole-image key points are first used for pre-inspection, and then the local key points are used for secondary analysis, and the overall time consumption and effect can be guaranteed, which can not only reduce the possibility of false alarms of actions, but also improve the accuracy of action detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0021] Figure 1 It is a flowchart of an implementation method of the motion detection method of the present application;

[0022] Figure 2 yes Figure 1 A schematic flow chart of an implementation method of step S2 in FIG.

[0023] Figure 3 yes Figure 1 A schematic flow chart of another implementation of step S2;

[0024] Figure 4 yes Figure 1 A schematic flow chart of another implementation method of step S2;

[0025] Figure 5 yes Figure 1 A schematic flow chart of an implementation method of step S3 in FIG.

[0026] Figure 6 yes Figure 5 A schematic flow chart of an implementation method of step S40;

[0027] Figure 7 yes Figure 1 A schematic flow chart of another implementation of step S3;

[0028] Figure 8 yes Figure 7 A schematic diagram of a flow chart of an implementation method of step S50;

[0029] Fig. 9 It is a structural schematic diagram of an implementation method of the motion detection system of the present application;

[0030] Fig.10 It is a schematic diagram of a framework of an implementation method of an action detection system of the present application;

[0031] Fig.11 It is a schematic diagram of a framework of an implementation scheme of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] See also Figure 1 , Figure 1 1 is a flow chart of an embodiment of an action detection method of the present application. The action detection method comprises:

[0034] S1: Obtain the first key point information of each target among multiple targets contained in the current video frame.

[0035] Specifically, in this embodiment, a panoramic camera is installed in the middle of the examination room or classroom that needs to be monitored. When all candidates enter the examination room and take their seats, the algorithm analysis device receives the real-time video stream of the panoramic camera and starts the analysis. After the algorithm receives the real-time video stream, it obtains the first key point information of each of the multiple targets contained in the current video frame. In this embodiment, the multiple key points include key points of the human body such as the head, shoulders, elbows, and wrists.

[0036] Specifically, in this embodiment, in the process of obtaining the first key point information, the head-shoulder frame coordinates of each target can also be obtained. Of course, in other embodiments, the head-shoulder frame coordinates can also be determined based on the head key point, the left shoulder key point and the right shoulder key point, which is not limited in this application. Specifically, assuming that there are M examinees in the examination room or classroom, the left shoulder key point position of the i-th examinee in the j-th frame is Right shoulder key point position Left elbow key point position Right elbow key point position Left wrist key point position Right wrist key point position The width of the head and shoulder frame is obtained according to the coordinates of the head and shoulder frame: The height of the head and shoulders frame is Of course, in addition to the above key point position information, other key point position information can also be obtained, which will not be repeated here.

[0037] S2: Based on the first key point information of each target and the second key point information of each target, targets whose actions executed in each target meet the requirements of similarity with the target action are determined as candidate targets.

[0038] Specifically, the second key point information of a target is determined based on the key point information of a target in the historical video frame. In this embodiment, the target includes a human body, and the target action includes the action of the human head moving in a preset direction, or the target action includes the action of the human hand contacting the hand of another human body. Of course, in other embodiments, the target can also be other, and the target action can also be other, which is not limited in this application. Since students are bowing their heads to answer questions or looking up to think about problems in a normal examination state, we believe that there is a possibility of suspected cheating for the action of the human head moving in a preset direction, the action of the human hand contacting the hand of another human body, etc., so we first perform a preliminary inspection analysis on each target in the video frame. When the target looks back, the aspect ratio of the head and shoulders will change to a certain extent, and the left and right shoulder key points of the human body will also change accordingly; when the target passes an object, the positional relationship between the three key points of the shoulder, elbow and wrist of the arm will become obvious; based on these change information, we design three models to determine the target whose action performed by each target meets the requirements of the similarity between the target action and the target action as a candidate target.

[0039] In one embodiment, when the target action includes the action of the human head moving in a preset direction, refer to Figure 2 , Figure 2 yes Figure 1 Schematic diagram of a flow chart of an implementation method of step S2 in FIG. Specifically, step S2 specifically includes:

[0040] S10: obtaining an actual value of the head-to-shoulder aspect ratio of each target in the current video frame based on the first key point information, and obtaining an average value of the head-to-shoulder aspect ratio of each target in multiple historical video frames before the current video frame based on the second key point information.

[0041] Specifically, the actual value of the head-to-shoulder aspect ratio of each target in the current video frame is The average head-to-shoulder aspect ratio of each target in multiple historical video frames before the current video frame is

[0042] S11: Determine whether a first deviation between the actual value of the head-to-shoulder aspect ratio and the average value of the head-to-shoulder aspect ratio is greater than a first threshold.

[0043] Specifically, in this embodiment, the first offset degree is the square of the difference between the actual value of the head-to-shoulder aspect ratio and the average value of the head-to-shoulder aspect ratio: The variance of the head-to-shoulder aspect ratio in multiple historical video frames is The first threshold is the product of the first coefficient and the variance of the head-to-shoulder aspect ratio in multiple historical video frames: The first coefficient λ1 is set by the staff based on experience, and is not limited in this application. Specifically, the judgment model in step S11 is as follows:

[0044] S12: If yes, the target is determined as a candidate target.

[0045] Specifically, in this embodiment, when the first deviation between the actual value of the head-to-shoulder aspect ratio and the average value of the head-to-shoulder aspect ratio is greater than the first threshold, that is, when When the target is detected, it indicates that the target may be capable of moving the human head in a preset direction, and the target is determined as a candidate target. At this time, a warning message may be triggered. Of course, the warning message may not be triggered, and this application does not limit this.

[0046] S13: Otherwise, return to the step of obtaining the first key point information of each target among the multiple targets included in the current video frame.

[0047] Specifically, when the first deviation between the actual value of the head-to-shoulder aspect ratio and the average value of the head-to-shoulder aspect ratio is less than or equal to the first threshold, that is, when When the target is not able to move the head of the human body in the preset direction, the process returns to step S1, which can reduce the overall time consumption of cheating detection. When performing the next round of pre-check analysis on the candidate target, all video frames before the candidate target are removed, and only the video frames after the current video frame are detected, which can ensure the accuracy of the calculation process, improve the accuracy of cheating detection, and reduce the overall time consumption of cheating detection.

[0048] In another embodiment, when the target action includes the action of the human head moving in a preset direction, the process of determining whether the target performs the action of the human head moving in the preset direction may also be other processes, for example, see Figure 3 , Figure 3 yes Figure 1 Schematic diagram of another implementation of step S2 in FIG. Specifically, step S2 further includes:

[0049] S20: Obtain the actual distance between the left and right shoulder key points of each target in the current video frame based on the first key point information, and obtain the average distance between the left and right shoulder key points of each target in multiple historical video frames before the current video frame based on the second key point information.

[0050] Specifically, the actual distance between the left and right shoulder key points of each target in the current video frame is The average distance between the left and right shoulder key points of each target in multiple historical video frames before the current video frame is

[0051] S21: Determine whether a second offset degree between the actual distance between the left and right shoulder key points and the average distance between the left and right shoulder key points is greater than a second threshold.

[0052] Specifically, in this embodiment, the second offset degree is the square of the difference between the actual distance between the left and right shoulder key points and the average distance between the left and right shoulder key points: The variance of the head-to-shoulder aspect ratio in multiple historical video frames is The second threshold is the product of the second coefficient and the variance of the head-to-shoulder aspect ratio in multiple historical video frames: The second coefficient λ2 is set by the staff based on experience, and is not limited in this application. Specifically, the judgment model in step S21 is as follows:

[0053] S22: If yes, determine the target as a candidate target.

[0054] Specifically, in this embodiment, when the second deviation degree between the actual distance between the left and right shoulder key points and the average distance between the left and right shoulder key points is greater than the second threshold, that is, when When the target is detected, it indicates that the target may be capable of moving the human head in a preset direction, and the target is determined as a candidate target. At this time, a warning message may be triggered. Of course, the warning message may not be triggered, and this application does not limit this.

[0055] S23: Otherwise, return to the step of obtaining the first key point information of each target among the multiple targets included in the current video frame.

[0056] Specifically, when the second deviation degree between the actual distance between the left and right shoulder key points and the average distance between the left and right shoulder key points is less than or equal to the second threshold, that is, when When , it indicates that the target has no possibility of executing the action of moving the human head in the preset direction, and then returns to step S1, which can reduce the overall time consumption of cheating detection.

[0057] In addition, in this embodiment, when performing the next round of pre-inspection analysis on the candidate target, all video frames before the candidate target are eliminated, and only the video frames after the current video frame are detected. This can ensure the accuracy of the calculation process, improve the accuracy of cheating detection, and reduce the overall time consumption of cheating detection.

[0058] In yet another embodiment, when the target action includes an action of a human hand contacting another human hand, refer to Figure 4 , Figure 4 yes Figure 1 Schematic diagram of another implementation of step S2 in FIG. Specifically, step S2 further includes:

[0059] S30: Obtaining the actual value of the elbow angle of each target in the current video frame based on the first key point information, and obtaining the average value of the elbow angle of each target in multiple historical video frames before the current video frame based on the second key point information.

[0060] Specifically, the actual value of the elbow angle of each target in the current video frame is The above formula gives the actual value of the left arm elbow angle of each target. Of course, the actual value of the right arm elbow angle can also be calculated as The average elbow angle of each target in multiple historical video frames before the current video frame is

[0061] S31: Determine whether a third deviation degree between the actual value of the elbow angle and the average value of the elbow angle is greater than a third threshold.

[0062] Specifically, in this embodiment, the third offset degree is the square of the difference between the actual value of the elbow angle and the average value of the elbow angle: The variance of the head-to-shoulder aspect ratio in multiple historical video frames is The third threshold is the product of the third coefficient and the variance of the elbow angle in multiple historical video frames: The third coefficient λ3 is set by the staff based on experience, and is not limited in this application. Specifically, the judgment model in step S31 is as follows:

[0063] S32: If yes, determine the target as a candidate target.

[0064] Specifically, in this embodiment, when the third deviation degree between the actual value of the elbow angle and the average value of the elbow angle is greater than the third threshold, that is, when When the target is judged to be in the action of contacting the hand of a human body with the hand of another human body, it indicates that the target may be in the action of performing the action of contacting the hand of a human body with the hand of another human body, and the target is determined as a candidate target. At this time, the warning information can be triggered. Of course, the warning information may not be triggered, and this application does not limit it here. Of course, the warning information when judging that the target has the action of contacting the hand of a human body with the hand of another human body and the warning information when judging that the target has the action of moving the head of a human body in a preset direction may be the same or different, and this application does not limit it here.

[0065] S33: Otherwise, return to the step of obtaining the first key point information of each target among the multiple targets included in the current video frame.

[0066] Specifically, when the third deviation degree between the actual value of the elbow angle and the average value of the elbow angle is less than or equal to the third threshold, that is, when When , it indicates that the target has no possibility of performing the action of the human body's hand contacting the hand of another human body, and then returns to step S1, which can reduce the overall time consumption of cheating detection.

[0067] In addition, in this embodiment, when performing the next round of pre-inspection analysis on the candidate target, all video frames before the candidate target are eliminated, and only the video frames after the current video frame are detected. This can ensure the accuracy of the calculation process, improve the accuracy of cheating detection, and reduce the overall time consumption of cheating detection.

[0068] S3: Extracting image regions containing each determined candidate target from the current video frame, and determining a target for performing the target action based on the image regions of each determined candidate target.

[0069] Specifically, if the target that performs the target action is determined based on the image area of ​​each candidate target, a cheating alarm message is issued to remind the invigilator that the candidate target actually performs the cheating action. This method can greatly improve the accuracy of cheating detection; if the target that performs the target action is not determined based on the image area of ​​each candidate target, it means that the first pre-check judgment is wrong and the candidate target does not perform the cheating action, then return to the step of obtaining the first key point information of each target in the multiple targets contained in the current video frame, which not only greatly reduces the possibility of false alarm of the action, improves the accuracy of action detection, but also reduces the overall time consumption of action detection.

[0070] In one embodiment, when the candidate target performs an action of moving the head of the human body in a preset direction, refer to Figure 5 , Figure 5 yes Figure 1 Schematic diagram of a flow chart of an implementation method of step S3 in FIG. Specifically, step S3 includes:

[0071] S40: extracting first image regions containing each determined candidate target from the current video frame respectively.

[0072] Specifically, the first image region includes the head and shoulders of the candidate target.

[0073] Specifically, in this embodiment, please refer to Figure 6 , Figure 6 yes Figure 5Schematic diagram of a flow chart of an implementation method of step S40 in FIG. Specifically, step S40 includes:

[0074] S400: extracting first detection frames containing each determined candidate target from the current video frame respectively.

[0075] Specifically, the first detection frame includes the head and shoulders of the candidate target, wherein the result of the first pre-detection of the candidate target is an action of moving the head of the human body in a preset direction.

[0076] S401: Expand the first detection frame to obtain a first image region of each determined candidate target.

[0077] Specifically, for example, the first detection frame can be expanded by 30% on the left and right and 20% on the top and bottom, and then cut out from the current video frame to obtain the first image area and send it to the target classification network. The first image area obtained after expansion can more accurately reflect the key point information of the candidate target, thereby improving the accuracy of action detection and reducing the overall time consumption of action detection. Of course, in other embodiments, the expansion ratio can also be other, and this application is not limited to this.

[0078] S41: Determine from the current video frame and a plurality of subsequent video frames after the current video frame based on the first image area that a first offset between an actual value of the head-to-shoulders aspect ratio and an average value of the head-to-shoulders aspect ratio is greater than a first threshold, or a second offset between an actual distance between left and right shoulder key points and an average distance between left and right shoulder key points is greater than a second threshold.

[0079] Specifically, the specific steps of determining whether the candidate target performs the action of moving the human head in a preset direction can be referred to steps S10-S11 or S20-S21, which will not be repeated in this application. Of course, other methods can also be used to determine whether the candidate target performs the action of moving the human head in a preset direction, which is not limited in this application. In this embodiment, the number of subsequent video frames can be set according to actual conditions, which is not limited in this application.

[0080] Specifically, in this embodiment, if it is determined from the current video frame and the multiple subsequent video frames after the current video frame that the candidate target turns back, it is determined that the candidate target is indeed performing the action of moving the human head in a preset direction, and a cheating alarm message is issued to remind the invigilator that the target actually has this behavior. This method can greatly improve the accuracy of action detection. It can prevent the candidate target from being judged as cheating due to an action such as turning the head at a certain moment, which can not only reduce the possibility of false alarms, improve the accuracy of cheating detection, but also reduce the overall time consumption of cheating detection.

[0081] In addition, in this embodiment, when performing the next round of detection and analysis on the candidate target, all video frames before the candidate target are eliminated, and only the video frames after the current video frame are detected. This can ensure the accuracy of the calculation process, improve the accuracy of cheating detection, and reduce the overall time consumption of cheating detection.

[0082] In another embodiment, when the candidate target performs an action in which a human hand contacts another human hand, refer to Figure 7 , Figure 7 yes Figure 1 Schematic diagram of another implementation of step S3 in FIG. Specifically, step S3 includes:

[0083] S50: extracting from the current video frame a second image region including each determined candidate target and a third image region including adjacent targets located around each determined candidate target.

[0084] Specifically, the second image region includes the wrist of the candidate target, the third image region includes the wrist of the candidate target, the number of neighboring targets and the third image region are four respectively, and the four neighboring targets are respectively located in four directions around the candidate target.

[0085] Specifically, in this embodiment, please refer to Figure 8 , Figure 8 yes Figure 7 Schematic diagram of a flow chart of an implementation method of step S50 in FIG. Specifically, step S50 includes:

[0086] S500: extracting from the current video frame a second detection frame including each determined candidate target and a third detection frame including neighboring targets around each determined candidate target.

[0087] Specifically, the second detection frame includes the wrist of the candidate target, and the third detection frame includes the wrist of the neighboring target. The result of the first pre-check of the candidate target is an action of the hand of the executing human body contacting the hand of another human body.

[0088] S501: Expand the second detection frame to obtain a second image region, and expand the third detection frame to obtain a third image region.

[0089] Specifically, for example, the second detection frame can be expanded 100% to the left and right and 20% to the top and bottom, then cut out from the current video frame to obtain the second image area and send it to the single-person key point network; the third detection frame can be expanded 100% to the left and right and 20% to the top and bottom, then cut out from the current video frame to obtain the third image area and send it to the single-person key point network. The second image area and the third image area obtained after the expansion can more accurately reflect the key point position information of the current target, thereby improving the accuracy of action detection and reducing the overall time consumption of action detection. Of course, in other embodiments, the expansion ratio can also be other, and this application is not limited to this.

[0090] S51: Obtaining the distance between the wrist of each determined candidate target and the wrist of each corresponding neighboring target based on the second image area and the third image area.

[0091] Specifically, in this embodiment, the distance between the wrist of the candidate target and the wrist of each corresponding neighboring target is acquired from the second image area and the third image area.

[0092] S52: Determine, based on the distance, a target whose distance is less than or equal to a preset threshold.

[0093] Specifically, a target less than or equal to a preset threshold is determined based on the distance between the wrist of the candidate target and the wrist of each corresponding adjacent target in step S51, that is, whether the hands of the candidate target and the adjacent target are in contact. If the candidate target performs an action of the hand of a human body in contact with the hand of another human body, a cheating alarm message is issued to remind the invigilator that the target actually has the cheating behavior. This method can prevent the candidate target from being judged as cheating due to normal actions such as stretching out the hand, which can not only reduce the possibility of false alarms and improve the accuracy of cheating detection, but also reduce the overall time consumption of cheating detection.

[0094] With this design approach, we first use the full-image key points for pre-inspection, and then use the local key points for secondary analysis. The overall time consumption and effect can be guaranteed, which not only reduces the possibility of false alarms, but also improves the accuracy of motion detection.

[0095] See also Fig. 9 , Fig. 9 This is a structural diagram of an implementation method of a cheating detection system of the present application. The cheating detection system specifically includes:

[0096] The obtaining module 10 is used to obtain the first key point information of each target among the multiple targets contained in the current video frame.

[0097] The processing module 12 is coupled to the acquisition module 10 and is used to determine the target whose action performed in each target meets the requirements of similarity with the target action based on the first key point information of each target and the second key point information of each target as a candidate target; wherein the second key point information of a target is determined based on the key point information of a target in the historical video frame. In addition, the processing module 12 is also used to extract the image areas containing each determined candidate target from the current video frame, and determine the target that performs the target action based on the image areas of each determined candidate target.

[0098] See also Fig.10 , Fig.10 1 is a schematic diagram of a framework of an embodiment of the motion detection system of the present application. The motion detection system includes a memory 20 and a processor 22 coupled to each other. Specifically, in the present embodiment, the memory 20 stores program instructions, and the processor 22 is used to execute the program instructions to implement the motion detection method mentioned in any of the above embodiments.

[0099] Specifically, the processor 22 may also be referred to as a CPU (Central Processing Unit). The processor 22 may be an integrated circuit chip having the ability to process signals. The processor 22 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. In addition, the processor 22 may be implemented by a plurality of integrated circuit chips.

[0100] See also Fig.11 , Fig.11It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 30 stores a computer program 300, which can be read by a computer, and the computer program 300 can be executed by a processor to implement the motion detection method mentioned in any of the above embodiments. Among them, the computer program 300 can be stored in the above-mentioned computer-readable storage medium 30 in the form of a software product, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The computer-readable storage medium 30 with a storage function can be a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, RandomAccess Memory), a disk or an optical disk, etc., which can store program codes, or a terminal device such as a computer, a server, a mobile phone, and a tablet.

[0101] In summary, different from the prior art, the cheating detection method provided by the present application includes: obtaining the first key point information of each target among the multiple targets contained in the current video frame; based on the first key point information of each target and the second key point information of each target, determining the target whose action performed in each target meets the requirements of similarity with the target action as a candidate target; wherein the second key point information of a target is determined based on the key point information of a target in the historical video frame; from the current video frame, respectively extracting the image areas containing each determined candidate target, and based on the image areas of each determined candidate target, determining the target that performs the target action. Through such a design method, first use the key points of the whole image for pre-inspection, and then use the local key points of the precise target for secondary analysis, the overall time consumption and effect can be guaranteed, which can not only reduce the possibility of false alarm of action, but also improve the accuracy of action detection.

[0102] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A motion detection method, characterized in that: include: Obtaining the first key point information of each target among multiple targets contained in the current video frame; Based on the first key point information of each target and the second key point information of each target, a target whose action executed in each target meets the requirement of similarity with the target action is determined as a candidate target; wherein the second key point information of a target is determined based on the key point information of the target in the historical video frame; Extracting image regions containing each of the determined candidate targets from the current video frame, and determining a target for performing the target action based on the image regions of each of the determined candidate targets; The step of determining, based on the first key point information of each target and the second key point information of each target, targets whose actions executed in each target meet the requirements of similarity with the target action as candidate targets includes: The target action includes an action of the target's head moving in a preset direction, obtaining an actual value of the head-to-shoulder aspect ratio of each target in the current video frame based on the first key point information, and obtaining an average value of the head-to-shoulder aspect ratio of each target in multiple historical video frames before the current video frame based on the second key point information; in response to a first deviation degree between the actual value of the head-to-shoulder aspect ratio and the average value of the head-to-shoulder aspect ratio being greater than a first threshold, determining the target as a candidate target; and / or, The target action includes an action of moving the head of the target in a preset direction, obtaining the actual distance between the left and right shoulder key points of each target in the current video frame based on the first key point information, and obtaining the average distance between the left and right shoulder key points of each target in multiple historical video frames before the current video frame based on the second key point information; in response to a second offset degree between the actual distance between the left and right shoulder key points and the average distance between the left and right shoulder key points being greater than a second threshold, determining the target as a candidate target; and / or, The target action includes the action of the target's hand contacting the hands of other targets, obtaining the actual value of the elbow angle of each target in the current video frame based on the first key point information, and obtaining the average value of the elbow angle of each target in multiple historical video frames before the current video frame based on the second key point information; in response to a third offset degree between the actual value of the elbow angle and the average value of the elbow angle being greater than a third threshold, the target is determined as a candidate target.

2. The motion detection method according to claim 1, characterized in that: The first deviation degree is the square of the difference between the actual value of the head-to-shoulder aspect ratio and the average value of the head-to-shoulder aspect ratio, and the first threshold is the product of the first coefficient and the variance of the head-to-shoulder aspect ratio in the multiple historical video frames.

3. The motion detection method according to claim 1, characterized in that: The second offset degree is the square of the difference between the actual distance between the left and right shoulder key points and the average distance between the left and right shoulder key points, and the second threshold is the product of the second coefficient and the variance of the distance between the left and right shoulder key points in the multiple historical video frames.

4. The motion detection method according to any one of claims 2 or 3, characterized in that: The step of extracting image regions containing each determined candidate target from the current video frame, and determining a target for performing the target action based on the determined image regions of each candidate target, comprises: In response to the candidate target executing an action of moving the head of the human body in a preset direction, extracting first image regions containing each determined candidate target from the current video frame; wherein the first image region contains the head and shoulders of the candidate target; A target is determined based on the first image area from the current video frame and a plurality of subsequent video frames after the current video frame that a first offset between an actual value of the head-to-shoulders aspect ratio and an average value of the head-to-shoulders aspect ratio is greater than a first threshold, or a second offset between an actual distance between the left and right shoulder key points and an average distance between the left and right shoulder key points is greater than a second threshold.

5. The motion detection method according to claim 4, characterized in that: The step of respectively extracting first image regions containing each determined candidate target from the current video frame comprises: Extracting first detection frames containing each of the determined candidate targets from the current video frame, respectively; wherein the first detection frames contain the head and shoulders of the candidate targets; The first detection frame is expanded to obtain a first image region of each determined candidate target.

6. The motion detection method according to claim 1, characterized in that: The third deviation degree is the square of the difference between the actual value of the elbow angle and the average value of the elbow angle, and the third threshold is the product of the third coefficient and the variance of the elbow angles in the multiple historical video frames.

7. The motion detection method according to claim 6, characterized in that: The step of extracting image regions containing each determined candidate target from the current video frame, and determining a target for performing the target action based on the determined image regions of each candidate target, comprises: In response to the candidate target performing an action in which the target's hand contacts the hands of other targets, extracting from the current video frame a second image region containing each determined candidate target and a third image region containing neighboring targets located around each determined candidate target; wherein the second image region contains the wrist of the candidate target, the third image region contains the wrist of the neighboring target, the number of the neighboring targets and the number of the third image region are four respectively, and the four neighboring targets are located in four directions around the candidate target respectively; respectively obtaining the distance between the wrist of each determined candidate target and the wrist of each of the adjacent targets corresponding thereto based on the second image area and the third image area; A target whose distance is less than or equal to a preset threshold is determined according to the distance.

8. The motion detection method according to claim 7, characterized in that: The step of respectively extracting from the current video frame a second image region containing each determined candidate target and a third image region containing adjacent targets located around each determined candidate target comprises: Extracting from the current video frame a second detection frame including each of the determined candidate targets and a third detection frame including adjacent targets around each of the determined candidate targets; wherein the second detection frame includes the wrist of the candidate target, and the third detection frame includes the wrist of the adjacent target; The second detection frame is expanded to obtain the second image region, and the third detection frame is expanded to obtain the third image region.

9. A motion detection system, characterized in that: It comprises a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the action detection method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to implement the action detection method according to any one of claims 1 to 8.

Images