Examination room supervision method and system based on human table model

Abstract

The invention relates to the technical field of computer artificial intelligence, and particularly discloses an examination room supervision method and system based on a human table model, and the method comprises the steps: inputting a video image into the human table model, outputting a table detection frame and a head detection frame, and sorting tables through the table detection frame; and matching the head detection frame with the sorted tables based on the head detection frame and the table detection frame so as to match and associate the examinee to each table, inputting the video data into a behavior recognition model to recognize a target examinee having an abnormal examination behavior in the examination room, generating abnormal examination behavior alarm data based on the table associated with the target examinee, and sending the abnormal examination behavior alarm data to the user. Accurate matching of the examinee and the table can be completed through the video data, the method can be suitable for different systems, binding of a seat number and the examinee through examinee data of an examination system is not needed, identification of examinee identities through high-definition videos is not needed, the method is suitable for scenes with low camera definition in an examination room, alarm information can be generated for alarming, and the method is convenient to use. And the examination notarization and fairness are ensured.

Description

Technical Field

[0001] The present invention belongs to the field of computer artificial intelligence technology, and in particular relates to an examination room supervision method and system based on a human-desk model. Background Art

[0002] With the continuous development of AI video analysis technology, in the field of educational examinations, the content of examination room surveillance videos is detected and analyzed based on AI algorithms. By extracting key information from the video, marking it, and generating corresponding event alarms, cheating incidents in examination rooms can be effectively curbed.

[0003] In the existing technology, the positioning of candidates in the examination room is to combine the inspection system, identity recognition system and comprehensive examination system. The three systems belong to different companies, and the construction and system docking costs are very high. In addition, the confidentiality requirements of the examination room mean that the camera cannot achieve ultra-high definition such as 4K and 8K, which leads to insufficient accuracy of candidate identity recognition and difficulty in accurately positioning and focusing on monitoring candidates. In addition, even if there are anti-cheating measures in the examination room such as on-site supervision, signal blockers and video playback, there are still violations of regulations and disciplines due to inadequate monitoring, negligent proctoring and other supervision situations, which seriously affect the fairness and justice of the examination. Summary of the invention

[0004] In view of the above problems, the embodiments of the present invention provide an examination room supervision method and system based on a person-desk model, which overcome the above problems or at least partially solve the above problems.

[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions: A method for monitoring an examination room based on a person-desk model, specifically comprising the following steps: Acquire examination room video data, and capture video images from the examination room video data; Input the video image into a pre-trained human-table model to output a table detection frame and a head detection frame; Determine the table order based on the table detection frame; Based on the head detection frame and the table detection frame, matching the head detection frame with the sorted tables to associate the examinee corresponding to each head detection frame with each table; Inputting the video data into a behavior recognition model to identify target examinees in the examination room who have abnormal examination behavior; Based on the table associated with the target examinee, abnormal examination behavior alarm data is generated, and the abnormal examination behavior alarm data is sent to the monitoring terminal.

[0006] As a further technical solution of the present invention, determining the table order based on the table detection frame specifically includes the following steps: Determine the camera position of the camera that collects the video data in the examination room; Determining a table detection frame of a first table based on the camera orientation; Taking the first table as the starting point, the tables in the examination room are sorted according to the table detection frames to obtain the table sorting result.

[0007] As a further technical solution of the present invention, based on the head detection frame and the table detection frame, the head detection frame is matched with the sorted tables to associate the examinee corresponding to each head detection frame with each table, which specifically includes the following steps: Remove the invigilator's head detection frame from the head detection frame to obtain the examinee's head detection frame; Determine a first candidate candidate head detection frame that meets a preset first matching rule from the candidate head detection frame; Determine a preliminary matching result between the candidate and the table corresponding to the head detection frame of the first candidate candidate based on a second matching rule corresponding to the camera orientation; The preliminary matching result is corrected based on the camera orientation corresponding correction rule to obtain the final matching result between the examinee and the table.

[0008] As a further technical solution of the present invention, removing the invigilator's head detection frame from the head detection frame to obtain the examinee's head detection frame specifically includes the following steps: Calculate the center distance between any two head detection frames in all head detection frames; When the center distance is less than half of the sum of the widths of the two head detection frames, determining that the two head detection frames include the examinee's head detection frame and the invigilator's head detection frame; The head detection frame with the larger coordinates is removed from the two head detection frames to remove the invigilator's head detection frame, thereby obtaining the examinee's head detection frame.

[0009] As a further technical solution of the present invention, the first matching rule satisfies the following conditions at the same time: The x-coordinate of the center point of the candidate's head detection frame is within the x-coordinate range of the left and right sides of the table detection frame; The y coordinate of the center point of the candidate's head detection frame is above the bottom edge of the table detection frame The distance between the y coordinate of the center point of the candidate's head detection frame and the top of the table detection frame is less than the product of the table height and the preset coefficient; The y coordinate of the bottom edge of the candidate's head detection frame is above the y coordinate of the bottom edge of the table detection frame.

[0010] As a further technical solution of the present invention, determining a preliminary matching result between the candidate and the table corresponding to the head detection frame of the first candidate candidate based on the second matching rule corresponding to the camera orientation specifically includes the following steps: Filtering a second candidate head detection frame from the first candidate candidate head detection frame based on a second matching rule corresponding to the camera orientation; Determine the optimal candidate head detection frame for each table from the second candidate head detection frame based on the third matching rule corresponding to the camera orientation, and match the candidate corresponding to the optimal candidate head detection frame with the table to obtain a preliminary matching result; When the camera is on the left side of the examination room, the second matching rule is as follows: If the distance between the bottom edge of the candidate's head detection frame and the bottom edge of the table detection frame in the first row of candidate's head detection frames is greater than 1.5 times the height of the candidate's head detection frame, the candidate's head detection frame will be removed and no matching will be performed; If the distance from the bottom edge of the candidate's head detection frame to the bottom edge of the table detection frame in the first row of candidate's head detection frames plus 1 / 10 of the height of the candidate's head detection frame is greater than the height of the bottom edge of the table detection frame, the candidate's head detection frame will be removed and no matching will be performed; If the bottom edge of the first candidate's head detection frame in the first column exceeds the bottom edge of the table detection frame, the candidate's head detection frame will be removed and no matching will be performed; If the bottom edge of the second candidate's head detection frame in the first column exceeds the center y coordinate of the table detection frame, the candidate's head detection frame is removed and no matching is performed; If the right side of the head detection frame of all candidates in the second column exceeds the right side of the table detection frame, the head detection frame of the candidate is removed and no matching is performed; When the camera is on the right side of the examination room, the second matching rule is as follows: If the first table is not the first column of table detection frames, and the candidate's head detection frame is at least partially within the range of the previous column of table detection frames, then the candidate's head detection frame is removed and no matching is performed; If the x-coordinate of the center point of the candidate's head detection frame is within the right range of the table detection frame, a match is performed; If the left side of the candidate's head detection frame exceeds the right side of the table detection frame and it is the last row of candidate's head detection frames, then a match is performed; When the camera orientation is that the camera is in the middle of the examination room, the second matching rule is equal to the first matching rule; The third matching rule is as follows: When the camera is on the left side of the examination room, the distance between the left side of the candidate's head detection frame and the left side of the table detection frame is calculated, and the one with the smallest distance is matched; When the camera is on the right side of the examination room, the distance between the right side of the examinee's head detection frame and the right side of the table detection frame is calculated, and the distance is the smallest and the examinee's head detection frame is within the visible range of the table. When the camera is in the middle of the examination room, calculate the distance between the center of the candidate's head detection frame and the center of the table detection frame, and choose the one with the smallest distance.

[0011] As a further technical solution of the present invention, the preliminary matching result is corrected based on the camera orientation corresponding correction rule to obtain the final matching result between the examinee and the table, which specifically includes the following steps: Traverse each table in order of tables, and determine whether the examinee on each table should be exchanged with an empty seat on the next table according to preset exchange conditions, wherein the exchange conditions include the camera orientation, the distance between the examinee's head detection frame and the table detection frame of the table it preliminarily matches, the relationship between the examinee's head detection frame and other table detection frames, and the relationship between the examinee's head detection frame and other examinee's head detection frames; If so, exchange is performed; If not, no exchange is performed.

[0012] As a further technical solution of the present invention, generating abnormal examination behavior warning data based on the table associated with the target examinee specifically includes the following steps: Extracting video clips including abnormal examinee behavior from the video data; Determine the seat number of the table associated with the target candidate; Generate alarm data including the video clip, seat number and category of abnormal candidate behavior.

[0013] As a further technical solution of the present invention, the human-table model and the behavior recognition model are specifically trained by the following steps: Acquire a training image, wherein the training image is annotated with a first head label, a first table label, and a first line label; Inputting the training image into the human-table model to obtain a second head label and a second table label; Inputting the training image into a behavior recognition model to obtain a second behavior label; Using the first head tag, the first table tag, the second head tag, and the second table tag to adjust the person-table model until the person-table model converges; The behavior recognition model is adjusted using the first behavior label and the second behavior label until the behavior recognition model converges.

[0014] An examination room supervision system based on a person-desk model specifically includes the following units: A video image acquisition unit, used to acquire examination room video data and capture video images from the examination room video data; A table and head detection unit, used for inputting the video image into a pre-trained human-table model to output a table detection frame and a head detection frame; A table sorting unit, configured to determine a table sorting based on the table detection frame; An examinee and table matching unit, configured to match the head detection frame with the sorted tables based on the head detection frame and the table detection frame, so as to associate the examinee corresponding to each head detection frame with each table; An abnormal examination behavior identification unit, used for inputting the video data into a behavior identification model to identify target examinees in the examination room who have abnormal examination behavior; The alarm data generating and sending unit is used to generate abnormal examination behavior alarm data based on the table associated with the target examinee, and send the abnormal examination behavior alarm data to the monitoring terminal.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The embodiment of the present invention captures a video image from the video data of the examination room, inputs the video image into the human-table model to output a table detection frame and a head detection frame, sorts the tables by the table detection frame, and matches the head detection frame with the sorted tables based on the head detection frame and the table detection frame, so as to associate the examinee corresponding to each head detection frame with each table, inputs the video data into the behavior recognition model to identify the target examinee with abnormal examination behavior in the examination room, generates abnormal examination behavior alarm data based on the table associated with the target examinee, and sends the abnormal examination behavior alarm data to the monitoring terminal. On the one hand, only video data needs to be obtained, which can be applied to systems of different companies. Through video data, the examinee and the table can be accurately matched, and there is no need to identify the table to obtain the seat number and then bind the seat number to the examinee through the examinee data of the examination system, nor is there any need to identify the examinee's identity through the face through high-definition video, which is applicable to the scene where the camera clarity in the examination room is not high. On the other hand, the abnormal examination behavior in the examination room can be monitored in real time through video data to issue an alarm, and the alarm information can be generated through the matching relationship between the examination and the table to issue an alarm, so as to ensure the notarization and fairness of the examination. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0017] Figure 1 A flow chart of an examination room supervision method based on a person-desk model provided by an embodiment of the present invention is shown.

[0018] Figure 2 This is a schematic diagram of the location of the camera in the examination room; Figure 3 A schematic diagram for sorting the table; Figure 4 A schematic diagram showing abnormal exam behavior for real-time monitoring; Figure 5 The application architecture diagram of an examination room supervision system based on a person-desk model provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention more clear, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the relevant information or data such as the examination information (such as examination room related information, etc.), video data acquisition (such as installing a camera in the examination room to collect video or images, etc.), face recognition (such as identifying the invigilator or extracting identity information from the examination face, etc.) involved in the implementation of the technical solution of the present invention comply with the relevant national laws and regulations and undergo necessary information desensitization processing.

[0020] Figure 1 The flowchart of the examination room supervision method based on the person-desk model provided by the embodiment of the present invention is shown. Specifically, the examination room supervision method based on the person-desk model of the embodiment of the present invention specifically includes the following steps: Step S101, obtaining examination room video data, and capturing video images from the examination room video data.

[0021] The examination room video data of this embodiment may be data collected by a camera during the examination in the examination room. In the examination room, the camera is usually set at the front of the examination room to collect video data from the front of the examinee. Figure 2 Figure a shows a schematic diagram of the camera on the left side of the examination room. Figure 2 The diagram b in the middle shows a schematic diagram of the camera in the middle of the examination room. Figure 2 Figure c in the middle shows a schematic diagram of the camera on the right side of the examination room.

[0022] The camera can collect examination room video data at a preset frame rate. In this embodiment, the examination room video data can be collected at the beginning of the examination. The examination room video data can be used to first realize the matching association between the examinee and the table. For example, the examination room video data is collected in the first 5 minutes of the examination, and video images are captured from the examination room video data at N frames per second.

[0023] Step S102: input the video image into a pre-trained human-table model to output a table detection frame and a head detection frame.

[0024] In this embodiment, the person-table model is used to identify the heads of people (including examinees and invigilators) in the examination room video data and the neural network for identifying tables. The behavior recognition model can be a neural network that identifies abnormal behaviors of examinees and invigilators in the examination room through the examination room video data. In one example, the person-table model and the behavior recognition model can be YOLO V8 (YouOnly Look Once Version 8), and of course can also be other neural networks. This embodiment does not limit the type of neural network used by the person-table model and the behavior recognition model.

[0025] In this embodiment, a training image can be first obtained, where the training image is annotated with a first head label, a first table label, and a first behavior label, and the training image is input into the person-table model to obtain a second head label and a second table label, and the training image is input into the behavior recognition model to obtain a second behavior label, and the first head label, the first table label, the second head label, and the second table label are used to adjust the person-table model until the person-table model converges, and the first behavior label and the second behavior label are used to adjust the behavior recognition model until the behavior recognition model converges.

[0026] For example, you can collect and copy real examination room pictures and video surveillance materials from various examination rooms, or record examination room pictures and videos in offline simulated examination rooms to obtain the original images, and then define five target label tags: head, table, front invigilator, middle invigilator, and back invigilator, and define 10 action behavior label tags: raising hands, putting hands under the table, collecting and distributing papers, standing, carrying examination paper tables, passing examination papers, suspicious items on the podium, invigilator's inattention, examinees carrying suspicious items, and invigilators carrying suspicious items.

[0027] Then, we manually screen out pictures and videos with behavioral actions, people, and tables on a small scale for image annotation, model training, and algorithm integration. When annotating images, we can use the Fast Forward Moving PictureExperts Group (FFmpeg) codec library to cut and extract frame video images, such as extracting three frames per second, and use the AutoLabelImg annotation software to automatically batch annotate pictures with behavioral actions, people, and tables. On this basis, we use the LabelImg annotation software to manually annotate and check the marking of people and tables, and mark the behaviors of candidates and invigilators.

[0028] The above-mentioned labeled images are then used for training on a training machine equipped with an NVIDIA GeForce RTX 3060 graphics card and a Compute Unified Device Architecture Version 11.8 (CUDA V11.8) parallel computing platform to obtain the trained person-table model and behavior recognition model, which are then integrated and deployed.

[0029] After training and deploying the person-table model and behavior recognition model, when the examination room video data is obtained and the video image is captured, the video image can be input into the person-table model to obtain the table detection frame and the head detection frame.

[0030] Step S103, determining the table order based on the table detection frame.

[0031] In one embodiment, the camera position of the camera that collects video data in the examination room is determined, and the table detection frame of the first table is determined based on the camera position. Taking the first table as the starting point, the tables in the examination room are sorted according to the table detection frame to obtain the table sorting result. The specific method of sorting the tables can be referred to the patent application document with application number 202311349856.5, which will not be described in detail here.

[0032] Of course, the center coordinates of the table detection frame can also be converted into coordinates in the camera coordinate system based on the camera coordinate system, and further converted into coordinates in the examination room coordinate system based on the position of the camera in the examination room to obtain the real position of the detected table in the examination room, and the detected tables can be sorted according to the real position and the preset table arrangement, such as Figure 3 Shown is a schematic diagram of the arrangement of tables when the camera is on the right side of the examination room.

[0033] Step S104, based on the head detection frame and the table detection frame, the head detection frame is matched with the sorted tables to associate the examinee corresponding to each head detection frame with each table.

[0034] In one embodiment, after the tables are sorted, each examinee needs to be matched and associated with each table one by one. Step S104 may specifically include the following sub-steps: Sub-step S1041, remove the invigilator's head detection frame from the head detection frame to obtain the examinee's head detection frame.

[0035] Specifically, the center distance between any two head detection frames among all the head detection frames can be calculated. When the center distance is less than half of the sum of the widths of the two head detection frames, it is determined that the two head detection frames include the candidate's head detection frame and the invigilator's head detection frame. The head detection frame with the larger coordinates is removed from the two head detection frames, and the invigilator's head detection frame is removed to obtain the candidate's head detection frame. That is, in the examination room, the invigilator may walk around in the examination room, resulting in the scene where the invigilator's head overlaps with the candidate's head when the video data is collected. Therefore, the invigilator's head detection frame can be removed, leaving the candidate's head detection frame, thereby avoiding the influence of the invigilator's head detection frame on the matching of the candidate and the table.

[0036] Sub-step S1042, determining a first candidate candidate head detection frame that meets a preset first matching rule from the candidate head detection frames.

[0037] In this embodiment, a first matching rule can be configured according to the relationship between the examinee's head and the table when the examinee is sitting in the seat, that is, a unified rule common to cameras in all directions, and the first candidate examinee head detection frame is first screened out through the unified rule. Specifically, the first matching rule includes the following conditions: Condition 1: The x-coordinate of the center point of the candidate's head detection frame is within the x-coordinate range of the left and right sides of the table detection frame, that is, the candidate's head is within the left and right range of the table; Condition 2: The y coordinate of the center point of the candidate's head detection frame is above the bottom edge of the table detection frame, that is, the candidate's head is above the table; Condition 3: The distance between the y coordinate of the center point of the candidate's head detection frame and the top of the table detection frame is less than the product of the table height and the preset coefficient, that is, the distance between the candidate's head and the table is greater than the product of the table height and the preset coefficient; Condition 4: The y coordinate of the bottom edge of the candidate's head detection frame is above the y coordinate of the bottom edge of the table detection frame, that is, the bottom of the candidate's head is above the bottom of the table.

[0038] After selecting the candidate's head detection frame that meets the above four conditions as the first candidate head detection frame through the above conditions, preliminary matching can be further performed according to the second matching rules of cameras in different orientations, that is, executing sub-step S1043.

[0039] Sub-step S1043, based on the second matching rule corresponding to the camera orientation, determine the preliminary matching result between the candidate and the table corresponding to the head detection frame of the first candidate.

[0040] Specifically, the second candidate head detection frame can be screened out from the first candidate candidate head detection frame based on the second matching rule corresponding to the camera orientation, and the optimal candidate head detection frame for each table can be determined from the second candidate head detection frame based on the third matching rule corresponding to the camera orientation, and the candidate corresponding to the optimal candidate head detection frame is matched with the table to obtain a preliminary matching result.

[0041] In one example, when the camera orientation is that the camera is on the left side of the examination room, the second matching rule includes the following conditions: If the distance between the bottom edge of the candidate's head detection frame and the bottom edge of the table detection frame in the first row of candidate's head detection frames is greater than 1.5 times the height of the candidate's head detection frame, the candidate's head detection frame will be removed and no matching will be performed; If the distance from the bottom edge of the candidate's head detection frame to the bottom edge of the table detection frame in the first row of candidate's head detection frames plus 1 / 10 of the height of the candidate's head detection frame is greater than the height of the bottom edge of the table detection frame, the candidate's head detection frame will be removed and no matching will be performed; If the bottom edge of the first candidate's head detection frame in the first column exceeds the bottom edge of the table detection frame, the candidate's head detection frame will be removed and no matching will be performed; If the bottom edge of the second candidate's head detection frame in the first column exceeds the center y coordinate of the table detection frame, the candidate's head detection frame is removed and no matching is performed; If the right side of the head detection frame of all candidates in the second column exceeds the right side of the table detection frame, the head detection frame of the candidate is removed and no matching is performed; After eliminating the above conditions, the second candidate head detection frame in the video image captured by the left camera is obtained, and the candidate and the table corresponding to the second candidate head detection frame are matched.

[0042] In another example, when the camera orientation is that the camera is on the right side of the examination room, the second matching rule includes the following conditions: If the first table is not the first column of table detection frames, and the candidate's head detection frame is at least partially within the range of the previous column of table detection frames, then the candidate's head detection frame is removed and no matching is performed; If the x-coordinate of the center point of the candidate's head detection frame is within the right range of the table detection frame, a match is performed; If the left side of the candidate's head detection frame exceeds the right side of the table detection frame and it is the last row of candidate's head detection frames, a match will be performed.

[0043] After eliminating the above conditions, the second candidate head detection frame in the video image captured by the right camera is obtained, and the candidate and the table corresponding to the second candidate head detection frame are matched.

[0044] In another example, when the camera is in the middle of the examination room, the second matching rule is equal to the first matching rule, that is, when the camera is in the middle of the examination room, all the first candidate candidate head detection frames that meet the first matching rule are used as the second candidate candidate head detection frames for matching with the table.

[0045] In this embodiment, the third matching rule is that when the camera is positioned on the left side of the examination room, the distance between the left side of the candidate's head detection frame and the left side of the table detection frame is calculated, and the match with the smallest distance is selected; when the camera is positioned on the right side of the examination room, the distance between the right side of the candidate's head detection frame and the right side of the table detection frame is calculated, and the match with the smallest distance and the candidate's head detection frame is within the visible range of the table; when the camera is positioned in the middle of the examination room, the distance between the center of the candidate's head detection frame and the center of the table detection frame is calculated, and the match with the smallest distance is selected.

[0046] After the above preliminary matching, the candidate's head detection frame is obtained for each table. In order to avoid mismatching due to failure to recognize the candidate's head, the preliminary matching result needs to be corrected. Specifically, sub-step S1044 can be executed.

[0047] Sub-step S1044, correct the preliminary matching result based on the camera orientation corresponding correction rule to obtain the final matching result between the examinee and the table.

[0048] Specifically, each table can be traversed in order, and the preset exchange conditions can be used to determine whether the candidate on each table should be exchanged with the empty seat on the next table. If so, the exchange is performed; if not, the exchange is not performed. After the traversal is completed, the final matching result is obtained.

[0049] Among them, the exchange conditions include the camera orientation, the distance between the candidate's head detection frame and the table detection frame of the table it preliminarily matches, the relationship between the candidate's head detection frame and other table detection frames, and the relationship between the candidate's head detection frame and other candidate's head detection frames.

[0050] For example, all tables can be traversed, and each time a table is traversed, it is checked whether the candidate on each table should be exchanged with the empty seat on the next table. The judgment conditions are as follows: For the left camera: ① If it is the first table and the distance between the two tables is less than half the width of the head, no matching is performed (i.e. no exchange is performed); ② If there is another table above the current table (not the last table in each column), and the vertical coordinate of the bottom edge of the head is smaller than the center vertical coordinate of the previous table, no matching is performed; ③If it is the second table after the first table, and the ordinate of the top edge of the head is lower than the ordinate of the bottom edge of the previous table, no matching is performed; ④ If it is the second to last table after the first table, and the vertical coordinate of the bottom edge of the head is lower than the center vertical coordinate of the current table, no matching is performed; ⑤ If it is the first table after the second table, and the difference between the vertical coordinates of the top edge of the current table and the bottom edge of the head is greater than one-fourth of the height of the table, no matching is performed; ⑥ If it is a table other than the first table after the second table, and the bottom edge of the head is lower than half of the vertical coordinate of the center of the previous table, it will not be matched; ⑦ If it is the second table after the second table, and the bottom edge of the head is lower than the center vertical coordinate of the previous table, no matching is performed; ⑧If it is the first table after the first table, and the bottom edge of the head is lower than the top edge of the current table, no matching is performed; ⑨If the remaining tables meet the conditions, they will be exchanged.

[0051] For the right camera: ① If it is the first table and the distance between the two tables is less than half the width of the head, no matching is performed; ② If there is another table after the current table (not the last table in each column), and the bottom edge of the head is lower than the center vertical coordinate of the previous table, and the current table is not the second table, no matching is performed; ③If it is the second table after the first table, and the top edge of the head is lower than the bottom edge of the previous table, no matching is performed; ④ If it is the last table, and the current table is not the first table, and the bottom edge of the head is lower than the center vertical coordinate of the previous table plus one tenth of the height, and the difference between the top edge of the current table and the bottom edge of the previous table is greater than one tenth of the height of the previous table, no matching is performed; ⑤ If it is the second table before the last table, and the top edge of the head is lower than the bottom edge of the previous author, and the bottom edge of the head is within the current table, then swap directly; ⑥ If the remaining conditions are met, they will be exchanged.

[0052] For the middle camera: ① If it is the first table and the distance between the two tables is less than half the width of the head, no matching is performed; ② If there is another table above the current table, and the top edge of the head is lower than the center vertical coordinate of the previous table, and the current table is not the second table, no matching is performed; ③If the remaining tables meet the conditions, they will be exchanged.

[0053] After the above matching and correction operations, the incorrectly identified and matched candidates can be repaired to achieve correct matching of candidates, invigilators and tables, until the candidates and 30 tables are completely matched and associated, and the final matching result is obtained, so that each candidate is accurately matched to a table. There is no need to obtain the candidate's seat number through the examination system for matching after the tables are sorted. The matching of candidates and tables is achieved only through video data, and the candidates in the examination room can be intelligently and accurately located and monitored during the examination.

[0054] Step S105, inputting the video data into a behavior recognition model to identify target examinees with abnormal examination behaviors in the examination room.

[0055] After achieving accurate matching association between candidates and tables, during the examination, the collected video data can be input into the behavior recognition model to identify target candidates with abnormal examination behavior in the examination room, that is, the head detection frame where the abnormal examination behavior occurs is marked, and the seat number is determined by the table detection frame matched by the head detection frame, and the seat number is used as the identity of the candidate with abnormal examination behavior.

[0056] like Figure 4 Shown is a schematic diagram of the monitoring interface, in which the detection boxes of candidates who have exhibited abnormal examination behavior and the seat numbers of the matched tables are displayed on the left side of the figure, and the abnormality category can also be displayed in the detection box (not shown in the figure). A schematic diagram of the table order and seat numbers is displayed on the right side, and the tables where the candidates who have exhibited abnormal examination behavior are located are highlighted and displayed with the abnormality category, such as candidates with seat numbers 1, 7, 17, 19, 13, and 25 who have stopped answering, and the candidate with seat number 23 who has put his hands under the table and buried his head.

[0057] Step S106, generating abnormal examination behavior alarm data based on the table associated with the target examinee, and sending the abnormal examination behavior alarm data to the monitoring terminal.

[0058] Specifically, a video clip including abnormal candidate behavior can be captured from the video data, the seat number of the table associated with the target candidate can be determined, alarm data including the video clip, seat number and abnormal candidate behavior category can be generated, and the alarm data can be sent to the monitoring terminal for display.

[0059] The embodiment of the present invention captures a video image from the video data of the examination room, inputs the video image into the human-table model to output a table detection frame and a head detection frame, sorts the tables by the table detection frame, and matches the head detection frame with the sorted tables based on the head detection frame and the table detection frame, so as to associate the examinee corresponding to each head detection frame with each table, inputs the video data into the behavior recognition model to identify the target examinee with abnormal examination behavior in the examination room, generates abnormal examination behavior alarm data based on the table associated with the target examinee, and sends the abnormal examination behavior alarm data to the monitoring terminal. On the one hand, only video data needs to be obtained, which can be applied to systems of different companies. Through video data, the examinee and the table can be accurately matched, and there is no need to identify the table to obtain the seat number and then bind the seat number to the examinee through the examinee data of the examination system, nor is there any need to identify the examinee's identity through the face through high-definition video, which is applicable to the scene where the camera clarity in the examination room is not high. On the other hand, the abnormal examination behavior in the examination room can be monitored in real time through video data to issue an alarm, and the alarm information can be generated through the matching relationship between the examination and the table to issue an alarm, so as to ensure the notarization and fairness of the examination.

[0060] Figure 5 The application architecture diagram of the examination room supervision system based on the person-desk model provided by an embodiment of the present invention is shown. The examination room supervision system based on the person-desk model of this embodiment specifically includes the following units: The video image acquisition unit 501 is used to acquire the examination room video data and capture the video image from the examination room video data; A table and head detection unit 502, used to input the video image into a pre-trained human-table model to output a table detection frame and a head detection frame; A table sorting unit 503, configured to determine a table sorting based on the table detection frame; An examinee and table matching unit 504, configured to match the head detection frame with the sorted tables based on the head detection frame and the table detection frame, so as to associate the examinee corresponding to each head detection frame with each table; The abnormal examination behavior identification unit 505 is used to input the video data into a behavior identification model to identify target examinees with abnormal examination behavior in the examination room; The alarm data generating and sending unit 506 is used to generate abnormal examination behavior alarm data based on the table associated with the target examinee, and send the abnormal examination behavior alarm data to the monitoring terminal.

[0061] As a further technical solution of the present embodiment, the table sorting unit 503 specifically includes the following modules: A camera position determination module, used to determine the camera position of the camera collecting the video data in the examination room; a first table determination module, configured to determine a table detection frame of a first table based on the camera orientation; The table sorting module is used to sort the tables in the examination room according to the table detection frame with the first table as the starting point to obtain the table sorting result.

[0062] As a further technical solution of the present embodiment, the examinee and table matching unit 504 specifically includes the following modules: A deduplication module, used for removing the invigilator's head detection frame from the head detection frame to obtain the examinee's head detection frame; A candidate head detection frame screening module is used to determine a first candidate candidate head detection frame that meets a preset first matching rule from the candidate head detection frame; A matching module, used to determine a preliminary matching result between the candidate and the table corresponding to the head detection frame of the first candidate based on a second matching rule corresponding to the camera orientation; The correction module is used to correct the preliminary matching result based on the camera orientation corresponding correction rule to obtain the final matching result between the examinee and the table.

[0063] As a further technical solution of the present embodiment, the deduplication module is specifically used for: Calculate the center distance between any two head detection frames in all head detection frames; When the center distance is less than half of the sum of the widths of the two head detection frames, determining that the two head detection frames include the examinee's head detection frame and the invigilator's head detection frame; The head detection frame with the larger coordinates is removed from the two head detection frames to remove the invigilator's head detection frame, thereby obtaining the examinee's head detection frame.

[0064] As a further technical solution of the present embodiment, the first matching rule satisfies the following conditions at the same time: The x-coordinate of the center point of the candidate's head detection frame is within the x-coordinate range of the left and right sides of the table detection frame; The y coordinate of the center point of the candidate's head detection frame is above the bottom edge of the table detection frame The distance between the y coordinate of the center point of the candidate's head detection frame and the top of the table detection frame is less than the product of the table height and the preset coefficient; The y coordinate of the bottom edge of the candidate's head detection frame is above the y coordinate of the bottom edge of the table detection frame.

[0065] As a further technical solution of the present embodiment, the matching module is specifically used for: Filtering a second candidate head detection frame from the first candidate candidate head detection frame based on a second matching rule corresponding to the camera orientation; Determine the optimal candidate head detection frame for each table from the second candidate head detection frame based on the third matching rule corresponding to the camera orientation, and match the candidate corresponding to the optimal candidate head detection frame with the table to obtain a preliminary matching result; When the camera is on the left side of the examination room, the second matching rule is as follows: If the distance between the bottom edge of the candidate's head detection frame and the bottom edge of the table detection frame in the first row of candidate's head detection frames is greater than 1.5 times the height of the candidate's head detection frame, the candidate's head detection frame will be removed and no matching will be performed; If the distance from the bottom edge of the candidate's head detection frame to the bottom edge of the table detection frame in the first row of candidate's head detection frames plus 1 / 10 of the height of the candidate's head detection frame is greater than the height of the bottom edge of the table detection frame, the candidate's head detection frame will be removed and no matching will be performed; If the bottom edge of the first candidate's head detection frame in the first column exceeds the bottom edge of the table detection frame, the candidate's head detection frame will be removed and no matching will be performed; If the bottom edge of the second candidate's head detection frame in the first column exceeds the center y coordinate of the table detection frame, the candidate's head detection frame is removed and no matching is performed; If the right side of the head detection frame of all candidates in the second column exceeds the right side of the table detection frame, the head detection frame of the candidate is removed and no matching is performed; When the camera is on the right side of the examination room, the second matching rule is as follows: If the first table is not the first column of table detection frames, and the candidate's head detection frame is at least partially within the range of the previous column of table detection frames, then the candidate's head detection frame is removed and no matching is performed; If the x-coordinate of the center point of the candidate's head detection frame is within the right range of the table detection frame, a match is performed; If the left side of the candidate's head detection frame exceeds the right side of the table detection frame and it is the last row of candidate's head detection frames, then a match is performed; When the camera orientation is that the camera is in the middle of the examination room, the second matching rule is equal to the first matching rule; The third matching rule is as follows: When the camera is on the left side of the examination room, the distance between the left side of the candidate's head detection frame and the left side of the table detection frame is calculated, and the one with the smallest distance is matched; When the camera is on the right side of the examination room, the distance between the right side of the examinee's head detection frame and the right side of the table detection frame is calculated, and the distance is the smallest and the examinee's head detection frame is within the visible range of the table. When the camera is in the middle of the examination room, calculate the distance between the center of the candidate's head detection frame and the center of the table detection frame, and choose the one with the smallest distance.

[0066] As a further technical solution of the present embodiment, the correction module is specifically used for: Traverse each table in order of tables, and determine whether the examinee on each table should be exchanged with an empty seat on the next table according to preset exchange conditions, wherein the exchange conditions include the camera orientation, the distance between the examinee's head detection frame and the table detection frame of the table it preliminarily matches, the relationship between the examinee's head detection frame and other table detection frames, and the relationship between the examinee's head detection frame and other examinee's head detection frames; If so, exchange is performed; If not, no exchange is performed.

[0067] As a further technical solution of the present embodiment, the alarm data generating and sending unit 506 specifically includes the following modules: A video clip capture module, used to capture video clips including abnormal examinee behavior from the video data; A seat number determination module, used to determine the seat number of the table associated with the target examinee; The alarm data generation module is used to generate alarm data including the video clip, seat number and abnormal candidate behavior category.

[0068] As a further technical solution of the present embodiment, the human-table model and the behavior recognition model are specifically trained by the following units: A training image acquisition unit, used to acquire a training image, wherein the training image is annotated with a first head label, a first table label, and a first behavior label; A head and table prediction unit, used for inputting the training image into a person-table model to obtain a second head label and a second table label; A behavior prediction unit, used for inputting the training image into a behavior recognition model to obtain a second behavior label; a person-table model adjustment unit, configured to adjust the person-table model by using the first head tag, the first table tag, the second head tag, and the second table tag until the person-table model converges; A behavior recognition model adjustment unit is used to adjust the behavior recognition model using the first behavior label and the second behavior label until the behavior recognition model converges.

[0069] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0070] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0071] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0072] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention patent shall be subject to the attached claims.

[0073] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for monitoring examination rooms based on a person-desk model, characterized in that: The specific steps include: Acquire examination room video data, and capture video images from the examination room video data; Input the video image into a pre-trained human-table model to output a table detection frame and a head detection frame; Determine the table order based on the table detection frame; Based on the head detection frame and the table detection frame, matching the head detection frame with the sorted tables to associate the examinee corresponding to each head detection frame with each table; Inputting the video data into a behavior recognition model to identify target examinees in the examination room who have abnormal examination behavior; Based on the table associated with the target examinee, abnormal examination behavior alarm data is generated, and the abnormal examination behavior alarm data is sent to the monitoring terminal.

2. The examination room supervision method based on the person-desk model according to claim 1 is characterized in that: Determining the table order based on the table detection frame specifically includes the following steps: Determine the camera position of the camera that collects the video data in the examination room; Determining a table detection frame of a first table based on the camera orientation; Taking the first table as the starting point, the tables in the examination room are sorted according to the table detection frames to obtain the table sorting result.

3. The examination room supervision method based on the person-desk model according to claim 2 is characterized in that: Based on the head detection frame and the table detection frame, the head detection frame is matched with the sorted tables to associate the examinee corresponding to each head detection frame with each table, specifically comprising the following steps: Remove the invigilator's head detection frame from the head detection frame to obtain the examinee's head detection frame; Determine a first candidate candidate head detection frame that meets a preset first matching rule from the candidate head detection frame; Determine a preliminary matching result between the candidate and the table corresponding to the head detection frame of the first candidate candidate based on a second matching rule corresponding to the camera orientation; The preliminary matching result is corrected based on the camera orientation corresponding correction rule to obtain the final matching result between the examinee and the table.

4. The examination room supervision method based on the person-desk model according to claim 3 is characterized in that: The invigilator's head detection frame is removed from the head detection frame to obtain the examinee's head detection frame, which specifically includes the following steps: Calculate the center distance between any two head detection frames in all head detection frames; When the center distance is less than half of the sum of the widths of the two head detection frames, determining that the two head detection frames include the examinee's head detection frame and the invigilator's head detection frame; The head detection frame with the larger coordinates is removed from the two head detection frames to remove the invigilator's head detection frame, thereby obtaining the examinee's head detection frame.

5. The examination room supervision method based on the person-desk model according to claim 3 is characterized in that: The first matching rule is to satisfy the following conditions at the same time: The x-coordinate of the center point of the candidate's head detection frame is within the x-coordinate range of the left and right sides of the table detection frame; The y coordinate of the center point of the candidate's head detection frame is above the bottom edge of the table detection frame The distance between the y coordinate of the center point of the candidate's head detection frame and the top of the table detection frame is less than the product of the table height and the preset coefficient; The y coordinate of the bottom edge of the candidate's head detection frame is above the y coordinate of the bottom edge of the table detection frame.

6. The examination room supervision method based on the person-desk model according to claim 3 is characterized in that: Determining a preliminary matching result between the candidate and the table corresponding to the head detection frame of the first candidate candidate based on the second matching rule corresponding to the camera orientation specifically includes the following steps: Filtering a second candidate head detection frame from the first candidate candidate head detection frame based on a second matching rule corresponding to the camera orientation; Determine the optimal candidate head detection frame for each table from the second candidate head detection frame based on the third matching rule corresponding to the camera orientation, and match the candidate corresponding to the optimal candidate head detection frame with the table to obtain a preliminary matching result; When the camera is on the left side of the examination room, the second matching rule is as follows: If the distance between the bottom edge of the candidate's head detection frame and the bottom edge of the table detection frame in the first row of candidate's head detection frames is greater than 1.5 times the height of the candidate's head detection frame, the candidate's head detection frame will be removed and no matching will be performed; If the distance from the bottom edge of the candidate's head detection frame to the bottom edge of the table detection frame in the first row of candidate's head detection frames plus 1 / 10 of the height of the candidate's head detection frame is greater than the height of the bottom edge of the table detection frame, the candidate's head detection frame will be removed and no matching will be performed; If the bottom edge of the first candidate's head detection frame in the first column exceeds the bottom edge of the table detection frame, the candidate's head detection frame will be removed and no matching will be performed; If the bottom edge of the second candidate's head detection frame in the first column exceeds the center y coordinate of the table detection frame, the candidate's head detection frame is removed and no matching is performed; If the right side of the head detection frame of all candidates in the second column exceeds the right side of the table detection frame, the head detection frame of the candidate is removed and no matching is performed; When the camera is on the right side of the examination room, the second matching rule is as follows: If the first table is not the first column of table detection frames, and the candidate's head detection frame is at least partially within the range of the previous column of table detection frames, then the candidate's head detection frame is removed and no matching is performed; If the x-coordinate of the center point of the candidate's head detection frame is within the right range of the table detection frame, a match is performed; If the left side of the candidate's head detection frame exceeds the right side of the table detection frame and it is the last row of candidate's head detection frames, then a match is performed; When the camera orientation is that the camera is in the middle of the examination room, the second matching rule is equal to the first matching rule; The third matching rule is as follows: When the camera is on the left side of the examination room, the distance between the left side of the candidate's head detection frame and the left side of the table detection frame is calculated, and the one with the smallest distance is matched; When the camera is on the right side of the examination room, the distance between the right side of the examinee's head detection frame and the right side of the table detection frame is calculated, and the distance is the smallest and the examinee's head detection frame is within the visible range of the table. When the camera is in the middle of the examination room, calculate the distance between the center of the candidate's head detection frame and the center of the table detection frame, and choose the one with the smallest distance.

7. The examination room supervision method based on the person-desk model according to claim 3 is characterized in that: The preliminary matching result is corrected based on the camera orientation corresponding correction rule to obtain the final matching result between the examinee and the table, which specifically includes the following steps: Traverse each table in order of tables, and determine whether the examinee on each table should be exchanged with an empty seat on the next table according to preset exchange conditions, wherein the exchange conditions include the camera orientation, the distance between the examinee's head detection frame and the table detection frame of the table it preliminarily matches, the relationship between the examinee's head detection frame and other table detection frames, and the relationship between the examinee's head detection frame and other examinee's head detection frames; If so, exchange is performed; If not, no exchange is performed.

8. The examination room supervision method based on the person-desk model according to any one of claims 1 to 7, characterized in that: Based on the table associated with the target examinee, abnormal examination behavior warning data is generated, specifically including the following steps: Extracting video clips including abnormal examinee behavior from the video data; Determine the seat number of the table associated with the target candidate; Generate alarm data including the video clip, seat number and category of abnormal candidate behavior.

9. The examination room supervision method based on the person-desk model according to any one of claims 1 to 7, characterized in that: The person-table model and the behavior recognition model are specifically trained through the following steps: Acquire a training image, wherein the training image is annotated with a first head label, a first table label, and a first line label; Inputting the training image into the human-table model to obtain a second head label and a second table label; Inputting the training image into a behavior recognition model to obtain a second behavior label; Using the first head tag, the first table tag, the second head tag, and the second table tag to adjust the person-table model until the person-table model converges; The behavior recognition model is adjusted using the first behavior label and the second behavior label until the behavior recognition model converges.

10. An examination room supervision system based on a person-table model, characterized in that: It includes the following units: A video image acquisition unit, used to acquire examination room video data and capture video images from the examination room video data; A table and head detection unit, used for inputting the video image into a pre-trained human-table model to output a table detection frame and a head detection frame; A table sorting unit, configured to determine a table sorting based on the table detection frame; An examinee and table matching unit, configured to match the head detection frame with the sorted tables based on the head detection frame and the table detection frame, so as to associate the examinee corresponding to each head detection frame with each table; An abnormal examination behavior identification unit, used for inputting the video data into a behavior identification model to identify target examinees in the examination room who have abnormal examination behavior; The alarm data generating and sending unit is used to generate abnormal examination behavior alarm data based on the table associated with the target examinee, and send the abnormal examination behavior alarm data to the monitoring terminal.

Images