Online Examination Behavior Detection Method Based on Edge Computing and Multimodal Fusion

Abstract

An embodiment of the present invention discloses an online examination behavior detection method based on edge computing and multimodal fusion. Among them, the method includes: determining an initial population suspected of group cheating according to the co-occurrence of cheating behaviors in each time period in the examination room; constructing a reachability matrix with each target candidate in the initial population as rows and columns, and setting the matrix element of the row and column to 1 if any target candidate in any row and any column has visual reachability; determining the rows or columns with all elements empty, and for each empty row or column respectively: defining a neighborhood A in the examination room with the current target candidate as the center; assigning different first weights to each candidate in the neighborhood A according to visual reachability, and assigning second weights to each candidate according to the nearest reachable distance between each candidate in the neighborhood A and other target candidates in the reachability matrix; the candidate with the largest product of the first weight and the second weight and the initial population together constitute the final population suspected of group cheating. This embodiment can automatically identify group cheating behaviors.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of online education technology, and in particular, to an online exam behavior detection method based on edge computing and multimodal fusion. Background Art

[0002] With the popularization of online education, it is possible to accurately obtain the action behaviors of individual examinees through video streams, screen streams, audio streams, etc. of online exam devices, and then detect whether there are cheating suspicions among the examinees in the examination room.

[0003] However, the existing online exam behavior detections mostly target individual examinees separately and cannot accurately identify situations such as group cheating. For example, Patent CN116883953A provides an online exam anti-cheating method system and storage medium, and Patent CN115311735A provides an abnormal behavior intelligent recognition and early warning method, both of which cannot identify group cheating situations. Summary of the Invention

[0004] The embodiments of the present invention provide an online exam behavior detection method based on edge computing and multimodal fusion to solve the above technical problems.

[0005] In a first aspect, the embodiments of the present invention provide an online exam behavior detection method based on edge computing and multimodal fusion, which is applied to edge devices for examination rooms;

[0006] The method includes:

[0007] Obtain the co-occurrence situation of cheating behaviors in each time period in the examination room;

[0008] Determine an initial population suspected of group cheating according to the co-occurrence situation;

[0009] Construct an accessibility matrix with each target examinee in the initial population as row and column indices. If the target examinee in any row has visual accessibility to the target examinee in any column, set the matrix element at the intersection of the any row and any column to 1;

[0010] Determine the rows or columns with all elements empty, and perform the following operations on the target examinees corresponding to each empty row or empty column respectively: delimit a neighborhood A in the examination room with the current target examinee as the center; assign different first weights to each examinee in the neighborhood A according to visual accessibility, and assign second weights to each examinee according to the nearest accessibility distance between each examinee in the neighborhood A and other target examinees in the accessibility matrix;

[0011] The examinee with the largest product of the first weight and the second weight and the initial population together form the final population suspected of group cheating, which is pushed to the background server for assisting multimodal data fusion analysis and displayed on the monitoring interface.

[0012] In a second aspect, an embodiment of the present invention provides an electronic device, which includes:

[0013] One or more processors;

[0014] A memory for storing one or more programs,

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the online exam behavior detection method based on edge computing and multimodal fusion described in any embodiment.

[0016] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the online exam behavior detection method based on edge computing and multimodal fusion described in any embodiment.

[0017] An embodiment of the present invention provides an online exam behavior detection method based on edge computing and multimodal fusion, which identifies group cheating behaviors in an online exam venue. Since group cheating often involves the co-occurrence of cheating behaviors of multiple candidates, this embodiment determines an initial population suspected of group cheating based on the sharing of cheating behaviors. For the case of missed identification where a candidate participates in group cheating but no specific cheating behavior is detected, this embodiment constructs a reachability matrix between every two students in the initial population according to the characteristic that information is passed one by one between candidates in group cheating. As long as one person in the initial population can establish visual contact with another person in the population, it is considered that they have the conditions to participate in group cheating; for a target candidate who cannot establish visual contact with anyone in the initial population, by considering the possibility and difficulty of establishing visual contact between the target candidate and neighboring candidates, an intermediate candidate who may help the two target candidates indirectly transmit information is found from the neighboring candidates, and the intermediate candidate is also included in the final population of group cheating and provided to the background server and monitoring interface together for invigilators to further identify. The entire method can automatically identify group cheating behaviors in the exam venue, fully consider the situation of missed identification, and improve the accuracy of identifying group cheating behaviors. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0019] Figure 1It is a flowchart of an online exam behavior detection method based on edge computing and multimodal fusion provided by an embodiment of the present invention;

[0020] Figure 2 It is a schematic diagram of the position distribution of some online exam devices and candidates in the examination room provided by an embodiment of the present invention;

[0021] Figure 3 It is another schematic diagram of the position distribution of some online exam devices and candidates in the examination room provided by an embodiment of the present invention;

[0022] Figure 4 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0023] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0024] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0025] In the description of the present invention, it should also be noted that unless otherwise clearly defined and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the internal communication of two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0026] An embodiment of the present invention provides an online exam behavior detection method based on edge computing and multimodal fusion. To illustrate this method, an online exam behavior detection system based on edge computing applied by this method is introduced first. The system includes an edge node, a background server, and an invigilation panel. By deploying an edge node at the exam end, the behavior of examinees is monitored in real time and abnormal behaviors are automatically reported. At the same time, the background periodically performs fusion analysis on multimodal data such as video streams, screen streams, audio streams, and environmental data, and pushes it to the invigilation teacher for key attention.

[0027] The method of this embodiment is applicable to the situation where multiple online exam devices are deployed inside a certain examination room to organize examinees to take a unified online exam. In this case, the edge node can adopt a two-level structure. Among them, the first-level edge node can be deployed on each examinee device, responsible for real-time collection of video streams (such as video streams collected by a computer camera), screen streams, audio streams, and environmental data. A lightweight behavior detection model (such as face recognition, eye tracking, action detection) runs in this edge node to analyze the behavior of examinees in real time. When an abnormal behavior (such as cheating, leaving the seat, using unauthorized devices) is detected, each first-level edge node automatically generates an abnormal event and reports it to the second-level edge node; at the same time, an additional video collection device can also be deployed in the examination room, responsible for collecting video streams, audio streams, and environmental data of the entire examination room from a high place. This device can also be used as a first-level edge node and report the overall data corresponding to the abnormal event to the second-level edge node. The second-level edge node can be deployed in the computer room corresponding to the examination room, responsible for analyzing whether there is group cheating behavior in this examination room according to the reported data of each examination room, and reporting the video streams, screen streams, audio streams, and environmental data related to the detected group cheating behavior and individual cheating behavior to the background server.

[0028] The background server periodically receives the video streams, screen streams, audio streams, and environmental data uploaded by the edge node, and uses a multimodal data fusion algorithm (such as a deep learning model) to comprehensively analyze the data and generate an examinee behavior report. At the same time, the background server generates a key attention list according to the multimodal data analysis results and pushes it to the invigilation teacher. The invigilation teacher can view the detailed data and video clips of abnormal behaviors through the invigilation panel.

[0029] Based on the above system architecture, Figure 1 is a flowchart of an online exam behavior detection method based on edge computing and multimodal fusion provided by an embodiment of the present invention. This method is applicable to the situation of identifying group cheating behaviors in an online exam examination room, and is executed by the second-level edge node or other electronic devices in the above system. The following will take the second boundary node executing this method as an example for illustration. As Figure 1 shown, this method specifically includes:

[0030] S110. Obtain the co-occurrence of cheating behaviors in each time period in the examination room.

[0031] As described above, by deploying the first edge node of each online test device in the examination room, the cheating behavior of each examinee can be detected. The cheating behavior does not need to be particularly accurate, and can be understood as abnormal behavior with a high possibility of cheating, such as frequent lowering of the head, looking around, body movement, long-term non-operation, and shooting of display images. Optionally, the first edge node can use a lightweight deep learning model (such as MobileNet) for facial recognition and motion detection, or use eye tracking technology (such as Gaze Tracking) to monitor the examinee's line of sight. When any of facial recognition, motion detection, or line of sight monitoring considers that the examinee has the above abnormal behavior, it is considered that a cheating behavior is detected, and the timestamp, behavior type, and corresponding student ID of the cheating behavior are all reported to the second-level edge node.

[0032] Then, the number of candidates and the number of times cheating occurs at the same time in each time period are recorded. Specifically, the second edge node divides the entire examination process into multiple overlapping or non-overlapping time periods, and records the number of candidates and the number of times cheating occurs at the same time in each time period. Optionally, if the time difference between cheating behaviors of different students is less than a set threshold (such as 1 minute, 2 minutes, etc.), these cheating behaviors are considered to occur at the same time.

[0033] S120: Determine an initial group of people suspected of group cheating based on the co-occurrence situation.

[0034] Optionally, if multiple examinees cheat at the same time for multiple consecutive times within any period of time, the set of examinees that cheat at the same time each time is recorded; and the intersection of each set is taken as the initial population of suspected group cheating.

[0035] For example, assuming that the duration of each time period is 15 minutes, if multiple candidates cheat at the same time for three consecutive times within a certain time period (for example, multiple candidates cheated in the first minute, multiple candidates cheated in the second minute, and multiple candidates cheated again in the third minute), then the sets U1, U2 and U3 of candidates who cheated each time will be recorded respectively, and the intersection of the three sets will be taken, and the intersection will be used as the initial population suspected of group cheating.

[0036] By taking the intersection, individual cheating behaviors that happen within these 3 minutes can be excluded, improving the accuracy of the initial group prediction for group cheating. However, there may be some candidates who participated in group cheating but whose specific cheating behaviors were not detected in the first edge node, resulting in the omission of candidates in the cheating group. These omitted candidates will be identified in S130-S140.

[0037] S130. Construct a reachability matrix with each target candidate in the initial population as the row and column indices. If a target candidate in any row has visual reachability with a target candidate in any column, set the matrix element at the intersection of the arbitrary row and column to 1.

[0038] To distinguish from other candidates, in this embodiment, the candidates in the initial population suspected of group cheating are all called target candidates, and the reachability matrix is used to describe whether it is possible for any two target candidates to transmit cheating information to each other by peeking at the screen of the online examination device, candidate actions, etc. This embodiment refers to this possibility as visual reachability.

[0039] Specifically, each row and each column in the reachability matrix correspond to a target candidate, and the matrix element represents whether there is visual reachability between the target candidate in the row where the element is located and the target candidate in the column where the element is located. Exemplarily, for the target candidate in the i-th row of the matrix, if it has visual reachability with the target candidate in the j-th column of the matrix, set the element at the position (i, j) of the matrix to 1.

[0040] In a specific implementation manner, in combination with Figure 2 , the element value of the i-th row of the matrix can be determined through the following steps:

[0041] Step 1. Take the target candidate C in the i-th row as the center and delimit a neighborhood in the examination room. Optionally, Figure 2 FIG. is a schematic diagram of the position distribution of some online examination devices and candidates in the examination room. Each solid rectangle therein represents the screen of an online examination device, and each solid circle represents a candidate. Assume that the target candidate corresponding to the i-th row in the reachability matrix and his / her examination device are a red circle and a red rectangle respectively. Then, taking this candidate as the center and the average visual range of a person as the radius to delimit a region, the neighborhood B in this step is obtained ( Figure 2 only a part of the neighborhood B is shown in FIG.).

[0042] Step 2. Emit rays in different directions starting from the target candidate C. The candidates corresponding to the front-row candidates or front-row online examination devices in the neighborhood B that the rays first reach, the rear-row candidates in the neighborhood B that the rays first reach, and the candidates in the same row as the target candidate in the neighborhood B are all regarded as candidates who have visual reachability with the target candidate C in any row. In combination with Figure 2, the front row, same row and back row here are all relative to the target candidate C represented by the red circle. For the front row, the target candidate can see the test equipment screen and the candidates in the front row, and can complete the cheating information transmission through the screen content or the candidates' movements, eyes, etc.; for the same row, the candidates can see each other by moving their bodies forward and backward, and thus complete the cheating information transmission through movements, eyes, etc.; for the back row, the target candidate cannot see the screen of the test equipment, and can only complete the cheating information transmission through movements, eyes, etc. between the candidates. Therefore, in this step, different judgment criteria are used for the visual accessibility of the front row, same row and back row of the target test, and the candidates in the neighborhood that meet the respective criteria are all candidates with visual accessibility. Exemplarily, with Figure 2 For example, assuming that the candidates shown in the figure are all candidates within the neighborhood B of the target candidate, then in the area to the left of the target candidate, the candidates with visual accessibility are represented by the green circle, and the other candidates in the area to the left are initially considered to not have visual accessibility.

[0043] Step 3: If the target candidate in the jth column of the reachability matrix is ​​a candidate who has visual reachability with the target candidate in the i-th row of the reachability matrix determined in step 2, then the element at position (i, j) in the reachability matrix is ​​set to 1. The elements on the diagonal of the reachability matrix are set to 0 by default.

[0044] After performing the above operations on each row of the target candidate in the reachable matrix, the elements corresponding to each row and column with visual reachability in the reachable matrix are set to 1, and the reachable matrix is ​​constructed.

[0045] S140. Determine the rows or columns whose elements are all empty, and perform the following operations on the target candidates corresponding to each empty row or column: define a neighborhood A in the examination room with the current target candidate as the center; assign different weights to each candidate in the neighborhood A according to visual accessibility, and assign another weight to each candidate according to the nearest reachable distance between each candidate in the neighborhood A and other target candidates in the reachable matrix.

[0046] After S130, if all the elements in a row or a column in the reachable matrix are 0, it indicates that the target candidate corresponding to the row or the column has no visual reachability with other target candidates in the initial population. In this case, the target candidate may indirectly transmit information with other target candidates in the initial population through other candidates outside the reachable matrix. This embodiment aims to identify these intermediate candidates who indirectly transmit information.

[0047] In a specific implementation, the following steps may be performed for any blank row of target examinee D:

[0048] Step 1: Define a candidate activity range in the examination room with the current target candidate D as the center.Figure 3 Use a red circle and a red rectangle to represent the target candidate D of the current empty row and their examination equipment. Then, other candidates and examination equipment shown in the figure are all within the neighborhood A of the target candidate. Determine the activity range as shown by the red dotted line with the red circle as the center. This range represents the body movement range of the central target candidate D, such as the range that can be reached by moving the upper body.

[0049] Step 2: Emit rays in different directions starting from the current target candidate D and each point on the boundary line of the activity range. Then, for the front-row candidates and the candidates corresponding to the front-row online examination equipment within the neighborhood A that are first reached by each ray, the back-row candidates within the neighborhood A that are first reached by each ray, and other candidates in the same row as the target candidate within the neighborhood A, they are all candidates with visual accessibility to the current target candidate D, and a weight of 1 is assigned to each of them. Combined with Figure 3 By judging visual accessibility with the help of the candidate's activity range, the range of candidates with visual accessibility can be expanded. For example, in addition to the candidate represented by the green circle, the candidate represented by the blue dot also has visual accessibility, and a weight of 1 is assigned to each of them. The remaining candidates within the neighborhood A without visual accessibility are assigned a weight of 0.

[0050] Step 3: For each candidate with a weight of 1, correct each weight according to the angle of the intersecting rays and the distance from the ray starting point to the intersection point to obtain new non-zero weights. This step further differentiates each candidate with a weight of 1. Although these candidates all have visual accessibility to the current target candidate D, the difficulty of information transmission between them and the current target candidate D is different. This step measures this difficulty based on the rays used to determine visual accessibility and corrects the weights according to the difficulty.

[0051] Optionally, the larger the angle between the intersecting ray corresponding to a candidate with a weight of 1 and the normal forward-looking line of sight of the target candidate without turning the head, and the larger the distance from the ray starting point to the intersection point, the smaller the corrected non-zero weight. Combined with Figure 3 The green arrow represents the direction of the normal forward-looking line of sight of the target candidate. Then, the figure exemplarily shows the angles between the intersecting rays corresponding to two non-zero weights and this line of sight direction. This angle affects the difficulty of information transmission between the current target candidate D and the candidate with a weight of 1 at the ray intersection point. The larger the angle, the higher the difficulty. Therefore, the original weight is corrected to a smaller value. At the same time, the distance from the ray starting point to the intersection point also affects the difficulty of information transmission between the current target candidate D and the candidate with a weight of 1 at the ray intersection point. The larger the distance, the higher the difficulty. Therefore, the original weight is also corrected to a smaller value. Under the above two basic correction principles, the specific correction method can be flexibly set according to needs, and this embodiment does not make specific restrictions.

[0052] Step 4: For each candidate E within the neighborhood A, determine whether there is visual accessibility between the current candidate E and other target candidates in the reachability matrix. If so, select the target candidate with the shortest distance from those with visual accessibility as the nearest reachable distance of the current candidate E. The greater the nearest reachable distance of each candidate E, the greater the difficulty of information transmission, and thus a smaller weight is assigned to each candidate E. If a certain candidate E has no visual accessibility with other target candidates in the reachability matrix, a weight of 0 is assigned to this candidate E.

[0053] In summary, in S140, for each candidate E within the neighborhood A of the target candidate D corresponding to any blank row, a weight (the corrected weight in Step 3) is assigned to each candidate E through Steps 1 to 3, and this weight represents the possibility of information transmission between candidate E and target candidate D; a weight is assigned to each candidate E through Step 4, and this weight represents the possibility of information transmission between candidate E and other target candidates other than D. For the convenience of distinction and description, in this embodiment, the above two weights are respectively referred to as the first weight and the second weight. The two weights adopt different calculation methods mainly for the consideration of the amount of calculation, to maintain a balance between prediction accuracy and execution efficiency.

[0054] S150: The candidates with the largest product of the first weight and the second weight and the initial population jointly form the final population suspected of group cheating and are pushed to the monitoring interface.

[0055] In this embodiment, the product of the first weight and the second weight is used to comprehensively represent the possibility and difficulty of each candidate E within the neighborhood A indirectly transmitting information for the target candidate, and at least one intermediate candidate E with the largest weight product is selected as the candidate most likely to have participated in group cheating but was missed in the identification, and together with the initial population, forms the final population of group cheating and reports it to the background server; at the same time, the video stream, screen stream, audio stream, environmental data of each candidate in the final population, as well as the global video stream of the entire examination room are reported to the background server. The background server adds the final population to the key attention list and displays the key attention list and its related video, audio and other data on the monitoring interface for the invigilator to further judge and identify group cheating behavior.

[0056] It should be noted that the examination data (including video stream, audio stream, screen stream, etc.) involved in this application are all information and data authorized by the candidates or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0057] In summary, this embodiment provides an online exam behavior detection method based on edge computing and multimodal fusion to identify group cheating behaviors in an online exam venue. Since group cheating often involves the co-occurrence of cheating behaviors of multiple candidates, this embodiment records the situations where multiple people cheat continuously for multiple times, and takes the intersection of the students who cheat simultaneously each time as the initial population suspected of group cheating. By taking the intersection, individual cheating behaviors that happen to occur within the same time period can be excluded (i.e., excluding over-identification situations), improving the accuracy of predicting the initial population. For the missed-identification situations where a candidate participates in group cheating but no specific cheating behavior is detected, based on the characteristic that candidates in group cheating transmit information to each other one by one, this embodiment constructs a reachability matrix between every two students in the initial population. As long as one person in the initial population can establish visual contact with another person in the population, it is considered that they meet the conditions for participating in group cheating; for a target candidate who cannot establish visual contact with anyone in the initial population, based on the possibility and difficulty of establishing visual contact between the target candidate and neighboring candidates, an intermediate candidate who may help the two target candidates indirectly transmit information is found from the neighboring candidates, and this intermediate candidate is also included in the final population of group cheating and provided to the background server and the monitoring interface together for the invigilator to further identify. The entire method can automatically identify group cheating behaviors in the exam venue, fully consider the situations of over-identification and missed-identification, and improve the accuracy of identifying group cheating behaviors.

[0058] Figure 4 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present invention, as Figure 4 shown. The device includes a processor 60, a memory 61, an input device 62, and an output device 63; the number of processors 60 in the device can be one or more, Figure 4 and here one processor 60 is taken as an example; the processor 60, the memory 61, the input device 62, and the output device 63 in the device can be connected through a bus or other means, Figure 4 and here the connection through the bus is taken as an example.

[0059] The memory 61, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to the online exam behavior detection method based on edge computing and multimodal fusion in the embodiment of the present invention. The processor 60 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 61, that is, implements the above-mentioned online exam behavior detection method based on edge computing and multimodal fusion.

[0060] The memory 61 may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 61 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some instances, the memory 61 may further include a memory remotely provided with respect to the processor 60, and these remote memories may be connected to the device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0061] The input device 62 can be used to receive input digital or character information, and generate key signal inputs related to the user settings and function controls of the device. The output device 63 may include a display device such as a display screen.

[0062] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the online examination behavior detection method based on edge computing and multimodal fusion in any embodiment.

[0063] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.

[0064] The computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such a propagated data signal may take various forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable medium may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device.

[0065] The program code contained on a computer-readable medium can be transmitted with any appropriate medium, including but not limited to wireless, wire, optical fiber cable, RF, etc., or any suitable combination of the above.

[0066] The computer program code for performing the operations of the present invention can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the C language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., connected through the Internet using an Internet service provider).

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. An online examination behavior detection method based on edge computing and multimodal fusion, characterized in that: Applied to edge devices in examination rooms; The method comprises: Obtaining the co-occurrence of cheating behaviors at different time periods in the examination room, wherein the cheating behaviors include looking around; According to the co-occurrence situation, determine the initial population of suspected group cheating; specifically, if multiple examinees cheat at the same time for multiple consecutive times in any period, record the set of examinees who cheat at the same time each time; take the intersection of each set as the initial population of suspected group cheating; A reachable matrix is ​​constructed with each target examinee in the initial population as a row and column index. If a target examinee in any row has visual reachability with a target examinee in any column, the matrix element where any row and any column intersect is set to 1; Determine the rows or columns where all elements are empty, and perform the following operations on the target candidates corresponding to each empty row or column: define a neighborhood in the examination room with the current target candidate as the center A ; According to visual accessibility, the neighborhood A Each candidate in the neighborhood is given a different first weight, and according to the A The closest reachable distance between each candidate and other target candidates in the reachable matrix is ​​used to assign a second weight to each candidate; The examinees whose product of the first weight and the second weight is the largest and the initial group together constitute the final group of suspected group cheating, which is pushed to the background server to assist in multimodal data fusion analysis and displayed on the monitoring interface for the invigilator to further judge and identify the group cheating behavior.

2. The method according to claim 1, characterized in that The method of obtaining the co-occurrence of cheating behaviors at different time periods in the examination room includes: Through each online test device in the examination room, determine whether the examinees corresponding to each device have cheated; Record the number of candidates and the number of times cheating occurred in each time period.

3. The method according to claim 1, characterized in that If the target examinee in any row and the target examinee in any column have visual accessibility, the matrix elements where the any row and any column intersect are set to 1, including: Define a neighborhood in the examination room with the target candidate in any row as the center B ; Starting from the target candidate, rays in different directions are emitted, and the first rays that are located in the neighborhood are B The first row of candidates or candidates corresponding to the first row of online examination devices in the neighborhood where each ray first reaches B The candidates in the back row, as well as the neighborhood B All candidates who are in the same row as the target candidate are considered candidates who can have visual accessibility to the target candidate; If the target examinee in any column belongs to the examinee with visual accessibility, the matrix element where any row and any column intersect is set to 1.

4. The method according to claim 1, characterized in that: The neighborhood is divided into A Different first weights are assigned to each candidate within the category, including: Taking the current target examinee as the center, define an examinee activity range in the examination room; The rays in different directions are respectively emitted from the current target examinee and each point on the boundary line of the activity range as the starting point, and the points in the neighborhood where each ray first reaches are A The first row of candidates or candidates corresponding to the first row of online examination devices in the neighborhood where each ray first reaches A The candidates in the back row, as well as the neighborhood A Other candidates in the same row as the target candidate are given a weight of 1; For each candidate with a weight of 1, each weight is corrected according to the angle of the intersecting ray and the distance from the starting point of the ray to the intersection point to obtain each non-zero first weight.

5. The method according to claim 1, characterized in that The weights are corrected according to the angle of the intersecting rays and the distance from the starting point of the rays to the intersection point to obtain the first weights that are not zero, including: The larger the angle between the intersecting ray corresponding to each candidate with a weight of 1 and the candidate's normal line of sight, the smaller the weight of each candidate is corrected to; The greater the distance from the starting point of the ray corresponding to each candidate with a weight of 1 to the intersection point, the smaller the weight of each candidate is corrected to.

6. The method according to claim 1, characterized in that The neighborhood A The closest reachable distance between each candidate and other target candidates in the reachable matrix is ​​used to assign a second weight to each candidate, including: For the neighborhood A for each candidate in the reachable matrix, respectively judging whether the current candidate has visual reachability with each other target candidate in the reachable matrix, selecting the one with the shortest distance from the other target candidates with visual reachability, and taking the distance as the shortest reachable distance; The larger the closest reachable distance of each candidate is, the smaller the second weight is assigned to each candidate.

7. The method according to claim 2, characterized in that: The method of judging whether the examinees corresponding to each online examination device in the examination room have cheated includes: Collect video streams, screen streams, audio streams and environmental data in real time through various online test devices in the examination room; Run the lightweight behavior detection model to determine whether the examinees corresponding to each device have cheated.

8. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the online examination behavior detection method based on edge computing and multimodal fusion as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, which, when executed by a processor, implements the online examination behavior detection method based on edge computing and multimodal fusion as described in any one of claims 1-7.

Images