Online examination behavior detection method based on edge calculation and multi-modal fusion

Abstract

The embodiment of the invention discloses an online examination behavior detection method based on edge calculation and multi-modal fusion. The method comprises the following steps: determining an initial group of suspected group cheating according to a cheating behavior co-occurrence condition of each time period in an examination room; constructing a reachable matrix by taking each target examinee in the initial crowd as a row and a column, and if the target examinees in any row and any column have visual reachability, setting the matrix element of the row and the column to be 1; rows or columns with all elements being empty are determined, and for each empty row or column, a neighborhood A in the examination room is delimited with the current target examinee as the center; different first weights are given to the examinees in the visual reachability neighborhood A, and second weights are given to the examinees according to the nearest reachable distances between the examinees in the neighborhood A and other target examinees in the reachability matrix; and enabling the examinee with the maximum product of the first weight and the second weight and the initial crowd to jointly form a final crowd suspected to cheat. According to the embodiment, the group cheating behavior can be automatically identified.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of online education technology, and in particular to an online examination behavior detection method based on edge computing and multimodal fusion. Background Art

[0002] With the popularization of online education, the video stream, screen stream, audio stream, etc. of online examination equipment can more accurately obtain the movements and behaviors of individual candidates, and then detect whether the candidates are suspected of cheating in the examination room.

[0003] However, the existing online examination behavior detection technology is mostly carried out on individual candidates, and it is not possible to accurately identify group cheating. For example, patent CN116883953A provides an online examination anti-cheating method system and storage medium, and patent CN115311735A provides an abnormal behavior intelligent identification and early warning method, but both cannot identify group cheating. Summary of the invention

[0004] An embodiment of the present invention provides an online examination behavior detection method based on edge computing and multimodal fusion to solve the above technical problems.

[0005] In a first aspect, an embodiment of the present invention provides an online examination behavior detection method based on edge computing and multimodal fusion, which is applied to edge devices for examination rooms;

[0006] The method comprises:

[0007] Obtain the co-occurrence of cheating behaviors at different time periods in the examination room;

[0008] Determine an initial group of people suspected of group cheating based on the co-occurrence situation;

[0009] A reachable matrix is ​​constructed with each target examinee in the initial population as a row and column index. If a target examinee in any row has visual reachability with a target examinee in any column, the matrix element where any row and any column intersect is set to 1;

[0010] Determine rows or columns with all elements being empty, and perform the following operations on the target examinees corresponding to each empty row or column: define a neighborhood A in the examination room with the current target examinee as the center; assign different first weights to each examinee in the neighborhood A according to visual accessibility, and assign a second weight to each examinee according to the closest accessible distance between each examinee in the neighborhood A and other target examinees in the reachable matrix;

[0011] The examinees whose product of the first weight and the second weight is the largest and the initial group together constitute the final group of suspected group cheating, which is pushed to the background server to assist multimodal data fusion analysis and displayed on the monitoring interface.

[0012] In a second aspect, an embodiment of the present invention provides an electronic device, the electronic device comprising:

[0013] one or more processors;

[0014] a memory for storing one or more programs,

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the online examination behavior detection method based on edge computing and multimodal fusion described in any embodiment.

[0016] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the online examination behavior detection method based on edge computing and multimodal fusion as described in any embodiment.

[0017] The embodiment of the present invention is an online examination behavior detection method based on edge computing and multimodal fusion, which identifies group cheating behavior in an online examination room. Since group cheating is often accompanied by the co-occurrence of cheating behaviors of multiple examinees, this embodiment determines the initial group of suspected group cheating according to the sharing of cheating behaviors. For the missed identification situation where the examinee participated in group cheating but no specific cheating behavior was detected, this embodiment constructs a reachable matrix between two students in the initial group according to the characteristics of one-to-one information transmission between examinees in group cheating. As long as one person in the initial group can establish a visual connection with another person in the group, they are considered to have the conditions to participate in group cheating; and for the target examinee who cannot establish a visual connection with anyone in the initial group, through the possibility and difficulty of establishing a visual connection between the examinee and the neighboring examinee, find the intermediate examinee from the neighboring examinees who may help the two target examinees to indirectly transmit information, and include the intermediate examinee in the final group cheating population, and provide them to the background server and monitoring interface for further identification by the invigilator. The whole method can consciously identify group cheating behaviors in the examination room, and fully considers the situation of missed identification, thereby improving the accuracy of identifying group cheating behaviors. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1It is a flow chart of an online examination behavior detection method based on edge computing and multimodal fusion provided by an embodiment of the present invention;

[0020] Figure 2 A schematic diagram of the position distribution of some online examination equipment and examinees in an examination room provided by an embodiment of the present invention;

[0021] Figure 3 Another schematic diagram of the position distribution of some online examination equipment and examinees in the examination room provided by an embodiment of the present invention;

[0022] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0024] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0025] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0026] The embodiment of the present invention provides an online examination behavior detection method based on edge computing and multimodal fusion. In order to illustrate the method, an online examination behavior detection system based on edge computing to which the method is applied is first introduced. The system includes edge nodes, background servers, and invigilation panels. By deploying edge nodes at the examination end, the behavior of examinees is monitored in real time and abnormal behavior is automatically reported. At the same time, the background periodically performs fusion analysis on multimodal data such as video streams, screen streams, audio streams, and environmental data, and pushes them to the invigilator for special attention.

[0027] The method of this embodiment is applicable to the case where multiple online examination devices are deployed in a certain examination room and candidates are organized to take a unified online examination. In this case, the edge node can adopt a two-level structure. Among them, the first-level edge node can be deployed on each candidate's device, responsible for real-time acquisition of video streams (such as video streams acquired by computer cameras), screen streams, audio streams and environmental data. The edge node runs a lightweight behavior detection model (such as facial recognition, eye tracking, motion detection) to analyze the candidate's behavior in real time. When abnormal behavior (such as cheating, leaving the seat, using unauthorized devices) is detected, each first-level edge node automatically generates an abnormal event and reports it to the second-level edge node; at the same time, an additional video acquisition device can also be deployed in the examination room, responsible for collecting video streams, audio streams and environmental data of the entire examination room at a high altitude. The device can also be used as a first-level edge node to report the overall data corresponding to the abnormal event to the second-level edge node. The second-level edge node can be deployed in the computer room corresponding to the examination room. It is responsible for analyzing whether there is group cheating in the examination room based on the reported data of each examination room, and reporting the detected group cheating behavior and individual cheating-related video streams, screen streams, audio streams and environmental data to the background server.

[0028] The backend server regularly receives video streams, screen streams, audio streams, and environmental data uploaded by edge nodes, and uses multimodal data fusion algorithms (such as deep learning models) to conduct comprehensive analysis of the data and generate a report on the examinee's behavior. At the same time, the backend server generates a focus list based on the results of multimodal data analysis and pushes it to the invigilator. The invigilator can view detailed data and video clips of abnormal behavior through the invigilation panel.

[0029] Based on the above system architecture, Figure 1 This is a flow chart of an online examination behavior detection method based on edge computing and multimodal fusion provided by an embodiment of the present invention. This method is applicable to the case of identifying group cheating behavior in an online examination room, and is executed by the second-level edge node or other electronic devices in the above system. The following will take the second edge node executing this method as an example to illustrate. Figure 1 As shown, the method specifically includes:

[0030] S110. Obtain the co-occurrence of cheating behaviors in each time period in the examination room.

[0031] As described above, by deploying the first edge node of each online test device in the examination room, the cheating behavior of each examinee can be detected. The cheating behavior does not need to be particularly accurate, and can be understood as abnormal behavior with a high possibility of cheating, such as frequent lowering of the head, looking around, body movement, long-term non-operation, and shooting of display images. Optionally, the first edge node can use a lightweight deep learning model (such as MobileNet) for facial recognition and motion detection, or use eye tracking technology (such as Gaze Tracking) to monitor the examinee's line of sight. When any of facial recognition, motion detection, or line of sight monitoring considers that the examinee has the above abnormal behavior, it is considered that a cheating behavior is detected, and the timestamp, behavior type, and corresponding student ID of the cheating behavior are all reported to the second-level edge node.

[0032] Then, the number of candidates and the number of times cheating occurs at the same time in each time period are recorded. Specifically, the second edge node divides the entire examination process into multiple overlapping or non-overlapping time periods, and records the number of candidates and the number of times cheating occurs at the same time in each time period. Optionally, if the time difference between cheating behaviors of different students is less than a set threshold (such as 1 minute, 2 minutes, etc.), these cheating behaviors are considered to occur at the same time.

[0033] S120: Determine an initial group of people suspected of group cheating based on the co-occurrence situation.

[0034] Optionally, if multiple examinees cheat at the same time for multiple consecutive times within any period of time, the set of examinees that cheat at the same time each time is recorded; and the intersection of each set is taken as the initial population of suspected group cheating.

[0035] For example, assuming that the duration of each time period is 15 minutes, if multiple candidates cheat at the same time for three consecutive times within a certain time period (for example, multiple candidates cheated in the first minute, multiple candidates cheated in the second minute, and multiple candidates cheated again in the third minute), then the sets U1, U2 and U3 of candidates who cheated each time will be recorded respectively, and the intersection of the three sets will be taken, and the intersection will be used as the initial population suspected of group cheating.

[0036] By taking the intersection, individual cheating behaviors that happen within these 3 minutes can be excluded, improving the accuracy of the initial group prediction for group cheating. However, there may be some candidates who participated in group cheating but whose specific cheating behaviors were not detected in the first edge node, resulting in the omission of candidates in the cheating group. These omitted candidates will be identified in S130-S140.

[0037] S130. Construct a reachable matrix using the target candidates in the initial population as row and column indexes. If the target candidate in any row is visually reachable to the target candidate in any column, set the matrix element where any row and any column intersect to 1.

[0038] In order to distinguish them from other candidates, in this embodiment, the candidates in the initial group suspected of group cheating are called target candidates, and a reachable matrix is ​​used to describe whether it is possible for each two target candidates to transmit cheating information to each other by secretly glancing at the screen of the online examination device, the candidates' movements, etc. This embodiment calls this possibility visual reachability.

[0039] Specifically, each row and each column in the reachability matrix corresponds to a target candidate, and the matrix element represents whether the target candidate in the row where the element is located has visual reachability with the target candidate in the column where the element is located. For example, for the target candidate in the i-th row of the matrix, if he has visual reachability with the target candidate in the j-th column of the matrix, the element at the (i, j) position of the matrix is ​​set to 1.

[0040] In one embodiment, in combination Figure 2 , the element value of the i-th row of the matrix can be determined by the following steps:

[0041] Step 1: Define a neighborhood in the examination room with the target candidate C in the i-th row as the center. Optional: Figure 2 The figure is a schematic diagram of the location distribution of some online test equipment and examinees in the test room. Each solid rectangle represents the screen of the online test equipment, and each solid circle represents the examinee. Assuming that the target examinee and his test equipment corresponding to the i-th row in the reachable matrix are the red circle and the red rectangle respectively, an area is defined with the examinee as the center and the average visual range of a person as the radius, and the neighborhood B in this step is obtained ( Figure 2 Only a part of neighborhood B is shown in the figure).

[0042] Step 2: Starting from the target candidate C, rays are emitted in different directions, and the front row candidates or candidates corresponding to the front row online test devices located in the neighborhood B that are first reached by each ray, the back row candidates located in the neighborhood B that are first reached by each ray, and the candidates in the same row as the target candidate in the neighborhood B are all regarded as candidates who can have visual accessibility to the target candidate C in any row. Figure 2, the front row, same row and back row here are all relative to the target candidate C represented by the red circle. For the front row, the target candidate can see the test equipment screen and the candidates in the front row, and can complete the cheating information transmission through the screen content or the candidates' movements, eyes, etc.; for the same row, the candidates can see each other by moving their bodies forward and backward, and thus complete the cheating information transmission through movements, eyes, etc.; for the back row, the target candidate cannot see the screen of the test equipment, and can only complete the cheating information transmission through movements, eyes, etc. between the candidates. Therefore, in this step, different judgment criteria are used for the visual accessibility of the front row, same row and back row of the target test, and the candidates in the neighborhood that meet the respective criteria are all candidates with visual accessibility. Exemplarily, with Figure 2 For example, assuming that the candidates shown in the figure are all candidates within the neighborhood B of the target candidate, then in the area to the left of the target candidate, the candidates with visual accessibility are represented by the green circle, and the other candidates in the area to the left are initially considered to not have visual accessibility.

[0043] Step 3: If the target candidate in the jth column of the reachability matrix is ​​a candidate who has visual reachability with the target candidate in the i-th row of the reachability matrix determined in step 2, then the element at position (i, j) in the reachability matrix is ​​set to 1. The elements on the diagonal of the reachability matrix are set to 0 by default.

[0044] After performing the above operations on each row of the target candidate in the reachable matrix, the elements corresponding to each row and column with visual reachability in the reachable matrix are set to 1, and the reachable matrix is ​​constructed.

[0045] S140. Determine the rows or columns whose elements are all empty, and perform the following operations on the target candidates corresponding to each empty row or column: define a neighborhood A in the examination room with the current target candidate as the center; assign different weights to each candidate in the neighborhood A according to visual accessibility, and assign another weight to each candidate according to the nearest reachable distance between each candidate in the neighborhood A and other target candidates in the reachable matrix.

[0046] After S130, if all the elements in a row or a column in the reachable matrix are 0, it indicates that the target candidate corresponding to the row or the column has no visual reachability with other target candidates in the initial population. In this case, the target candidate may indirectly transmit information with other target candidates in the initial population through other candidates outside the reachable matrix. This embodiment aims to identify these intermediate candidates who indirectly transmit information.

[0047] In a specific implementation, the following steps may be performed for any blank row of target examinee D:

[0048] Step 1: Define a candidate activity range in the examination room with the current target candidate D as the center. Figure 3 , the red circle and red rectangle are used to represent the target candidate D and his / her test equipment in the current empty row, and the other candidates and test equipment shown in the figure are all located in the neighborhood A of the target candidate. The activity range shown by the red dotted line is determined with the red circle as the center, which represents the body movement range of the central target candidate D, such as the range that can be reached by moving the upper body.

[0049] Step 2: Use the current target candidate D and each point on the boundary line of the activity range as starting points to emit rays in different directions. The front row candidates and candidates corresponding to the front row online test devices located in the neighborhood A that are first reached by each ray, the back row candidates located in the neighborhood A that are first reached by each ray, and other candidates in the same row as the target candidate in the neighborhood A are all candidates who have visual accessibility to the current target candidate D, and all of them are given a weight of 1. Figure 3 , judging visual accessibility by the activity range of candidates, can expand the range of candidates with visual accessibility. For example, in addition to the candidates represented by green circles, the candidates represented by blue dots also have visual accessibility and are all assigned a weight of 1. The remaining candidates in neighborhood A who do not have visual accessibility are assigned a weight of 0.

[0050] Step 3: For each candidate with a weight of 1, the weights are modified according to the angle of the intersecting ray and the distance from the starting point of the ray to the intersection point to obtain new non-zero weights. This step further distinguishes the candidates with a weight of 1. Although these candidates have visual accessibility with the current target candidate D, the difficulty of information transmission between them and the current target candidate D is different. This step measures the difficulty according to the ray used to determine the visual accessibility, and modifies the weight according to the difficulty.

[0051] Optionally, the larger the angle between the intersection ray corresponding to a candidate with a weight of 1 and the normal horizontal sight line of the target candidate without turning his head, and the larger the distance from the starting point of the ray to the intersection point, the smaller the corrected non-zero weight. Figure 3 , the green arrow indicates the direction of the normal horizontal line of sight of the target candidate, and the figure exemplarily shows the angle between the two intersecting rays corresponding to non-zero weights and the line of sight direction. This angle affects the difficulty of information transmission between the current target candidate D and the candidate with a weight of 1 at the intersection of the rays. The larger the angle, the higher the difficulty, so the original weight is corrected to a smaller value. At the same time, the distance from the starting point of the ray to the intersection also affects the difficulty of information transmission between the current target candidate D and the candidate with a weight of 1 at the intersection of the rays. The larger the distance, the higher the difficulty, so the original weight is also corrected to a smaller value. Under the above two basic correction principles, the specific correction method can be flexibly set as needed, and this embodiment does not make specific restrictions.

[0052] Step 4: for each candidate E in the neighborhood A, determine whether the current candidate E has visual accessibility with other target candidates in the reachable matrix. If so, select the one with the closest distance from other target candidates with visual accessibility, and use the distance as the closest reachable distance of the current candidate E. The larger the closest reachable distance of each candidate E, the greater the difficulty of information transmission, and a smaller weight is assigned to each candidate E. If a candidate E does not have visual accessibility with other target candidates in the reachable matrix, another weight of 0 is assigned to the candidate E.

[0053] In summary, in S140, for each candidate E in the neighborhood A of the target candidate D corresponding to any blank row, a weight (referring to the weight corrected in step three) is assigned to each candidate E through steps one to three, and the weight represents the possibility of information transmission between candidate E and the target candidate D; and a weight is assigned to each candidate E through step four, and the weight represents the possibility of information transmission between candidate E and other target candidates other than D. For the convenience of distinction and description, the above two weights are referred to as the first weight and the second weight, respectively, in this embodiment. The two weights are calculated using different methods, mainly for the consideration of the amount of calculation, to maintain a balance between prediction accuracy and execution efficiency.

[0054] S150. The examinees whose product of the first weight and the second weight is the largest and the initial group together constitute the final group of people suspected of group cheating, and push it to the monitoring interface.

[0055] This embodiment uses the product of the first weight and the second weight to comprehensively represent the possibility and difficulty of each candidate E in the neighborhood A indirectly transmitting information to the target candidate, and selects at least one intermediate candidate E with the largest weight product as the candidate who is most likely to have participated in the group cheating but was missed to form the final group of group cheating together with the initial group and report it to the backend server; at the same time, the video stream, screen stream, audio stream, environmental data of each candidate in the final group, and the global video stream of the entire examination room are all reported to the backend server, and the backend server adds the final group to the key focus list, and displays the key focus list and its related video, audio and other data on the monitoring interface, so that the invigilator can further judge and identify the group cheating behavior.

[0056] It should be noted that the test data involved in this application (including video streams, audio streams, screen streams, etc.) are all information and data authorized by the candidates or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0057] In summary, this embodiment provides an online examination behavior detection method based on edge computing and multimodal fusion to identify group cheating behaviors in online examination rooms. Since group cheating is often accompanied by the co-occurrence of cheating behaviors by multiple examinees, this embodiment records the situation where multiple people cheat for many consecutive times, and takes the intersection of students who cheat each time as the initial population of suspected group cheating. By taking the intersection, individual cheating behaviors that happen to occur in the same time period can be excluded (i.e., multiple identification situations are excluded), thereby improving the accuracy of the initial population prediction. As for the missed identification of candidates who participated in group cheating but no specific cheating behavior was detected, this embodiment constructs a reachable matrix between two students in the initial group based on the characteristics of one-to-one information transmission between candidates in group cheating. As long as one person in the initial group can establish a visual connection with another person in the group, they are considered to have the conditions to participate in group cheating; and for the target candidate who cannot establish a visual connection with anyone in the initial group, the possibility and difficulty of establishing a visual connection between him and the neighboring candidates are used to find the intermediate candidates who may help the two target candidates to indirectly transmit information, and the intermediate candidates are also included in the final group of group cheating, and are provided to the background server and monitoring interface for further identification by the invigilator. The whole method can consciously identify the group cheating behavior in the examination room, and fully considers the situations of multiple identification and missed identification, thereby improving the identification accuracy of group cheating behavior.

[0058] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown in FIG. Figure 4 As shown, the device includes a processor 60, a memory 61, an input device 62 and an output device 63; the number of processors 60 in the device can be one or more. Figure 4 A processor 60 is taken as an example; the processor 60, the memory 61, the input device 62 and the output device 63 in the device can be connected by a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0059] The memory 61, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the online test behavior detection method based on edge computing and multimodal fusion in the embodiment of the present invention. The processor 60 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 61, that is, realizing the above-mentioned online test behavior detection method based on edge computing and multimodal fusion.

[0060] The memory 61 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 61 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 61 may further include a memory remotely arranged relative to the processor 60, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0061] The input device 62 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 63 may include a display device such as a display screen.

[0062] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the online examination behavior detection method based on edge computing and multimodal fusion of any embodiment.

[0063] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0064] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0065] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0066] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. An online examination behavior detection method based on edge computing and multimodal fusion, characterized in that: Applied to edge devices for examination rooms; the method comprises: Obtain the co-occurrence of cheating behaviors at different time periods in the examination room; Determine an initial group of people suspected of group cheating based on the co-occurrence situation; A reachable matrix is ​​constructed with each target examinee in the initial population as a row and column index. If a target examinee in any row has visual reachability with a target examinee in any column, the matrix element where any row and any column intersect is set to 1; Determine rows or columns with all elements being empty, and perform the following operations on the target examinees corresponding to each empty row or column: define a neighborhood A in the examination room with the current target examinee as the center; assign different first weights to each examinee in the neighborhood A according to visual accessibility, and assign a second weight to each examinee according to the closest accessible distance between each examinee in the neighborhood A and other target examinees in the reachable matrix; The examinees whose product of the first weight and the second weight is the largest and the initial group together constitute the final group of suspected group cheating, which is pushed to the background server to assist multimodal data fusion analysis and displayed on the monitoring interface.

2. The method according to claim 1, characterized in that The method of obtaining the co-occurrence of cheating behaviors at different time periods in the examination room includes: Through each online test device in the examination room, determine whether the examinees corresponding to each device have cheated; Record the number of candidates and the number of times cheating occurred in each time period.

3. The method according to claim 1, characterized in that Based on the co-occurrence, the initial group of people suspected of group cheating is determined, including: If multiple candidates cheat at the same time for multiple times in a period of time, the set of candidates who cheat at the same time will be recorded; Take the intersection of each set as the initial population suspected of group cheating.

4. The method according to claim 1, characterized in that If the target examinee in any row and the target examinee in any column have visual accessibility, the matrix elements where the any row and any column intersect are set to 1, including: Taking any row of target examinees as the center, define a neighborhood B in the examination room; Taking the target examinee as the starting point, rays in different directions are emitted, and the examinees in the front row or the examinees corresponding to the front row online test devices located in the neighborhood B that are reached by each ray for the first time, the examinees in the back row located in the neighborhood B that are reached by each ray for the first time, and the examinees in the same row as the target examinee in the neighborhood B are all regarded as examinees who can have visual accessibility with the target examinee; If the target examinee in any column belongs to the examinee with visual accessibility, the matrix element where any row and any column intersect is set to 1.

5. The method according to claim 1, characterized in that The step of assigning different first weights to each examinee in the neighborhood A according to visual accessibility includes: Taking the current target examinee as the center, define an examinee activity range in the examination room; Taking the current target examinee and each point on the boundary line of the activity range as starting points, rays in different directions are emitted, and a weight of 1 is assigned to the front row examinee or the examinee corresponding to the front row online test device in the neighborhood A reached for the first time by each ray, the back row examinee in the neighborhood A reached for the first time by each ray, and other examinees in the same row as the target examinee in the neighborhood A; For each candidate with a weight of 1, each weight is corrected according to the angle of the intersecting ray and the distance from the starting point of the ray to the intersection point to obtain each non-zero first weight.

6. The method according to claim 1, characterized in that The weights are corrected according to the angle of the intersecting rays and the distance from the starting point of the rays to the intersection point to obtain the first weights that are not zero, including: The larger the angle between the intersecting ray corresponding to each candidate with a weight of 1 and the candidate's normal line of sight, the smaller the weight of each candidate is corrected to; The greater the distance from the starting point of the ray corresponding to each candidate with a weight of 1 to the intersection point, the smaller the weight of each candidate is corrected to.

7. The method according to claim 1, characterized in that The assigning a second weight to each candidate according to the closest reachable distance between each candidate in the neighborhood A and other target candidates in the reachable matrix includes: For each examinee in the neighborhood A, determine whether the current examinee has visual reachability with each other target examinee in the reachability matrix, select the one with the shortest distance from the other target examinees with visual reachability, and use the distance as the shortest reachable distance; The larger the closest reachable distance of each candidate is, the smaller the second weight is assigned to each candidate.

8. The method according to claim 2, characterized in that: The method of judging whether the examinees corresponding to each online examination device in the examination room have cheated includes: Collect video streams, screen streams, audio streams and environmental data in real time through various online test devices in the examination room; Run the lightweight behavior detection model to determine whether the examinees corresponding to each device have cheated.

9. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the online examination behavior detection method based on edge computing and multimodal fusion as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, which, when executed by a processor, implements the online examination behavior detection method based on edge computing and multimodal fusion as described in any one of claims 1-8.

Images