An anomaly detection method, system, device, storage medium, and electronic device.
By acquiring the row-level pixel density vectors of textbook pages, analyzing reading patterns and dividing content blocks, accumulating reading time and speed, and generating a document density map, the shortcomings of existing technologies such as optical character recognition and hash value matching are solved, and accurate detection of abnormal reading behavior is achieved.
Patent Information
- Application Number
- CN202610968019.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-01
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2046-07-01
AI Technical Summary
In existing technologies, optical character recognition cannot effectively recognize images and formulas, consumes a lot of computing resources, and the hash value matching of the whole frame image cannot accurately locate the reading position. Based on the total screen dwell time, it is impossible to distinguish the actual reading time of different areas of the textbook, resulting in low accuracy in detecting abnormal reading behavior.
By obtaining the row-level pixel density vector of the textbook page, the correlation between the current and previous reading frames is analyzed, content blocks are divided, effective reading time and speed are accumulated, a document density map is generated, and anomaly detection is performed by combining the group baseline data from the education service terminal.
It enables precise location of specific reading positions within the textbook page, distinguishes the actual reading situation in different areas of the textbook, and improves the accuracy of detecting abnormal reading behavior.
Smart Images

Figure CN122531032B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of educational technology, and in particular to an anomaly detection method, system, device, storage medium, and electronic device. Background Technology
[0002] In smart classroom teaching, students read the same electronic textbook simultaneously through tablet terminals, and abnormal reading behavior needs to be detected during the reading process.
[0003] Currently, existing anomaly detection technologies typically use optical character recognition (OCR) or whole-frame image hash value matching to locate reading content, and use the total dwell time on the screen of the electronic textbook application as the reading time to complete reading behavior recording and anomaly detection.
[0004] However, optical character recognition can only recognize text content and cannot effectively recognize images or formulas, and the recognition process consumes a lot of computational resources; whole-frame image hash value matching can only locate the reading page and cannot accurately locate the specific reading position on the page; reading time is calculated based on the total screen dwell time, which cannot distinguish the actual reading time in different areas of the textbook, resulting in low accuracy in detecting abnormal reading behavior. Summary of the Invention
[0005] In view of this, this application provides an anomaly detection method, system, device, storage medium, and electronic device. The main purpose is to improve the technical problems of existing technologies that use optical character recognition methods, which can only recognize text content and cannot effectively recognize images and formulas, and that the recognition process consumes a lot of computational resources; use whole-frame image hash value matching methods, which can only locate the reading page and cannot accurately locate the specific reading position on the page; and use total screen dwell time to calculate reading time, which cannot distinguish the actual reading time of different areas of the textbook, resulting in low accuracy in detecting abnormal reading behavior.
[0006] Firstly, this application provides an anomaly detection method for use in educational terminals, comprising: Obtain the textbook page of each student in the target class who is reading the electronic textbook, and compress the textbook page by horizontal projection line by line to obtain the row-level pixel density vector corresponding to the textbook page; By performing correlation analysis on the row-level pixel density vectors of the current reading frame and the previous reading frame of the textbook page, the current reading mode of each student is identified, and the textbook page is divided into multiple content blocks according to pixel density. Based on the reading progress data of each student in the current reading mode, the reading position of each student in the textbook page and the reading content block corresponding to the reading position are determined. When the reading content block is within the effective reading area of the educational terminal's screen, the reading time spent by each student in the reading content block is accumulated, and the actual reading situation of each student in the reading content block within the effective reading area of the screen is analyzed to obtain the effective reading time and effective reading speed of each student in the reading content block. The effective reading time and effective reading speed are concatenated in each reading content block according to the row-level pixel density vector of each reading content block to obtain the document density map of each student. The document density map of each student is then sent to the education server so that the education server can detect abnormal reading behavior of each student in the target class based on the group baseline data of the target class and the document density map of each student.
[0007] Optionally, the step of identifying the current reading mode of each student by performing correlation analysis on the row-level pixel density vectors of the current reading frame and the previous reading frame of the textbook page, dividing the textbook page into multiple content blocks according to pixel density, and determining the reading position of each student on the textbook page and the reading content block corresponding to the reading position based on the reading progress data of each student in the current reading mode, includes: Calculate the correlation coefficient between the row-level pixel density vectors of the current reading frame and the previous reading frame of the textbook page, and analyze the pixel density similarity between the current reading frame and the previous reading frame; When the correlation coefficient satisfies the correlation condition, the pixel density similarity between the current reading frame and the previous reading frame is high. The pixel offset between the current reading frame and the previous reading frame is analyzed. The pixel offset is the amount of pixels that the current reading frame has shifted in the vertical direction relative to the previous reading frame. When the pixel offset satisfies the scrolling condition, the current reading frame is shifted by pixels in the vertical direction relative to the previous reading frame, and the current reading mode is determined to be the scrolling mode. The row-level pixel density vectors of the current reading frame and the previous reading frame are concatenated to obtain the first document density map corresponding to each student. The pixel offsets of the current reading frame and the previous reading frame in the first document density map are analyzed, and the pixel offsets are determined as the scroll displacement of the current reading frame relative to the previous reading frame. The historical reading progress data is updated based on the scroll displacement, and the reading position of each student is determined based on the updated reading progress data. Among the multiple content blocks, determine the reading content block corresponding to the reading position of each student.
[0008] Optionally, the method further includes: If the correlation coefficient does not meet the correlation conditions, the pixel density similarity between the current reading frame and the previous reading frame is low. The current reading mode is determined to be a page-turning mode. The initial position information corresponding to the page-turning page after page-turning is determined as the page-turning reading progress data corresponding to the page-turning mode. The reading position corresponding to the page-turning reading progress data is determined as the page-turning reading position of each student. The reading content block corresponding to the page-turning reading position is determined among the multiple content blocks. If the pixel offset does not meet the scrolling condition, and the current reading frame does not have a pixel translation in the vertical direction relative to the previous reading frame, the current reading mode is determined to be a static mode, the historical reading progress data is determined to be the reading progress data in the static mode, the reading position corresponding to the historical reading progress data is determined to be the static reading position of each student, and the reading content block corresponding to the static reading position is determined among the multiple content blocks.
[0009] Optionally, the textbook page can be divided into multiple content blocks according to pixel density, including: In the row-level pixel density vector corresponding to the textbook page, continuous low-density regions are identified, and the textbook page is divided into multiple blocks based on the continuous low-density regions; Based on the content data and row-level pixel density vectors of the multiple blocks, the content features and pixel density features of the multiple blocks are extracted, and the block type of the multiple blocks is identified according to the content features and pixel density features to obtain the content blocks corresponding to the multiple blocks.
[0010] Optionally, when the reading content block is within the effective reading area of the educational terminal's screen, the reading time spent by each student in the reading content block is accumulated, and the actual reading situation of each student in the reading content block within the effective reading area of the screen is analyzed to obtain the effective reading time and effective reading speed of each student in the reading content block, including: When the front-facing camera of the educational terminal is available, the human eye and eyeball area of each student are located by the front-facing camera to obtain the pupil gaze direction of each student, and the gaze area corresponding to the pupil gaze direction is mapped on the screen of the educational terminal to obtain the effective reading area of each student on the screen of the educational terminal. When the front-facing camera of the educational terminal is unavailable, the target center area of the screen in the vertical direction is determined as the effective reading area of the screen. Within the target sampling time, determine the vertical overlap between the reading content block corresponding to the reading position and the effective reading area. If the vertical overlap satisfies the overlap condition, the reading content block is within the effective reading area of the educational terminal's screen. The target sampling time, the vertical overlap, and the block height of the reading content block corresponding to the reading position are fused together to analyze the actual reading situation of each student in the effective reading area of the screen for the reading content block, and to obtain the effective screen reading time of each student in the reading content block corresponding to the reading position. The effective reading speed of the reading content block corresponding to the reading position within the effective reading time is calculated based on the effective reading time and the number of lines of text contained in the reading content block corresponding to the reading position.
[0011] Secondly, this application provides an anomaly detection method applied to an educational service provider, including: The reading content blocks, effective reading time, and effective reading speed of each student in the document density map of the target class are fused together to obtain the group baseline data of the target class. The target reading time for each reading content block is determined from the effective reading time. The target reading time is compared with the median reading time data in the group baseline data. The deviation of each student's reading time for each reading content block from the median reading time of the target class is analyzed to obtain the time difference data of each student reading each content block. By comparing the duration difference data with the discrete duration data in the group baseline data, the degree of deviation of each student's reading time from the target duration of the target class is analyzed, and the reading deviation data of each student is obtained. Based on the reading deviation data of each student, the reading behavior of each student is detected, and reading behaviors that meet the abnormal behavior conditions are identified as abnormal reading behaviors, and alarms are issued for the abnormal reading behaviors.
[0012] Optionally, the process of fusing the reading content blocks, effective reading time, and effective reading speed in the document density map of each student in the target class to obtain the group baseline data of the target class includes: Analyze the pixel similarity between reading content blocks in the document density map of each student, merge the content blocks that meet the similarity conditions, and obtain multiple merged content blocks of the target class; The multiple merged content blocks of the target class are arranged according to the order of reading position to obtain the second document density map corresponding to the electronic textbook; Based on the textbook content structure in the second document density map, the reading content block, effective reading time, and reading speed corresponding to each student's reading position are fused to obtain the group baseline data of the target class.
[0013] Optionally, the step of detecting the reading behavior of each student based on the reading deviation data of each student, identifying reading behaviors that meet the abnormal behavior conditions as abnormal reading behaviors, and issuing an alarm for the abnormal reading behaviors includes: Based on the reading progress data of each student, the reading order of the reading content blocks is marked to obtain the reading order data of each student; Based on the screen operation behavior data of each student, it is determined whether the face and eyeballs of each student are within the screen range of the educational terminal, and the student's attention detection data is obtained; Based on the reading deviation data, reading order data, and attention detection data of each student, the system analyzes whether there are abnormalities in the dwell time, reading order, and attention state of the reading content blocks in each student's reading behavior. Reading behaviors that meet the abnormal behavior conditions are identified as abnormal reading behaviors, and alarms are issued for the abnormal reading behaviors.
[0014] Thirdly, this application provides an anomaly detection device for use in educational terminals, comprising: The acquisition module is configured to acquire the textbook page of each student in the target class reading the electronic textbook, and to perform line-by-line horizontal projection compression on the textbook page to obtain the row-level pixel density vector corresponding to the textbook page; The determination module is configured to identify the current reading mode of each student by performing correlation analysis on the row-level pixel density vectors of the current reading frame and the previous reading frame of the textbook page, divide the textbook page into multiple content blocks according to pixel density, and determine the reading position of each student in the textbook page and the reading content block corresponding to the reading position based on the reading progress data of each student in the current reading mode. The analysis module is configured to, when the reading content block is in the effective reading area of the screen of the educational terminal, accumulate the reading time of each student in the reading content block, analyze the actual reading situation of each student in the reading content block in the effective reading area of the screen, and obtain the effective reading time of each student in the reading content block and the effective reading speed within the effective reading time. The sending module is configured to concatenate the effective reading time and the effective reading speed in each reading content block according to the row-level pixel density vector of each reading content block to obtain the document density map of each student, and send the document density map of each student to the education server so that the education server can detect abnormal reading behavior of each student in the target class based on the group baseline data of the target class and the document density map of each student.
[0015] Fourthly, this application provides an anomaly detection device for use in educational service providers, comprising: The fusion module is configured to fuse the reading content blocks, effective reading time, and effective reading speed of each student in the document density map of the target class to obtain the group baseline data of the target class. The analysis module is configured to compare the effective reading time of each student in the reading content block with the median reading time data in the group baseline data of the target class, analyze the individual reading time deviation of each student and the target class, and obtain the reading time difference data. The analysis module is also configured to analyze the degree of deviation of each student's individual reading time relative to the overall deviation of the target class based on the discrete time data in the time difference data and the group baseline data of the target class, and determine the reading deviation data of each student relative to the target class. The detection module is configured to detect the reading behavior of each student based on the reading deviation data of each student, identify reading behaviors that meet the abnormal behavior conditions as abnormal reading behaviors, and issue an alarm for the abnormal reading behaviors.
[0016] Using the above technical solution, this application provides an anomaly detection method, device, storage medium, and electronic device, comprising: acquiring the textbook page of each student in a target class reading an electronic textbook, and compressing the textbook page line by line using horizontal projection to obtain the row-level pixel density vector corresponding to the textbook page; identifying the current reading mode of each student by performing correlation analysis on the row-level pixel density vectors of the current reading frame and the previous reading frame of the textbook page, and dividing the textbook page into multiple content blocks according to pixel density; determining the reading position of each student in the textbook page and the reading content block corresponding to the reading position based on the reading progress data of each student in the current reading mode; and displaying the reading content block on the screen of the educational terminal. Within the effective reading area, the reading time of each student in the reading content block is accumulated, and the actual reading situation of each student in the reading content block within the effective reading area of the screen is analyzed to obtain the effective reading time and effective reading speed of each student in the reading content block. Based on the row-level pixel density vector of each reading content block, the effective reading time and effective reading speed are concatenated in each reading content block to obtain the document density map of each student. The document density map of each student is then sent to the education server so that the education server can detect abnormal reading behavior of each student in the target class based on the group baseline data of the target class and the document density map of each student. Compared with existing technologies, this application identifies student reading patterns by performing correlation analysis on the row-level pixel density vectors of consecutive reading frames, divides textbook content into blocks based on pixel density, and matches student reading positions with corresponding content blocks to achieve precise positioning of specific reading locations within the textbook page; by only counting the reading time of content blocks within the effective reading area of the screen and calculating the effective reading duration and effective reading speed, it distinguishes the actual reading situation in different areas of the textbook and achieves accurate accumulation of effective reading time; by stitching together the reading features of each content block to generate a student document density map and reporting it to the education service terminal, it achieves personalized abnormal reading behavior detection and effectively improves the accuracy of detecting abnormal reading behavior in electronic textbooks. Attached Figure Description
[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1A flowchart illustrating an anomaly detection method provided in an embodiment of this application is shown. Figure 2 A flowchart illustrating an anomaly detection method provided in an embodiment of this application is shown. Figure 3 A flowchart illustrating an anomaly detection method provided in an embodiment of this application is shown. Figure 4 A flowchart illustrating an anomaly detection method provided in an embodiment of this application is shown. Figure 5 This paper shows a schematic diagram of the structure of an anomaly detection device provided in an embodiment of this application; Figure 6 This paper shows a schematic diagram of the structure of an anomaly detection device provided in an embodiment of this application; Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation
[0020] The embodiments of this application will now be described in more detail with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.
[0021] To address the shortcomings of existing technologies, such as optical character recognition (OCR) which only recognizes text and cannot effectively recognize images or formulas, and which consumes significant computational resources; frame-by-frame image hash value matching which can only locate the page but not the specific reading position within the page; and reading time calculated based on total screen dwell time which cannot differentiate between actual reading time in different areas of the textbook, resulting in low accuracy in detecting abnormal reading behavior, this embodiment provides an anomaly detection method. Figure 1 As shown, applied to a teaching terminal, the method includes: Step 101: Obtain the textbook page of each student in the target class who is reading the electronic textbook, and compress the textbook page by horizontal projection line by line to obtain the row-level pixel density vector corresponding to the textbook page.
[0022] In this embodiment of the application, the target class can be a class that conducts teaching with electronic textbooks. For example, in this embodiment of the application, the target class can specifically be a teaching class in the K-12 stage.
[0023] In this embodiment of the application, the electronic teaching material can be a digital teaching text material applied to an educational terminal. The electronic teaching material can include various content formats such as text paragraphs, titles, images, charts, and formulas. For example, the electronic teaching material in this embodiment can specifically be a classroom-synchronized teaching material loaded into a third-party application.
[0024] In this embodiment, the educational terminal can be a tablet device used by students, and the educational terminal can support functions such as electronic textbook display, screen capture, eye detection, and data reporting.
[0025] In the embodiments of this application, the teaching material page can be a single-page visual content of an electronic teaching material displayed on an educational terminal screen.
[0026] In the embodiments of this application, line-by-line horizontal projection compression can be a process of performing horizontal pixel statistics and compression on the pixel data of the textbook page line by line in the vertical direction. Line-by-line horizontal projection compression can convert a two-dimensional textbook page image into one-dimensional feature data.
[0027] In the embodiments of this application, the row-level pixel density vector can be a one-dimensional feature vector that represents the foreground distribution of pixels in each row of the textbook page. The row-level pixel density vector can be used as the density fingerprint of the textbook page content. The density fingerprint can be used for content block matching and alignment between different students.
[0028] In this embodiment of the application, the textbook page of each student in the target class reading the electronic textbook can be obtained through the system-level screenshot API of the education terminal. In order to reduce the power consumption of the device, an adaptive sampling strategy can be used to collect screen frames. The adaptive sampling strategy can include active mode, changing mode, static mode and idle mode. Different modes can correspond to different collection frame rates to balance detection accuracy and device power consumption.
[0029] In the embodiments of this application, after obtaining the textbook page, the textbook page can be processed sequentially by grayscale conversion, adaptive binarization, line-by-line horizontal projection calculation, and normalization to obtain the row-level pixel density vector corresponding to the textbook page. This process has low computational complexity and can reduce resource consumption and save computing resources compared to traditional optical character recognition schemes.
[0030] Step 102: By performing correlation analysis on the row-level pixel density vectors of the current reading frame and the previous reading frame of the textbook page, the current reading mode of each student is identified, and the textbook page is divided into multiple content blocks according to pixel density. Based on the reading progress data of each student in the current reading mode, the reading position of each student in the textbook page and the reading content block corresponding to the reading position are determined.
[0031] In this embodiment, the current reading frame can be the screen frame of the textbook page currently collected by the educational terminal, and the previous reading frame can be the screen frame of the textbook page collected by the educational terminal at the previous moment. The current reading frame and the previous reading frame can be used to compare and analyze changes in reading behavior.
[0032] In this embodiment of the application, correlation analysis can be performed by analyzing the similarity and displacement relationship between the row-level pixel density vectors of two reading frames. Correlation analysis can be used to identify students' reading patterns. For example, the correlation analysis in this embodiment can specifically employ a normalized cross-correlation calculation method, including but not limited to the Pearson correlation coefficient calculation method.
[0033] In this embodiment, the reading mode can be the operation mode for students to read electronic textbooks. The reading mode can include scrolling mode, page turning mode and static mode. Different reading modes can correspond to different reading progress update rules.
[0034] In the embodiments of this application, the content block can be a content unit with independent semantics that is divided into pixels according to the pixel density characteristics of the teaching material page. The content block can include text paragraphs, titles, images, charts, formulas, blank areas and other types.
[0035] In this embodiment of the application, the reading progress data can be quantitative data representing the student's position in reading the electronic textbook, and the reading progress data can be updated in real time with the student's scrolling, page turning and other operations.
[0036] In the embodiments of this application, the reading position can be the position of the content that the student is currently reading on the textbook page or in the overall structure of the electronic textbook.
[0037] In this embodiment, the correlation coefficient between the current reading frame and the row-level pixel density vector of the previous reading frame can be calculated first. The pixel density similarity can be determined based on the correlation coefficient, and the current reading mode can be determined by combining the pixel offset. Continuous low-density regions can be identified in the row-level pixel density vector of the textbook page, and the textbook page can be divided into multiple blocks. The content features and pixel density features of the blocks can be extracted to identify the block type, resulting in multiple content blocks. Based on the reading progress data corresponding to the reading mode, the student's reading position and the matching content block can be determined.
[0038] Step 103: When the reading content block is within the effective reading area of the educational terminal screen, accumulate the reading time of each student in the reading content block, analyze the actual reading situation of each student in the reading content block within the effective reading area of the screen, and obtain the effective reading time and effective reading speed of each student in the reading content block.
[0039] In this embodiment of the application, the reading dwell time can be the total dwell time of a student on a single reading content block.
[0040] In this embodiment, the effective reading time can be the cumulative time of the reading content block within the effective reading area of the screen. The effective reading time can accurately reflect the student's actual reading time.
[0041] In the embodiments of this application, the effective reading speed can be a reading rate calculated based on the effective reading time and the number of lines of text in the content block. The effective reading speed can be used to assess the reading speed of students.
[0042] In this embodiment, the vertical overlap between the reading content block and the effective reading area of the screen can be calculated within the target sampling time. When the vertical overlap meets the overlap condition, the effective reading time is obtained by combining the target sampling time, the vertical overlap, and the height of the content block. Then, the effective reading speed is calculated based on the effective reading time and the number of lines of text in the content block.
[0043] Step 104: Based on the row-level pixel density vector of each reading content block, the effective reading time and effective reading speed are concatenated in each reading content block to obtain the document density map of each student. The document density map of each student is then sent to the education server so that the education server can detect abnormal reading behavior of each student in the target class based on the group baseline data of the target class and the document density map of each student.
[0044] In this embodiment, the document density map can be an electronic textbook content density feature map formed by stitching together row-level pixel density vectors. The document density map can record the complete density distribution and reading data of the electronic textbook.
[0045] In the embodiments of this application, the first document density map can be a personal reading density map formed by stitching together individual student terminals. The first document density map can be used to locate the reading position and scroll displacement of individual students.
[0046] In this embodiment of the application, the group baseline data can be group reading reference data obtained by the education server based on the reading data of all students in the target class. The group baseline data can include median duration data, discrete duration data, median speed data, etc.
[0047] In the embodiments of this application, abnormal reading behavior can be the behavior of students deviating from the group reading baseline. Abnormal reading behavior can include reading too fast, reading pauses, skipping, looking back, getting distracted, leaving the seat, etc., which will not be listed one by one here.
[0048] In this embodiment, the row-level pixel density vector corresponding to each reading content block can be used as the basis for splicing, and the effective reading time and effective reading speed of the student in the block can be associated and spliced to form the first document density map corresponding to a single student.
[0049] In this embodiment of the application, after generating the first document density map, it can be encoded with lightweight features such as reading mode, scrolling position, effective reading area mode, and list of completed blocks and reported to the education server. The amount of data reported per packet is lower than the transmission overhead of screenshots, while protecting student privacy by not uploading original screenshots and text content. The education server can combine the baseline data of the target class with the first document density map of each student to detect and determine anomalies in students' reading behavior.
[0050] Compared with existing technologies, this embodiment identifies student reading patterns by performing correlation analysis on the row-level pixel density vectors of consecutive reading frames, divides textbook content into blocks based on pixel density, and matches student reading positions with corresponding content blocks to achieve precise positioning of specific reading locations within the textbook page; by only counting the reading time of content blocks within the effective reading area of the screen and calculating the effective reading duration and effective reading speed, it distinguishes the actual reading situation in different areas of the textbook and achieves accurate accumulation of effective reading time; by stitching together the reading features of each content block to generate a student document density map and reporting it to the education server, it achieves personalized abnormal reading behavior detection and effectively improves the accuracy of detecting abnormal reading behavior in electronic textbooks.
[0051] As an optional approach, when performing the task of "identifying each student's current reading mode by performing correlation analysis on the row-level pixel density vectors of the current reading frame and the previous reading frame of the textbook page, dividing the textbook page into multiple content blocks according to pixel density, and determining each student's reading position and the corresponding reading content block on the textbook page based on each student's reading progress data in the current reading mode," the following methods can be used, but are not limited to: Figure 2 As shown, the method includes: Step 201: Calculate the correlation coefficient between the row-level pixel density vectors of the current reading frame and the previous reading frame on the textbook page, and analyze the pixel density similarity between the current reading frame and the previous reading frame.
[0052] In this embodiment, the correlation coefficient can be a normalized cross-correlation coefficient. The correlation coefficient can be used to quantify the overall similarity between the row-level pixel density vectors of the current reading frame and the previous reading frame. The formula for calculating the correlation coefficient is shown in Formula 1, where... It can represent the normalized cross-correlation coefficient, with a value range of [-1, 1], where 1 represents complete cross-correlation and 0 represents complete uncorrelation. This can represent the pixel density of the current reading frame in the y-th row, and the value can be in the range of [0, 1]. This can represent the pixel density of the previous reading frame in the y-th row, and the value can be in the range of [0,1]. It can represent the average pixel density of the previous reading frame; It can represent the average pixel density of the current reading frame.
[0053] (Formula 1) Step 202: If the correlation coefficient meets the correlation conditions, the pixel density similarity between the current reading frame and the previous reading frame is high. Analyze the pixel offset between the current reading frame and the previous reading frame. The pixel offset is the amount of pixels that the current reading frame has shifted in the vertical direction relative to the previous reading frame.
[0054] In this embodiment, the relevant condition can be a preset correlation coefficient threshold, which can be used to determine whether the pixel densities of two frames have high similarity. For example, in this embodiment, the relevant condition can specifically be a correlation coefficient greater than or equal to 0.3.
[0055] In the embodiments of this application, the pixel offset can be the vertical translation pixel value of the current reading frame relative to the previous reading frame, and the pixel offset can be used to determine whether a scrolling operation has occurred.
[0056] Step 203: When the pixel offset meets the scrolling condition, the current reading frame is shifted by pixels in the vertical direction relative to the previous reading frame, and the current reading mode is determined to be the scrolling mode.
[0057] In this embodiment, the scrolling condition can be a preset pixel offset threshold, which can be used to determine whether to trigger the scrolling mode. For example, in this embodiment, the scrolling condition can specifically be that the absolute value of the pixel offset is greater than 5 pixels.
[0058] In the embodiments of this application, the scrolling mode can be a reading mode in which students swipe the screen vertically, and the scrolling mode can include two sub-modes: scrolling down and scrolling back up.
[0059] Step 204: Concatenate the row-level pixel density vectors of the current reading frame and the previous reading frame to obtain the first document density map corresponding to each student, and analyze the pixel offset of the current reading frame and the previous reading frame in the first document density map. The pixel offset is determined as the scroll displacement of the current reading frame relative to the previous reading frame.
[0060] In this embodiment, the scroll displacement can be a precise inter-frame translation amount determined based on the first document density map, and the scroll displacement can be used to update the student's reading progress data.
[0061] For the embodiments of this application, the formula for calculating the pixel offset can be as shown in Formula 2, where offset can represent the optimal offset (that is, the pixel offset in the embodiments of this application). It can represent taking the value that maximizes the expression. value; It can represent the candidate offset, with a value range of [-H / 2, H / 2], where H can represent the number of pixels in the screen height; corr() can represent the Pearson correlation coefficient calculation; This can indicate that the fingerprint from the previous frame will be displayed. Vertical translation The resulting vector after [number] pixels.
[0062] (Formula 2) In this embodiment, the calculation formula for document density map stitching can be as shown in Formula 3, where doc_map can represent the document density map; scroll_pos can represent the current cumulative scroll position; and H can represent the number of pixels in screen height. This can represent the density fingerprint vector of the current frame. When concatenating row-level pixel density vectors, the vectors can be written into the corresponding intervals of the first document density map according to the scroll position. For overlapping areas, the average of multiple samples can be taken to improve data stability.
[0063] (Formula 3) In the embodiments of this application, if the peak value of the cross-correlation function at the optimal offset is lower than a preset threshold, it indicates that the two frames do not overlap sufficiently (such as when the user scrolls rapidly). This can be marked as a rapid jump event, and no incremental displacement update is performed. The reading position is then repositioned in the document density map.
[0064] Step 205: Update the historical reading progress data according to the scroll displacement, and determine the reading position of each student based on the updated reading progress data.
[0065] In this embodiment of the application, the historical reading progress data can be the quantified reading position data of the previous moment, and the historical reading progress data can be accumulated and updated in real time with the scroll displacement.
[0066] For the embodiments of this application, the calculation formula for the rolling position update can be as shown in Formula 4, wherein, It can represent the cumulative scroll position at frame t, and its value can be greater than or equal to 0; It can represent the cumulative scroll position of the previous frame, and the value can be greater than or equal to 0; offset can represent the pixel displacement; by superimposing the scroll displacement onto the historical reading progress data, the updated reading progress data can be obtained, and thus the student's current reading position can be determined.
[0067] (Formula 4) Step 206: Among multiple content blocks, determine the reading content block corresponding to each student's reading position.
[0068] In this embodiment, the coordinates of the reading position in the electronic textbook can be matched with the location range of multiple content blocks to determine the corresponding reading content block; if the reading position spans multiple content blocks, multiple corresponding reading content blocks can be matched simultaneously.
[0069] Optionally, but not limited to, the following methods may be used, which include: when the correlation coefficient does not meet the correlation condition, the pixel density similarity between the current reading frame and the previous reading frame is low, the current reading mode is determined to be a page-turning mode, the initial position information corresponding to the page-turning page after page-turning is determined as the page-turning reading progress data corresponding to the page-turning mode, the reading position corresponding to the page-turning reading progress data is determined as the page-turning reading position for each student, and the reading content block corresponding to the page-turning reading position is determined among multiple content blocks; when the pixel offset does not meet the scrolling condition, the current reading frame does not show pixel translation in the vertical direction relative to the previous reading frame, the current reading mode is determined to be a static mode, the historical reading progress data is determined as the reading progress data in the static mode, the reading position corresponding to the historical reading progress data is determined as the static reading position for each student, and the reading content block corresponding to the static reading position is determined among multiple content blocks.
[0070] In this embodiment of the application, the page-turning mode can be a reading mode in which students switch textbook pages horizontally, and the reading progress in the page-turning mode can be switched to the initial position of the new page.
[0071] In the embodiments of this application, the static mode can be a reading mode in which students do not perform scrolling or page turning operations, and the reading progress can remain unchanged according to historical data in the static mode.
[0072] In the embodiments of this application, the page-turning reading position can be the initial reading position of the new page after page turning, and the static reading position can be the historical reading position that remains unchanged.
[0073] As an optional approach, when performing the task of "dividing the textbook page into multiple content blocks according to pixel density", the following methods can be used, but are not limited to: identifying continuous low-density regions in the row-level pixel density vector corresponding to the textbook page, and dividing the textbook page into multiple blocks based on the continuous low-density regions; extracting content features and pixel density features of the multiple blocks based on the content data and row-level pixel density vector, and identifying the block type of the multiple blocks based on the content features and pixel density features, thereby obtaining the content blocks corresponding to the multiple blocks.
[0074] In this embodiment, a continuous low-density region can be a continuous pixel row region in the row-level pixel density vector where the density value is lower than a preset threshold, and the continuous low-density region can serve as the dividing line for block division.
[0075] For the embodiments of this application, the calculation formula for grayscale processing can be as shown in Formula 5, where Gray(x,y) can represent the grayscale value at coordinate (x,y), and the value range can be [0, 255]; R(x,y) can represent the red channel value at coordinate (x,y), and the value range can be [0, 255]; G(x,y) can represent the green channel value at coordinate (x,y), and the value range can be [0, 255]; B(x,y) can represent the blue channel value at coordinate (x,y), and the value range can be [0, 255]; x can represent the horizontal coordinate of the pixel, and the value range can be [0, W-1]; y can represent the vertical coordinate of the pixel, and the value range can be [0, H-1]; 0.299, 0.587, and 0.114 can represent the brightness weighting coefficients; W can represent the width of the screen frame in pixels; and H can represent the height of the screen frame in pixels.
[0076] (Formula 5) In this embodiment of the application, the formula for calculating row-level pixel density can be as shown in Formula 6, where density(y) can represent the proportion of foreground pixels in the y-th row, and the value range can be [0, 1]; W can represent the width of the screen frame in pixels; H can represent the height of the screen frame in pixels; B(x, y) can represent the binarized pixel value in the y-th row and x-th column, and the value can be 0 or 1.
[0077] (Formula 6) For the embodiments of this application, the formula for calculating the normalized density can be as shown in Formula 7, where density_norm(y) can represent the normalized density value of the y-th row, and the value range can be [0, 1]; density(y) can represent the original density value of the y-th row. It can represent the maximum density in all rows of the entire frame, and the value range can be (0,1).
[0078] (Formula 7) In this embodiment of the application, the content features may include the number of lines of text in the block, the peak spacing variance, etc., and the pixel density features may include the average density of the block, the density pattern vector, etc., and the density fingerprint of each block may be a downsampled 32-dimensional density pattern vector.
[0079] For the embodiments of this application, the block type can be automatically identified based on features. A text paragraph can be a block with a density fingerprint (i.e., the row-level pixel density vector in the embodiments of this application) having a peak number greater than or equal to 3 and a small line spacing variance; a title can be a block with a density fingerprint having a peak number of 1 to 2 and a high average density; an image or chart can be a block with a density fingerprint having a peak number of 0 and a smooth density; a formula can be a block with a density fingerprint having a low peak number and a large line spacing variance; and a blank space can be a block with a density fingerprint having an average density of approximately 0.
[0080] As an optional approach, when performing the task of "accumulating the reading time of each student within the effective reading area of the reading content block on the educational terminal's screen, analyzing the actual reading situation of each student within the effective reading area of the screen, and obtaining the effective reading time and effective reading speed of each student within the effective reading time," the following method can be used, but is not limited to: when the front-facing camera of the educational terminal is available, locating each student's eyes and eyeball area using the front-facing camera to obtain the pupil gaze direction of each student, and mapping the gaze area corresponding to the pupil gaze direction on the screen of the educational terminal to obtain the effective reading area of each student on the educational terminal screen; when the front-facing camera of the educational terminal is unavailable... In this scenario, the target center area of the educational terminal screen in the vertical direction is defined as the effective reading area. Within the target sampling time, the vertical overlap between the reading content block corresponding to the reading position and the effective reading area is determined. If the vertical overlap meets the overlap condition, the reading content block is within the effective reading area of the educational terminal screen. The target sampling time, vertical overlap, and block height of the reading content block corresponding to the reading position are fused together to analyze the actual reading situation of each student in the effective reading area, obtaining the effective reading time of each student in the reading content block corresponding to the reading position. Based on the effective reading time and the number of lines of text contained in the reading content block corresponding to the reading position, the effective reading speed of the reading content block corresponding to the reading position within the effective reading time is calculated.
[0081] For the embodiments of this application, it can be first determined whether the front-facing camera of the education terminal is available. If it is available, the human eye and eyeball area can be located through the front-facing camera to obtain the pupil gaze direction and map it as the effective reading area of the screen. If it is not available, the target center area in the vertical direction of the education terminal screen can be determined as the effective reading area of the screen. The determination of the effective reading area of the screen can adopt a three-level strategy, and can be combined with degradation and recovery mechanisms to ensure the detection availability in different scenarios.
[0082] In this embodiment, the vertical overlap can be the number of pixels that overlap between the reading content block and the effective reading area of the screen in the vertical direction. The vertical overlap can be used to determine whether the content block is within the effective reading area. The formula for calculating the vertical overlap is shown in Formula 8, where, It can represent the number of pixels that overlap between the reading content block Block_i and the effective reading area of the screen in the vertical direction, and the value can be greater than or equal to 0; It can represent the top y-coordinate of block_i in the current screen frame, and its value range can be [0, H); The value can represent the bottom y-coordinate of block_i in the current screen frame, and the value range can be (0, H]; erz_top can represent the top y-coordinate of the effective reading area of the screen; erz_bottom can represent the bottom y-coordinate of the effective reading area of the screen.
[0083] (Formula 8) For the embodiments of this application, the formula for calculating the effective reading time can be as shown in Formula Nine, where, It can represent the cumulative effective dwell time of the reading content block Block_i (that is, the effective reading time in this embodiment of the application), and the value can be greater than or equal to 0; This can represent the time interval between two samples (i.e., the target sampling time in the embodiments of this application); It can represent the number of pixels that overlap between the reading content block and the effective reading area of the screen; It can represent the total height in pixels of block_i, and its value can be greater than 0.
[0084] (Formula 9) In this embodiment of the application, the formula for calculating reading speed can be as shown in Formula 10, wherein, It can represent the estimated reading speed of block_i, and its value can be greater than 0; It can represent the number of lines of text contained in block_i, and the value can be greater than or equal to 1; It can represent the final effective dwell time of block_i (i.e., the effective reading time in this embodiment), and the value can be greater than 0.
[0085] (Formula 10) For the embodiments of this application, the vertical overlap can satisfy the overlap condition by the vertical overlap being greater than 0. The effective reading time can be obtained by multiplying the target sampling time by the ratio of the vertical overlap to the block height. The effective reading speed can be the ratio of the number of lines of text to the effective reading time. The reading behavior can be marked as sequential reading, skipping, looking back, pausing, and quickly flipping through.
[0086] Compared with existing technologies, this embodiment determines the scrolling reading mode by using the correlation coefficient of inter-frame pixel density vectors and pixel offset, calculates the scrolling displacement and updates the reading progress by combining the document density map, and achieves dynamic and accurate correction of the student's reading position in scrolling reading scenarios; it distinguishes between page-turning mode and static mode by using correlation coefficient thresholds and pixel offset thresholds and matches the corresponding reading positions, achieving accurate reading position recognition covering all reading operation scenarios; it divides the textbook into blocks by continuous low-density regions of row-level pixel density vectors and identifies block types by combining multi-dimensional features, achieving automatic classification of electronic textbook page content blocks; and it determines the effective reading area of the screen through a multi-level method of camera detection and default area adaptation, and calculates reading data by combining block overlap and block height, achieving accurate calculation of student reading time and reading speed.
[0087] This embodiment provides an anomaly detection method, such as Figure 3 As shown, applied to the education server, this method includes: Step 301: Merge the reading content blocks, effective reading time, and effective reading speed of each student in the document density map of the target class to obtain the group baseline data of the target class.
[0088] In this embodiment, the education server can be a device that receives and processes data reported by education terminals. The education server can be used for group data statistics and anomaly detection. The overall architecture can adopt a two-layer structure of terminal analysis plus server-side group aggregation and judgment.
[0089] In the embodiments of this application, the fusion process can be to aggregate, statistically analyze and calibrate the reading data of all students in the class, and the fusion process can be used to generate standardized group baseline data.
[0090] Step 302: Determine the target reading time for each reading content block from the effective reading time, compare the target reading time with the median reading time data in the group baseline data, analyze the deviation of each student's reading time for each reading content block from the median reading time of the target class, and obtain the time difference data for each student's reading time for each content block.
[0091] In this embodiment of the application, the target reading time can be the effective reading time of a student in a single content block, and the median reading time data can be the median of the effective reading time of all students in the class in that content block.
[0092] In the embodiments of this application, the duration difference data can be the difference between the target reading time and the median reading time data, and the duration difference data can characterize the absolute deviation of the student's reading time from the average level of the group.
[0093] Step 303: Compare the duration difference data with the discrete duration data in the group baseline data, analyze the degree of deviation of each student's reading time from the target time of the target class, and obtain the reading deviation data of each student.
[0094] In this embodiment of the application, the discrete duration data can be the interquartile range of the effective reading time of all students in the class in the content block, and the discrete duration data can characterize the degree of dispersion of the group's reading time.
[0095] For the embodiments of this application, the formula for calculating the reading deviation can be as shown in Formula 11, where, This can represent the degree of deviation of a student from block_i; This can represent the effective dwell time of a student on block_i; It can represent the median of the effective dwell time of the entire class on block_i; It can represent the interquartile range of the effective dwell time of the whole class on block_i; this formula can standardize the duration difference data by dividing the discrete duration data, so that the deviation of different blocks is comparable.
[0096] (Formula Eleven) Step 304: Based on the reading deviation data of each student, detect the reading behavior of each student, identify the reading behavior that meets the abnormal behavior conditions as abnormal reading behavior, and issue an alarm for abnormal reading behavior.
[0097] In this embodiment of the application, the abnormal behavior condition can be a preset deviation threshold, reading order condition, or attention state condition. The abnormal behavior condition can be used to determine abnormal reading behavior. Abnormal types can include reading too fast, skimming over images, getting stuck while reading, possibly losing focus, skipping, falling behind in progress, staring at the screen without operation, leaving the seat, frequently revisiting, etc.
[0098] In this embodiment of the application, the alarm may be to push an anomaly notification to the teacher. The alarm content may include the anomaly type, block information, and comparison data between the actual data and the group baseline.
[0099] Compared with existing technologies, this embodiment constructs class group baseline data by integrating class student document density map data, and judges anomalies by combining reading time deviation and standardized reading deviation, thereby improving the accuracy of reading anomaly detection based on group data.
[0100] As an optional approach, when performing the process of "merging the reading content blocks, effective reading time, and effective reading speed in the document density map of each student in the target class to obtain the group baseline data of the target class," the following methods can be used, but are not limited to: Figure 4 As shown, the method includes: Step 401: Analyze the pixel similarity between reading content blocks in the document density map of each student, merge the content blocks that meet the similarity conditions, and obtain multiple merged content blocks for the target class.
[0101] In this embodiment of the application, pixel similarity can be the density feature matching degree of content blocks in different student document density maps, and the similarity condition can be that the similarity is higher than a preset matching threshold.
[0102] In the embodiments of this application, merging content blocks can be the aggregation of the same textbook content blocks reported by different students, and merging content blocks can be used to construct a unified textbook content structure.
[0103] Step 402: Arrange the multiple merged content blocks of the target class according to the order of reading position to obtain the second document density map corresponding to the electronic textbook.
[0104] In this embodiment, the second document density map can be a global density map of electronic textbooks constructed by the education server based on the data of all students in the class, and the second document density map can be used to generate group baseline data.
[0105] In the embodiments of this application, the second document density map can also be generated through three cold start methods: pre-construction on the teacher's end, progressive stitching, and delayed judgment. When the map coverage reaches 80%, anomaly detection is initiated to avoid misjudgment of abnormal behavior caused by incomplete baseline.
[0106] Step 403: Based on the textbook content structure in the second document density map, the reading content block, effective reading time, and reading speed corresponding to each student's reading position are fused to obtain the group baseline data of the target class.
[0107] In this embodiment, the group baseline data can be adaptively adjusted for different block types, text paragraphs can use reading speed as the core indicator, images and charts can use dwell time as the core indicator, and titles and formulas can use a relaxed threshold.
[0108] For the embodiments of this application, the population baseline data may include median duration, interquartile range (i.e., discrete duration data in the embodiments of this application), median velocity, median arrival time, and other data.
[0109] As an optional approach, when performing the task of "detecting each student's reading behavior based on each student's reading deviation data, identifying reading behaviors that meet the abnormal behavior conditions as abnormal reading behaviors, and issuing alerts for abnormal reading behaviors," the following methods can also be used, but are not limited to: marking the reading order of reading content blocks based on each student's reading progress data to obtain each student's reading order data; determining whether each student's face and eyes are within the screen range of the educational terminal based on each student's screen operation behavior data to obtain the student's attention detection data; and analyzing whether there are abnormalities in the dwell time, reading order, and attention state of each student's reading behavior based on each student's reading deviation data, reading order data, and attention detection data, identifying reading behaviors that meet the abnormal behavior conditions as abnormal reading behaviors, and issuing alerts for abnormal reading behaviors.
[0110] In this embodiment, reading sequence data can mark students' sequential reading, skipping, and rereading behaviors, while attention detection data can determine whether students are looking at the screen or leaving their seats.
[0111] For the embodiments of this application, abnormal dwell time may be reading too fast or too slow with a deviation exceeding a threshold, abnormal reading order may be skipping or frequent rereading, and abnormal attention state may be daydreaming or leaving one's seat.
[0112] For example, if student A's reading deviation from text block X is lower than the lower limit of the reading deviation data, it indicates that student A's reading speed is much higher than the group baseline reading speed, and it can be determined that the student has abnormal behavior. The alarm content can be that student A reads too fast in block X. If student A's reading deviation from text image X is lower than the lower limit of the reading deviation data, it indicates that student A's effective reading time is much lower than the group baseline reading time, and it can be determined that the student has abnormal behavior. The alarm content can be that student A skipped image X.
[0113] For example, if student A's reading deviation from text block X exceeds the upper limit of the reading deviation data, and the effective reading area of the screen is in eye-calibration mode, it indicates that student A has been looking at block X, confirming abnormal behavior. The alert message could be: Student A is having difficulty with block X. If student A's reading deviation from text block X exceeds the upper limit of the reading deviation data, and the effective reading area of the screen is in fixed default mode, it cannot be confirmed whether the student is looking at block X, but abnormal behavior can be confirmed. The alert message could be: Student A may be distracted. If student A has not left a reading time or reading speed record for block X, abnormal behavior can be confirmed. The alert message could be: Student A skipped block X. If student A's reading position lags behind the median of the group baseline data, abnormal behavior can be confirmed. The alert message could be: Student A is behind in reading progress.
[0114] For example, if student A has no screen operation for a long time and the front-facing camera detects a face, it can be determined that the student has engaged in abnormal behavior, and the alarm content can be: student A has not operated for N minutes (still facing the screen); if student A has no screen operation for a long time and the front-facing camera does not detect a face, it can be determined that the student has engaged in abnormal behavior, and the alarm content can be: student A may have left his seat; if student A reviews block X more times than the review threshold, it can be determined that the student has engaged in abnormal behavior, and the alarm content can be: student A repeatedly reviews block X.
[0115] Compared with existing technologies, this embodiment constructs a global document density map by merging and sorting block similarity data and integrates reading data from multiple students to accurately construct group baseline data that is adapted to the structure of the entire electronic textbook. By combining reading deviation, reading order data and attention detection data to comprehensively determine anomalies, it can identify abnormal reading behaviors of students from multiple dimensions such as reading time, reading order and attention state, further improving the dimensions of anomaly detection and enhancing the comprehensiveness and accuracy of anomaly detection.
[0116] Furthermore, as Figure 1 and Figure 2 The specific implementation of the method shown in this embodiment provides an anomaly detection device, such as... Figure 5 As shown, the device includes: an acquisition module 51, a determination module 52, an analysis module 53, and a generation module 54.
[0117] The acquisition module 51 is configured to acquire the textbook page of each student in the target class reading the electronic textbook, and to perform line-by-line horizontal projection compression on the textbook page to obtain the row-level pixel density vector corresponding to the textbook page; The determination module 52 is configured to identify the current reading mode of each student by performing correlation analysis on the row-level pixel density vectors of the current reading frame and the previous reading frame of the textbook page, and to divide the textbook page into multiple content blocks according to pixel density. Based on the reading progress data of each student in the current reading mode, the module determines the reading position of each student in the textbook page and the reading content block corresponding to the reading position. The analysis module 53 is configured to accumulate the reading time of each student in the reading content block when the reading content block is in the effective reading area of the screen of the educational terminal, analyze the actual reading situation of each student in the reading content block in the effective reading area of the screen, and obtain the effective reading time and effective reading speed of each student in the reading content block. The sending module 54 is configured to concatenate the effective reading time and effective reading speed in each reading content block according to the row-level pixel density vector of each reading content block to obtain a document density map for each student, and send the document density map of each student to the education server so that the education server can detect abnormal reading behavior of each student in the target class based on the group baseline data of the target class and the document density map of each student.
[0118] In some examples of this embodiment, the determining module 52 is specifically configured to calculate the correlation coefficient between the row-level pixel density vectors of the current reading frame and the previous reading frame of the textbook page, and analyze the pixel density similarity between the current reading frame and the previous reading frame; if the correlation coefficient meets the correlation condition, the pixel density similarity between the current reading frame and the previous reading frame is high, and the pixel offset between the current reading frame and the previous reading frame is analyzed, where the pixel offset is the amount of pixel translation of the current reading frame relative to the previous reading frame in the vertical direction; if the pixel offset meets the scrolling condition, the current reading frame is relatively horizontally shifted relative to the previous reading frame in the vertical direction. An upward pixel shift determines the current reading mode as scrolling mode. The row-level pixel density vectors of the current and previous reading frames are concatenated to obtain the first document density map for each student. The pixel offsets of the current and previous reading frames in the first document density map are analyzed, and these pixel offsets are determined as the scroll displacement of the current reading frame relative to the previous reading frame. The historical reading progress data is updated based on the scroll displacement, and the reading position of each student is determined based on the updated reading progress data. Among multiple content blocks, the reading content block corresponding to each student's reading position is determined.
[0119] In some examples of this embodiment, the determining module 52 is further configured to: when the correlation coefficient does not meet the correlation conditions, the pixel density similarity between the current reading frame and the previous reading frame is low; determine the current reading mode as a page-turning mode; determine the initial position information corresponding to the page-turning page after page-turning as the page-turning reading progress data corresponding to the page-turning mode; determine the reading position corresponding to the page-turning reading progress data as the page-turning reading position for each student; and determine the reading content block corresponding to the page-turning reading position among multiple content blocks. When the pixel offset does not meet the scrolling conditions, the current reading frame does not show pixel translation in the vertical direction relative to the previous reading frame; determine the current reading mode as a static mode; determine the historical reading progress data as the reading progress data in the static mode; determine the reading position corresponding to the historical reading progress data as the static reading position for each student; and determine the reading content block corresponding to the static reading position among multiple content blocks.
[0120] In some examples of this embodiment, the determining module 52 is further configured to identify continuous low-density regions in the row-level pixel density vector corresponding to the textbook page, and divide the textbook page into multiple blocks based on the continuous low-density regions; extract the content features and pixel density features of the multiple blocks based on the content data and row-level pixel density vector of the multiple blocks, and identify the block type of the multiple blocks according to the content features and pixel density features, thereby obtaining the content blocks corresponding to the multiple blocks.
[0121] In some examples of this embodiment, the analysis module 52 is specifically configured to: when the front-facing camera of the educational terminal is available, locate the eyes and eyeballs of each student using the front-facing camera to obtain the pupil gaze direction of each student, and map the gaze area corresponding to the pupil gaze direction onto the screen of the educational terminal to obtain the effective reading area of each student on the screen of the educational terminal; when the front-facing camera of the educational terminal is unavailable, determine the target center area of the screen of the educational terminal in the vertical direction as the effective reading area of the screen; within the target sampling time, determine the vertical overlap between the reading content block corresponding to the reading position and the effective reading area, and if the vertical overlap meets the overlap condition, the reading content block is in the effective reading area of the screen of the educational terminal; fuse the target sampling time, the vertical overlap, and the block height of the reading content block corresponding to the reading position to analyze the actual reading situation of each student on the reading content block in the effective reading area of the screen, and obtain the effective reading time of each student on the screen corresponding to the reading content block; calculate the effective reading speed of the reading content block corresponding to the reading position within the effective reading time based on the effective reading time and the number of lines of text contained in the reading content block corresponding to the reading position.
[0122] Furthermore, as Figure 3 and Figure 4 The specific implementation of the method shown in this embodiment provides an anomaly detection device, such as... Figure 6 As shown, the device includes: a fusion module 61, an analysis module 62, and a detection module 63.
[0123] The fusion module 61 is configured to fuse the reading content blocks, effective reading time and effective reading speed of each student in the document density map of the target class to obtain the group baseline data of the target class. Analysis module 62 is configured to compare the effective reading time of each student in the reading content block with the median reading time data in the group baseline data of the target class, analyze the individual reading time deviation of each student and the target class, and obtain the time difference data. Analysis module 62 is also configured to analyze the degree of deviation of each student's individual reading time relative to the overall deviation of the target class based on the discrete time data in the time difference data and the group baseline data of the target class, and determine the reading deviation data of each student relative to the target class. The detection module 63 is configured to detect each student's reading behavior based on each student's reading deviation data, identify reading behaviors that meet the abnormal behavior conditions as abnormal reading behaviors, and issue an alarm for abnormal reading behaviors.
[0124] In some examples of this embodiment, the fusion module 61 is specifically configured to analyze the pixel similarity between reading content blocks in the document density map of each student, merge the content blocks that meet the similarity conditions to obtain multiple merged content blocks of the target class; arrange the multiple merged content blocks of the target class according to the order of reading position to obtain the second document density map corresponding to the electronic textbook; based on the textbook content structure in the second document density map, fuse the reading content blocks corresponding to each student's reading position, effective reading time, and reading speed to obtain the group baseline data of the target class.
[0125] In some examples of this embodiment, the detection module 63 is specifically configured to mark the reading order of the reading content blocks based on each student's reading progress data to obtain each student's reading order data; to determine whether each student's face and eyes are within the screen range of the educational terminal based on each student's screen operation behavior data to obtain the student's attention detection data; and to analyze whether there are abnormalities in the dwell time of the reading content blocks, abnormal reading order, and abnormal attention state in each student's reading behavior based on each student's reading deviation data, reading order data, and attention detection data, to identify reading behaviors that meet the abnormal behavior conditions as abnormal reading behaviors, and to issue an alarm for abnormal reading behaviors.
[0126] Based on the above, Figure 1 , Figure 2 , Figure 3 and Figure 4 Accordingly, this embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method. Figure 1 , Figure 2 , Figure 3 and Figure 4 The method shown.
[0127] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause a computer device (such as personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of this application.
[0128] like Figure 7 The diagram shown is a hardware structure schematic of an electronic device according to the present invention, comprising: At least one processor 701; and, A memory 702 is communicatively connected to at least one processor 701; wherein, The memory 702 stores instructions that can be executed by at least one processor, which enables the at least one processor to perform the aforementioned anomaly detection method.
[0129] Figure 7 Take the 701 processor as an example.
[0130] The electronic device may also include an input device 703 and a display device 704.
[0131] The processor 701, memory 702, input device 703, and display device 704 can be connected via a bus or other means. Figure 7 Taking the example of a connection between China and Israel via a bus.
[0132] The memory 702, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the anomaly detection method in the embodiments of this application, for example, Figure 1 , Figure 2 , Figure 3 and Figure 4 The method flow is shown. The processor 701 executes various functional applications and data processing by running non-volatile software programs, instructions, and modules stored in the memory 702, thereby implementing the anomaly detection method in the above embodiments.
[0133] The memory 702 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the anomaly detection method. Furthermore, the memory 702 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 702 may optionally include memory remotely located relative to the processor 701, and these remote memories may be connected to the apparatus performing the anomaly detection method via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
[0134] The input device 703 can receive user clicks and generate signal inputs related to user settings and function control for the anomaly detection method. The display device 704 may include a display screen or other display device.
[0135] One or more modules are stored in memory 702, and when run by one or more processors 701, the anomaly detection method in any of the above method embodiments is executed.
[0136] Optionally, the aforementioned physical devices may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.
[0137] Those skilled in the art will understand that the physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.
[0138] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.
[0139] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platform, or it can be implemented by hardware. Compared with existing technologies, this embodiment identifies student reading patterns by performing correlation analysis on the row-level pixel density vectors of consecutive reading frames. It then divides textbook content into blocks based on pixel density and matches student reading positions with corresponding content blocks, achieving precise positioning of specific reading locations within the textbook page. By only counting the reading time of content blocks within the effective reading area of the screen and calculating effective reading duration and speed, it distinguishes the actual reading situation in different areas of the textbook, achieving precise accumulation of effective reading time. By stitching together the reading characteristics of each content block to generate a student document density map and reporting it to the education service terminal, it achieves personalized abnormal reading behavior detection, effectively improving the accuracy of detecting abnormal reading behavior in electronic textbooks. It determines scrolling reading patterns by using the correlation coefficient of inter-frame pixel density vectors and pixel offsets, calculates scrolling displacement based on the document density map, and updates reading progress, achieving dynamic and precise correction of student reading positions in scrolling reading scenarios. Finally, by integrating class student document density map data to construct class group baseline data, and combining reading duration deviation and standardized reading deviation to determine anomalies, it improves the accuracy of reading anomaly detection based on group data.
[0140] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.
[0141] The above are merely specific embodiments of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to these embodiments, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.
Claims
1. An anomaly detection method, characterized in that, Applications in educational terminals include: The textbook pages of each student in the target class are obtained and the textbook pages are compressed by horizontal projection line by line to obtain the row-level pixel density vector corresponding to the textbook page. The row-level pixel density vector is the density fingerprint of the textbook page content. The density fingerprint is used for content block matching and alignment between different students. By performing correlation analysis on the row-level pixel density vectors of the current reading frame and the previous reading frame of the textbook page, the current reading mode of each student is identified, and the textbook page is divided into multiple content blocks according to pixel density. Based on the reading progress data of each student in the current reading mode, the reading position of each student in the textbook page and the reading content block corresponding to the reading position are determined. When the reading content block is within the effective reading area of the educational terminal's screen, the reading time spent by each student in the reading content block is accumulated. The actual reading situation of each student in the reading content block within the effective reading area of the screen is analyzed to obtain the effective reading time and effective reading speed of each student in the reading content block. The effective reading time is the cumulative time of the reading content block within the effective reading area of the screen, reflecting the student's actual reading time. The effective reading speed is the reading rate calculated based on the effective reading time and the number of lines of text in the content block. The effective reading time and the effective reading speed are concatenated in each reading content block according to the row-level pixel density vector of each reading content block to obtain the document density map of each student. The document density map of each student is then sent to the education server so that the education server can detect abnormal reading behavior of each student in the target class based on the group baseline data of the target class and the document density map of each student. The process involves performing correlation analysis on the row-level pixel density vectors of the current and previous reading frames of the textbook page to identify the current reading mode of each student, dividing the textbook page into multiple content blocks according to pixel density, and determining the reading position of each student on the textbook page and the corresponding reading content block based on the reading progress data of each student in the current reading mode. This includes: Calculate the correlation coefficient between the row-level pixel density vectors of the current reading frame and the previous reading frame of the textbook page, and analyze the pixel density similarity between the current reading frame and the previous reading frame; When the correlation coefficient satisfies the correlation condition, the pixel density similarity between the current reading frame and the previous reading frame is high. The pixel offset between the current reading frame and the previous reading frame is analyzed. The pixel offset is the amount of pixels that the current reading frame has shifted in the vertical direction relative to the previous reading frame. When the pixel offset satisfies the scrolling condition, the current reading frame is shifted by pixels in the vertical direction relative to the previous reading frame, and the current reading mode is determined to be the scrolling mode. The row-level pixel density vectors of the current reading frame and the previous reading frame are concatenated to obtain the first document density map corresponding to each student. The pixel offsets of the current reading frame and the previous reading frame in the first document density map are analyzed, and the pixel offsets are determined as the scroll displacement of the current reading frame relative to the previous reading frame. The historical reading progress data is updated based on the scroll displacement, and the reading position of each student is determined based on the updated reading progress data. Among the multiple content blocks, determine the reading content block corresponding to each student's reading position; The textbook page is divided into multiple content blocks according to pixel density, including: In the row-level pixel density vector corresponding to the textbook page, continuous low-density regions are identified, and the textbook page is divided into multiple blocks based on the continuous low-density regions; Based on the content data and row-level pixel density vectors of the multiple blocks, the content features and pixel density features of the multiple blocks are extracted, and the block type of the multiple blocks is identified according to the content features and pixel density features to obtain the content blocks corresponding to the multiple blocks; When the reading content block is within the effective reading area of the educational terminal's screen, the reading time spent by each student in the reading content block is accumulated, and the actual reading situation of each student in the effective reading area of the screen is analyzed to obtain the effective reading time and effective reading speed of each student in the reading content block, including: When the front-facing camera of the educational terminal is available, the human eye and eyeball area of each student are located by the front-facing camera to obtain the pupil gaze direction of each student, and the gaze area corresponding to the pupil gaze direction is mapped on the screen of the educational terminal to obtain the effective reading area of each student on the screen of the educational terminal. When the front-facing camera of the educational terminal is unavailable, the target center area of the screen in the vertical direction is determined as the effective reading area of the screen. Within the target sampling time, the vertical overlap between the reading content block corresponding to the reading position and the effective reading area is determined. If the vertical overlap satisfies the overlap condition, the reading content block is within the effective reading area of the screen of the educational terminal. The vertical overlap is the number of overlapping pixels between the reading content block and the effective reading area of the screen in the vertical direction. The vertical overlap is used to determine whether the content block is within the effective reading area. The target sampling time, the vertical overlap, and the block height of the reading content block corresponding to the reading position are fused together to analyze the actual reading situation of each student in the effective reading area of the screen for the reading content block, and to obtain the effective screen reading time of each student in the reading content block corresponding to the reading position. The effective reading speed of the reading content block corresponding to the reading position within the effective reading time is calculated based on the effective reading time and the number of lines of text contained in the reading content block corresponding to the reading position.
2. The method according to claim 1, characterized in that, The method further includes: If the correlation coefficient does not meet the correlation conditions, the pixel density similarity between the current reading frame and the previous reading frame is low. The current reading mode is determined to be a page-turning mode. The initial position information corresponding to the page-turning page after page-turning is determined as the page-turning reading progress data corresponding to the page-turning mode. The reading position corresponding to the page-turning reading progress data is determined as the page-turning reading position of each student. The reading content block corresponding to the page-turning reading position is determined among the multiple content blocks. If the pixel offset does not meet the scrolling condition, and the current reading frame does not have a pixel translation in the vertical direction relative to the previous reading frame, the current reading mode is determined to be a static mode, the historical reading progress data is determined to be the reading progress data in the static mode, the reading position corresponding to the historical reading progress data is determined to be the static reading position of each student, and the reading content block corresponding to the static reading position is determined among the multiple content blocks.
3. The method according to claim 1, characterized in that, The method further includes: the education server is used for, The reading content blocks, effective reading time, and effective reading speed of each student in the document density map of the target class are fused together to obtain the group baseline data of the target class. The target reading time for each reading content block is determined from the effective reading time. The target reading time is compared with the median reading time data in the group baseline data. The deviation of each student's reading time for each reading content block from the median reading time of the target class is analyzed to obtain the time difference data of each student reading each content block. By comparing the duration difference data with the discrete duration data in the group baseline data, the degree of deviation of each student's reading time from the target duration of the target class is analyzed, and the reading deviation data of each student is obtained. Based on the reading deviation data of each student, the reading behavior of each student is detected, and reading behaviors that meet the abnormal behavior conditions are identified as abnormal reading behaviors, and alarms are issued for the abnormal reading behaviors.
4. The method according to claim 3, characterized in that, The process of fusing the reading content blocks, effective reading time, and effective reading speed from the document density map of each student in the target class to obtain the group baseline data of the target class includes: Analyze the pixel similarity between reading content blocks in the document density map of each student, merge the content blocks that meet the similarity conditions, and obtain multiple merged content blocks of the target class; The multiple merged content blocks of the target class are arranged according to the order of reading position to obtain the second document density map corresponding to the electronic textbook; Based on the textbook content structure in the second document density map, the reading content block, effective reading time, and reading speed corresponding to each student's reading position are fused to obtain the group baseline data of the target class.
5. The method according to claim 3, characterized in that, The process involves detecting each student's reading behavior based on their reading deviation data, identifying reading behaviors that meet the abnormal behavior criteria as abnormal reading behaviors, and issuing alerts for these abnormal reading behaviors, including: Based on the reading progress data of each student, the reading order of the reading content blocks is marked to obtain the reading order data of each student; Based on the screen operation behavior data of each student, it is determined whether the face and eyeballs of each student are within the screen range of the educational terminal, and the student's attention detection data is obtained; Based on the reading deviation data, reading order data, and attention detection data of each student, the system analyzes whether there are abnormalities in the dwell time, reading order, and attention state of the reading content blocks in each student's reading behavior. Reading behaviors that meet the abnormal behavior conditions are identified as abnormal reading behaviors, and alarms are issued for the abnormal reading behaviors.
6. An anomaly detection device, characterized in that, Applications in educational terminals include: The acquisition module is configured to acquire the textbook pages of each student in the target class who are reading the electronic textbook, and to perform line-by-line horizontal projection compression on the textbook pages to obtain the row-level pixel density vector corresponding to the textbook pages. The row-level pixel density vector is the density fingerprint of the textbook page content. The density fingerprint is used for content block matching and alignment between different students. The determination module is configured to identify the current reading mode of each student by performing correlation analysis on the row-level pixel density vectors of the current reading frame and the previous reading frame of the textbook page, divide the textbook page into multiple content blocks according to pixel density, and determine the reading position of each student in the textbook page and the reading content block corresponding to the reading position based on the reading progress data of each student in the current reading mode. The analysis module is configured to, when the reading content block is within the effective reading area of the educational terminal's screen, accumulate the reading time spent by each student on the reading content block, analyze the actual reading situation of each student on the reading content block within the effective reading area of the screen, and obtain the effective reading time and effective reading speed of each student on the reading content block. The effective reading time is the cumulative time of the reading content block within the effective reading area of the screen, and the effective reading time reflects the student's actual reading time. The effective reading speed is a reading rate calculated based on the effective reading time and the number of lines of text in the content block. The sending module is configured to concatenate the effective reading time and the effective reading speed in each reading content block according to the row-level pixel density vector of each reading content block to obtain the document density map of each student, and send the document density map of each student to the education server so that the education server can detect abnormal reading behavior of each student in the target class based on the group baseline data of the target class and the document density map of each student; The determination module is further configured to calculate the correlation coefficient between the row-level pixel density vectors of the current reading frame and the previous reading frame of the textbook page, and analyze the pixel density similarity between the current reading frame and the previous reading frame; if the correlation coefficient satisfies the correlation condition, the pixel density similarity between the current reading frame and the previous reading frame is high, and the pixel offset between the current reading frame and the previous reading frame is analyzed, where the pixel offset is the amount of pixel translation of the current reading frame relative to the previous reading frame in the vertical direction; if the pixel offset satisfies the scrolling condition, the current reading frame appears as a pixel in the vertical direction relative to the previous reading frame. The current reading mode is determined to be a scrolling mode by pixel translation; the row-level pixel density vectors of the current reading frame and the previous reading frame are concatenated to obtain a first document density map corresponding to each student, and the pixel offsets of the current reading frame and the previous reading frame in the first document density map are analyzed, and the pixel offsets are determined as the scroll displacement of the current reading frame relative to the previous reading frame; the historical reading progress data is updated according to the scroll displacement, and the reading position of each student is determined based on the updated reading progress data; among the multiple content blocks, the reading content block corresponding to the reading position of each student is determined; The determination module is further configured to identify continuous low-density regions in the row-level pixel density vector corresponding to the textbook page, and divide the textbook page into multiple blocks based on the continuous low-density regions; extract content features and pixel density features of the multiple blocks based on the content data and row-level pixel density vector of the multiple blocks, and identify the block type of the multiple blocks according to the content features and pixel density features to obtain the content blocks corresponding to the multiple blocks; The analysis module is further configured to, when the front-facing camera of the educational terminal is available, locate the eyes and eyeball areas of each student using the front-facing camera to obtain the pupil gaze direction of each student, and map the gaze area corresponding to the pupil gaze direction onto the screen of the educational terminal to obtain the effective reading area of each student on the screen of the educational terminal; when the front-facing camera of the educational terminal is unavailable, determine the target center area of the screen of the educational terminal in the vertical direction as the effective reading area; within the target sampling time, determine the vertical overlap between the reading content block corresponding to the reading position and the effective reading area, and if the vertical overlap meets the overlap condition, map the reading content block onto the screen of the educational terminal. In the effective reading area of the screen of the educational terminal, the vertical overlap is the number of pixels that overlap between the reading content block and the effective reading area of the screen in the vertical direction. The vertical overlap is used to determine whether the content block is within the effective reading area. The target sampling time, the vertical overlap, and the block height of the reading content block corresponding to the reading position are fused together to analyze the actual reading situation of each student on the reading content block in the effective reading area of the screen, and the effective reading time of each student on the reading content block corresponding to the reading position is obtained. Based on the effective reading time and the number of lines of text contained in the reading content block corresponding to the reading position, the effective reading speed of the reading content block corresponding to the reading position within the effective reading time is calculated.
Citation Information
Patent Citations
Electronic equipment and reading mode identification method thereof
CN110782854A
Student comprehensive quality evaluation system and method based on multi-dimensional data fusion
CN121279881A
