Computer data acquisition and analysis system and method

Abstract

The invention relates to the field of data acquisition and analysis, and particularly discloses a computer data acquisition and analysis system and method.The computer data acquisition and analysis system comprises the steps that firstly, school student monitoring videos acquired by cameras and student attendance check-in data acquired by intelligent equipment are acquired, and then feature extraction and correlation analysis are conducted on the school student monitoring videos and the student attendance check-in data through the deep learning technology; and finally, a classification result is obtained through a classifier to judge whether to send a student attendance early warning to a teacher end, so that measures can be quickly taken to deal with an absent condition, the efficiency and accuracy of school attendance management are improved, and the overall education quality is improved.

Description

Technical Field

[0001] The present application relates to the field of data collection and analysis, and more specifically, to a computer data collection and analysis system and method. Background Art

[0002] Computer data refers to the general term for all symbolic media that can be input into a computer and processed by a computer program in computer science. These data can be numbers, letters, symbols, analog quantities, etc., and have a certain meaning. Data is the manifestation and carrier of information, which can be symbols, text, numbers, voice, images, videos, etc., and computer-based data acquisition and analysis is the process of collecting data from various data sources (such as sensors, databases, and networks) for analysis. In the traditional school student attendance monitoring method, teachers usually record student attendance by calling the roll in class, which is time-consuming.

[0003] With the development of the Internet, online attendance systems have emerged. However, even with online attendance systems, students may ask classmates to sign for them or falsify attendance records.

[0004] Therefore, a computer data acquisition and analysis system and method are desired. Summary of the invention

[0005] In order to solve the above technical problems, this application is proposed. The embodiment of this application provides a computer data acquisition and analysis system and method, which first obtains school student monitoring videos collected by cameras and student attendance check-in data collected by smart devices, and then uses deep learning technology to perform feature extraction and correlation analysis on the two, and finally obtains classification results through a classifier to determine whether to send a student attendance warning to the teacher side, so as to facilitate rapid measures to deal with absences, improve the efficiency and accuracy of school attendance management, and improve the overall education quality.

[0006] According to one aspect of the present application, a computer data acquisition and analysis system is provided, comprising:

[0007] The school student data acquisition module is used to obtain school student monitoring videos collected by cameras and student attendance data collected by smart devices;

[0008] A school student data extraction module, used to extract school student facial salient global feature vectors and student attendance sign-in text-related semantic feature vectors from the school student monitoring video collected by the camera and the student attendance sign-in data collected by the smart device;

[0009] The student attendance warning analysis module is used to determine whether to send a student attendance warning to the teacher end based on the school student facial salient global feature vector and the student attendance sign-in text associated semantic feature vector.

[0010] According to another aspect of the present application, a computer data acquisition and analysis method is provided, comprising:

[0011] Obtain school student surveillance videos collected by cameras and student attendance data collected by smart devices;

[0012] Extracting school student facial salient global feature vectors and student attendance sign-in text-related semantic feature vectors from the school student monitoring video collected by the camera and the student attendance sign-in data collected by the smart device;

[0013] Based on the school student facial salient global feature vector and the student attendance sign-in text associated semantic feature vector, it is determined whether to send a student attendance warning to the teacher end.

[0014] Compared with the prior art, the computer data acquisition and analysis system and method provided by the present application first obtains the school student monitoring video collected by the camera and the student attendance check-in data collected by the smart device, and then uses the deep learning technology to perform feature extraction and correlation analysis on the two, and finally obtains the classification result through the classifier to determine whether to send a student attendance warning to the teacher side, so as to facilitate quick measures to deal with absences, improve the efficiency and accuracy of school attendance management, and improve the overall education quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0016] Figure 1 4 is a block diagram of a computer data acquisition and analysis system according to an embodiment of the present application.

[0017] Figure 2 It is a block diagram of a school student data extraction module in a computer data acquisition and analysis system according to an embodiment of the present application.

[0018] Figure 3 It is a block diagram of a student attendance check-in data feature extraction unit in a computer data acquisition and analysis system according to an embodiment of the present application.

[0019] Figure 4It is a block diagram of a student attendance warning analysis module in a computer data acquisition and analysis system according to an embodiment of the present application.

[0020] Figure 5 The figure is a flow chart of a computer data collection and analysis method according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0022] Figure 1 : is a block diagram of a computer data acquisition and analysis system according to an embodiment of the present application. Figure 1 As shown, according to the computer data acquisition and analysis system 100 of the embodiment of the present application, it includes: a school student data acquisition module 110, which is used to obtain the school student monitoring video collected by the camera and the student attendance check-in data collected by the smart device; a school student data extraction module 120, which is used to extract the school student facial significant global feature vector and the student attendance check-in text associated semantic feature vector from the school student monitoring video collected by the camera and the student attendance check-in data collected by the smart device; a student attendance warning analysis module 130, which is used to determine whether to send a student attendance warning to the teacher end based on the school student facial significant global feature vector and the student attendance check-in text associated semantic feature vector.

[0023] In the above-mentioned computer data acquisition and analysis system 100, the school student data acquisition module 110 is used to obtain the school student monitoring video collected by the camera and the student attendance check-in data collected by the smart device. It should be understood that in the field of computer science, data refers to the general term for all symbolic media that can be input into a computer and processed by a computer program. These data include numbers, letters, symbols, analog quantities, etc., and have specific meanings. Data is the form and carrier of information, which can be expressed as symbols, text, numbers, voice, images, videos, etc. Computer-based data acquisition and analysis is the process of collecting and analyzing data from various data sources (such as sensors, databases, networks, etc.). In traditional school student attendance monitoring methods, teachers usually record student attendance by calling the roll in class, which takes a long time. With the development of the Internet, online attendance systems have emerged. However, even with the use of online attendance systems, students may still circumvent attendance monitoring by asking classmates to sign on their behalf or forging attendance records.

[0024] In the technical solution of this application, the surveillance video can monitor the security situation on campus in real time, prevent the occurrence of accidents, and provide evidence for campus emergencies to ensure the safety of teachers and students. Intelligent devices automatically record information such as students' arrival, departure and recess activities through the attendance check-in system, helping teachers to understand students' attendance in a timely manner, reduce absenteeism and improve management efficiency. Among them, high-definition surveillance cameras transmit real-time video streams to the monitoring center through the network, usually combined with artificial intelligence technology, which can realize functions such as face recognition and abnormal behavior detection, and improve the intelligent level of monitoring. Secondly, intelligent attendance equipment such as fingerprint recognition devices, facial recognition systems or RFID card readers can automatically record students' identity information and generate attendance data when they arrive at school. These data are usually stored in the school's information management system, which is convenient for teachers and managers to check and analyze at any time. In this way, surveillance videos and attendance data can be effectively utilized to improve the scientificity and efficiency of campus management and create a harmonious learning environment. Therefore, in the technical solution of the present application, by obtaining the school student monitoring video collected by the camera and the student attendance check-in data collected by the smart device, and combining it with deep learning technology, it is determined whether it is necessary to send a student attendance warning to the teacher side, so as to take quick measures to deal with absenteeism, thereby improving the efficiency and accuracy of the school's attendance management, and then improving the overall education quality.

[0025] In the above-mentioned computer data acquisition and analysis system 100, the school student data extraction module 120 is used to extract the school student facial significant global feature vector and the student attendance sign-in text associated semantic feature vector from the school student monitoring video collected by the camera and the student attendance sign-in data collected by the smart device. It should be understood that by extracting the facial significant global feature vector in the school student monitoring video and the associated semantic feature vector in the attendance sign-in data, a more intelligent campus management system can be constructed. This system not only improves the accuracy of identity authentication, but also enhances the ability to analyze student attendance behavior, providing important support for achieving efficient campus management and security.

[0026] Figure 2 FIG. 1 is a block diagram of a school student data extraction module in a computer data collection and analysis system according to an embodiment of the present application. Figure 2As shown, in a specific embodiment of the present application, the school student data extraction module 120 includes: a school student monitoring data preprocessing unit 121, which is used to perform data preprocessing on the school student monitoring video collected by the camera to obtain multiple school student monitoring noise reduction key frames; a school student monitoring noise reduction key frame feature extraction unit 122, which is used to perform feature extraction on the multiple school student monitoring noise reduction key frames to obtain the school student facial salient global feature vector; a student attendance sign-in data feature extraction unit 123, which is used to perform feature extraction on the student attendance sign-in data collected by the smart device to obtain the student attendance sign-in text-associated semantic feature vector.

[0027] It should be understood that surveillance videos often have problems such as noise, blur, and lighting changes, which can affect the accuracy of face recognition, behavior analysis, and anomaly detection. Therefore, it is necessary to preprocess the surveillance video data. Among them, in the processing process, the video needs to be denoised first. Commonly used denoising techniques include spatial domain and frequency domain methods. Spatial domain methods such as median filtering and Gaussian filtering can effectively reduce image noise, while frequency domain methods such as Fourier transform can remove high-frequency noise in the image. Then, the key frame extraction operation is performed. The key frame represents an important static image in the video and usually contains the main information of the scene. By analyzing the time series changes of the video, the inter-frame difference method or optical flow method can be used to detect frames with significant changes and select these frames as key frames. These methods can effectively filter out repeated and irrelevant frames and retain only the most representative images. By preprocessing the school student surveillance video collected by the camera and obtaining multiple denoised key frames, the quality and effectiveness of the surveillance video can be significantly improved. This process not only helps to improve the accuracy and stability of face recognition, but also provides a reliable data basis for campus safety management and student behavior analysis, ensuring the efficient operation of the surveillance system.

[0028] Furthermore, feature extraction of noise reduction key frames from student monitoring in multiple schools can achieve effective identification of different students, thereby improving the accuracy and efficiency of the attendance system. It also provides important data support for school security management and attendance monitoring, ensuring the safety and harmony of the campus environment.

[0029] Furthermore, feature extraction of student attendance data collected by smart devices can help educational institutions gain a deeper understanding of student attendance behavior patterns, identify attendance anomalies, and provide data support for school decision-making. In addition, associating semantic feature vectors helps to achieve intelligent management of student attendance, thereby optimizing teaching and management processes.

[0030] In a specific embodiment of the present application, the school student monitoring data preprocessing unit 121 is used to: extract key frames from the school student monitoring video collected by the camera to obtain multiple school student monitoring key frames; pass the multiple school student monitoring key frames through a school student monitoring denoising module based on an autoencoder to obtain the multiple school student monitoring denoising key frames.

[0031] It should be understood that surveillance videos usually contain a large number of continuous frames, many of which are repeated or similar in time. Directly processing all frames not only wastes storage space and computing resources, but may also lead to inaccurate analysis results. Specifically, first, the original video needs to be preprocessed to ensure video quality. This includes denoising, adjusting brightness and contrast, etc., in order to improve the accuracy of subsequent analysis. Next, the difference between video frames is used to identify key frames. Commonly used methods include inter-frame difference method and optical flow method. Inter-frame difference method detects significantly changed frames by calculating the pixel difference between adjacent frames. When the difference between frames exceeds the set threshold, the frame is considered a key frame. The optical flow method focuses on motion information and identifies important frames by analyzing the motion trajectory of objects in the scene. Finally, the extracted key frames can be used to train and test subsequent deep learning models to realize functions such as student facial recognition and behavior analysis. This key frame not only retains important information in the video, but also reduces the amount of processed data, improving the response speed and accuracy of the system.

[0032] Furthermore, considering that monitoring key frames are usually affected by factors such as noise, blur, and lighting changes, these problems may hinder effective monitoring and analysis. Therefore, the use of denoising technology can improve the usability of images and enable monitoring systems to perform their functions more accurately. Among them, autoencoders are an effective deep learning model that is widely used in image denoising. Its working principle is to compress the input image into a low-dimensional feature representation through an encoder, and then reconstruct the features back to the original image through a decoder. In the process of denoising, the autoencoder can learn how to remove noise from the input image and retain important feature information. Specifically, the extracted monitoring key frames are first input into the trained autoencoder. The encoder part maps these key frames to a low-dimensional feature space, and in this process, the noise information will be effectively suppressed. Then, the decoder part reconstructs this feature representation into a denoised image. In this process, the autoencoder effectively removes the noise in the image by learning to extract effective features, generating a clearer and more realistic image. Finally, through such denoising processing, the school monitoring system can more accurately identify students' facial features and behavior patterns, thereby improving campus safety and management efficiency.

[0033] Specifically, the multiple school student monitoring key frames are passed through a school student monitoring denoising module based on an autoencoder to obtain the multiple school student monitoring denoising key frames, including: inputting the multiple school student monitoring key frames into an encoder of the school student monitoring denoising module based on an autoencoder, wherein the encoder uses a convolutional layer to perform explicit spatial encoding on the multiple school student monitoring key frames to obtain image features; and, inputting the image features into a decoder of the school student monitoring denoising module based on an autoencoder, wherein the decoder uses a deconvolution layer to perform deconvolution processing on the image features to obtain the multiple school student monitoring denoising key frames.

[0034] In a specific embodiment of the present application, the school student monitoring denoising key frame feature extraction unit 122 includes: passing the multiple school student monitoring denoising key frames through a school student face interest target detection network to obtain multiple school student face interest regions; passing the multiple school student face interest regions through a school student face salient target detector to obtain a school student face salient global feature map; performing global mean pooling on the school student face salient global feature map along the channel dimension to obtain the school student face salient global feature vector.

[0035] It should be understood that although the keyframes after monitoring denoising have eliminated many interferences in the image, it is still necessary to further extract the specific areas of the student's face for more in-depth analysis. Among them, the facial object of interest detection network is usually based on deep learning technology, especially convolutional neural network, to achieve efficient detection of faces in images. The network is trained with a large amount of labeled training data and can learn the diversity of facial features and their changes under different environmental conditions. Specifically, first, the denoised keyframes are input into the pre-trained face detection network. These networks, such as YOLO (You Only Look Once) or Faster R-CNN, can quickly and accurately identify the face area in the image. The network will output the position coordinates of each detected face (usually represented by a rectangular box) and the corresponding confidence by analyzing the pixel features of the image. Next, the network will filter out valid face detection results according to the set threshold. At this stage, the detection results below the threshold will be filtered out to ensure that only high-confidence face areas are retained. In this way, it can be ensured that the extracted facial interest areas have high accuracy and credibility. Finally, the detected regions of interest on the students’ faces are cropped and labeled for subsequent processing and analysis.

[0036] Specifically, the plurality of school student monitoring denoised key frames are passed through a school student facial interest target detection network to obtain a plurality of school student facial interest regions, including: using the school student facial interest target detection network to process the plurality of school student monitoring denoised key frames using the following target detection formula to obtain the plurality of school student facial interest regions;

[0037] Among them, the target detection formula is:

[0038] ROI=(cls(ψ det ,,B),Regr(ψ det , B))

[0039] Among them, ψ det Monitor denoising key frames for students from multiple schools, B is the anchor frame, ROI is the facial region of interest for students from multiple schools, cls(ψ det ,B) indicates classification, Regr(ψ det ,B) indicates regression.

[0040] In another embodiment of the present application, face detection can also be performed on the denoised key frames to locate the student's facial area. Commonly used detection algorithms include Haar feature cascade classifiers, HOG (Histogram of Directed Gradients) and deep learning methods (such as MTCNN or Dlib). These algorithms can quickly and effectively detect the face in the image and determine its position in the image. Then, the detected facial area is feature extracted using a deep learning model. Through a pre-trained model trained on a large-scale face data set, such as FaceNet or VGG-Face, each facial image can be converted into a high-dimensional feature vector. This feature vector contains the salient feature information of the face, such as the relative position and shape of the eyes, nose and mouth, which can effectively represent the identity characteristics of the student.

[0041] Furthermore, the facial regions of interest of multiple school students are processed by the school student facial salient object detector to obtain the salient global feature map of school students' faces, mainly to further improve the understanding and recognition ability of students' facial features. This global feature map can effectively integrate the important information in each student's facial region. Among them, the salient object detector is usually based on deep learning technology, focusing on extracting important features in the image, and can effectively distinguish the salient regions of the face from the background. By learning the facial features in the training data, the detector can automatically identify the key points of the face (such as eyes, nose, mouth, etc.) and generate a global feature map. Such a feature map contains important clues such as facial structural information and emotional expression. Specifically, first, the denoised facial region of interest is input into the salient object detector. The network will extract features from the input facial image through the convolution layer, and use the capabilities of the multi-layer neural network to analyze features at different levels. In this process, the salient object detector not only focuses on local features, but also captures global features through contextual information, thereby generating a comprehensive facial feature description. Subsequently, the detector will output a global feature map containing facial salient features. This feature map integrates key information in the facial region, such as facial expressions, postures, and lighting changes, and can provide rich feature representations for subsequent applications. In this way, schools can better identify students, monitor their emotional states, and perform behavioral analysis. Finally, these global feature maps can be used as input and further applied to facial recognition algorithms, sentiment analysis models, etc. Since the salient feature maps extract rich facial information, they can significantly improve the performance of these models, making schools more accurate and efficient in safety management, mental health monitoring, and personalized education.

[0042] Specifically, the multiple school student facial regions of interest are passed through a school student facial salient target detector to obtain a school student facial salient global feature map, including: using each layer of the school student facial salient target detector to respectively perform the following on the input data in the forward pass of the layer: using a first convolution kernel to perform convolution processing on the input data to obtain a first convolution feature map; using a second convolution kernel to perform convolution processing on the first convolution feature map to obtain a second convolution feature map, wherein the size of the first convolution kernel is larger than the size of the second convolution kernel; and, performing pooling processing on the second convolution feature map to obtain a pooled feature map; and, performing activation processing on the pooled feature map to obtain an activated feature map; wherein the output of the last layer of the school student facial salient target detector is the school student facial salient global feature map, and the input of the first layer of the school student facial salient target detector is the multiple school student facial regions of interest.

[0043] Furthermore, global mean pooling can effectively extract the global information of facial features by calculating the average value of each channel in the feature map, reduce the computational complexity, retain key information, and enhance the generalization ability of the model. Specifically, the salient global feature map of school students' faces generated by the salient object detector is input into the global mean pooling layer. Feature maps usually have multiple channels (for example, RGB images have three channels), and each channel represents different feature information. In the process of global mean pooling, the mean of each channel is calculated, that is, the feature map is averaged in the spatial dimensions (width and height). Specifically, for each channel, mean pooling adds all pixel values ​​in the feature map and then divides it by the number of pixels to obtain the average value of the channel. This process generates a single value for each channel and finally outputs a feature vector containing the mean of all channels. In this way, the obtained salient global feature vector of school students' faces will contain the global information of each channel. This way of compressing features reduces the computing resources required for subsequent processing, and through the operation of mean pooling, the main information in the facial features is highlighted, while noise and details are ignored.

[0044] Figure 3 FIG. 1 is a block diagram of a student attendance check-in data feature extraction unit in a computer data collection and analysis system according to an embodiment of the present application. Figure 3 As shown, in a specific embodiment of the present application, the student attendance check-in data feature extraction unit 123 includes: a student attendance check-in text semantic encoding subunit 1231, which is used to pass the student attendance check-in data collected by the smart device through a student attendance check-in text data semantic encoder to obtain multiple student attendance check-in text semantic feature vectors; a student attendance check-in text feature cascade subunit 1232, which is used to cascade the multiple student attendance check-in text semantic feature vectors into a student attendance check-in text associated semantic feature vector.

[0045] It should be understood that the student attendance check-in data collected by the smart device is processed by the student attendance check-in text data semantic encoder, which can effectively extract and represent the semantic information in the attendance data. Among them, the semantic encoder can be a model based on deep learning, such as the BERT (Bidirectional Encoder Representations from Transformers) model, which can understand the context and semantic relationship of the text. Specifically, the original check-in data is input into the semantic encoder. The encoder will perform word segmentation and embedding for each check-in information, and convert the text into a vector representation. In this process, the encoder will not only consider the individual meaning of each word, but also understand the relationship between words, and capture deeper semantic information through contextual information. Then, the encoder will output the semantic feature vectors of each check-in record. The dimensions of these vectors can vary according to the settings of the model, but are usually kept at a fixed size for batch processing. In this way, the rich information of the check-in data can be captured more comprehensively, rather than relying solely on the original text data.

[0046] Specifically, the student attendance check-in data collected by the smart device is passed through a student attendance check-in text data semantic encoder to obtain multiple student attendance check-in text semantic feature vectors, including: performing word segmentation processing on the student attendance check-in data collected by the smart device to obtain an attendance check-in word sequence; using the embedding layer of the student attendance check-in text data semantic encoder to respectively map each attendance check-in word in the attendance check-in word sequence to a word embedding vector to obtain a sequence of attendance check-in word embedding vectors; using the converter-based Bert model of the student attendance check-in text data semantic encoder to perform global context-based semantic encoding on the sequence of attendance check-in word embedding vectors to obtain multiple student attendance check-in text semantic feature vectors.

[0047] Furthermore, concatenating multiple student attendance sign-in text semantic feature vectors into a student attendance sign-in text associated semantic feature vector can enhance the expressive power of the feature vector, enabling the model to more comprehensively understand the potential patterns and relationships in the attendance data, thereby improving the accuracy of data analysis. Among them, a simple concatenation method can be used to arrange multiple feature vectors in sequence to form a longer vector.

[0048] In a specific embodiment of the present application, the student attendance sign-in text feature cascade subunit includes: initializing a priority queue PriorityQueue for storing multiple student attendance sign-in text semantic feature vectors, wherein the priority queue will sort according to the priority of each student attendance sign-in text semantic feature vector to ensure that it can be processed in priority order during cascading; inserting each student attendance sign-in text semantic feature vector into the priority queue; taking out the student attendance sign-in text semantic feature vectors from the priority queue in turn, and performing cascade operations in priority order; after cascading all the student attendance sign-in text semantic feature vectors, a student attendance sign-in text associated semantic feature vector is generated.

[0049] Part of the deployment code is shown below.

[0050]

[0051]

[0052] It should be understood that, first, by defining a PriorityQueue class, the heapq module is used to implement the function of the priority queue. This class contains a push method for inserting elements, a pop method for taking out the element with the highest priority, and an is_empty method for determining whether the queue is empty. Then, these multiple student attendance sign-in text semantic feature vectors and their corresponding priorities (such as timestamps) are inserted into the priority queue, where each feature vector is a numpy array. Next, the feature vectors are taken out from the priority queue in turn and cascaded in order of priority. The cascade operation is implemented by the np.concatenate function, which concatenates multiple feature vectors into a large feature vector. Finally, a student attendance sign-in text-associated semantic feature vector is generated, which contains the information of all feature vectors and is cascaded in order of priority.

[0053] By using the priority queue mechanism, this method can effectively cascade in a predefined priority order when processing the semantic feature vectors of student attendance check-in text, thereby improving the effect of feature fusion. Among them, the priority queue mechanism ensures that the feature vectors are cascaded in order of importance or time, avoiding the information loss or redundancy that may be caused by random or disordered cascading. This orderly cascading method can better retain the key information of the feature vector and improve the accuracy of feature fusion. Through the priority queue, the system can flexibly process feature vectors of different priorities to adapt to different application scenarios and requirements. For example, the priority can be determined according to timestamps, importance or other predefined rules, thereby enhancing the robustness and adaptability of the system. The priority queue mechanism makes the cascading process of feature vectors more efficient and orderly, reducing unnecessary calculations and resource consumption. This efficient cascading method can significantly improve the overall efficiency of attendance analysis, allowing the system to generate student attendance judgment classification feature vectors more quickly.

[0054] Here, by encapsulating the cascading process of feature vectors in a priority queue, the design and maintenance of the system becomes simpler and more intuitive. In this way, the use of the priority queue mechanism to cascade the semantic feature vectors of student attendance sign-in text not only improves the accuracy of feature fusion and the robustness of the system, but also significantly improves the efficiency of attendance analysis and simplifies the design and maintenance of the system. This mechanism provides strong technical support for the realization of intelligent student attendance management.

[0055] In the above-mentioned computer data acquisition and analysis system 100, the student attendance warning analysis module 130 is used to determine whether to send a student attendance warning to the teacher side based on the school student facial significant global feature vector and the student attendance sign-in text associated semantic feature vector. It should be understood that judging whether to send a student attendance warning to the teacher side based on the school student facial significant global feature vector and the student attendance sign-in text associated semantic feature vector is intended to improve the intelligence and accuracy of attendance management by integrating multi-dimensional information. This process, by combining facial recognition technology with attendance data analysis, can more comprehensively monitor student attendance, timely discover potential absence problems, and issue warnings to teachers to prompt them to take corresponding measures.

[0056] Figure 4 FIG. 1 is a block diagram of a student attendance warning analysis module in a computer data collection and analysis system according to an embodiment of the present application. Figure 4As shown, in a specific embodiment of the present application, the student attendance warning analysis module 130 includes: a student attendance data feature fusion unit 131, which is used to perform perspective co-projection matching based on node low-dimensional embedding expression on the school student's facial salient global feature vector and the student attendance sign-in text associated semantic feature vector to obtain a student attendance judgment classification feature vector; a student attendance warning classification judgment unit 132, which is used to pass the student attendance judgment classification feature vector through a classifier to obtain a classification result, and the classification result is used to determine whether to send a student attendance warning to the teacher end.

[0057] It should be understood that the facial salient global feature vector contains information such as the student's emotional state, attention and participation, which can reflect the student's true performance in class. The attendance sign-in text-associated semantic feature vector provides contextual information about the student's attendance record, such as the time, frequency and regularity of sign-in. The combination of these two feature vectors can help educators better identify possible absence risks and learning problems.

[0058] In particular, in the technical solution of the present application, the school student facial salient global feature vector is obtained by processing the facial features extracted from the surveillance video, emphasizing the facial expressions, postures and identity information of the students in the attendance scene. This type of feature vector is generated by facial recognition and salient target detection network, has strong image features and spatiotemporal dynamic information, reflects the real-time status and behavioral characteristics of the students, and focuses on the visual information at a specific time point. In contrast, the student attendance check-in text-associated semantic feature vector is obtained by semantically encoding the attendance check-in data collected by the smart device, focusing on the description of the check-in behavior and the understanding of the context, reflecting the logical structure of the attendance record, including the check-in time, place and its relevance to the student identity, mainly relying on the semantic information and structured features of the text data, so its information expression is relatively static and lacks dynamics in time and space. When performing feature fusion, the heterogeneity between the school student facial salient global feature vector and the student attendance check-in text-associated semantic feature vector may lead to the non-smoothness of the fused student attendance judgment classification feature vector manifold, affecting the certainty of feature expression. Among them, when the facial salient global feature vectors and the text-associated semantic feature vectors are unevenly distributed in the feature space or there are large differences, the classifier may face difficulties in capturing potential association patterns, which will reduce the accuracy and stability of the classification results. Especially in the attendance judgment process, the smoothness of the manifold is crucial to the effectiveness of the classification results. Any unevenness of the manifold may lead to information loss or misunderstanding, thereby affecting the final attendance warning decision. Therefore, in the technical solution of the present application, the school student facial salient global feature vectors and the student attendance sign-in text-associated semantic feature vectors are matched based on the perspective co-projection expression of the node low-dimensional embedding expression to obtain the student attendance judgment classification feature vector.

[0059] Among them, the perspective co-projection matching based on the node low-dimensional embedding expression is performed on the school student facial salient global feature vector and the student attendance sign-in text associated semantic feature vector to obtain the student attendance judgment classification feature vector, including: constructing an attendance distance topological matrix between the school student facial salient global feature vector and the student attendance sign-in text associated semantic feature vector; constructing an attendance feature value granularity association matrix between the school student facial salient global feature vector and the student attendance sign-in text associated semantic feature vector; performing topological association modulation on the attendance feature value granularity association matrix based on the attendance distance topological matrix to obtain an attendance topological modulation association matrix; using the attendance topological modulation association matrix as the perspective co-projection space, projecting the school student facial salient global feature vector and the student attendance sign-in text associated semantic feature vector to the perspective co-projection space to obtain the perspective modulated school student facial salient global feature vector and the perspective modulated student attendance sign-in text associated semantic feature vector; fusing the perspective modulated school student facial salient global feature vector and the perspective modulated student attendance sign-in text associated semantic feature vector to obtain the student attendance judgment classification feature vector.

[0060] The fusion step is specifically expressed as:

[0061] M t =D(V 1 , V 2 )

[0062] D(V 1 ,V 2 ) is the attendance distance topological matrix between the school student facial salient global feature vector and the student attendance sign-in text associated semantic feature vector, that is, d i,j =d(v 1i ,v 2j ), and V 1 and V 2 are column vectors.

[0063]

[0064] V c =α·V 1 ′++β·V 2 '

[0065] Among them, V 1 represents the significant global feature vector of the school students’ faces, V 2 represents the semantic feature vector associated with the student attendance check-in text, D(V 1 ,V 2 ) is the attendance distance topological matrix between the school student facial salient global feature vector and the student attendance sign-in text associated semantic feature vector, Mt represents the attendance distance topology matrix, T represents the transpose of the vector, Represents matrix multiplication, M v represents the attendance feature value granularity association matrix, M c represents the attendance topology modulation correlation matrix, V 1 ′ represents the global facial salient feature vector of students in the school modulated by the viewing angle, V 2 ′ represents the semantic feature vector associated with the student attendance check-in text modulated by perspective, α and β represent weighted hyperparameters, V c Represents the classification feature vector of student attendance judgment.

[0066] In the technical solution of the present application, the school student facial salient global feature vector and the student attendance sign-in text associated semantic feature vector have relatively significant feature heterogeneity, which results in that when the feature fusion of the school student facial salient global feature vector and the student attendance sign-in text associated semantic feature vector is performed, the feature manifold and feature space mismatch caused by the heterogeneity will cause the student attendance judgment classification feature vector to fail to have feature manifold smoothness, thereby affecting the certainty of its feature expression.

[0067] Based on this, in the technical solution of the present application, the school student facial significant global feature vector and the student attendance sign-in text-associated semantic feature vector are matched by perspective co-projection based on the low-dimensional embedding expression of nodes, which first constructs the attendance distance topological matrix between the school student facial significant global feature vector and the student attendance sign-in text-associated semantic feature vector. This step utilizes the idea of ​​graph theory, takes the characteristic values ​​of each position in the school student facial significant global feature vector and the student attendance sign-in text-associated semantic feature vector as nodes, and uses the distance metric function to determine the low-dimensional embedding expression of the edges between nodes, so as to construct a feature expression that can quantify the topological correlation relationship between the school student facial significant global feature vector and the student attendance sign-in text-associated semantic feature vector. This step is crucial for understanding the spatial distribution of data points and their mutual connections, and also provides basic structural information for subsequent steps.

[0068] Next, an attendance feature value granularity association matrix is ​​constructed between the school student facial salient global feature vector and the student attendance sign-in text associated semantic feature vector. This step focuses on exploring the correlation between the internal attributes of the feature vector. In a specific example, the product between the transposed vectors of the school student facial salient global feature vector and the student attendance sign-in text associated semantic feature vector is calculated to obtain the attendance feature value granularity association matrix. Furthermore, the attendance feature value granularity association matrix is ​​topologically associated modulated based on the attendance distance topological matrix to obtain an attendance topological modulation association matrix. That is, the feature value granularity association information of the school student facial salient global feature vector and the student attendance sign-in text associated semantic feature vector is driven to wander on the distance topological map to construct a low-dimensional modulated perspective co-projection space domain for mapping the school student facial salient global feature vector and the student attendance sign-in text associated semantic feature vector to a continuous high-dimensional regression space attribute.

[0069] Next, the attendance topology modulation association matrix is ​​used as the perspective co-projection space, and the school student facial salient global feature vector and the student attendance sign-in text-related semantic feature vector are projected into the perspective co-projection space to obtain the perspective-modulated school student facial salient global feature vector and the perspective-modulated student attendance sign-in text-related semantic feature vector. That is, the key to this technical solution is to find an implicit third space with modulation capability, in which as much information of the original high-dimensional data as possible is retained, so that the converted new feature vector not only reduces the dimension, but also maintains the relative position relationship between the original features.

[0070] Finally, the perspective-modulated school student facial salient global feature vector and the perspective-modulated student attendance sign-in text-associated semantic feature vector are fused to obtain the student attendance judgment classification feature vector. Fusion can be achieved in a variety of forms, ranging from simple arithmetic average to complex ensemble learning algorithms, depending on the application scenario and the goals pursued. The final student attendance judgment classification feature vector combines the main features of the two sets of input data and provides a more comprehensive data representation, which is not only conducive to improving the performance of machine learning models, but also promotes the ability of cross-modal or multi-source data analysis, thereby significantly enhancing the understanding and utilization efficiency of complex data structures.

[0071] Furthermore, the student attendance judgment classification feature vector contains a variety of information, including the student's facial features and semantic information of attendance behavior. These feature vectors are fed into the classifier, whose goal is to judge the student's attendance status based on the input feature vector, usually a binary classification, such as "normal attendance" or "absent". Among them, the prepared student attendance judgment classification feature vector must first be input into the selected classifier. Then, the classifier processes the input feature vector according to its internally learned model parameters and calculates the probability distribution of each possible category. Finally, the classifier selects the category with the highest probability as the classification result. For example, if the classification result shows "absent", the system will trigger an early warning mechanism and send a notification to the teacher. This early warning mechanism not only relies on the accuracy of the classification results, but can also be dynamically adjusted according to the confidence of the model, so as to more effectively manage attendance issues, pay attention to students' individual needs, and improve the quality of education.

[0072] In summary, the embodiment of the present application first obtains the school student monitoring video collected by the camera and the student attendance check-in data collected by the smart device, and then uses deep learning technology to perform feature extraction and correlation analysis on the two, and finally obtains the classification result through the classifier to determine whether to send a student attendance warning to the teacher side, so as to facilitate quick measures to deal with absenteeism, improve the efficiency and accuracy of school attendance management, and improve the overall education quality.

[0073] As described above, the computer data acquisition and analysis system 100 according to the embodiment of the present application can be implemented in various terminal devices. In one example, the computer data acquisition and analysis system 100 can be integrated into the terminal device as a software module and / or a hardware module. For example, the computer data acquisition and analysis system 100 can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the computer data acquisition and analysis system 100 can also be one of the many hardware modules of the terminal device.

[0074] Alternatively, in another example, the computer data acquisition and analysis system 100 and the terminal device may also be separate devices, and the computer data acquisition and analysis system 100 may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0075] Figure 5 Flowchart of the computer data collection and analysis method according to the embodiment of the present application. Figure 5As shown, according to the computer data collection and analysis method of the embodiment of the present application, it includes: S110, obtaining the school student monitoring video collected by the camera and the student attendance check-in data collected by the smart device; S120, extracting the school student's facial salient global feature vector and the student attendance check-in text-associated semantic feature vector from the school student monitoring video collected by the camera and the student attendance check-in data collected by the smart device; S130, based on the school student's facial salient global feature vector and the student attendance check-in text-associated semantic feature vector, judging whether to send a student attendance warning to the teacher end.

[0076] Here, those skilled in the art will appreciate that the specific operations of each step in the above-mentioned computer data acquisition and analysis method have been described in detail above. Figures 1 to 4 The computer data acquisition and analysis system has been described in detail, and therefore, its repeated description will be omitted.

Claims

1. A computer data acquisition and analysis system, characterized in that: include: The school student data acquisition module is used to obtain school student monitoring videos collected by cameras and student attendance data collected by smart devices; A school student data extraction module, used to extract school student facial salient global feature vectors and student attendance sign-in text-related semantic feature vectors from the school student monitoring video collected by the camera and the student attendance sign-in data collected by the smart device; The student attendance warning analysis module is used to determine whether to send a student attendance warning to the teacher end based on the school student facial salient global feature vector and the student attendance sign-in text associated semantic feature vector.

2. The computer data acquisition and analysis system according to claim 1, characterized in that: The school student data extraction module includes: A school student monitoring data preprocessing unit, used for performing data preprocessing on the school student monitoring video collected by the camera to obtain a plurality of school student monitoring noise reduction key frames; A school student monitoring denoising key frame feature extraction unit is used to extract features from the plurality of school student monitoring denoising key frames to obtain a significant global feature vector of the school student's face; The student attendance check-in data feature extraction unit is used to extract features from the student attendance check-in data collected by the smart device to obtain the student attendance check-in text-related semantic feature vector.

3. The computer data acquisition and analysis system according to claim 2, characterized in that: The school student monitoring data preprocessing unit is used to: Extracting key frames from the school student monitoring video collected by the camera to obtain a plurality of school student monitoring key frames; The multiple school student monitoring key frames are passed through a school student monitoring denoising module based on an autoencoder to obtain the multiple school student monitoring denoising key frames.

4. The computer data acquisition and analysis system according to claim 3, characterized in that: The school student monitoring denoising key frame feature extraction unit comprises: Passing the plurality of school student monitoring denoised key frames through a school student facial object of interest detection network to obtain a plurality of school student facial regions of interest; Passing the plurality of school student facial interest regions through a school student facial salient object detector to obtain a school student facial salient global feature map; The global significant facial feature map of the school students is globally mean pooled along the channel dimension to obtain the global significant facial feature vector of the school students.

5. The computer data acquisition and analysis system according to claim 4, characterized in that: The student attendance check-in data feature extraction unit includes: A student attendance sign-in text semantic encoding subunit, used for passing the student attendance sign-in data collected by the smart device through a student attendance sign-in text data semantic encoder to obtain a plurality of student attendance sign-in text semantic feature vectors; The student attendance sign-in text feature cascading subunit is used to cascade the multiple student attendance sign-in text semantic feature vectors into a student attendance sign-in text associated semantic feature vector.

6. The computer data acquisition and analysis system according to claim 5, characterized in that: The student attendance check-in text feature cascade sub-unit includes: Initialize a priority queue PriorityQueue to store multiple student attendance sign-in text semantic feature vectors. The priority queue will sort the semantic feature vectors of each student attendance sign-in text according to their priority to ensure that they can be processed in priority order during cascading. Insert the semantic feature vector of each student's attendance check-in text into the priority queue; Take out the semantic feature vectors of student attendance check-in text from the priority queue in sequence, and perform cascade operations in order of priority; After cascading all the semantic feature vectors of student attendance check-in text, a student attendance check-in text associated semantic feature vector is generated.

7. The computer data acquisition and analysis system according to claim 6, characterized in that: The student attendance early warning analysis module includes: The student attendance data feature fusion unit is used to perform perspective co-projection matching based on the low-dimensional embedding expression of nodes on the school student facial salient global feature vector and the student attendance sign-in text-related semantic feature vector to obtain the student attendance judgment classification feature vector; The student attendance warning classification judgment unit is used to pass the student attendance judgment classification feature vector through a classifier to obtain a classification result, and the classification result is used to determine whether to send a student attendance warning to the teacher end.

8. The computer data acquisition and analysis system according to claim 7, characterized in that: The student attendance data feature fusion unit includes: Constructing an attendance distance topological matrix between the school student facial salient global feature vectors and the student attendance sign-in text associated semantic feature vectors; Constructing an attendance feature value granularity association matrix between the school student facial salient global feature vector and the student attendance check-in text associated semantic feature vector; Performing topological correlation modulation on the attendance eigenvalue granularity correlation matrix based on the attendance distance topological matrix to obtain an attendance topological modulation correlation matrix; Taking the attendance topology modulation association matrix as the perspective co-projection space, projecting the school student facial salient global feature vector and the student attendance sign-in text-associated semantic feature vector into the perspective co-projection space to obtain the perspective-modulated school student facial salient global feature vector and the perspective-modulated student attendance sign-in text-associated semantic feature vector; The view-modulated school student facial salient global feature vector and the view-modulated student attendance sign-in text-associated semantic feature vector are fused to obtain the student attendance judgment classification feature vector.

9. A computer data collection and analysis method, characterized in that: include: Obtain school student surveillance videos collected by cameras and student attendance data collected by smart devices; Extracting school student facial salient global feature vectors and student attendance sign-in text-related semantic feature vectors from the school student monitoring video collected by the camera and the student attendance sign-in data collected by the smart device; Based on the school student facial salient global feature vector and the student attendance sign-in text associated semantic feature vector, it is determined whether to send a student attendance warning to the teacher end.

10. The computer data collection and analysis method according to claim 9, characterized in that: Extracting school student facial salient global feature vectors and student attendance sign-in text-related semantic feature vectors from the school student monitoring video collected by the camera and the student attendance sign-in data collected by the smart device, including: Performing data preprocessing on the school student monitoring video collected by the camera to obtain a plurality of school student monitoring noise reduction key frames; Performing feature extraction on the plurality of school student monitoring denoising key frames to obtain a significant global feature vector of the school student's face; Feature extraction is performed on the student attendance check-in data collected by the smart device to obtain a semantic feature vector associated with the student attendance check-in text.

Images