An AI-based method for counting students' attendance in classrooms

Abstract

The invention discloses an AI-based method for counting the number of students in a classroom, and relates to the technical field of intelligent video surveillance. The method has the following implementation steps: constructing a classroom student data set; building a lightweight YOLOv7 network model; training an AI personnel detection model using an improved loss function; inputting a verification set into the AI ​​personnel detection model, and obtaining an optimal weight through experimental training; defining a classroom detection area, and collecting an RTSP video stream about the detection area; performing filtering gain processing on the collected RTSP video stream to reduce the signal-to-noise ratio and reduce image noise, and inputting the processed RTSP video stream into the AI ​​personnel detection model to obtain a final detection result. The invention greatly reduces the amount of calculation and the amount of parameters of a deep learning model, and also improves the accuracy of personnel detection, meets the real-time requirements of classroom student detection, and ensures the smooth progress of university teaching and other related work.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent video surveillance, and in particular to an AI-based method for counting the number of students in a classroom. Background Art

[0002] How to effectively count student attendance is an important teaching management task in colleges and universities. Attendance can effectively constrain students in time and space and ensure their participation in various educational and teaching activities arranged by the school. Teaching activities are one of the most important tasks of colleges and universities, and one of the most basic ways is classroom teaching. Therefore, strengthening the management of college students' attendance has become an important issue that needs to be urgently solved in college teaching and management.

[0003] In most universities in my country, the main methods of student attendance are as follows: ① by teachers calling the roll; ② by the teaching management department spot-checking student attendance; ③ by student cadres tracking attendance and calling the roll. The main problems with the above methods are as follows: ① teachers' manual roll-calling will take up normal teaching time and affect the teaching progress. If spot-checking is carried out, substitution often occurs; ② the teaching management department's spot-checking of student attendance by class will affect the classroom teaching order of teachers and increase the workload of the management, making it difficult to implement; ③ since universities usually link class attendance with class and individual evaluation, student cadres may have human factors in calling the roll of their own classes. At present, the attendance methods of the attendance system mainly include card swiping attendance, fingerprint attendance and face recognition attendance. If card swiping attendance is used in universities, every university student is required to bring a card to class, but this method is obviously not suitable for students. On the one hand, there are many students, and on the other hand, the situation of punching in for others cannot be avoided. At the same time, the card is not easy to save and easy to lose. The fingerprint attendance recognition result is affected by the problem of clear lines, the impact of seasonal changes on the skin, and the impact of finger wear. There are many college students, and the attendance time required for multiple people to line up in a short period of time is too long. It is also not applicable. The single face recognition attendance method is better than the previous two methods, but students queuing to use the attendance machine for face recognition will also be affected by factors such as outdoor light, facial posture, height, etc., and the time consumed is also too long. Based on the above shortcomings, the present invention proposes an AI-based classroom student attendance counting method. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes an AI-based method for counting the number of students in a classroom.

[0005] To achieve the above object, according to an embodiment of the first aspect of the present invention, a method for counting the number of students in a classroom based on AI is proposed, comprising the following steps:

[0006] Step 1: Build a classroom student data set, perform data enhancement on the images in the data set, and then divide the data set into a training set, a validation set, and a test set;

[0007] Step 2: Use the AI ​​processing module to build a lightweight YOLOv7 network model;

[0008] Step 3: After data normalization, the training set is input into the lightweight YOLOv7 network model, and the AI ​​personnel detection model is obtained by training using the improved loss function;

[0009] Step 4: Input the validation set into the AI ​​person detection model, obtain the optimal weight through experimental training, and test the images in the test set;

[0010] Step 5: Define the classroom detection area and use the video acquisition module to collect the RTSP video stream of the detection area;

[0011] Step 6: Use the video preprocessing module to perform filter gain processing on the collected RTSP video stream to reduce the signal-to-noise ratio and image noise; and input the processed RTSP video stream into the AI ​​personnel detection model to obtain the final detection result;

[0012] Step 7: Design the host computer monitoring platform.

[0013] Furthermore, the specific processing steps of the video preprocessing module are:

[0014] S61: Read the RTSP video stream frame by frame as the target image, and perform denoising and enhancement on the acquired target image data through a denoising and enhancement program; specifically:

[0015] Convert the read target image information into a digital signal, filter the converted digital signal, and obtain the amplitude information of the digital signal; collect the amplitude information of the digital signal according to the preset collection interval time, and generate an amplitude information group FZi;

[0016] The reference amplitude FM is calculated based on the received i amplitude information, where i≥10; specifically:

[0017] The average value of the amplitude information group is calculated according to the average value calculation formula as FZavg;

[0018] The standard deviation of the amplitude information group is calculated according to the standard deviation calculation formula as f;

[0019] Compare the standard deviation f with the set value; if f < the set value, let the latest collected amplitude information be FZt, and use the formula FM = (FZt + FZavg) / 2 to calculate the reference amplitude FM;

[0020] If f≥the set value, the amplitude information group FZi is traversed, the maximum amplitude information is marked as FZmax, and the minimum amplitude information is marked as FZmin; the difference ratio Cb is calculated using the formula Cb=(FZmax-FZmin) / FZmax;

[0021] The reference amplitude FM is calculated using the formula FM=FZavg×(1+f×Cb×a1); where a1 is the preset equalization coefficient;

[0022] Obtain the i+1th amplitude information and mark it as FZ(i+1); compare FZ(i+1) with the reference amplitude FM; if |FZ(i+1)-FM|≥preset difference, generate an adjustment signal; if |FZ(i+1)-FM|<preset difference, generate a normal signal;

[0023] When receiving the adjustment signal, the video preprocessing module adjusts the gain of the digital signal by controlling the programmable gain amplifier circuit to adjust the amplitude of the digital signal to the reference amplitude FM; then set i=i+1, and so on;

[0024] S62: converting the target image data with color into a grayscale image; determining the pixel value of the grayscale image according to the grayscale target image data, and establishing a binary image;

[0025] S63: performing edge detection and matching on the acquired target image data; splicing two or more images through an image mosaic program, and cropping the acquired image data through an image cropping program.

[0026] Furthermore, the data enhancement processing specifically includes:

[0027] S11: The dataset is cropped from random positions to 1 / 4 of the original image size as a supplementary dataset;

[0028] S12: The original images in the dataset are flipped from 6 angles as a supplementary dataset;

[0029] S13: The original images in the dataset are randomly scaled to different sizes as a supplementary dataset;

[0030] S14: Add noise to the original images in the dataset as a supplementary dataset.

[0031] Furthermore, the lightweight YOLOv7 network model includes:

[0032] Backbone network, using Ghostnet network to design YOLOv7's lightweight Backbone for feature extraction;

[0033] The prediction network Head is used to classify and locate features to obtain the final prediction results.

[0034] Furthermore, the improved loss function is to use CIoU Loss to define the anchor box loss function, and the CIoU Loss formula is as follows:

[0035]

[0036]

[0037]

[0038]

[0039]

[0040] Where ɑ is the weight function; U is used to measure the consistency of aspect ratio; ρ 2 (A, B) represents the center distance between the prediction box A and the target box B; c represents the diagonal distance between the minimum union interval of A and B; IoU is equal to the intersection of A and B divided by the union of the two; arctan h ω is the inclination angle of the rectangle diagonal.

[0041] Furthermore, the classroom detection area is achieved by drawing a rectangular area covering the classroom seats on the video using Opencv, so that the AI ​​personnel detection model only outputs the recognition results within the area and counts the number of attendees; the video acquisition module is composed of multiple groups of optical lenses, the direction of the optical lenses can be adjusted, and RTSP video streams about the detection area are collected from various angles.

[0042] Furthermore, two or more images are spliced ​​by an image mosaicking program, and the acquired image data is cropped by an image cropping program. The specific process is as follows:

[0043] According to the acquired target image data, a corresponding image set is established; in the image set, a corresponding reference image is determined, and the reference image is used as a reference for outputting a mosaic image;

[0044] According to the benchmark of the output mosaic image, the contrast matching of the mosaic image, the pixel size and data type of the image are determined, and the hue of the image is processed using the histogram equalization method and the color smoothing method.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] The present invention constructs a classroom student data set, performs data enhancement processing on the pictures in the data set, and then divides the data set into a training set, a validation set, and a test set; uses an AI processing module to build a lightweight YOLOv7 network model; inputs the training set into the lightweight YOLOv7 network model after data normalization, and obtains an AI personnel detection model by training with an improved loss function; inputs the validation set into the AI ​​personnel detection model, obtains the optimal weight through experimental training, and tests the pictures in the test set; defines a classroom detection area, uses a video acquisition module to collect an RTSP video stream about the detection area, uses a video preprocessing module to perform filtering gain processing on the collected RTSP video stream, reduces the signal-to-noise ratio, and reduces image noise; and inputs the processed RTSP video stream into the AI ​​personnel detection model to obtain the final detection result; the calculation amount and parameter amount of the deep learning model are greatly reduced, and the accuracy of personnel detection is also improved, so as to meet the real-time requirements of classroom student detection and ensure the smooth progress of university teaching and other related work. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 This is a principle block diagram of an AI-based classroom student attendance counting method of the present invention.

[0049] Figure 2 It is the lightweight YOLOv7 backbone network designed in the present invention.

[0050] Figure 3 This is a flow chart of the implementation method of the AI ​​classroom student attendance statistics method in the present invention. DETAILED DESCRIPTION

[0051] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] like Figure 1 As shown, an AI-based classroom student attendance counting method includes:

[0053] Step 1: Build a classroom student data set, perform data enhancement processing on the images in the data set, and then divide the data set into a training set, a validation set, and a test set; the data enhancement processing specifically includes:

[0054] S11: The dataset is cropped from random positions to 1 / 4 of the original image size as a supplementary dataset;

[0055] S12: The original images in the dataset are flipped from 6 angles as a supplementary dataset;

[0056] S13: The original images in the dataset are randomly scaled to different sizes as a supplementary dataset;

[0057] S14: Add noise to the original images in the dataset as a supplementary dataset;

[0058] Step 2: Use the AI ​​processing module to build a lightweight YOLOv7 network model;

[0059] The lightweight YOLOv7 network model includes:

[0060] Backbone network uses Ghostnet network to design YOLOv7's Backbone light weight, such as Figure 2 As shown, it is used for feature extraction;

[0061] Prediction network Head, used to classify and locate features to obtain the final prediction results;

[0062] Step 3: After data normalization, the training set is input into the lightweight YOLOv7 network model, and the AI ​​personnel detection model is obtained by training using the improved loss function;

[0063] Among them, the improved loss function is to use CIoU Loss to define the anchor box loss function. The CIoU Loss formula is as follows:

[0064]

[0065]

[0066]

[0067]

[0068]

[0069] Where ɑ is the weight function; U is used to measure the consistency of aspect ratio; ρ 2(A, B) represents the center distance between the prediction box A and the target box B; c represents the diagonal distance between the minimum union interval of A and B; IoU is equal to the intersection of A and B divided by the union of the two; arctan h ω is the inclination angle of the rectangle diagonal;

[0070] Step 4: Input the validation set into the AI ​​person detection model, obtain the optimal weight through experimental training, and test the images in the test set;

[0071] Step 5: Define the classroom detection area, and use the video acquisition module to collect the RTSP video stream of the detection area; the classroom detection area is to use Opencv to draw a rectangular area covering the classroom seats on the video, so that the AI ​​personnel detection model only outputs the recognition results within the area and counts the attendance; the video acquisition module is composed of multiple sets of optical lenses, the direction of the optical lenses can be adjusted, and the RTSP video stream of the detection area can be collected from various angles, making the collected RTSP video stream richer;

[0072] Step 6: Use the video preprocessing module to perform filter gain processing on the collected RTSP video stream to reduce the signal-to-noise ratio and image noise; and input the processed RTSP video stream into the AI ​​personnel detection model to obtain the final detection result;

[0073] Step 7: Design the host computer monitoring platform.

[0074] Among them, the specific processing steps of the video preprocessing module are:

[0075] S61: Read the RTSP video stream frame by frame as the target image, and perform denoising and enhancement on the acquired target image data through a denoising and enhancement program; specifically:

[0076] Convert the read target image information into a digital signal, filter the converted digital signal, and obtain the amplitude information of the digital signal; collect the amplitude information of the digital signal according to the preset collection interval time, and generate an amplitude information group FZi;

[0077] The reference amplitude FM is calculated based on the received i amplitude information, where i≥10; specifically:

[0078] The average value of the amplitude information group is calculated according to the average value calculation formula as FZavg;

[0079] The standard deviation of the amplitude information group is calculated according to the standard deviation calculation formula as f;

[0080] Compare the standard deviation f with the set value; if f < the set value, let the latest collected amplitude information be FZt, and use the formula FM = (FZt + FZavg) / 2 to calculate the reference amplitude FM;

[0081] If f≥the set value, the amplitude information group FZi is traversed, the maximum amplitude information is marked as FZmax, and the minimum amplitude information is marked as FZmin; the difference ratio Cb is calculated using the formula Cb=(FZmax-FZmin) / FZmax;

[0082] The reference amplitude FM is calculated using the formula FM=FZavg×(1+f×Cb×a1); where a1 is the preset equalization coefficient;

[0083] Obtain the i+1th amplitude information and mark it as FZ(i+1); compare FZ(i+1) with the reference amplitude FM; if |FZ(i+1)-FM|≥preset difference, generate an adjustment signal; if |FZ(i+1)-FM|<preset difference, generate a normal signal;

[0084] When receiving the adjustment signal, the video preprocessing module adjusts the gain of the digital signal by controlling the programmable gain amplifier circuit to adjust the amplitude of the digital signal to the reference amplitude FM; then set i=i+1, and so on;

[0085] S62: converting the target image data with color into a grayscale image; determining the pixel value of the grayscale image according to the grayscale target image data, and establishing a binary image;

[0086] S63: edge detection and matching are performed on the acquired target image data; two or more images are spliced ​​by an image mosaic program, and the acquired image data is cropped by an image cropping program; the specific process is as follows:

[0087] According to the acquired target image data, a corresponding image set is established; in the image set, a corresponding reference image is determined, and the reference image is used as a reference for outputting a mosaic image;

[0088] According to the benchmark of the output mosaic image, the contrast matching of the mosaic image, the pixel size and data type of the image are determined, and the hue of the image is processed using the histogram equalization method and the color smoothing method.

[0089] Figure 3 This is a flowchart for the implementation of the AI ​​classroom student attendance statistics method. It introduces in detail the overall architecture of the AI ​​classroom student detection model and the network training process. The feature extraction network in the algorithm includes training parameter initialization and data preprocessing, etc.; the student detection network in the algorithm mainly includes loss function design and confidence threshold selection.

[0090] The above formulas are all calculated by removing dimensions and taking numerical values. The formula is a formula that is closest to the actual situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and preset thresholds in the formula are set by technical personnel in this field according to actual conditions or obtained by simulating a large amount of data.

[0091] Working principle of the present invention:

[0092] The invention discloses an AI-based method for counting the number of students in a classroom. When working, the method comprises: constructing a classroom student data set, performing data enhancement processing on pictures in the data set, and then dividing the data set into a training set, a verification set, and a test set; using an AI processing module to build a lightweight YOLOv7 network model; inputting the training set into the lightweight YOLOv7 network model after data normalization, and obtaining an AI personnel detection model by training with an improved loss function; inputting the verification set into the AI ​​personnel detection model, obtaining an optimal weight through experimental training, and testing pictures in the test set; defining a classroom detection area, using a video acquisition module to collect an RTSP video stream about the detection area, and using a video preprocessing module to perform filtering gain processing on the collected RTSP video stream to reduce the signal-to-noise ratio and reduce image noise; and inputting the processed RTSP video stream into the AI ​​personnel detection model to obtain a final detection result. The invention greatly reduces the amount of calculation and the amount of parameters of a deep learning model, and also improves the accuracy of personnel detection, meets the real-time requirements of classroom student detection, and ensures the smooth progress of university teaching and other related work.

[0093] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0094] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for counting the number of students in a classroom based on AI, characterized in that: include: Step 1: Build a classroom student data set, perform data enhancement on the images in the data set, and then divide the data set into a training set, a validation set, and a test set; Step 2: Use the AI ​​processing module to build a lightweight YOLOv7 network model; Step 3: After data normalization, the training set is input into the lightweight YOLOv7 network model, and the AI ​​personnel detection model is obtained by training using the improved loss function; Step 4: Input the validation set into the AI ​​person detection model, obtain the optimal weight through experimental training, and test the images in the test set; Step 5: Define the classroom detection area and use the video acquisition module to collect the RTSP video stream of the detection area; Step 6: Use the video preprocessing module to perform filter gain processing on the collected RTSP video stream to reduce the signal-to-noise ratio and image noise; and input the processed RTSP video stream into the AI ​​personnel detection model to obtain the final detection result; The specific processing steps of the video preprocessing module are as follows: S61: Read the RTSP video stream frame by frame as the target image, and perform denoising and enhancement on the acquired target image data through a denoising and enhancement program; specifically: Convert the read target image information into a digital signal, filter the converted digital signal, and obtain the amplitude information of the digital signal; collect the amplitude information of the digital signal according to the preset collection interval time, and generate an amplitude information group FZi; The reference amplitude FM is calculated based on the received i amplitude information, where i≥10; specifically: The average value of the amplitude information group is calculated according to the average value calculation formula as FZavg; According to the standard deviation calculation formula, the standard deviation of the amplitude information group is calculated as ƒ; Compare the standard deviation ƒ with the set value; if ƒ < the set value, let the latest collected amplitude information be FZt, and use the formula FM = (FZt + FZavg) / 2 to calculate the reference amplitude FM; If ƒ≥ the set value, traverse the amplitude information group FZi, mark the maximum amplitude information as FZmax, and mark the minimum amplitude information as FZmin; use the formula Cb=(FZmax-FZmin) / FZmax to calculate the difference ratio Cb; The reference amplitude FM is calculated using the formula FM=FZavg×(1+ƒ×Cb×a1); where a1 is the preset equalization coefficient; Obtain the i+1th amplitude information and mark it as FZ(i+1); compare FZ(i+1) with the reference amplitude FM; if |FZ(i+1)-FM|≥preset difference, generate an adjustment signal; if |FZ(i+1)-FM|<preset difference, generate a normal signal; When receiving the adjustment signal, the video preprocessing module adjusts the gain of the digital signal by controlling the programmable gain amplifier circuit to adjust the amplitude of the digital signal to the reference amplitude FM; then set i=i+1, and so on; S62: converting the target image data with color into a grayscale image; determining the pixel value of the grayscale image according to the grayscale target image data, and establishing a binary image; S63: performing edge detection and matching on the target image data; splicing two or more images through an image mosaic program, and cropping the acquired image data through an image cropping program; Step 7: Design the host computer monitoring platform.

2. According to the AI-based classroom student attendance counting method of claim 1, it is characterized in that: The data enhancement processing specifically includes: S11: The dataset is cropped from random positions to 1 / 4 of the original image size as a supplementary dataset; S12: The original images in the dataset are flipped from 6 angles as a supplementary dataset; S13: The original images in the dataset are randomly scaled to different sizes as a supplementary dataset; S14: Add noise to the original images in the dataset as a supplementary dataset.

3. According to the AI-based classroom student attendance counting method of claim 1, it is characterized in that: The lightweight YOLOv7 network model includes: Backbone network, using Ghostnet network to design YOLOv7's lightweight Backbone for feature extraction; The prediction network Head is used to classify and locate features to obtain the final prediction results.

4. According to the AI-based classroom student attendance counting method of claim 1, it is characterized in that: in, The improved loss function is to use CIoU Loss to define the anchor box loss function. The CIoU Loss formula is as follows: ， Where, ɑ is the weight function; U is used to measure the consistency of aspect ratio; Represents the center distance between the prediction box A and the target box B; c represents the diagonal distance between the minimum union interval of A and B; IoU is equal to the intersection of A and B divided by the union of the two; is the inclination angle of the rectangle diagonal.

5. The AI-based classroom student attendance counting method according to claim 1 is characterized in that: The classroom detection area is achieved by drawing a rectangular area covering the classroom seats on the video using Opencv, so that the AI ​​personnel detection model only outputs the recognition results within the area and counts the number of attendees; the video acquisition module is composed of multiple groups of optical lenses, the direction of the optical lenses can be adjusted, and the RTSP video stream about the detection area is collected from various angles.

6. The AI-based classroom student attendance counting method according to claim 1 is characterized in that: Two or more images are stitched together through the image mosaicking program, and the acquired image data is cropped through the image cropping program. The specific process is as follows: According to the acquired target image data, a corresponding image set is established; in the image set, a corresponding reference image is determined, and the reference image is used as a reference for outputting a mosaic image; According to the benchmark of the output mosaic image, the contrast matching of the mosaic image, the pixel size and data type of the image are determined, and the hue of the image is processed using the histogram equalization method and the color smoothing method.

Images