A classroom device control method based on image recognition

Abstract

This invention relates to the field of classroom equipment control technology, and more particularly to a classroom equipment control method based on image recognition. The method includes: acquiring images of target objects in the classroom using a camera, and labeling the images to obtain labeled images; controlling the equipment in the classroom using a target tracking algorithm and the machine learning model, and optimizing the machine learning model in real time according to actual conditions; determining whether the control stability of the classroom equipment meets requirements based on the false recognition rate of the target object; determining whether it is necessary to reduce the recognition weight of blurred areas and increase the recognition weight of clear areas; obtaining the tracking interruption frequency of the target object to determine whether the anti-interference ability of the machine learning model meets requirements; determining whether it is necessary to reduce the dynamic energy density threshold; and determining the number of verifications for consecutive image frames based on the occlusion rate of the target object's core features. This invention improves the control stability of classroom equipment.

Description

Technical Field

[0001] This invention relates to the field of classroom equipment control technology, and in particular to a classroom equipment control method based on image recognition. Background Technology

[0002] With the development of intelligent education, automated control of classroom equipment has become an important means to improve teaching efficiency and optimize the learning environment. Traditional classroom equipment control relies heavily on manual operation (such as manually switching lights, projectors, and air conditioners on and off) or timed control, which suffers from problems such as slow response, resource waste, and cumbersome operation. For example, teachers need to frequently interrupt teaching to operate the equipment, affecting the continuity of teaching; and after class, it is easy to forget to turn off the equipment, resulting in energy waste. Therefore, intelligent control technology based on image recognition has gradually become a research hotspot. It automatically identifies target objects in the teaching scene (such as teachers, students, and equipment status) through visual perception, realizing adaptive adjustment of the equipment, and has advantages such as timely response, energy saving, and reduced manual intervention.

[0003] Chinese Patent Publication No. CN110620921A discloses a method and apparatus for detecting classroom equipment faults. The method includes: detecting whether a camera in the classroom is functioning normally; if the camera is functioning normally, turning off all light-emitting devices and opening a curtain control device to control the curtains to be in a fully obscured state; sequentially turning on each light-emitting device and acquiring a set of images recorded by the camera within a first preset continuous time period before and after each light-emitting device is turned on; calculating a test curve of the brightness change in the classroom before and after each light-emitting device is turned on over time based on the acquired sets of images; and determining whether the switching function of each light-emitting device is normal based on the test curve of the brightness change in the classroom before and after each light-emitting device is turned on over time. Therefore, it is evident that the classroom equipment fault detection method and apparatus suffer from a decrease in control stability due to interference factors such as changes in light intensity during image recognition. Summary of the Invention

[0004] Therefore, the present invention provides a classroom equipment control method based on image recognition to overcome the problem in the prior art where the control stability of classroom equipment decreases due to interference factors such as changes in light during the image recognition process.

[0005] To achieve the above objectives, the present invention provides a classroom equipment control method based on image recognition, comprising:

[0006] Images of target objects in the classroom are captured using a camera, and the images are labeled to obtain labeled images. The labeled images are then used to train an initial model to obtain a machine learning model.

[0007] The target tracking algorithm and the machine learning model are used to control the equipment in the classroom, and the machine learning model is optimized in real time according to the actual situation.

[0008] The false positive rate of the target object is obtained, and the control stability of the classroom equipment is determined based on the false positive rate of the target object.

[0009] If the control stability does not meet the requirements, then determine whether it is necessary to reduce the recognition weight of the blurred areas of the image and increase the recognition weight of the clear areas;

[0010] If it is not necessary to reduce the recognition weight of blurred areas and increase the recognition weight of clear areas, then obtain the tracking interruption frequency of the target object to determine whether the anti-interference ability of the machine learning model meets the requirements;

[0011] If the anti-interference capability does not meet the requirements, determine whether it is necessary to reduce the dynamic energy density threshold.

[0012] If it is not necessary to reduce the dynamic energy density threshold, the number of verifications for consecutive frames of the image is determined based on the occlusion rate of the core features of the target object.

[0013] Furthermore, determining whether the control stability of the classroom equipment meets the requirements based on the misjudgment rate of the target object includes:

[0014] The false positive rate of the target object is compared with the preset first false positive rate;

[0015] If the misjudgment rate of the target object is less than or equal to the preset first misjudgment rate, then the control stability of the classroom equipment is determined to meet the requirements.

[0016] If the misjudgment rate of the target object is greater than the preset first misjudgment rate, then the control stability of the classroom equipment is determined to be unsatisfactory.

[0017] Further, determine whether it is necessary to reduce the recognition weight of blurred areas of the image while increasing the recognition weight of sharp areas, including:

[0018] The false positive rate of the target object is compared with the preset first false positive rate and the preset second false positive rate, respectively;

[0019] If the false positive rate of the target object is greater than the preset second false positive rate, it is determined that the recognition weight of the blurred area of ​​the image needs to be reduced, while the recognition weight of the clear area needs to be increased.

[0020] If the misjudgment rate of the target object is greater than a preset first misjudgment rate and less than or equal to a preset second misjudgment rate, then it is determined that there is no need to reduce the recognition weight of the blurred area of ​​the image, while increasing the recognition weight of the clear area.

[0021] Furthermore, the reduction in the recognition weight of the blurred region of the image is determined by the difference between the false recognition rate of the target object and the preset second false recognition rate.

[0022] Furthermore, the increase in the recognition weight of the clear region is proportional to the decrease in the recognition weight of the blurred region of the image.

[0023] Furthermore, the robustness of the machine learning model is determined based on the frequency of tracking interruptions of the target object, including:

[0024] Compare the target object tracking interruption frequency with the preset first interruption frequency;

[0025] If the tracking interruption frequency of the target object is less than or equal to the preset first interruption frequency, then it is determined that the anti-interference ability of the machine learning model meets the requirements, and it is determined whether the recognition weight of the blurred area and the recognition weight of the clear area of ​​the image meet the requirements.

[0026] If the tracking interruption frequency of the target object is greater than the preset first interruption frequency, then the anti-interference capability of the machine learning model is determined to be unacceptable.

[0027] Further, determine whether it is necessary to reduce the dynamic energy density threshold, including:

[0028] The tracking interruption frequency of the target object is compared with the preset first interruption frequency and the preset second interruption frequency, respectively.

[0029] If the tracking interruption frequency of the target object is greater than the preset first interruption frequency and less than or equal to the preset second interruption frequency, then it is determined that the dynamic energy density threshold needs to be reduced, and the dynamic energy density threshold is reduced accordingly.

[0030] If the tracking interruption frequency of the target object is greater than the preset second interruption frequency, then it is determined that there is no need to reduce the dynamic energy density threshold.

[0031] Furthermore, the reduction in the dynamic energy density threshold is determined by the difference between the tracking interruption frequency of the target object and the preset first interruption frequency.

[0032] Furthermore, the number of verifications for consecutive image frames is determined based on the occlusion rate of the target object's core features, including:

[0033] Compare the occlusion rate of the core features of the target object with the preset occlusion rate;

[0034] If the occlusion rate of the core features of the target object is less than or equal to the preset occlusion rate, then the occlusion robustness of the target tracking algorithm is determined to meet the requirements, and it is not necessary to increase the number of verifications for consecutive frames of the image. It is also determined whether the dynamic energy density threshold meets the requirements.

[0035] If the occlusion rate of the core features of the target object is greater than the preset occlusion rate, then it is determined that the number of verifications for consecutive image frames needs to be increased, and the number of verifications for consecutive image frames needs to be increased.

[0036] Furthermore, the increase in the number of verifications for consecutive frames of the image is determined by the difference between the core feature occlusion rate of the target object and the preset occlusion rate.

[0037] Compared with existing technologies, the beneficial effects of this invention are as follows: The method of this invention adjusts the recognition weights of blurred and sharp regions of an image based on the false recognition rate of the target object. Since chalk dust floating on the surface of the camera can partially cover the image, it may cause local blurring, leading to algorithmic misjudgments. By reducing the recognition weight of blurred regions and increasing the recognition weight of sharp regions, the interference of chalk dust can be offset as much as possible, making the device control logic more closely match the actual usage conditions of a classroom. The dynamic energy density threshold is adjusted based on the tracking interruption frequency of the target object. Since high-speed instantaneous movements may occur in a classroom, causing moving targets to appear between frames... Motion trails or jumps can cause algorithms to misjudge actions as noise rather than valid actions. By reducing the dynamic energy density threshold, actions that were originally misjudged as noise can be included in the valid judgment range, improving the detection rate. The number of verifications for consecutive frames of the image is adjusted according to the occlusion rate of the target object's core features. Since people in the classroom may briefly cover the core feature area of ​​the target, making it impossible to accurately identify features for control, increasing the number of verifications for consecutive frames can extend the verification time window, filter out recognition fluctuations caused by brief occlusions, and ensure that the system can re-identify based on complete features after the occlusions disappear, thereby maintaining the accuracy of control and improving the control stability of the classroom equipment.

[0038] Furthermore, the method of the present invention adjusts the recognition weights of blurred areas and clear areas of an image by setting a preset first misjudgment rate and a preset second misjudgment rate. Since chalk dust floats on the surface of the camera and covers some areas, it may cause local blurring of the image, which may lead to algorithm misjudgment. By reducing the recognition weight of blurred areas and increasing the recognition weight of clear areas, the interference of chalk dust can be offset as much as possible, making the device control logic more in line with the actual use of the classroom, and further improving the control stability of the classroom equipment.

[0039] Furthermore, the method of the present invention adjusts the dynamic energy density threshold by setting a preset first interrupt frequency and a preset second interrupt frequency. Since there may be high-speed instantaneous movements in the classroom, the moving target may appear as a blur or jump between frames, which may cause the algorithm to misjudge it as noise rather than effective movement. By reducing the dynamic energy density threshold, the movements that were originally misjudged as noise can be included in the effective judgment range, thereby improving the detection rate and further improving the control stability of the classroom equipment.

[0040] Furthermore, the method of the present invention adjusts the number of verifications for consecutive frames of an image by setting a preset occlusion rate. Since people in the classroom may briefly cover the core feature area of ​​the target, making it impossible to accurately identify the features and thus control the system, increasing the number of verifications for consecutive frames can extend the verification time window, filter out the recognition fluctuations caused by brief occlusions, and ensure that the system can re-identify based on complete features after the occlusions disappear, thereby maintaining the accuracy of control and further improving the control stability of the classroom equipment. Attached Figure Description

[0041] Figure 1 This is a flowchart of a classroom equipment control method based on image recognition, according to an embodiment of the present invention.

[0042] Figure 2 This is a flowchart illustrating the process of determining whether it is necessary to reduce the recognition weight of blurred areas and increase the recognition weight of clear areas in the classroom equipment control method based on image recognition according to an embodiment of the present invention.

[0043] Figure 3 This is a flowchart illustrating the process of determining whether the dynamic energy density threshold needs to be reduced in the classroom equipment control method based on image recognition according to an embodiment of the present invention.

[0044] Figure 4 This is a flowchart illustrating the process of determining the number of verifications for consecutive image frames in the classroom equipment control method based on image recognition, as described in an embodiment of the present invention. Detailed Implementation

[0045] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0046] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0047] Please see Figure 1 , Figure 2 , Figure 3 as well as Figure 4 The flowcharts shown are respectively the classroom equipment control method based on image recognition according to the embodiments of the present invention, the process of determining whether it is necessary to reduce the recognition weight of the blurred area of ​​the image while increasing the recognition weight of the clear area, the process of determining whether it is necessary to reduce the dynamic energy density threshold, and the process of determining the number of verifications for consecutive frames of the image.

[0048] The classroom equipment control method based on image recognition according to embodiments of the present invention includes:

[0049] Step S1: Use a camera to capture images of target objects in the classroom, label the images to obtain labeled images, and use the labeled images to train the initial model to obtain a machine learning model;

[0050] Step S2: Use the target tracking algorithm and the machine learning model to control the equipment in the classroom, and optimize the machine learning model in real time according to the actual situation;

[0051] Step S3: Obtain the misjudgment rate of the target object, and determine whether the control stability of the classroom equipment meets the requirements based on the misjudgment rate of the target object;

[0052] Step S4: If the control stability does not meet the requirements, determine whether it is necessary to reduce the recognition weight of the blurred area and increase the recognition weight of the clear area.

[0053] Step S5: If it is not necessary to reduce the recognition weight of blurred areas and increase the recognition weight of clear areas, then obtain the tracking interruption frequency of the target object to determine whether the anti-interference ability of the machine learning model meets the requirements.

[0054] Step S6: If the anti-interference performance does not meet the requirements, determine whether it is necessary to reduce the dynamic energy density threshold.

[0055] Step S7: If it is not necessary to reduce the dynamic energy density threshold, then determine the number of verifications for consecutive frames of the image based on the occlusion rate of the core features of the target object.

[0056] Specifically, the images of the target objects include images of people in the classroom, images of lighting equipment, and images of projection equipment.

[0057] Specifically, labeling images includes labeling people in the image as teachers or students, labeling devices with their corresponding names, and labeling the location of the devices.

[0058] Specifically, machine learning models can be object detection models, semantic segmentation models, and behavior recognition models.

[0059] Specifically, target tracking algorithms can be SORT algorithm, KCF algorithm, or ByteTrack algorithm.

[0060] Specifically, the equipment in the classroom includes lighting fixtures, projectors, and air conditioning.

[0061] Specifically, the robustness of a machine learning model refers to its ability to maintain stable recognition accuracy and decision reliability in the complex and dynamic environment of a classroom, when faced with various interfering factors (such as changes in lighting, occlusion, background noise, and fluctuations in the state of the equipment itself).

[0062] Specifically, the dynamic energy density threshold is a key parameter used for image analysis and processing. It is a threshold that is dynamically adjusted according to the local or overall energy distribution of the image and is mainly used to distinguish different regions or target objects from the background in the image.

[0063] Specifically, the number of verifications for consecutive image frames is determined by repeatedly identifying and verifying multiple consecutive image frames. Only when the identification results that meet the preset conditions (such as the presence of the target, state changes, action commands, etc.) remain consistent in N consecutive image frames is the validity of the result confirmed and the device control command is triggered, where N is the number of verifications.

[0064] In implementation, the method of this invention adjusts the recognition weights of blurred and sharp regions of the image based on the false recognition rate of the target object. Since chalk dust floating on the camera surface can partially cover the image, it may cause local blurring, leading to algorithmic misjudgments. By reducing the recognition weight of blurred regions and increasing the recognition weight of sharp regions, the interference of chalk dust can be offset as much as possible, making the device control logic more closely match the actual usage conditions of a classroom. The dynamic energy density threshold is adjusted based on the tracking interruption frequency of the target object. Because high-speed instantaneous movements may occur in a classroom, causing moving targets to exhibit ghosting or jumps between frames, this leads to… The algorithm is prone to misjudging actions as noise rather than valid actions. By reducing the dynamic energy density threshold, actions that were originally misjudged as noise can be included in the valid judgment range, thus improving the detection rate. The number of verifications for consecutive frames of the image is adjusted according to the occlusion rate of the core features of the target object. Since people in the classroom may briefly cover the core feature area of ​​the target, making it impossible to accurately identify features and thus control the system, increasing the number of verifications for consecutive frames can extend the verification time window, filter out the recognition fluctuations caused by brief occlusions, and ensure that the system can re-identify based on complete features after the occlusions disappear, thereby maintaining the accuracy of control and improving the control stability of the classroom equipment.

[0065] Specifically, determining whether the control stability of the classroom equipment meets the requirements based on the misjudgment rate of the target object includes:

[0066] The false positive rate of the target object is compared with the preset first false positive rate;

[0067] If the misjudgment rate of the target object is less than or equal to the preset first misjudgment rate, then the control stability of the classroom equipment is determined to meet the requirements.

[0068] If the misjudgment rate of the target object is greater than the preset first misjudgment rate, then the control stability of the classroom equipment is determined to be unsatisfactory.

[0069] The reasons why the control stability of the classroom equipment does not meet the requirements may be that the anti-interference ability of the machine learning model is not up to standard, or that the recognition weights of blurred areas and sharp areas of the image are not up to standard. The next step is to determine which specific cause it is, which is also the process of determining whether to reduce the recognition weight of blurred areas and increase the recognition weight of sharp areas.

[0070] Specifically, determining whether it is necessary to reduce the recognition weight of blurred areas of the image while increasing the recognition weight of sharp areas includes:

[0071] The false positive rate of the target object is compared with the preset first false positive rate and the preset second false positive rate, respectively;

[0072] If the false positive rate of the target object is greater than the preset second false positive rate, it is determined that the recognition weight of the blurred area of ​​the image needs to be reduced, while the recognition weight of the clear area needs to be increased.

[0073] If the misjudgment rate of the target object is greater than a preset first misjudgment rate and less than or equal to a preset second misjudgment rate, then it is determined that there is no need to reduce the recognition weight of the blurred area of ​​the image, while increasing the recognition weight of the clear area.

[0074] Specifically, when the false positive rate of the target object exceeds a preset second false positive rate, it is determined that the reason for the classroom equipment's control stability failure is that the recognition weights of blurred and sharp image regions are not up to standard. Therefore, it is necessary to reduce the recognition weights of blurred regions and increase the recognition weights of sharp regions. When the false positive rate of the target object is greater than a preset first false positive rate but less than or equal to a preset second false positive rate, it can be preliminarily determined that the machine learning model's anti-interference capability is not up to standard. The next step is to determine the final compatibility of the machine learning model's anti-interference capability based on the target object's tracking interruption frequency, i.e., to determine whether the reason for the classroom equipment's control stability failure is due to the machine learning model's inadequate anti-interference capability.

[0075] Understandably, the preset first false positive rate is lower than the preset second false positive rate. The three intervals divided by the preset first and second false positive rates correspond to three different scenarios:

[0076] The first interval is when the false positive rate of the target object is less than or equal to the preset first false positive rate. The corresponding situation is: the control stability of the classroom equipment meets the requirements, and no adjustment is needed.

[0077] The second interval is when the misjudgment rate of the target object is greater than the preset first misjudgment rate and less than or equal to the preset second misjudgment rate. The corresponding situation is: because there may be high-speed instantaneous actions in the classroom, the moving target may appear as a blur or jump between frames, which may cause the algorithm to misjudge it as noise rather than effective action. At this time, it is necessary to further judge whether the anti-interference ability of the machine learning model meets the requirements.

[0078] The third interval is when the false recognition rate of the target object is greater than the preset second false recognition rate. The corresponding situation is: because chalk dust floats on the surface of the camera and causes some areas to be covered, it may cause local blurring of the image, which may lead to the algorithm's misjudgment. At this time, it is necessary to adjust the recognition weight of the blurred area and the recognition weight of the clear area of ​​the image.

[0079] Understandably, in image recognition-based classroom equipment control methods, the first and second misjudgment rates are used to characterize control stability. The core principle is to achieve a refined description of the stability state through tiered thresholds, clearly defining the acceptable standard for stability and distinguishing the degree of intervention required when unstable, thus avoiding the limitations of a single threshold. The first misjudgment rate is the passing grade for control stability, its core function being to determine whether the system is in an acceptable stable state. The second misjudgment rate is the critical value requiring active optimization to restore stability, its core function being to distinguish between slight and severe instability, avoiding excessive or insufficient intervention. The preset first and second misjudgment rates can be set according to actual operating conditions. The setting of the preset first and second misjudgment rates aims to improve the control stability and practicality of the classroom equipment. Optionally, the preset first and second misjudgment rates are determined through a limited number of experiments by evaluating the control effect of different judgment accuracies on the classroom equipment. The determined preset first and second misjudgment rates should be neither too low nor cause excessive interference to the control process of the classroom equipment. For example, the preset first false positive rate is generally selected in the range of [3%, 7%], and the preset second false positive rate is generally selected in the range of [8%, 12%].

[0080] Preferably, the first false positive rate is 5% in the preferred embodiment, and the second false positive rate is 10% in the preferred embodiment.

[0081] Specifically, the misjudgment rate of the target object is the ratio of the number of incorrect judgments in a series of judgments to the total number of judgments.

[0082] In practice, the method of the present invention determines the control stability of classroom equipment by setting a preset first misjudgment rate and a preset second misjudgment rate, thereby reducing the impact of the decrease in control accuracy of classroom equipment due to inaccurate determination of control stability and further improving the control stability of classroom equipment.

[0083] Specifically, the reduction in the recognition weight of the blurred region of the image is determined by the difference between the false recognition rate of the target object and the preset second false recognition rate.

[0084] Specifically, when the difference between the false positive rate of the target object and the preset second false positive rate is within 3%, the recognition weight of the blurred image region is reduced to 0.9 times the original value. When the difference between the false positive rate of the target object and the preset second false positive rate exceeds 3%, the recognition weight of the blurred image region is reduced by 0.05 for every 1% increase beyond the original value of 0.9 times the original value. For example, if the difference between the false positive rate of the target object and the preset second false positive rate is 5%, the current recognition weight of the blurred image region is 0.8, and the reduced recognition weight of the blurred image region is 0.8×0.9-0.05×2=0.62.

[0085] In practice, the method of the present invention adjusts the recognition weight of blurred areas and clear areas of the image by setting a preset first misjudgment rate and a preset second misjudgment rate. Since chalk dust floats on the surface of the camera and causes some areas to be covered, it may cause local blurring of the image, which may lead to algorithm misjudgment. By reducing the recognition weight of blurred areas and increasing the recognition weight of clear areas, the interference of chalk dust can be offset as much as possible, making the device control logic more in line with the actual use of the classroom, and further improving the control stability of the classroom equipment.

[0086] Specifically, the increase in the recognition weight of the clear region is proportional to the decrease in the recognition weight of the blurred region of the image.

[0087] Specifically, the ratio of the increase in the recognition weight of the clear region to the decrease in the recognition weight of the blurred region is 1:1. That is, if the recognition weight of the blurred region decreases by 0.1, the recognition weight of the clear region increases by 0.1.

[0088] Specifically, determining whether the machine learning model's robustness meets requirements based on the frequency of tracking interruptions of the target object includes:

[0089] Compare the target object tracking interruption frequency with the preset first interruption frequency;

[0090] If the tracking interruption frequency of the target object is less than or equal to the preset first interruption frequency, then it is determined that the anti-interference ability of the machine learning model meets the requirements, and it is determined whether the recognition weight of the blurred area and the recognition weight of the clear area of ​​the image meet the requirements.

[0091] If the tracking interruption frequency of the target object is greater than the preset first interruption frequency, then the anti-interference capability of the machine learning model is determined to be unacceptable.

[0092] When the tracking interruption frequency of the target object is less than or equal to the preset first interruption frequency, it is determined that the anti-interference ability of the machine learning model meets the requirements. However, it has been previously determined that the control stability of the classroom equipment does not meet the requirements. Therefore, it is necessary to further determine whether the recognition weights of the blurred areas and the recognition weights of the clear areas of the image meet the requirements.

[0093] In implementation, the recognition weights of blurred and clear regions of an image are compared with a predetermined blur weight threshold and a predetermined clarity weight threshold to determine whether they meet the requirements. If the actual recognition weight of a blurred region is greater than the predetermined blur weight threshold, or the recognition weight of a clear region is less than the predetermined clarity weight threshold, then the recognition weights of blurred and clear regions of the image are determined to be unacceptable. The predetermined blur weight threshold and clarity weight threshold are the average values ​​of the recognition weights of blurred and clear regions of the image monitored in the previous three months.

[0094] If the recognition weights of blurred and clear regions of an image do not meet the requirements, the recognition weights of blurred regions are reduced while the recognition weights of clear regions are increased. If the recognition weights of blurred and clear regions meet the requirements, the false recognition rate of the target object is re-acquired, and the control stability of the classroom equipment is re-evaluated.

[0095] When the tracking interruption frequency of the target object exceeds the preset first interruption frequency, it can be determined that the reason for the unsatisfactory control stability of the classroom equipment is that the anti-interference capability of the machine learning model is not up to standard. The reasons for the unsatisfactory anti-interference capability of the machine learning model may be that the dynamic energy density threshold is not up to standard, or that the occlusion robustness of the target tracking algorithm is not up to standard. The next step is to determine which specific cause it is, which is also the process of determining whether the dynamic energy density threshold needs to be reduced.

[0096] Specifically, determining whether the dynamic energy density threshold needs to be reduced includes:

[0097] The tracking interruption frequency of the target object is compared with the preset first interruption frequency and the preset second interruption frequency, respectively.

[0098] If the tracking interruption frequency of the target object is greater than the preset first interruption frequency and less than or equal to the preset second interruption frequency, then it is determined that the dynamic energy density threshold needs to be reduced, and the dynamic energy density threshold is reduced accordingly.

[0099] If the tracking interruption frequency of the target object is greater than the preset second interruption frequency, then it is determined that there is no need to reduce the dynamic energy density threshold.

[0100] Specifically, when the tracking interruption frequency of the target object is greater than a preset first interruption frequency but less than or equal to a preset second interruption frequency, it is determined that the reason for the machine learning model's failure to meet the anti-interference requirements is that the dynamic energy density threshold does not meet the requirements, and therefore the dynamic energy density threshold needs to be reduced. When the tracking interruption frequency of the target object is greater than the preset second interruption frequency, it can be preliminarily determined that the occlusion robustness of the target tracking algorithm does not meet the requirements. Next, it is necessary to make a final determination on whether the occlusion robustness of the target tracking algorithm meets the requirements based on the occlusion rate of the core features of the target object, that is, to determine whether the reason for the machine learning model's failure to meet the anti-interference requirements is that the occlusion robustness of the target tracking algorithm does not meet the requirements.

[0101] It is understandable that the preset first interrupt frequency is less than the preset second interrupt frequency, and the three intervals divided by the preset first interrupt frequency and the preset second interrupt frequency correspond to three different situations:

[0102] The first interval is when the tracking interruption frequency of the target object is less than or equal to the preset first interruption frequency. The corresponding situation is: the anti-interference ability of the machine learning model meets the requirements. At this time, it is necessary to further determine whether the recognition weight of the blurred area and the recognition weight of the clear area of ​​the image meet the requirements.

[0103] The second interval is when the tracking interruption frequency of the target object is greater than the preset first interruption frequency and less than or equal to the preset second interruption frequency. The corresponding situation is: because there may be high-speed instantaneous movements in the classroom, the moving target will have a trailing or jump between frames, which will cause the algorithm to easily misjudge it as noise rather than effective movement. At this time, it is necessary to adjust the dynamic energy density threshold.

[0104] The third interval is when the tracking interruption frequency of the target object is greater than the preset second interruption frequency. The corresponding situation is: because people in the classroom may briefly cover the core feature area of ​​the target, it is impossible to accurately identify the features and thus control the target. In this case, it is necessary to further determine whether the occlusion robustness of the target tracking algorithm meets the requirements.

[0105] Understandably, in image recognition-based classroom equipment control methods, using preset first and second interruption frequencies as thresholds to characterize the anti-interference ability of the machine learning model is fundamentally about achieving refined evaluation and targeted adjustment of the model's anti-interference capability through hierarchical quantization. The first interruption frequency is the basic threshold for model anti-interference, used to determine whether the model meets the basic requirements of classroom equipment control. The second interruption frequency is a higher threshold than the first, used to determine whether the tracking interruption caused by interference is severe enough to require urgent optimization of core parameters. The preset first and second interruption frequencies can be set according to actual operating conditions. The setting of the preset first and second interruption frequencies aims to ensure the control stability and practicality of the classroom equipment. Optionally, the preset first and second interruption frequencies are determined through a limited number of experiments by evaluating the control effect of different interruption conditions on the classroom equipment. The determined preset first and second interruption frequencies should be neither too low nor cause excessive interference to the control process of the classroom equipment. For example, the preset first interruption frequency is generally selected in the range of [1 time / hour, 3 times / hour], and the preset second interruption frequency is generally selected in the range of [4 times / hour, 6 times / hour].

[0106] Preferably, the preferred embodiment of the preset first interruption frequency is 2 times / hour, and the preferred embodiment of the preset second interruption frequency is 5 times / hour.

[0107] In practice, the method of the present invention determines the anti-interference capability of the machine learning model by setting a preset first interruption frequency and a preset second interruption frequency, thereby reducing the impact of the decrease in control stability of classroom equipment due to inaccurate determination of the anti-interference capability of the machine learning model, and further improving the control stability of classroom equipment.

[0108] Specifically, the reduction in the dynamic energy density threshold is determined by the difference between the tracking interruption frequency of the target object and a preset first interruption frequency.

[0109] Specifically, when the difference between the tracking interruption frequency of the target object and the preset first interruption frequency is within 2 times / hour, the dynamic energy density threshold is reduced to 0.9 times the original value. When the difference between the tracking interruption frequency of the target object and the preset first interruption frequency exceeds 2 times / hour, in addition to reducing it to 0.9 times the original value, the dynamic energy density threshold is reduced by 2 pixels / frame for every 1 time / hour exceeding the original value. For example, if the difference between the tracking interruption frequency of the target object and the preset first interruption frequency is 4 times / hour, and the current dynamic energy density threshold is 50 pixels / frame, the reduced dynamic energy density threshold is 50×0.9-2×2=41 pixels / frame.

[0110] In practice, the method of the present invention adjusts the dynamic energy density threshold by setting a preset first interrupt frequency and a preset second interrupt frequency. Since there may be high-speed instantaneous movements in the classroom, the moving target may appear as a blur or jump between frames, which may cause the algorithm to misjudge it as noise rather than effective movement. By reducing the dynamic energy density threshold, the movements that were originally misjudged as noise can be included in the effective judgment range, thereby improving the detection rate and further improving the control stability of the classroom equipment.

[0111] Specifically, the number of verifications for consecutive image frames is determined based on the occlusion rate of the target object's core features, including:

[0112] Compare the occlusion rate of the core features of the target object with the preset occlusion rate;

[0113] If the occlusion rate of the core features of the target object is less than or equal to the preset occlusion rate, then the occlusion robustness of the target tracking algorithm is determined to meet the requirements, and it is not necessary to increase the number of verifications for consecutive frames of the image. It is also determined whether the dynamic energy density threshold meets the requirements.

[0114] If the occlusion rate of the core features of the target object is greater than the preset occlusion rate, then it is determined that the number of verifications for consecutive image frames needs to be increased, and the number of verifications for consecutive image frames needs to be increased.

[0115] Specifically, when the occlusion rate of the core features of the target object is less than or equal to the preset occlusion rate, it is determined that the occlusion robustness of the target tracking algorithm meets the requirements. However, if the anti-interference ability of the machine learning model has been determined to be unacceptable, it is necessary to further determine whether the dynamic energy density threshold meets the requirements.

[0116] In practice, the dynamic energy density threshold is determined to meet the requirements based on the comparison between the actual dynamic energy density threshold and the predetermined threshold. If the actual dynamic energy density threshold is greater than the predetermined threshold, the dynamic energy density threshold is determined to not meet the requirements. The predetermined threshold is the average value of the dynamic energy density threshold monitored in the previous three months of the historical period.

[0117] If the dynamic energy density threshold does not meet the requirements, the dynamic energy density threshold is reduced; if the dynamic energy density threshold meets the requirements, the tracking interruption frequency of the target object is re-acquired, and the anti-interference ability of the machine learning model is re-evaluated.

[0118] When the occlusion rate of the core features of the target object is greater than the preset occlusion rate, it can be determined that the reason why the anti-interference ability of the machine learning model does not meet the requirements is that the occlusion robustness of the target tracking algorithm does not meet the requirements. Therefore, it is necessary to increase the number of verifications for consecutive frames of the image.

[0119] It is understandable that the two intervals of the preset occlusion rate division correspond to two different situations:

[0120] The first interval is when the occlusion rate of the core features of the target object is less than or equal to the preset occlusion rate. The corresponding situation is: the occlusion robustness of the target tracking algorithm meets the requirements. At this time, it is necessary to further determine whether the dynamic energy density threshold meets the requirements.

[0121] The second interval is when the occlusion rate of the core features of the target object is greater than the preset occlusion rate. The corresponding situation is that people in the classroom may briefly cover the core feature area of ​​the target, making it impossible to accurately identify the features and thus control the target. In this case, it is necessary to adjust the number of verifications for consecutive frames of the image.

[0122] Understandably, the core logic of target tracking is to extract the target's core features (such as the teacher's limb contours, the student's head area, key components of the equipment, etc.) for matching and localization. When the target is occluded, the core features will be partially or completely obscured. The occlusion rate is essentially a quantitative indicator of the degree to which occlusion damages the core features, directly reflecting the severity of occlusion interference, which is highly compatible with the anti-interference requirements of tracking algorithms. The core feature occlusion rate is directly related to the algorithm's dependence on feature integrity and is the most intuitive quantitative indicator for measuring its stability in occluded scenarios. The preset occlusion rate can be set according to actual working conditions. The setting of the preset occlusion rate aims to determine the control stability and practicality of the classroom equipment. Optionally, the preset occlusion rate is determined through a limited number of trials by evaluating the control effect of the classroom equipment under non-occluded conditions. The determined preset occlusion rate should be neither too small nor cause excessive interference to the control process of the classroom equipment. For example, the preset occlusion rate is generally selected in the range of [25%, 35%].

[0123] Preferably, the preset occlusion rate is 30% in the preferred embodiment.

[0124] Specifically, the occlusion robustness of a target tracking algorithm refers to its ability to maintain continuous tracking of the target and avoid tracking interruption or loss when the target object (such as a teacher, student, or key component of equipment) is partially or completely occluded.

[0125] In practice, the method of the present invention determines the occlusion robustness of the target tracking algorithm by setting a preset occlusion rate, thereby reducing the impact of inaccurate determination of the occlusion robustness of the target tracking algorithm on the control stability of the classroom equipment and further improving the control stability of the classroom equipment.

[0126] Specifically, the increase in the number of verifications for consecutive frames of the image is determined by the difference between the core feature occlusion rate of the target object and the preset occlusion rate.

[0127] Specifically, when the difference between the core feature occlusion rate of the target object and the preset occlusion rate is within 5%, the number of verifications for consecutive image frames increases to 1.2 times the original value. When the difference between the core feature occlusion rate of the target object and the preset occlusion rate exceeds 5%, in addition to increasing to 1.2 times the original value, the number of verifications for consecutive image frames increases by 1 for every 2% exceeding the original value. For example, if the difference between the core feature occlusion rate of the target object and the preset occlusion rate is 9%, the current number of verifications for consecutive image frames is 5, and the increased number of verifications for consecutive image frames is 5 × 1.2 + 1 × 2 = 8.

[0128] In practice, the method of the present invention adjusts the number of verifications for consecutive frames of an image by setting a preset occlusion rate. Since people in the classroom may briefly cover the core feature area of ​​the target, making it impossible to accurately identify the features and thus control the system, increasing the number of verifications for consecutive frames can extend the verification time window, filter out the recognition fluctuations caused by brief occlusions, and ensure that the system can re-identify based on complete features after the occlusions disappear, thereby maintaining the accuracy of control and further improving the control stability of the classroom equipment.

[0129] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. An image recognition-based classroom device control method, characterized by, include: Images of target objects in the classroom are captured using a camera, and the images are labeled to obtain labeled images. The labeled images are then used to train an initial model to obtain a machine learning model. The target tracking algorithm and the machine learning model are used to control the equipment in the classroom, and the machine learning model is optimized in real time according to the actual situation. The false positive rate of the target object is obtained, and the control stability of the classroom equipment is determined based on the false positive rate of the target object. If the control stability does not meet the requirements, then determine whether it is necessary to reduce the recognition weight of the blurred areas of the image and increase the recognition weight of the clear areas; If it is not necessary to reduce the recognition weight of blurred areas and increase the recognition weight of clear areas, then obtain the tracking interruption frequency of the target object to determine whether the anti-interference ability of the machine learning model meets the requirements; If the anti-interference capability does not meet the requirements, determine whether it is necessary to reduce the dynamic energy density threshold. If it is not necessary to reduce the dynamic energy density threshold, the number of verifications for consecutive frames of the image is determined based on the occlusion rate of the core features of the target object.

2. The image recognition-based classroom equipment control method according to claim 1, characterized by, Determining whether the control stability of the classroom equipment meets the requirements based on the misjudgment rate of the target object includes: The false positive rate of the target object is compared with the preset first false positive rate; If the misjudgment rate of the target object is less than or equal to the preset first misjudgment rate, then the control stability of the classroom equipment is determined to meet the requirements. If the misjudgment rate of the target object is greater than the preset first misjudgment rate, then the control stability of the classroom equipment is determined to be unsatisfactory.

3. The image recognition-based classroom equipment control method according to claim 2, characterized by, Determine whether it is necessary to reduce the recognition weight of blurred areas of the image while increasing the recognition weight of sharp areas, including: The false positive rate of the target object is compared with the preset first false positive rate and the preset second false positive rate, respectively; If the false positive rate of the target object is greater than the preset second false positive rate, it is determined that the recognition weight of the blurred area of ​​the image needs to be reduced, while the recognition weight of the clear area needs to be increased. If the misjudgment rate of the target object is greater than a preset first misjudgment rate and less than or equal to a preset second misjudgment rate, then it is determined that there is no need to reduce the recognition weight of the blurred area of ​​the image, while increasing the recognition weight of the clear area.

4. The image recognition-based classroom equipment control method according to claim 3, characterized by, The reduction in the recognition weight of the blurred region of the image is determined by the difference between the false positive rate of the target object and the preset second false positive rate.

5. The image recognition-based classroom equipment control method according to claim 4, characterized by, The increase in the recognition weight of the clear region is proportional to the decrease in the recognition weight of the blurred region of the image.

6. The image recognition-based classroom equipment control method according to claim 3, characterized by, Determining whether the anti-interference capability of the machine learning model meets the requirements based on the tracking interruption frequency of the target object includes: Compare the target object tracking interruption frequency with the preset first interruption frequency; If the tracking interruption frequency of the target object is less than or equal to the preset first interruption frequency, then it is determined that the anti-interference ability of the machine learning model meets the requirements, and it is determined whether the recognition weight of the blurred area and the recognition weight of the clear area of ​​the image meet the requirements. If the tracking interruption frequency of the target object is greater than the preset first interruption frequency, then the anti-interference capability of the machine learning model is determined to be unacceptable.

7. The image recognition-based classroom equipment control method according to claim 6, characterized by, Determining whether the dynamic energy density threshold needs to be reduced includes: The tracking interruption frequency of the target object is compared with the preset first interruption frequency and the preset second interruption frequency, respectively. If the tracking interruption frequency of the target object is greater than the preset first interruption frequency and less than or equal to the preset second interruption frequency, then it is determined that the dynamic energy density threshold needs to be reduced, and the dynamic energy density threshold is reduced accordingly. If the tracking interruption frequency of the target object is greater than the preset second interruption frequency, then it is determined that there is no need to reduce the dynamic energy density threshold.

8. The image recognition-based classroom equipment control method according to claim 7, characterized by, The reduction in the dynamic energy density threshold is determined by the difference between the tracking interruption frequency of the target object and the preset first interruption frequency.

9. The image recognition-based classroom equipment control method according to claim 8, characterized by, The number of verifications for consecutive image frames is determined based on the occlusion rate of the target object's core features, including: Compare the occlusion rate of the core features of the target object with the preset occlusion rate; If the occlusion rate of the core features of the target object is less than or equal to the preset occlusion rate, then the occlusion robustness of the target tracking algorithm is determined to meet the requirements, and it is not necessary to increase the number of verifications for consecutive frames of the image. It is also determined whether the dynamic energy density threshold meets the requirements. If the occlusion rate of the core features of the target object is greater than the preset occlusion rate, then it is determined that the number of verifications for consecutive image frames needs to be increased, and the number of verifications for consecutive image frames needs to be increased.

10. The image recognition-based classroom device control method according to claim 9, characterized by, The increase in the number of verifications for consecutive frames of the image is determined by the difference between the core feature occlusion rate of the target object and the preset occlusion rate.

Images