Motion identification system for online education

Abstract

The invention relates to an action recognition system for online education, and the system comprises a real-time snapshot device which is disposed at a client of remote education and is used for carrying out the real-time snapshot processing of an environment where a remote education user is located, so as to obtain and output a corresponding snapshot processing image; and the contact notification mechanism is used for executing on-site broadcasting of notification information corresponding to small action recognition when the contact between the head and the hand is intelligently recognized based on various image processing data by adopting the multiple trained Hough neural network. According to the invention, the Hough neural network after multiple times of training can be introduced to intelligently identify whether the head and the hand are in contact based on various image processing data, and when the head and the hand are intelligently identified to be in contact, on-site broadcasting of notification information corresponding to small action identification is executed. Therefore, intelligent detection and warning of illegal small actions of remote education customers are realized.

Description

Action recognition system for online education Technical Field

[0001] This invention relates to the field of distance education, and more particularly to a motion recognition system for online education. Background Technology

[0002] The various aspects of image processing are interconnected. A practical image processing system often combines several image processing techniques to achieve the desired results. Image digitization is the first step in transforming an image into a form suitable for computer processing. Image encoding techniques are used for transmitting and storing images. Image enhancement and restoration can be the final goal of image processing or a preparation for further processing. Image features derived from image segmentation can serve as the final result or as the basis for subsequent image analysis.

[0003] Image matching, description, and recognition compare and register images. By extracting features and their relationships, a symbolic description of the image is obtained and then compared with a model to determine its classification. Image matching attempts to establish a geometric correspondence between two images, measuring their similarity or difference. Matching is used for registration between images or between images and maps, such as detecting changes in scenery between images taken at different times or finding the trajectories of moving objects.

[0004] However, image processing is a numerical processing model, and mature and reliable application solutions need to be continuously developed in various sub-fields. For example, image processing is needed to intelligently analyze whether the head and hands of remote education users are in contact, and to perform targeted recognition and warnings for small actions such as touching the head. Obviously, there is a lack of corresponding technical solutions in the existing technology, which makes it impossible to provide valuable reference data for the intelligent management of remote education. Summary of the Invention

[0005] To address the technical problems in existing technologies, this invention provides an action recognition system for online education. By introducing a Hofmeister neural network trained multiple times, it intelligently identifies whether the head and hands are in contact based on the number of noise types in a dynamically adjusted image, the maximum noise amplitude of the dynamically adjusted image, the color component values ​​of each pixel in the head within the dynamically adjusted image, and the color component values ​​of each pixel in the hand within the dynamically adjusted image. The number of training iterations of the Hofmeister neural network is proportional to the maximum noise amplitude of the dynamically adjusted image, thus ensuring the reliability and stability of the recognition results after multiple training iterations. Furthermore, a contact notification mechanism installed on the remote education client is used to broadcast the corresponding notification information when head and hand contact is intelligently detected, thereby achieving intelligent detection and warning of inappropriate small actions by remote education clients.

[0006] According to the present invention, an action recognition system for online education is provided, the system comprising:

[0007] The real-time image capture device is installed on the client side of the distance education system to perform real-time image capture processing on the environment where the distance education user is located, so as to obtain and output the corresponding captured images.

[0008] An interpolation operation device, connected to the real-time capture device, is used to perform bicubic interpolation processing on the received capture image with an 8-pixel × 8-pixel neighborhood to obtain and output the corresponding interpolated image.

[0009] A maximum value filtering device, connected to the interpolation operation device, is used to perform maximum value filtering processing on the received interpolation operation image to obtain and output the corresponding maximum value filtered image;

[0010] A dynamic adjustment device, connected to the maximum value filtering device, is used to perform dynamic range expansion processing on the received maximum value filtered image to obtain and output a corresponding dynamic adjustment image;

[0011] A continuous processing mechanism, located on the client side of the distance education system and connected to the dynamic adjustment device, includes a pixel detection unit, a quantity analysis unit, and an identifier conversion unit. These units are connected sequentially. The continuous processing mechanism is used to detect the constituent pixels of the head and hand in the dynamically adjusted image based on their respective imaging characteristics. A Hofit neural network, trained multiple times, is introduced to intelligently identify whether the head and hand are in contact based on the noise type and quantity, the maximum noise amplitude, the color component values ​​of the constituent pixels of the head, and the color component values ​​of the constituent pixels of the hand. The number of training iterations of the Hofit neural network is proportional to the maximum noise amplitude of the dynamically adjusted image.

[0012] The contact notification mechanism, connected to the continuous processing mechanism, is used to execute the on-site broadcast of notification information corresponding to small action recognition when the intelligent system detects head and hand contact, and to pause the on-site broadcast of notification information corresponding to small action recognition when the intelligent system detects no head and hand contact.

[0013] Therefore, the technical features of this invention are as follows:

[0014] First, a continuous processing mechanism including a pixel detection unit, a quantity analysis unit, and an identifier conversion unit is introduced to complete the intelligent identification of whether the head and hand are in contact. The pixel detection unit, the quantity analysis unit, and the identifier conversion unit are connected in sequence.

[0015] Secondly, a Hofit neural network, after multiple training iterations, is introduced to intelligently identify whether the head and hand are in contact based on the number of noise types in the dynamically adjusted image, the maximum noise amplitude of the dynamically adjusted image, the color component values ​​of each constituent pixel of the head in the dynamically adjusted image, and the color component values ​​of each constituent pixel of the hand in the dynamically adjusted image. The number of training iterations of the Hofit neural network is proportional to the maximum noise amplitude of the dynamically adjusted image, thereby ensuring the reliability and stability of the identification results of the Hofit neural network after multiple training iterations.

[0016] Furthermore, a contact notification mechanism is installed on the remote education client to execute on-site broadcast of corresponding notification information when the head and hand make contact, thereby realizing intelligent detection and warning of unauthorized small actions by remote education clients. Attached Figure Description

[0017] The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0018] Figure 1 is a schematic diagram of the internal structure of an action recognition system for online education according to a first embodiment of the present invention.

[0019] Figure 2 is a schematic diagram of the internal structure of a motion recognition system for online education according to a second embodiment of the present invention.

[0020] Figure 3 is a schematic diagram of the internal structure of a motion recognition system for online education according to a third embodiment of the present invention. Detailed Implementation

[0021] The embodiments of the motion recognition system for online education of the present invention will now be described in detail with reference to the accompanying drawings.

[0022] First Embodiment

[0023] Figure 1 is a schematic diagram of the internal structure of a motion recognition system for online education according to a first embodiment of the present invention. The system includes:

[0024] The real-time image capture device is installed on the client side of the distance education system to perform real-time image capture processing on the environment where the distance education user is located, so as to obtain and output the corresponding captured images.

[0025] An interpolation operation device, connected to the real-time capture device, is used to perform bicubic interpolation processing on the received capture image with an 8-pixel × 8-pixel neighborhood to obtain and output the corresponding interpolated image.

[0026] Specifically, the interpolation operation device, connected to the real-time capture device, is used to perform bicubic interpolation processing of an 8-pixel × 8-pixel neighborhood on the received capture processing image to obtain and output the corresponding interpolated operation image. This includes: optionally using a PAL chip to implement the interpolation operation device, which is connected to the real-time capture device and used to perform bicubic interpolation processing of an 8-pixel × 8-pixel neighborhood on the received capture processing image to obtain and output the corresponding interpolated operation image.

[0027] A maximum value filtering device, connected to the interpolation operation device, is used to perform maximum value filtering processing on the received interpolation operation image to obtain and output the corresponding maximum value filtered image;

[0028] A dynamic adjustment device, connected to the maximum value filtering device, is used to perform dynamic range expansion processing on the received maximum value filtered image to obtain and output a corresponding dynamic adjustment image;

[0029] A continuous processing mechanism, located on the client side of the distance education system and connected to the dynamic adjustment device, includes a pixel detection unit, a quantity analysis unit, and an identifier conversion unit. These units are connected sequentially. The continuous processing mechanism is used to detect the constituent pixels of the head and hand in the dynamically adjusted image based on their respective imaging characteristics. A Hofit neural network, trained multiple times, is introduced to intelligently identify whether the head and hand are in contact based on the noise type and quantity, the maximum noise amplitude, the color component values ​​of the constituent pixels of the head, and the color component values ​​of the constituent pixels of the hand. The number of training iterations of the Hofit neural network is proportional to the maximum noise amplitude of the dynamically adjusted image.

[0030] The contact notification mechanism is connected to the continuous processing mechanism and is used to execute the on-site broadcast of the notification information corresponding to the small action recognition when the intelligent recognition detects head and hand contact, and to pause the on-site broadcast of the notification information corresponding to the small action recognition when the intelligent recognition detects no head and hand contact.

[0031] The continuous processing mechanism is used to detect each constituent pixel of the head and each constituent pixel of the hand in the dynamically adjusted image based on the imaging characteristics of the head and hand, respectively. The Hofit neural network, which has been trained multiple times, is introduced to intelligently identify whether the head and hand are in contact based on the number of noise types in the dynamically adjusted image, the maximum noise amplitude of the dynamically adjusted image, the color component values ​​of each constituent pixel of the head in the dynamically adjusted image, and the color component values ​​of each constituent pixel of the hand in the dynamically adjusted image. The imaging characteristics of the head and hand are the contour imaging features of the head and hand.

[0032] The continuous processing mechanism is used to detect each constituent pixel of the head and each constituent pixel of the hand in the dynamically adjusted image based on the imaging characteristics of the head and hand, respectively. It then introduces a Hofit neural network that has undergone multiple training iterations to intelligently identify whether the head and hand are in contact based on the number of noise types in the dynamically adjusted image, the maximum noise amplitude of the dynamically adjusted image, the color component values ​​of each constituent pixel of the head in the dynamically adjusted image, and the color component values ​​of each constituent pixel of the hand in the dynamically adjusted image. This further includes: normalizing the number of noise types in the dynamically adjusted image, the maximum noise amplitude of the dynamically adjusted image, the color component values ​​of each constituent pixel of the head in the dynamically adjusted image, and the color component values ​​of each constituent pixel of the hand in the dynamically adjusted image before inputting them into the Hofit neural network that has undergone multiple training iterations.

[0033] Second Embodiment

[0034] Figure 2 is a schematic diagram of the internal structure of a motion recognition system for online education according to a second embodiment of the present invention.

[0035] Unlike Figure 1, the motion recognition system for online education in Figure 2 may also include the following components:

[0036] A height sensor is connected to the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism, respectively, and is used to measure the current real-time height values ​​of each of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism.

[0037] The height sensor is connected to the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism, respectively, and is used to measure the current real-time height values ​​of each of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism. The height sensor includes multiple height measurement units, each connected to the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism, to respectively measure the current real-time height values ​​of each of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism.

[0038] The height sensor includes multiple height measurement units, which are respectively connected to the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism to measure the current real-time height values ​​of each of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism. The multiple height measurement units are multiple height sensors, which are respectively connected to the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism to measure the current real-time height values ​​of each of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism.

[0039] The plurality of height measurement units are plurality of height sensors, which are respectively connected to the interpolation operation device, the maximum value filtering device, the dynamic adjustment device and the continuous processing mechanism to complete the separate measurement of the current real-time height values ​​of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device and the continuous processing mechanism, including: the plurality of height sensors have the same structure;

[0040] The plurality of height measurement units are plurality of height sensors, which are respectively connected to the interpolation operation device, the maximum value filtering device, the dynamic adjustment device and the continuous processing mechanism to complete the separate measurement of the current real-time height values ​​of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device and the continuous processing mechanism. The plurality of height sensors have the same upper limit value and lower limit value for height measurement.

[0041] Third Embodiment

[0042] Figure 3 is a schematic diagram of the internal structure of a motion recognition system for online education according to a third embodiment of the present invention.

[0043] Unlike Figure 1, the motion recognition system for online education in Figure 3 may also include the following components:

[0044] A playback processing device is connected to multiple height sensors of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism, respectively, for synchronously playing the current real-time height values ​​of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism.

[0045] The playback processing device is connected to multiple height sensors of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism, respectively, and is used to synchronously play the current real-time height values ​​of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism. The playback processing device is a character display device.

[0046] The playback processing device is connected to multiple height sensors of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism, respectively, and is used to synchronously play the current real-time height values ​​of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism. The playback processing device is a voice playback device.

[0047] Furthermore, in the action recognition system for online education, the continuous processing mechanism is used to detect each constituent pixel of the head and each constituent pixel of the hand in the dynamically adjusted image based on the imaging characteristics of the head and hand, respectively. Introducing a Hofit neural network that has undergone multiple training iterations to intelligently identify whether the head and hand are in contact based on the number of noise types in the dynamically adjusted image, the maximum noise amplitude of the dynamically adjusted image, the color component values ​​of each constituent pixel of the head in the dynamically adjusted image, and the color component values ​​of each constituent pixel of the hand in the dynamically adjusted image also includes: using the MATLAB toolbox to simulate and test the process of intelligently identifying whether the head and hand are in contact based on the number of noise types in the dynamically adjusted image, the maximum noise amplitude of the dynamically adjusted image, the color component values ​​of each constituent pixel of the head in the dynamically adjusted image, and the color component values ​​of each constituent pixel of the hand in the dynamically adjusted image.

[0048] The action recognition system for online education of this invention addresses the technical problem in the prior art that it is difficult to specifically identify small actions of remote users that affect the effectiveness of remote education. By introducing a Hofit neural network that has been trained multiple times, it can intelligently identify whether the head and hands are in contact based on various image processing data. When the intelligent identification of head and hand contact occurs, it executes the on-site broadcast of the corresponding notification information for the small action recognition, thereby realizing the intelligent detection and warning of illegal small actions of remote education customers.

[0049] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A motion recognition system for online education, characterized in that, The system includes: a real-time capture device, installed on the client side of the distance education system, for performing real-time capture processing on the environment where the distance education user is located, to obtain and output corresponding captured images; an interpolation device, connected to the real-time capture device, for performing bicubic interpolation processing on the received captured images using an 8-pixel × 8-pixel neighborhood, to obtain and output corresponding interpolated images; a maximum value filtering device, connected to the interpolation device, for performing maximum value filtering processing on the received interpolated images, to obtain and output corresponding maximum value filtered images; a dynamic adjustment device, connected to the maximum value filtering device, for performing dynamic range expansion processing on the received maximum value filtered images, to obtain and output corresponding dynamically adjusted images; and a continuous processing mechanism, installed on the client side of the distance education system and connected to the dynamic adjustment device, including a pixel detection unit, a quantity analysis unit, and an identifier conversion unit, which are connected sequentially. The continuous processing mechanism is used to detect each constituent pixel of the head and each constituent pixel of the hand in the dynamically adjusted image based on the imaging characteristics of the head and hand, respectively, and introduces a Hoffett neural network that has undergone multiple training iterations to dynamically adjust... The system intelligently identifies whether the head and hand are in contact by analyzing the number of noise types in the entire image, the maximum noise amplitude of the dynamically adjusted image, the color component values ​​of each pixel in the dynamically adjusted image (head and hand), and the color component values ​​of each pixel in the dynamically adjusted image. The number of training iterations of the Hofit neural network is proportional to the maximum noise amplitude of the dynamically adjusted image. A contact notification mechanism, connected to the continuous processing mechanism, is used to execute the on-site broadcast of the notification information corresponding to the small action recognition when the intelligent system identifies head and hand contact, and to pause the execution of the notification information corresponding to the small action recognition when the intelligent system identifies no contact between the head and hand. Live broadcast; wherein, the continuous processing mechanism is used to detect each constituent pixel of the head and each constituent pixel of the hand in the dynamically adjusted image based on the imaging characteristics of the head and hand respectively. The Hofit neural network, which has been trained multiple times, is introduced to intelligently identify whether the head and hand are in contact based on the number of noise types in the dynamically adjusted image, the maximum noise amplitude of the dynamically adjusted image, the color component values ​​of each constituent pixel of the head in the dynamically adjusted image, and the color component values ​​of each constituent pixel of the hand in the dynamically adjusted image. This includes: the imaging characteristics of the head and hand are the contour imaging features of the head and hand.

2. The action recognition system for online education as described in claim 1, characterized in that: The continuous processing mechanism is used to detect each constituent pixel of the head and each constituent pixel of the hand in the dynamically adjusted image based on the imaging characteristics of the head and hand, respectively. It introduces a Hofit neural network that has undergone multiple training iterations to intelligently identify whether the head and hand are in contact based on the number of noise types in the dynamically adjusted image, the maximum noise amplitude of the dynamically adjusted image, the color component values ​​of each constituent pixel of the head in the dynamically adjusted image, and the color component values ​​of each constituent pixel of the hand in the dynamically adjusted image. This also includes: normalizing the numbers of noise types in the dynamically adjusted image, the maximum noise amplitude of the dynamically adjusted image, the color component values ​​of each constituent pixel of the head in the dynamically adjusted image, and the color component values ​​of each constituent pixel of the hand in the dynamically adjusted image before inputting them into the Hofit neural network that has undergone multiple training iterations.

3. The action recognition system for online education as described in claim 2, characterized in that, The system further includes: a height sensor, connected to the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism respectively, for measuring the current real-time height values ​​of each of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism; wherein, the height sensor, connected to the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism respectively, for measuring the current real-time height values ​​of each of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism respectively, includes: the height sensor comprising multiple height measurement units, for connecting to the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism respectively, to complete the separate measurement of the current real-time height values ​​of each of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism.

4. The motion recognition system for online education as described in claim 3, characterized in that: The height sensor includes multiple height measurement units, which are respectively connected to the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism to separately measure the current real-time height values ​​of each of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism. The multiple height measurement units are multiple height sensors, respectively connected to the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism to separately measure the current real-time height values ​​of each of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism.

5. The action recognition system for online education as described in claim 4, characterized in that: The plurality of height measurement units are plurality of height sensors, which are respectively connected to the interpolation operation device, the maximum value filtering device, the dynamic adjustment device and the continuous processing mechanism to complete the separate measurement of the current real-time height values ​​of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device and the continuous processing mechanism, including: the plurality of height sensors have the same structure.

6. The action recognition system for online education as described in claim 5, characterized in that: The plurality of height measurement units are plurality of height sensors, which are respectively connected to the interpolation operation device, the maximum value filtering device, the dynamic adjustment device and the continuous processing mechanism to complete the separate measurement of the current real-time height values ​​of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device and the continuous processing mechanism. The plurality of height sensors have the same upper limit value and lower limit value for height measurement.

7. The action recognition system for online education as described in any one of claims 3-6, characterized in that, The system further includes a playback processing device, which is connected to multiple height sensors of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism, respectively, for synchronously playing the current real-time height values ​​of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism.

8. The action recognition system for online education as described in claim 7, characterized in that: The playback processing device is connected to multiple height sensors of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism, respectively, and is used to synchronously play the current real-time height values ​​of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism, including: the playback processing device is a character display device.

9. The action recognition system for online education as described in claim 7, characterized in that: A playback processing device is connected to multiple height sensors of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism, respectively, for synchronously playing the current real-time height values ​​of the interpolation operation device, the maximum value filtering device, the dynamic adjustment device, and the continuous processing mechanism, including: the playback processing device is a voice playback device.

Images