A real-time detection system and method for reading attention of user groups based on library shared space

Abstract

The present invention discloses a real-time detection system and method for reading attention of a user group in a shared space of a library. The detection system is arranged in the shared space, and comprises a stripe projection amplification unit, an image acquisition unit and a computer processing system. The stripe projection amplification unit projects a stripe light beam toward the user group in the shared space. The image acquisition unit acquires a stripe modulation image of the user group containing the stripe light beam in the shared space, and simultaneously inputs the acquired stripe modulation image into the computer processing system. The computer processing system is used to control the stripe projection amplification unit, and the computer processing system processes the acquired stripe modulation image to extract the number N of motion regions, the average motion amplitude and the average shape change speed. The number N of motion regions, the average motion amplitude and the average shape change speed are used to characterize the reading attention of the user group in the shared space.

Description

Technical Field

[0001] The present invention belongs to the field of new generation information technology, and in particular to a real-time detection system and method for reading attention based on recognition of changes in the limbs of user groups in a library shared space. Background Art

[0002] With the development of information technology, the Internet, new media, big data, artificial intelligence, mobile devices and other technological innovations have brought about changes in users' service requirements and resource demands for libraries, and many new attempts to build learning commons spaces in university libraries have emerged. Library learning commons spaces have become the most important place for users to connect with libraries. The emergence of artificial intelligence today is impacting various application fields. The service system of university libraries has shifted from "digital service 1.0" marked by document information services to "intelligent service 2.0" that focuses on mining collection resources. Different scenario constructions in smart spaces often affect users' reading effects, and different users have different requirements for their spatial scenario construction in different reading states. Therefore, the detection and identification of user reading states in library smart spaces has become an important research content to improve the efficiency of library services.

[0003] The reading attention of users in the shared space of the library is an important indicator to measure the emotions and value orientation between users and the library. Whether the library has high service efficiency, good service quality, and a strong academic atmosphere can be judged from the emotional reactions of users. Good emotions lead to good reading attention, and thus good reading effects. The reading attention of users is closely related to their body posture and its changes. The body posture is the user's expression. There are three main different ways of expressing people's expressions, namely, tone expression, body expression, and facial expression. Among them, facial expression is one of the most advantageous and convenient ways for people to convey emotions, cognition, intentions, and opinions to each other, so some detection technologies related to face and eyes have emerged. For example: a learning state recognition method based on outlier detection technology under data mining theory, which allocates limited educational resources to students with the most urgent needs. The density-based local outlier detection algorithm is used to mine the student test score data, find out the suspicious outliers, and then analyze the learning status of the suspicious outliers. For example, the cloud platform controls the high-definition dome camera with variable focus to quickly recognize the faces of the students in the classroom. It can quickly recognize the faces of all students in the classroom, and continuously scan the listening status. The acquired image and video data are uploaded to the cloud server in real time for analysis, and the structured data is processed by big data, and the statistical results are output. The background is visualized, and the system is bound to the student's WeChat or QQ to realize service functions such as real-time class status reminders. Another example is the use of video monitoring technology, Java programming technology, image processing technology, and statistical analysis technology to detect the learning status through the face state. This technology can classify and identify the four states of students in class: attending class seriously, being in a daze, sleeping, and playing with mobile phones. There is also a technology for analyzing the learning state based on the eye movement analysis algorithm of the neural network RNN. This technology proposes a method based on RNN-EMA (RNN-Eye Movement Analysis) eye movement analysis algorithm, which predicts students' learning behavior and completes the current learning status detection by analyzing the sequence eye movement vector; another example is the learning fatigue recognition and intervention method based on facial expression, which decomposes the user's expression features into different subspaces, recognizes them in the expression subspace, and finally recognizes the expression in the intelligent learning space; in addition, there is also the analysis of students' learning concentration based on micro-expressions, which is classified by the micro-expressions of users' eyes and faces, etc. The user's reading attention is closely related to his or her emotions, and his or her expression is always accompanied by the body's physical movements, such as dancing when happy and dejected when sad. The expression of emotional physical movements is called emotional body language by researchers. The study of emotional body language shows that when the brain recognizes physical stimuli and non-physical neutral things, the activation of the brain area for physical stimulation is similar to the recognition of the face. Emotions put the body at the core of emotional information processing, and believe that emotional experience and emotional processing are inseparable from body sensory resources.Therefore, some scholars have proposed a pixel processing method of mathematical difference of video images, which can identify the changes in the body movement state of users during reading, so as to judge the basic emotions of users. For example, some scholars have captured the coordinates of the user's posture joints through video images, and compared them with the standard posture for scoring to measure the current posture emotion. Therefore, the user's limb posture and its changes can be used as a basis for identifying the user's emotional state, thereby reflecting the user's reading attention. From the above research and analysis, it can be seen that whether it is face detection or facial expression detection, if it is only satisfied with the detection of static pictures, it is difficult to apply it to the real-time detection of library users, and the same is true for the emotion detection system. For example, the above method only uses the two-dimensional state of the user's body as the basis for judgment, which results in a large judgment error. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention proposes a real-time detection system and method for reading attention of user groups based on library shared space, which accurately obtains the user's reading status based on body shape change recognition.

[0005] The technical solution adopted by the present invention is as follows:

[0006] A real-time detection system for reading attention of user groups in a shared space of a library, wherein the detection system is arranged in the shared space, and comprises a fringe projection amplification unit, an image acquisition unit and a computer processing system; the fringe projection amplification unit projects a fringe light beam toward the user group in the shared space; the image acquisition unit acquires a fringe modulation image of the user group containing the fringe light beam in the shared space, and simultaneously inputs the acquired fringe modulation image into the computer processing system; the computer processing system is used to control the fringe projection amplification unit, and the computer processing system processes the acquired fringe modulation image to extract the number N of motion regions and the average motion amplitude and average shape change speed By the number of motion regions N and the average motion amplitude and average shape change speed Characterizes the reading attention of user groups in a shared space.

[0007] Furthermore, the fringe projection amplification unit includes a laser light source, a grating generator and a light-amplifying projector; the laser light source, the grating generator and the light-amplifying projector are placed in sequence from top to bottom above the shared space, and the laser light source, the grating generator and the light-amplifying projector are located on the same optical axis.

[0008] Furthermore, the laser light source is sequentially connected to the timing controller and the computer processing system through signal lines;

[0009] Furthermore, the image acquisition unit includes a camera, which is located next to the light amplifying projector and connected to a computer processing system via a signal line; the computer processing system and a timing controller synchronize the timing and frame extraction of the fringe image.

[0010] Furthermore, the fringe modulation image is processed in the computer processing system to obtain the number of motion regions N and the average motion amplitude. and average shape change speed And the number of motion regions N, the average motion amplitude and the and average shape change speed The threshold value.

[0011] A real-time detection method for reading attention of a user group based on a library shared space comprises the following steps:

[0012] S1, using the detection system to collect the stripe modulation image of the user group containing the stripe light beam in the shared space, and pre-processing the two adjacent frames of stripe modulation image in turn, and performing mathematical difference operation on the processed two frames of stripe modulation image, and the image after operation is high-pass filtered to form a preliminary result image.

[0013] S2, calculate the stripe area, number, pixel duty cycle, and sampling frequency in the preliminary result image to obtain the spatiotemporal distribution parameters of the user's limb posture changes in the image, which include the number of motion areas N, the average motion amplitude, and average shape change speed

[0014] S3, in the computer processing system, respectively set the number of motion regions N, the average motion amplitude and average shape change speed The judgment threshold is the number of motion areas N, the average motion amplitude and average shape change speed The comparison results of the above spatiotemporal distribution parameters and the thresholds are compared with the judgment thresholds respectively, and the comparison results of the above spatiotemporal distribution parameters and the thresholds represent the reading attention of the user group in the shared space.

[0015] Furthermore, the preprocessing includes monochrome filtering and binary fringe processing. The distribution functions of the two images after binary fringe processing are H 1(x,y) ,H 2(x,y) ; The difference function of the two images is obtained by difference processing: ΔH (x,y) =H 2(x,y) -H 1(x,y) The difference of 1 indicates the area where the shape and limbs have changed, and the difference of 0 indicates the area where the shape and limbs have not changed.

[0016] Furthermore, the method for obtaining the number of motion regions N is as follows: (x,y) The continuous areas with function value 1 are contoured, and the continuous areas are counted to obtain the number N of motion areas.

[0017] Further, the average motion amplitude is obtained as Among them, F i represents the change amplitude of the i-th limb, M i Represented as the i-th pixel.

[0018] Furthermore, the average shape change speed is S i is the shape change speed of the i-th motion area.

[0019] Beneficial effects of the present invention:

[0020] 1. The present invention discloses a real-time reading attention detection system based on the recognition of changes in the body shapes of user groups in a library shared space. By modulating the grating illumination light, the system can effectively highlight the three-dimensional changes in the body shapes of the users, and provide effective and accurate optical information for the detection of the users' reading attention.

[0021] 2. The present invention describes a real-time reading attention detection system based on the recognition of physical changes of a user group in a library shared space. The grating stripe image can be amplified by a light amplifying projector, thereby realizing full-field status detection of the user group in the library shared space.

[0022] 3. The real-time detection system for reading attention based on the recognition of limb changes of user groups in a shared space of a library described in the present invention can realize time-sharing stripe image illumination through a timing controller, thereby realizing online real-time frame detection of the reading attention status of the library user group.

[0023] 4. The present invention describes a real-time detection system for reading attention based on the recognition of changes in the limbs of user groups in a library shared space. The stripe image is captured by a camera under timing control, and the irradiation light is monochromatic light, which can accurately extract the stripe image without affecting the user's reading.

[0024] 5. The present invention describes a real-time reading attention detection system based on the recognition of body shape and limb changes of user groups in a library shared space. The system uses a computer processing system to calculate the stripe area, number, pixel duty cycle, and sampling frequency in a stripe image. Based on the relationship between the changes in user body shape and limb posture and the reading attention status, the three-level reading attention dynamic status of the user group can be accurately obtained, thereby providing a basis for the intelligent control of the library shared space and greatly improving the library service efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram of the real-time detection system of reading attention of the present invention;

[0026] In the figure, 1. laser light source; 2. grating generator; 3. light amplifying projector; 4. timing controller; 5. camera; 6. computer processing system. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0028] like Figure 1 A real-time detection system for reading attention of user groups in a shared space of a library is shown, and the detection system is arranged in the shared space; the detection system comprises a laser light source 1, a grating generator 2, a light amplifying projector 3, a timing controller 4, a camera 5 and a computer processing system 6. Among them, the laser light source 1, the grating generator 2 and the light amplifying projector 3 are placed in sequence from top to bottom above the shared space, and the laser light source 1, the grating generator 2 and the light amplifying projector 3 are located on the same optical axis, and a fringe projection amplification unit is formed by the laser light source 1, the grating generator 2 and the light amplifying projector 3. The laser light source 1 emits a collimated light beam, which passes through the grating generator 2 and the light amplifying projector 3 and then illuminates the user group object for imaging; the fringe projection amplification system projects the amplified fringe light beam toward the user group object in the shared space. The laser light source 1 is connected to the timing controller 4 and the computer processing system 6 in sequence through a signal line; the striped light beam generated by the laser light source 1 is controlled by the switch of the timing controller 4; the computer processing system 6 performs time-sharing control on the continuous light of the laser light source 1 through the timing controller 4; more specifically, the timing controller 4 controls the pulse width of the irradiated light passing through the grating generator 2, so that the width can be adjusted according to the detection frequency requirement.

[0029] The camera 5 is located next to the optical amplifying projector 3 and is used to capture the stripe modulated image of the user group objects containing the stripe light beam in the shared space; the camera 5 is connected to the computer processing system 6 via a signal line; the computer processing system 6 and the timing controller 4 synchronize the timing and frame extraction of the stripe image.

[0030] In addition to controlling the frame extraction of the grating stripe image, the computer processing system 6 processes the obtained image of the user group's reading posture under the stripe modulation, extracts two images from the images of the required attention period, and performs mathematical difference, filtering, contour line, pixel area, number and other parameter processing and calculation, and identifies and classifies the user group's reading attention based on the relationship between the reading attention and the user's posture, thereby achieving the purpose of the present technology.

[0031] Based on the above reading attention real-time detection system, this application also proposes a real-time detection method for reading attention based on a user group in a library shared space, including the following steps:

[0032] S1, the computer processing system sequentially performs monochrome filtering and binary fringe processing on two frames of images captured by the camera 5 that are adjacent in time, and performs mathematical difference operation on the processed two frames of fringe modulation images. After the operation, the image is subjected to high-pass filtering to form a preliminary result image. The specific operation is described as follows using two images:

[0033] The monochromatic filtering process is to assign 0 to the wavelength values ​​other than λ0 in the color image, where λ0 is the laser wavelength; and then perform conventional binarization. Suppose the distribution functions of the two images after processing are H 1(x,y) ,H 2(x,y) ; The difference function of the two images obtained by difference processing is ΔH (x,y) =H 2(x,y) -H 1(x,y) , where the difference value is 1, which indicates the area where the shape and limbs have changed, and the difference value is 0, which indicates the area where the shape and limbs have not changed.

[0034] S2, the computer processing system calculates the stripe area, number, pixel duty cycle, and sampling frequency in the preliminary result image to obtain the main parameters of the spatiotemporal distribution of the user's limb posture changes in the image. The specific operation instructions are as follows:

[0035] ΔH (x,y) The continuous area with the function value of 1 is contoured, and the continuous areas are counted to get the number N. This parameter represents the number of parts where the shape changes, called motion areas, which reflects the degree of agitation of the user group.

[0036] Assume that the amplitude of the limb change is F i , due to the change in the amplitude of the limb F i It is proportional to the number of pixels M with a value of 1 in the motion area of ​​the image, so the average motion amplitude can be defined as Among them, F i represents the change amplitude of the i-th limb, M i Represented as the i-th pixel.

[0037] Assume that the speed of shape change of the i-th motion area is S i , then the speed of shape change in the motion area is expressed as S i =F i / Δt i , where Δt i is the frame sampling time interval, from which the average shape change speed can be obtained as

[0038] S3. Set the number of motion regions N, the average motion amplitude, and the average limb change speed respectively in the computer processing system. and the judgment thresholds of the average limb change speed Compare the number of motion regions N, the average motion amplitude and the average limb change speed with their respective judgment thresholds. The comparison results between the above spatio-temporal distribution parameters and the thresholds characterize the reading attention of the user group in the shared space. Specifically:

[0039] Set the lowest threshold N1 and the highest threshold N2 for the number of motion regions N, and N1 < N2; when N < N1, it indicates that the change in the motion region is small and the attention is high; when N1 < N < N2, it indicates that the change in the motion region is medium and the attention is average; when N2 < N, it indicates that the change in the motion region is large and the attention is low.

[0040] Set the lowest threshold F1 and the highest threshold F2 for the average motion amplitude , and F1 < F2; when it indicates that the motion amplitude is small and the attention is high; when it indicates that the motion amplitude is medium and the attention is average; when it indicates that the motion amplitude is large and the attention is low.

[0041] Set the lowest threshold S1 and the highest threshold S2 for the average limb change speed , and S1 < S2; when it indicates that the limb change is small and the attention is high; it indicates that the limb change is medium and the attention is average; when it indicates that the limb change is large and the attention is low. That is, negative, neutral, and positive. According to the relationship between the reading attention and the change of the user's limb level (see the following table), set the thresholds for the parameters (N, F, S) representing the poor, medium, and good reading attention of the user respectively, and perform classification processing according to the thresholds to obtain the final result of the reading attention of the user group.

[0042] The relationship between the reading state and the reading attention based on the fringe pattern analysis

[0043]

[0044] The above embodiments are only used to illustrate the design ideas and characteristics of the present invention, and the purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made according to the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A real-time reading attention detection system based on library shared space user groups, characterized in that: The detection system is arranged in a shared space, and comprises a fringe projection amplification unit, an image acquisition unit and a computer processing system (6); the fringe projection amplification unit projects a fringe light beam toward a user group in the shared space; the image acquisition unit acquires a fringe modulation image of the user group containing the fringe light beam in the shared space, and simultaneously inputs the acquired fringe modulation image into the computer processing system (6); the computer processing system (6) is used to control the fringe projection amplification unit, and the computer processing system (6) processes the acquired fringe modulation image to extract the number N of motion regions and the average motion amplitude. and average shape change speed By the number of motion regions N and the average motion amplitude and average shape change speed Characterize the reading attention of user groups in the shared space; The fringe projection amplification unit comprises a laser light source (1), a grating generator (2) and a light amplifying projector (3); the laser light source (1), the grating generator (2) and the light amplifying projector (3) are placed in sequence from top to bottom above a shared space, and the laser light source (1), the grating generator (2) and the light amplifying projector (3) are located on the same optical axis; The image acquisition unit comprises a camera (5), the camera (5) is located next to the light amplifying projector (3), and the camera (5) is connected to a computer processing system (6) via a signal line; the computer processing system (6) and the timing controller (4) synchronize the timing and frame extraction of the fringe image; The fringe modulation image is processed in the computer processing system (6) to obtain the number N of motion regions and the average motion amplitude. and average shape change speed The number of motion regions N, the average motion amplitude, and and average shape change speed The threshold value.

2. According to claim 1, a real-time detection system for reading attention of user groups based on library shared space is characterized in that: The laser light source (1) is sequentially connected to a timing controller (4) and a computer processing system (6) via signal lines.

3. A real-time detection method for reading attention of user groups based on library shared space, characterized in that: Based on the real-time detection system of reading attention of user groups in library shared space described in claim 1, the method comprises the following steps: S1, using the detection system to collect the user group stripe modulation image containing the stripe light beam in the shared space, and pre-processing two adjacent frames of stripe modulation image in sequence, and performing mathematical difference operation on the processed two frames of stripe modulation image, and the image after operation is high-pass filtered to form a preliminary result image; S2, calculate the stripe area, number, pixel duty cycle, and sampling frequency in the preliminary result image to obtain the spatiotemporal distribution parameters of the user's limb posture changes in the image, which include the number of motion areas N, the average motion amplitude, and average shape change speed S3, in the computer processing system, respectively set the number of motion regions N, the average motion amplitude and average limb change speed The judgment threshold is the number of motion areas N, the average motion amplitude and average limb change speed The comparison results of the above spatiotemporal distribution parameters and the thresholds are respectively compared with the judgment thresholds, and the comparison results of the above spatiotemporal distribution parameters and the thresholds represent the reading attention of the user group in the shared space; The preprocessing includes monochrome filtering and binary fringe processing. The distribution functions of the two images after binary fringe processing are H 1(x,y) ,H 2(x,y) ; The difference function of the two images is obtained by difference processing: ΔH (x,y) =H 2(x,y) -H 1(x,y) , the difference of 1 area indicates the area where the shape limb changes, and the difference of 0 area indicates the area where the shape limb does not change; The method to obtain the number of motion regions N is as follows: (x,y) The continuous area with the function value of 1 is contoured, and the continuous areas are counted to obtain the number of motion areas N; The average motion amplitude is obtained as Among them, F i represents the change amplitude of the i-th limb, M i Represented as the i-th pixel; The average shape change speed is S i is the shape change speed of the i-th motion area.

Images