Screen adjustment method based on human body posture

Abstract

The present application discloses a screen adjustment method based on human body posture, which analyzes the user status monitoring video collected by a camera by applying computer vision technology and intelligent algorithms, and deeply mines the user's status characteristics in combination with multi-scale semantic information extraction, so as to identify the user's real-time status and behavior expression, and further respond to the user's status recognition result of speaking or thinking without controlling the display screen for adjustment.

Description

Technical Field

[0001] The present application relates to the field of intelligent adjustment, and more specifically, to a screen adjustment method based on human body posture. Background Art

[0002] With the rapid development of information technology, people are increasingly relying on computers and monitors for work and entertainment in their daily lives. However, prolonged use of computer monitors may cause users to experience various physical discomforts, such as neck, back, and eye fatigue. These discomforts are often related to improper adjustment of the monitor.

[0003] The traditional monitor adjustment method requires users to manually adjust the height, angle, etc. of the monitor according to their own feelings. This is not only inefficient, but users may fail to make adjustments in time due to lack of professional knowledge or ignoring their own feelings.

[0004] At present, there are also some micro-intelligent displays on the market that can self-adjust parameters such as height or angle to relieve users' physical discomfort. However, such displays often make misadjustments during actual use. That is, when the user looks up to think or talks with others, the display will mistakenly think that the user has changed his sitting position and make unnecessary adjustments to parameters such as height or angle.

[0005] Therefore, an optimized screen adjustment method based on human body posture is expected. Summary of the invention

[0006] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a screen adjustment method based on human body posture, which analyzes the user status monitoring video collected by the camera by applying computer vision technology and intelligent algorithms, and deeply mines the user's status characteristics in combination with multi-scale semantic information extraction, thereby identifying the user's real-time status and behavior expression, and further responding to the user's status recognition result of speaking or thinking, does not control the display screen for adjustment.

[0007] According to one aspect of the present application, a screen adjustment method based on human body posture is provided, which includes:

[0008] Obtain user status monitoring video collected by a camera deployed on the display screen;

[0009] Performing multi-scale state semantic feature extraction on the user state monitoring video to obtain a sequence of user state semantic feature vectors;

[0010] Performing feature aggregation on the sequence of user state semantic feature vectors to obtain a user state temporal semantic fusion feature vector;

[0011] Performing state recognition on the user state temporal semantic fusion feature vector to obtain a state recognition result;

[0012] In response to the state recognition result being speaking or thinking, the position of the display screen relative to the user is not controlled.

[0013] Compared with the prior art, the present application provides a screen adjustment method based on human posture, which analyzes the user status monitoring video collected by the camera by applying computer vision technology and intelligent algorithms, and deeply mines the user's status characteristics in combination with multi-scale semantic information extraction, thereby identifying the user's real-time status and behavior representation, and further responding to the user's status recognition result of speaking or thinking, without controlling the display screen for adjustment. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0015] Figure 1 is a flow chart of a screen adjustment method based on human body posture according to an embodiment of the present application;

[0016] Figure 2 is a system architecture diagram of a screen adjustment method based on human body posture according to an embodiment of the present application;

[0017] Figure 3 is a flowchart of sub-step S2 of the screen adjustment method based on human body posture according to an embodiment of the present application;

[0018] Figure 4 4 is a flowchart of sub-step S23 of the screen adjustment method based on human body posture according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0020] As stated in this application and claims, unless the context clearly indicates an exception,

[0021] The words "a", "an", "a kind" and / or "the" do not refer to the singular, but may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list, and the method or device may also include other steps or elements.

[0022] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0023] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0024] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0025] The traditional display adjustment method requires the user to manually adjust the height, angle, etc. of the display according to their own feelings, which is not only inefficient, but the user may fail to make timely adjustments due to lack of professional knowledge or neglect of their own feelings. At present, there are also some micro-intelligent displays on the market that can self-adjust parameters such as height or angle to alleviate the user's physical discomfort. However, this type of display often produces misadjustments during actual use. That is, when the user looks up to think or talks with others, the display will mistakenly think that the user has changed his sitting position and make unnecessary adjustments to parameters such as height or angle. Therefore, an optimized screen adjustment method based on human posture is expected.

[0026] In the technical solution of the present application, a screen adjustment method based on human body posture is proposed. Figure 1 The figure is a flow chart of a screen adjustment method based on human body posture according to an embodiment of the present application. Figure 2 FIG. 1 is a system architecture diagram of a screen adjustment method based on human body posture according to an embodiment of the present application. Figure 1 and Figure 2As shown, according to the embodiment of the present application, the screen adjustment method based on human posture includes the following steps: S1, obtaining a user status monitoring video collected by a camera deployed on a display screen; S2, performing multi-scale state semantic feature extraction on the user status monitoring video to obtain a sequence of user status semantic feature vectors; S3, performing feature aggregation on the sequence of user status semantic feature vectors to obtain a user status temporal semantic fusion feature vector; S4, performing state recognition on the user status temporal semantic fusion feature vector to obtain a state recognition result; S5, in response to the state recognition result being speaking or thinking, not controlling the position of the display screen relative to the user.

[0027] In particular, the S1 obtains the user status monitoring video collected by the camera deployed on the display screen. Here, the camera can capture the user's posture and behavior when using the display in real time, and provide necessary input data for the intelligent adjustment system. That is, by analyzing the collected user status monitoring video, the user's body language, including the position and angle of the head, neck, and arms, can be learned, so as to judge the user's comfort level and behavior pattern.

[0028] In particular, the S2 extracts multi-scale state semantic features from the user state monitoring video to obtain a sequence of user state semantic feature vectors. In particular, in a specific example of the present application, Figure 3 As shown, the S2 includes: S21, performing key frame sampling on the user state monitoring video to obtain a time series of user state monitoring key frames; S22, respectively passing each user state monitoring key frame in the time series of the user state monitoring key frames through a state semantic feature extractor based on a void convolutional neural network model to obtain a sequence of user state semantic feature graphs; S23, using a multi-scale semantic feature extraction module to perform multi-scale semantic information extraction on each user state semantic feature graph in the sequence of user state semantic feature graphs to obtain a sequence of user state semantic feature vectors.

[0029] Specifically, in S21, key frame sampling is performed on the user status monitoring video to obtain a time series of user status monitoring key frames. Considering that the amount of user status monitoring video data as video data is huge, there is a lot of redundant information therein, such as a large number of repeated frames and similar frames. Directly processing the user status monitoring video may increase the calculation time and cost. Therefore, in the technical solution of the present application, key frame sampling is performed on the user status monitoring video to obtain a time series of user status monitoring key frames. In this way, the more important and critical image frame information in the user status monitoring video can be extracted by key frame sampling, that is, the redundancy in the video data is removed, and only the information that is crucial to understanding the user status is retained.

[0030] Specifically, the S22, each user state monitoring key frame in the time series of the user state monitoring key frame is respectively passed through a state semantic feature extractor based on a hole convolutional neural network model to obtain a sequence of user state semantic feature maps. Among them, the atrous convolutional neural network (Atrous Convolutional Neural Network, referred to as ACNN or Dilated CNN) is a special convolutional neural network that increases the receptive field without increasing the amount of calculation by using hole convolution (also called dilated convolution or holed convolution). Among them, the core idea of ​​hole convolution is to introduce a dilation rate in the convolution operation, and the dilation rate determines the spacing between elements in the convolution kernel. By increasing the dilation rate, the receptive field of the network can be effectively increased without increasing the number of parameters, so that it can capture a wider range of contextual information. In the technical solution of the present application, the state semantic feature extractor is constructed using a hole convolutional neural network model to extract image features for each of the user state monitoring key frames, which is very helpful for understanding large-scale structures and patterns in images. Specifically, since the dilated convolutional network can adapt to larger scale features, this is especially important for tasks such as human posture recognition, because human posture may change at a wider scale and angle. In this way, the advantages of dilated convolution can be used to extract the posture and behavior features in each of the user status monitoring key frames, so as to understand and grasp the user's real-time status and behavior pattern.

[0031] Accordingly, in a possible implementation, each user state monitoring key frame in the time series of the user state monitoring key frames can be respectively passed through a state semantic feature extractor based on a hole convolutional neural network model to obtain a sequence of user state semantic feature graphs through the following steps, for example: extracting key frames from the user state monitoring video; serializing the extracted key frames to form a key frame data sequence; designing a state semantic feature extractor model based on a hole convolutional neural network to extract state semantic features from the key frames; taking each user state monitoring key frame as input, and extracting state semantic features through a hole convolutional neural network model; hole convolution can expand the receptive field, capture a wider range of contextual information, and help extract richer features; for each key frame, serialize the state semantic feature graph extracted by the hole convolutional neural network to obtain a sequence of user state semantic feature graphs.

[0032] Specifically, the S23 uses a multi-scale semantic feature extraction module to perform multi-scale semantic information extraction on each user state semantic feature graph in the sequence of user state semantic feature graphs to obtain a sequence of user state semantic feature vectors. It should be understood that although the atrous convolutional neural network can capture and recognize the user's posture changes at a wider scale and angle, in actual application scenarios, the user's posture and behavior may be manifested at different scales. For example, tiny finger movements may be performing fine operations, while larger body movements may indicate that the user is resting or changing postures. In addition, the postures and behaviors of different users may show different characteristics due to individual differences, and the use of the same feature extraction strategy may ignore important posture changes and behavioral features. Therefore, in the technical solution of the present application, it is expected to use a multi-scale semantic feature extraction module to perform multi-scale semantic information extraction on each user state semantic feature graph in the sequence of user state semantic feature graphs to flexibly respond to the user's diverse posture changes, thereby obtaining a sequence of user state semantic feature vectors. In particular, in a specific example of the present application, such as Figure 4 As shown, the S23 includes: S231, extracting global semantic feature information and local semantic feature information from the user state semantic feature graph to obtain a global user state semantic feature vector and a local user state semantic feature vector; S232, fusing the global user state semantic feature vector and the local user state semantic feature vector to obtain the user state semantic feature vector.

[0033] More specifically, S231 extracts global semantic feature information and local semantic feature information from the user state semantic feature map to obtain a global user state semantic feature vector and a local user state semantic feature vector. In a specific example, the user state semantic feature map is upsampled to obtain an upsampled user state semantic feature map; the upsampled user state semantic feature map is processed using a global semantic feature extraction module to obtain a global user state semantic extraction feature vector; the global user state semantic extraction feature vector is dimensionally adjusted to obtain the global user state semantic feature vector; the upsampled user state semantic feature map is processed using a local semantic feature extraction module to obtain a local user state semantic extraction feature vector; the local user state semantic extraction feature vector is dimensionally adjusted to obtain the local user state semantic feature vector; wherein the global semantic feature extraction module includes a global average pooling layer, a first point convolution layer, a first batch normalization layer, and a first activation layer, and the local semantic feature extraction module includes a second point convolution layer, a second batch normalization layer, and a second activation layer.

[0034] More specifically, the S232 fuses the global user state semantic feature vector and the local user state semantic feature vector to obtain the user state semantic feature vector. In a specific example, the global user state semantic feature vector and the local user state semantic feature vector are calculated by adding the elements by position to obtain a multi-scale user state semantic fusion feature vector; and the multi-scale user state semantic fusion feature vector is activated based on the sigmoid function to obtain the user state semantic feature vector. It should be understood that the Sigmoid function is a commonly used nonlinear activation function, which can map the input real number to the interval (0,1) and has a smooth S-shaped curve.

[0035] It is worth mentioning that in other specific examples of the present application, the multi-scale semantic feature extraction module can also be used in other ways to perform multi-scale semantic information extraction on each user state semantic feature map in the sequence of user state semantic feature maps to obtain a sequence of user state semantic feature vectors, for example: by using a multi-scale semantic feature extraction module, semantic information of different scales can be extracted from the sequence of user state semantic feature maps; convolution operations or pooling operations of different scales can be applied at different levels or branches to capture semantic information at different scales; semantic features extracted at different scales can be fused, and multi-scale features can be fused by splicing, weighted averaging, etc.; the fused multi-scale semantic feature vectors are serialized to obtain a sequence of user state semantic feature vectors.

[0036] It is worth mentioning that in other specific examples of the present application, multi-scale state semantic feature extraction can be performed on the user state monitoring video in other ways to obtain a sequence of user state semantic feature vectors, for example: input the user state monitoring video; sample video frames from the user state monitoring video, and sample at equal intervals or according to specific conditions as needed; use models such as convolutional neural networks (CNN) to perform multi-scale feature extraction on each video frame to capture semantic information at different scales; fuse features extracted at different scales by weighted averaging or other methods; serialize the fused multi-scale feature vectors; at each time step, use models such as recurrent neural networks (RNN) and long short-term memory networks (LSTM) to extract semantic features from multi-scale features to capture semantic information in the video; through the above steps, a sequence of user state semantic feature vectors is obtained, each vector representing the state semantic feature at a certain time point in the video.

[0037] In particular, the S3 performs feature aggregation on the sequence of the user state semantic feature vectors to obtain the user state temporal semantic fusion feature vector. In particular, in a specific example of the present application, the sequence of the user state semantic feature vectors is aggregated through a message passing neural network model to obtain the user state temporal semantic fusion feature vector. In the process of the above-mentioned void convolution coding and multi-scale semantic feature extraction, the action features and posture information expressed by each user state monitoring key frame in the user state monitoring video are learned and captured, but at this time each user state semantic feature vector only expresses the user's behavior and posture state at a specific time point, and lacks temporal context association. It should be understood that a single action or posture often cannot fully express the user's behavior pattern. In other words, the user's behavior is usually a combination of a series of actions, which are temporally coherent and have logical connections with each other. Therefore, in order to capture this temporal interaction and dependency, in the technical solution of the present application, the sequence of the user state semantic feature vectors is aggregated through a message passing neural network model to integrate the comprehensive representation information of the user state semantic features in the time dimension, thereby obtaining a user state temporal semantic fusion feature vector. In this way, modeling the long-term temporal dependency between the sequences of the user state semantic feature vectors through the message passing aggregation neural network model helps to understand the overall behavior pattern and posture change state of the user. In a specific example of the present application, the process of passing the sequence of the user state semantic feature vectors through the message passing aggregation neural network model to obtain the user state temporal semantic fusion feature vector includes: first, aggregating the user's overall state semantic feature information of the sequence of the user state semantic feature vectors to obtain the user state semantic aggregation feature vector; then, retaining the user state semantic feature information at the current time point to obtain the explicit user state semantic feature vector at the current time point; then, the user state semantic aggregation feature vector and the explicit user state semantic feature information at the current time point are message-fitted and passed to obtain the user state temporal semantic fusion feature vector. Wherein, aggregating the user's overall state semantic feature information of the sequence of the user state semantic feature vectors to obtain the user state semantic aggregation feature vector includes: calculating the positional sum of the sequence of the user state semantic feature vectors to obtain the user state semantic aggregation feature vector. Retaining user state semantic feature information at the current time point to obtain an explicit user state semantic feature vector at the current time point, including: constructing a retention weight based on a learnable scalar parameter; performing point multiplication of the retention weight and the last user state semantic feature vector in the sequence of user state semantic feature vectors to obtain the explicit user state semantic feature vector at the current time point.More specifically, the user state semantic aggregation feature vector and the current time point explicit user state semantic feature information are message-fitted and transmitted to obtain the user state temporal semantic fusion feature vector, including: adding points according to positions of the user state semantic aggregation feature vector and the current time point explicit user state semantic feature information, and then performing information fitting through a multi-layer perceptron to obtain the user state temporal semantic fusion feature vector, wherein the multi-layer perceptron is composed of fully connected layers.

[0038] In summary, in the above embodiment, the sequence of user state semantic feature vectors is subjected to feature aggregation to obtain the user state temporal semantic fusion feature vector, including: using a message passing aggregation neural network to perform feature aggregation on the sequence of user state semantic feature vectors according to the following formula to obtain the user state temporal semantic fusion feature vector; wherein the formula is:

[0039]

[0040] in, is the last user state semantic feature vector in the sequence of user state semantic feature vectors, is the user state temporal semantic fusion feature vector, ∈ (k) is the k-1th learnable scalar parameter, N(v) represents the sequence of the user state semantic feature vector, The user state semantic feature vector at the u-th position in the sequence of the user state semantic feature vectors is represented, and MLP is a multi-layer perceptron.

[0041] In particular, the S4 performs state recognition on the user state time series semantic fusion feature vector to obtain a state recognition result. In particular, in a specific example of the present application, the user state time series semantic fusion feature vector is passed through a state recognizer based on a classifier to obtain the state recognition result, and the state recognition result is used to represent the state type label. Here, the classifier can automatically make a decision based on the input feature vector, that is, the user state time series semantic fusion feature vector, to identify the current state of the user without manual intervention. Specifically, the classifier can map the user state time series semantic fusion feature vector to a discrete state label, so that the system can clearly identify what state the user is in, such as thinking, resting, talking, etc. Specifically, the user state time series semantic fusion feature vector is passed through a state recognizer based on a classifier to obtain the state recognition result, and the state recognition result is used to represent the state type label, including: using multiple fully connected layers of the classifier to fully connect the user state time series semantic fusion feature vector to obtain a coded classification feature vector; and, passing the coded classification feature vector through the Softmax classification function of the classifier to obtain the classification result.

[0042] In a preferred example, passing the user state temporal semantic fusion feature vector through a classifier-based state recognizer to obtain a state recognition result specifically includes the following steps: performing dot addition of the user state temporal semantic fusion feature vector with the square root of the length of the user state temporal semantic fusion feature vector and the inverse of the square root of the second norm of the user state temporal semantic fusion feature vector to obtain a user state temporal semantic fusion offset feature vector; calculating an exponential function of the user state temporal semantic fusion offset feature vector with a natural constant as the base to obtain a user state temporal semantic fusion offset prediction feature vector; performing dot multiplication of the user state temporal semantic fusion feature vector with the first norm and a weight hyperparameter of the user state temporal semantic fusion feature vector to obtain a user state temporal semantic fusion constraint feature vector; performing dot addition of the user state temporal semantic fusion offset prediction feature vector with the user state temporal semantic fusion constraint feature vector to obtain an optimized user state temporal semantic fusion feature vector; and, passing the optimized user state temporal semantic fusion feature vector through a classifier-based state recognizer to obtain a state recognition result. Specifically, considering the image semantic temporal transfer aggregation distribution of the user state temporal semantic fusion feature vector for the multi-scale image semantic features of the key frames of the user state monitoring video based on the image semantic temporal distribution and the overall distribution regression constraint of the classifier, in the above preferred example, the structured norm representation of the user state temporal semantic fusion feature vector is used as the local canonical coordinates for each eigenvalue of the user state temporal semantic fusion feature vector to determine the class rotation offset prediction direction of the vector overall distribution representation of the user state temporal semantic fusion feature vector relative to the eigenvalue for each eigenvalue of the user state temporal semantic fusion feature vector as the center, and the bounding box of the vector eigenvalue distribution of the user state temporal semantic fusion feature vector is used to perform eigenvalue constraints to improve the constraint of the user state temporal semantic fusion feature vector under the overall regression distribution, thereby improving the training speed of the model and the accuracy of the state recognition result obtained by the user state temporal semantic fusion feature vector through the state recognizer based on the classifier.

[0043] In particular, in S5, in response to the state recognition result being speaking or thinking, the position of the display screen relative to the user is not controlled. In the technical solution of the present application, after obtaining the state recognition result, in response to the state recognition result being speaking or thinking, the position of the display screen relative to the user is not controlled. Here, when the user is having a conversation or thinking, they may not need the parameters of the display to be changed. At this time, automatically adjusting the display may interrupt the user's thoughts or conversations, causing unnecessary interference.

[0044] In summary, according to the embodiment of the present application, the screen adjustment method based on human posture is explained, which analyzes the user status monitoring video collected by the camera by applying computer vision technology and intelligent algorithms, and deeply mines the user's status characteristics in combination with multi-scale semantic information extraction, thereby identifying the user's real-time status and behavior representation, and further responds to the user's status recognition result of speaking or thinking, without controlling the display screen for adjustment.

[0045] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A screen adjustment method based on human body posture, characterized in that: include: Obtain user status monitoring video collected by a camera deployed on the display screen; Performing multi-scale state semantic feature extraction on the user state monitoring video to obtain a sequence of user state semantic feature vectors; Performing feature aggregation on the sequence of user state semantic feature vectors to obtain a user state temporal semantic fusion feature vector; Performing state recognition on the user state temporal semantic fusion feature vector to obtain a state recognition result; In response to the state recognition result being speaking or thinking, not controlling the position of the display screen relative to the user; Performing multi-scale state semantic feature extraction on the user state monitoring video to obtain a sequence of user state semantic feature vectors, including: Performing key frame sampling on the user status monitoring video to obtain a time sequence of user status monitoring key frames; Each user state monitoring key frame in the time series of the user state monitoring key frames is passed through a state semantic feature extractor based on a dilated convolutional neural network model to obtain a sequence of user state semantic feature graphs; A multi-scale semantic feature extraction module is used to extract multi-scale semantic information from each user state semantic feature graph in the sequence of user state semantic feature graphs to obtain a sequence of user state semantic feature vectors.

2. The screen adjustment method based on human body posture according to claim 1, characterized in that: Using a multi-scale semantic feature extraction module to extract multi-scale semantic information from each user state semantic feature graph in the sequence of user state semantic feature graphs to obtain a sequence of user state semantic feature vectors, including: Extracting global semantic feature information and local semantic feature information from the user state semantic feature graph to obtain a global user state semantic feature vector and a local user state semantic feature vector; The global user state semantic feature vector and the local user state semantic feature vector are fused to obtain the user state semantic feature vector.

3. The screen adjustment method based on human body posture according to claim 2, characterized in that: Extracting global semantic feature information and local semantic feature information from the user state semantic feature graph to obtain a global user state semantic feature vector and a local user state semantic feature vector includes: Upsampling the user state semantic feature map to obtain an upsampled user state semantic feature map; Using a global semantic feature extraction module to process the upsampled user state semantic feature map to obtain a global user state semantic extraction feature vector; Performing dimension adjustment on the global user state semantic extraction feature vector to obtain the global user state semantic feature vector; Processing the upsampled user state semantic feature map using a local semantic feature extraction module to obtain a local user state semantic extraction feature vector; Performing dimension adjustment on the local user state semantic extraction feature vector to obtain the local user state semantic feature vector; Among them, the global semantic feature extraction module includes a global average pooling layer, a first point convolution layer, a first batch normalization layer and a first activation layer, and the local semantic feature extraction module includes a second point convolution layer, a second batch normalization layer and a second activation layer.

4. The screen adjustment method based on human body posture according to claim 3, characterized in that: Fusion of the global user state semantic feature vector and the local user state semantic feature vector to obtain the user state semantic feature vector includes: Calculate the position-by-position addition of the global user state semantic feature vector and the local user state semantic feature vector to obtain a multi-scale user state semantic fusion feature vector; The multi-scale user state semantic fusion feature vector is activated based on a sigmoid function to obtain the user state semantic feature vector.

5. The screen adjustment method based on human body posture according to claim 4, characterized in that: Performing feature aggregation on the sequence of user state semantic feature vectors to obtain a user state temporal semantic fusion feature vector, including: The sequence of the user state semantic feature vectors is aggregated through a message passing neural network model to obtain the user state temporal semantic fusion feature vector; The method of obtaining the user state time series semantic fusion feature vector by aggregating the sequence of the user state semantic feature vectors through a message passing aggregation neural network model comprises: Aggregating user overall state semantic feature information of the sequence of user state semantic feature vectors to obtain a user state semantic aggregate feature vector; Retaining the user state semantic feature information at the current time point to obtain the explicit user state semantic feature vector at the current time point; The user state semantic aggregation feature vector and the current time point explicit user state semantic feature information are message-fitted and transmitted to obtain the user state temporal semantic fusion feature vector.

6. The screen adjustment method based on human body posture according to claim 5, characterized in that: Aggregating the user overall state semantic feature information of the sequence of user state semantic feature vectors to obtain a user state semantic aggregate feature vector, including: The position-wise sum of the sequence of the user state semantic feature vectors is calculated to obtain the user state semantic aggregate feature vector.

7. The screen adjustment method based on human body posture according to claim 6, characterized in that: The user state semantic feature information at the current time point is retained to obtain the explicit user state semantic feature vector at the current time point, including: Constructing retention weights based on learnable scalar parameters; The retention weight is point-multiplied with the last user state semantic feature vector in the sequence of user state semantic feature vectors to obtain the explicit user state semantic feature vector at the current time point.

8. The screen adjustment method based on human body posture according to claim 7, characterized in that: The user state semantic aggregation feature vector and the current time point explicit user state semantic feature information are subjected to message fitting transmission to obtain the user state temporal semantic fusion feature vector, including: After adding points according to positions on the user state semantic aggregation feature vector and the current time point explicit user state semantic feature information, information fitting is performed through a multi-layer perceptron to obtain the user state temporal semantic fusion feature vector, wherein the multi-layer perceptron is composed of a fully connected layer.

9. The screen adjustment method based on human body posture according to claim 8, characterized in that: Performing state recognition on the user state temporal semantic fusion feature vector to obtain a state recognition result, including: The user state temporal semantic fusion feature vector is passed through a state recognizer based on a classifier to obtain the state recognition result, and the state recognition result is used to represent a state type label.

Images