Intelligent picture control method and related device

Abstract

The invention discloses an intelligent picture control method and a related device, and relates to the technical field of data analysis, and the method comprises the steps: carrying out the depth feature map and tensor feature extraction based on an image sequence, and carrying out the gait recognition in combination with a gravity center track, and obtaining the target gait data; performing user action prediction based on the target gait data in combination with brain-like semantic feature extraction; carrying out human voice audio separation based on the audio information in combination with non-negative matrix factorization so as to carry out voiceprint feature analysis, and obtaining voiceprint feature information; performing user identity detection based on the voiceprint feature information to obtain user identity information; and carrying out picture zooming analysis and picture switching analysis on the display screen based on the user identity information and the user action prediction information so as to adjust the picture of the display screen. According to the invention, the picture of the user can be tracked in real time, omission of the picture of the user is avoided, and a more ideal picture control effect is achieved.

Description

Technical Field

[0001] This invention relates to the field of data analysis technology, and in particular to an intelligent screen control method and related device. Background Technology

[0002] During presentations and conferences, it's often necessary to switch and zoom the screen based on the speaker. Currently, this is typically done by detecting the speaker's voice to identify the user and then switching the screen accordingly. While current methods primarily extract the fundamental frequency of audio frames to determine voiceprint features, the lack of periodicity in some audio signals within frames makes it difficult to guarantee the accuracy of voiceprint feature extraction, thus reducing the reliability of user identification. Furthermore, current methods usually only consider the speaker's identity when switching screens, rarely considering adaptively zooming the screen based on user actions. Most adjustments to the screen zoom and switching are made only after user actions, which introduces significant latency, preventing real-time tracking of the user's screen and failing to achieve the desired screen control. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art. This invention provides an intelligent screen control method and related device that can track the user's screen in real time, avoid missing the user's screen, and achieve a more ideal screen control effect.

[0004] To address the aforementioned technical problems, this invention provides an intelligent screen control method, the method comprising: Based on the sensor combination, the user's image sequence and audio information are collected, and depth feature map and tensor feature extraction are performed on the image sequence to obtain depth feature map and tensor feature information. Based on the depth feature map and tensor feature information, gait recognition is performed in combination with the center of gravity trajectory to obtain target gait data. Based on the target gait data and brain-like semantic feature extraction, user action prediction is performed to obtain user action prediction information. Based on the audio information, human voice audio is separated by non-negative matrix decomposition to obtain human voice audio information, and voiceprint feature analysis is performed based on the human voice audio information to obtain voiceprint feature information. User identity is detected based on the voiceprint feature information to obtain user identity information; Based on the user identity information and user action prediction information, screen scaling analysis and screen switching analysis are performed on the display screen to obtain screen scaling information and screen switching information, and the screen is adjusted based on the screen scaling information and screen switching information.

[0005] Optionally, the step of extracting depth feature maps and tensor features based on the image sequence to obtain depth feature map and tensor feature information, and performing gait recognition based on the depth feature map and tensor feature information combined with the center of gravity trajectory to obtain target gait data, includes: Human image sequences are extracted based on the image sequences, and tensor feature extraction is performed based on the human image sequences using genetic algorithms and Tucker decomposition to obtain tensor feature information; Based on the image sequence, a depth map sequence is extracted to obtain a target depth map sequence. The target depth map sequence is then cropped and normalized to obtain a normalized target depth map sequence. Feature map extraction is performed on the normalized target depth map sequence to obtain depth feature maps; The key point information is obtained by performing key point recognition on the human image sequence based on the key point recognition model. Based on the key point information, center of gravity trajectory analysis is performed to obtain center of gravity trajectory information, and a gait energy map is generated based on the human image sequence. Gait recognition is performed based on the gait energy map, center of gravity trajectory information, depth feature map, and tensor feature information to obtain target gait data.

[0006] Optionally, generating a gait energy map based on the human image sequence includes: Based on the human image sequence, gait sequence image analysis is performed using a preset recursive algorithm to obtain the target gait image sequence. Determine the time-matching data of the target gait image sequence; Based on the target gait image sequence, a gait energy map is generated using the time-matched data and the minimum cumulative distance algorithm.

[0007] Optionally, the step of predicting user actions based on the target gait data combined with brain-like semantic feature extraction to obtain user action prediction information includes: Based on the target gait data, brain-like semantic features of gait behavior are extracted to obtain brain-like semantic feature information; Based on the target gait data and the image sequence, the connection analysis of skeletal key points is performed to obtain the corresponding connection data, and the user's movement direction is analyzed based on the connection data to obtain the target user's movement direction. Based on the aforementioned brain-like semantic feature information and the target user's action direction, user action prediction information is obtained.

[0008] Optionally, the step of performing human voice audio separation based on the audio information combined with nonnegative matrix factorization to obtain human voice audio information, and performing voiceprint feature analysis based on the human voice audio information to obtain voiceprint feature information, includes: Based on the audio information, a discrete Fourier transform is performed using a preset window function to obtain the time-frequency matrix; The time-frequency matrix is ​​decomposed based on nonnegative matrix factorization to obtain the basis matrix and activation coefficient matrix. The audio information is then initially separated based on the basis matrix and activation coefficient matrix combined with a classification algorithm to obtain the initially separated audio information. Human voice audio information is determined based on the initially separated audio information combined with an audio analysis model; The human voice audio information is subjected to spectrogram extraction to obtain spectrogram information, and frame-level feature information is extracted based on the spectrogram information; Phoneme feature analysis is performed based on the frame-level feature information to obtain phoneme feature information; Based on the phoneme feature information, voiceprint feature analysis is performed to obtain voiceprint feature information.

[0009] Optionally, the step of performing user identity detection based on the voiceprint feature information to obtain user identity information includes: Obtain historical voiceprint feature information and user personal information corresponding to the historical voiceprint feature information, and construct a voiceprint information database based on the historical voiceprint feature information and user personal information; Calculate the cosine value of the angle between each historical voiceprint feature and the voiceprint feature in the voiceprint information database, and calculate the cosine distance between each historical voiceprint feature and the voiceprint feature based on the cosine value of the angle. Based on the cosine distance, determine the voiceprint feature information and voiceprint authentication data of each historical voiceprint feature information, and determine the user identity information based on the voiceprint authentication data.

[0010] Optionally, the step of performing screen scaling analysis and screen switching analysis based on the user identity information and user action prediction information to obtain screen scaling information and screen switching information includes: A screen number is generated based on the user identity information and user action prediction information, and screen switching information is determined based on the screen number. Based on the user action prediction information and user identity information, the screen focus object is determined, and the screen adjustment amount of the screen focus object is determined based on the approximation algorithm. The screen adjustment amount includes vertical tilt adjustment information, horizontal displacement adjustment information and vertical displacement adjustment information, and screen scaling information is determined based on the screen adjustment amount.

[0011] In addition, the present invention also provides an intelligent screen control device, the device comprising: Gait recognition module: used to collect image sequences and audio information of users based on sensor combination, and extract depth feature maps and tensor features based on the image sequences to obtain depth feature map and tensor feature information, and perform gait recognition based on the depth feature map and tensor feature information combined with the center of gravity trajectory to obtain target gait data; Action prediction module: used to predict user actions based on the target gait data and brain-like semantic feature extraction, and obtain user action prediction information; Voiceprint feature analysis module: used to separate human voice audio based on the audio information and non-negative matrix decomposition to obtain human voice audio information, and to perform voiceprint feature analysis based on the human voice audio information to obtain voiceprint feature information; User identity detection module: used to detect user identity based on the voiceprint feature information and obtain user identity information; Screen control module: used to perform screen scaling analysis and screen switching analysis on the display screen based on the user identity information and user action prediction information, obtain screen scaling information and screen switching information on the display screen, and adjust the screen display screen based on the screen scaling information and screen switching information.

[0012] In addition, the present invention also provides an electronic device, which includes a processor and a memory. The memory is used to store instructions, and the processor is used to call the instructions in the memory to cause the electronic device to execute the above-described intelligent screen control method.

[0013] In addition, the present invention also provides a computer-readable storage medium that stores computer instructions, which, when executed on an electronic device, cause the electronic device to perform the above-described intelligent screen control method.

[0014] In this embodiment of the invention, depth feature maps and tensor features are extracted based on image sequences to obtain depth feature map and tensor feature information for gait recognition. User action prediction is then performed based on target gait data, effectively improving the accuracy of user action prediction. Voiceprint feature analysis is performed based on audio information combined with non-negative matrix factorization, and user identity detection is performed based on voiceprint feature information, improving the accuracy of voiceprint feature extraction and ensuring the reliability of user identity detection. Screen scaling and switching analysis are performed based on user identity information and user action prediction information to adjust the screen display, enabling real-time tracking of the user's screen, avoiding omissions of user screen views, and achieving a more ideal screen control effect. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating the intelligent screen control method in an embodiment of the present invention; Figure 2 This is a flowchart illustrating an intelligent screen control method according to another embodiment of the present invention; Figure 3 This is a schematic diagram of the structural composition of the intelligent screen control device in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structural composition of the electronic device in an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1 Please see Figure 1 , Figure 1 This is a flowchart illustrating the intelligent screen control method in an embodiment of the present invention, the method comprising: S11: Collect image sequences and audio information of the user based on sensor combination, and extract depth feature maps and tensor features based on the image sequences to obtain depth feature map and tensor feature information. Then, combine the depth feature map and tensor feature information with the center of gravity trajectory to perform gait recognition and obtain target gait data. In the specific implementation of this invention, a human image sequence is extracted based on the image sequence, and tensor feature extraction is performed using a genetic algorithm and Tucker decomposition based on the human image sequence to obtain tensor feature information; a depth map sequence is extracted based on the image sequence to obtain a target depth map sequence, and the target depth map sequence is cropped and normalized to obtain a normalized target depth map sequence; feature map extraction is performed on the normalized target depth map sequence to obtain a depth feature map; key point recognition is performed on the human image sequence based on a key point recognition model to obtain key point information; center of gravity trajectory analysis is performed based on the key point information to obtain center of gravity trajectory information, and a gait energy map is generated based on the human image sequence; gait recognition is performed based on the gait energy map, center of gravity trajectory information, depth feature map, and tensor feature information to obtain target gait data, providing more comprehensive and accurate data for gait recognition and avoiding excessive deviation between the obtained gait data and the actual gait.

[0019] S12: Based on the target gait data and brain-like semantic feature extraction, perform user action prediction to obtain user action prediction information; In the specific implementation of this invention, brain-like semantic features of gait behavior are extracted based on target gait data to obtain brain-like semantic feature information; by combining target gait data with image sequences, connection analysis of skeletal key points is performed to obtain corresponding connection data, and user action direction analysis is performed based on the connection data to obtain the target user action direction; user action prediction is performed based on brain-like semantic feature information and target user action direction, which can effectively improve the accuracy of user action prediction and provide more accurate data support for subsequent screen switching and scaling parameter analysis.

[0020] S13: Based on the audio information, perform human voice audio separation by combining non-negative matrix decomposition to obtain human voice audio information, and perform voiceprint feature analysis based on the human voice audio information to obtain voiceprint feature information; In the specific implementation of this invention, a discrete Fourier transform is performed based on audio information combined with a preset window function to obtain a time-frequency matrix; the time-frequency matrix is ​​decomposed based on non-negative matrix decomposition to obtain a basis matrix and an activation coefficient matrix, which are then combined with a classification algorithm to initially separate the audio information; the human voice audio information is determined based on the initially separated audio information and an audio analysis model; spectrograms are extracted from the human voice audio information to obtain spectrogram information for extracting frame-level feature information; phoneme feature analysis is performed based on the frame-level feature information to obtain phoneme feature information for voiceprint feature analysis, thereby improving the reliability of voiceprint feature extraction.

[0021] S14: Perform user identity detection based on the voiceprint feature information to obtain user identity information; In the specific implementation of this invention, historical voiceprint feature information and user personal information corresponding to the historical voiceprint feature information are acquired to construct a voiceprint information database; the cosine value of the angle between each historical voiceprint feature information and the voiceprint feature information in the voiceprint information database is calculated, and the cosine distance between each historical voiceprint feature information and the voiceprint feature information is calculated based on the cosine value of the angle; the voiceprint authentication data of each historical voiceprint feature information and the voiceprint feature information is determined based on the cosine distance, and the user identity information is determined based on the voiceprint authentication data, which ensures the reliability of user identity detection and can provide sufficient analysis data for subsequent screen control.

[0022] S15: Based on the user identity information and user action prediction information, perform screen scaling analysis and screen switching analysis to obtain screen scaling information and screen switching information, and adjust the screen display based on the screen scaling information and screen switching information.

[0023] In the specific implementation of this invention, a pointing screen number is generated based on user identity information and user action prediction information, and screen switching information is determined based on the pointing screen number; the screen focus object is determined based on user action prediction information and user identity information, and the screen adjustment amount of the screen focus object is determined based on the approximation algorithm. The screen adjustment amount includes vertical tilt adjustment information, horizontal displacement adjustment information and vertical displacement adjustment information, and screen scaling information is determined based on the screen adjustment amount, which can better track the user screen and make the screen adjustment more in line with the actual scene.

[0024] In this embodiment of the invention, depth feature maps and tensor features are extracted based on image sequences to obtain depth feature map and tensor feature information for gait recognition. User action prediction is then performed based on target gait data, effectively improving the accuracy of user action prediction. Voiceprint feature analysis is performed based on audio information combined with non-negative matrix factorization, and user identity detection is performed based on voiceprint feature information, improving the accuracy of voiceprint feature extraction and ensuring the reliability of user identity detection. Screen scaling and switching analysis are performed based on user identity information and user action prediction information to adjust the screen display, enabling real-time tracking of the user's screen, avoiding omissions of user screen views, and achieving a more ideal screen control effect.

[0025] Example 2 Please see Figure 2 , Figure 2 This is a flowchart illustrating an intelligent screen control method according to another embodiment of the present invention, the method comprising: S201: Collect user image sequences and audio information based on sensor combination, extract human image sequences based on the image sequences, and extract tensor features based on the human image sequences using genetic algorithm and Tucker decomposition to obtain tensor feature information; In the specific implementation of this invention, image sequences and audio information of users are collected through a sensor combination, including image sensors and audio sensors. Users include speakers and hosts, etc. Human images are segmented from the image sequence using background subtraction and dynamic thresholding techniques to form a human image sequence. A genetic algorithm, a metaheuristic algorithm, typically relies on biologically inspired operators such as mutation, crossover, and selection to generate high-quality solutions to optimization and search problems. A support vector machine model is constructed using Tucker decomposition, a form of tensor decomposition that decomposes a high-order tensor into a core tensor and the product of multiple factor matrices. The genetic algorithm, combined with the support vector machine model constructed from Tucker decomposition, extracts tensor feature information with classification functions from the human image sequence.

[0026] S202: Extract depth map sequence based on the image sequence to obtain target depth map sequence, and perform cropping and normalization processing on the target depth map sequence to obtain normalized target depth map sequence; In the specific implementation of this invention, a depth estimation model is used to extract depth map sequences from the image sequence to obtain a target depth map sequence. The depth estimation model can be a Depth Anything model. The target depth map sequence is then cropped and normalized. Background cropping is performed on the target depth map sequence to obtain a background-cropped target depth map sequence. The background-cropped target depth map sequence is then normalized, using batch normalization and group normalization methods to obtain a normalized target depth map sequence.

[0027] S203: Extract feature maps from the normalized target depth map sequence to obtain depth feature maps; In the specific implementation of this invention, the normalized target depth map sequence is input into the depth map feature extractor for feature map extraction to obtain a depth feature map. The depth map feature extractor is a convergent model obtained by inputting the sample dataset into a deep neural network for training.

[0028] S204: Perform key point recognition on the human image sequence based on the key point recognition model to obtain key point information; In the specific implementation of this invention, the human image sequence is input into the key point recognition model to identify key points and obtain key point information, including human skeletal key points, etc. The key point recognition model is a convergent model obtained by inputting the sample dataset into a deep neural network for training.

[0029] S205: Perform center of gravity trajectory analysis based on the key point information to obtain center of gravity trajectory information, and generate a gait energy map based on the human image sequence; In a specific implementation of the present invention, generating a gait energy map based on the human image sequence includes: performing gait sequence image analysis based on the human image sequence and a preset recursive algorithm to obtain a target gait image sequence; determining the time matching data of the target gait image sequence; and generating a gait energy map based on the target gait image sequence using the time matching data and a minimum cumulative distance algorithm.

[0030] Specifically, based on the key point information, center of gravity trajectory analysis is performed. The key point information is scaled to the same coordinate standard, and the center of gravity coordinates are obtained from the key point information according to the preset center of gravity trajectory. The center of gravity trajectory curve is formed by combining the time frames of the human image sequence with the center of gravity coordinates, thus obtaining the center of gravity trajectory information. Based on the human image sequence, gait sequence image analysis is performed using a preset recursive algorithm. The preset recursive algorithm is used to determine the images in each gait time sequence within the image. By analyzing the gait images in the time sequence of the human image sequence using the preset recursive algorithm, the target gait image sequence is obtained. Time matching data for the target gait image sequence is determined; this time matching data is the time data matched to each image in the gait image sequence. Based on the target gait image sequence, a gait energy map is generated using the time matching data and a minimum cumulative distance algorithm. The time path of the gait image sequence is determined using the minimum cumulative distance algorithm combined with the time matching data. The target gait image sequence is then time-aligned according to the determined time path to form the gait energy map.

[0031] S206: Based on the gait energy map, center of gravity trajectory information, depth feature map and tensor feature information, perform gait recognition to obtain target gait data; In the specific implementation of this invention, gait energy map, center of gravity trajectory information, depth feature map and tensor feature information are input into a preset deep learning model for gait recognition to obtain target gait data.

[0032] S207: Based on the target gait data and combined with brain-like semantic feature extraction, perform user action prediction to obtain user action prediction information; In a specific implementation of this invention, the step of predicting user actions based on the target gait data combined with brain-like semantic feature extraction to obtain user action prediction information includes: extracting brain-like semantic features of gait behavior based on the target gait data to obtain brain-like semantic feature information; performing connection analysis of skeletal key points based on the target gait data and the image sequence to obtain corresponding connection data, and performing user action direction analysis based on the connection data to obtain the target user action direction; and predicting user actions based on the brain-like semantic feature information and the target user action direction to obtain user action prediction information.

[0033] Specifically, based on the target gait data, brain-like semantic features of gait behavior are extracted. This extraction involves analyzing and understanding the semantic features of gait behavior that simulate the hierarchical structure of the human brain. The target gait data is input into a preset model for brain-like semantic feature extraction. This preset model can be a deep learning model. This model selects and extracts features from the target gait data, retaining brain-like semantic features with high weights during feature extraction, thus obtaining brain-like semantic feature information. Based on the target gait data and the image sequence, skeletal keypoint connection analysis is performed. Specifically, based on the target gait data in each frame of the image sequence, corresponding skeletal keypoints are connected according to the image frame time to obtain the corresponding connection data. Based on this connection data, user movement direction analysis is performed. The user's torso direction relative to the horizontal plane and relative to the vertical plane are determined based on the connection data between the skeletal keypoints. The user's forward direction is then determined based on these directions, thus obtaining the target user's movement direction.

[0034] User action prediction is performed based on the aforementioned brain-like semantic feature information and the target user's action direction. This information is then input into a Long Short-Term Memory (LSTM) network for further action prediction. The LSM network can perform time-series action prediction based on the relevant data, thus obtaining user action prediction information. This effectively improves the accuracy of user action prediction, providing more accurate data support for subsequent analysis of screen switching and scaling parameters.

[0035] S208: Based on the audio information, perform human voice audio separation by combining non-negative matrix decomposition to obtain human voice audio information, and perform voiceprint feature analysis based on the human voice audio information to obtain voiceprint feature information; In the specific implementation of this invention, the step of separating human voice audio based on the audio information combined with non-negative matrix decomposition to obtain human voice audio information, and performing voiceprint feature analysis based on the human voice audio information to obtain voiceprint feature information, includes: performing discrete Fourier transform based on the audio information combined with a preset window function to obtain a time-frequency matrix; decomposing the time-frequency matrix based on non-negative matrix decomposition to obtain a basis matrix and an activation coefficient matrix, and performing initial separation of the audio information based on the basis matrix and activation coefficient matrix combined with a classification algorithm to obtain initially separated audio information; determining human voice audio information based on the initially separated audio information combined with an audio analysis model; extracting spectrograms from the human voice audio information to obtain spectrogram information, and extracting frame-level feature information based on the spectrogram information; performing phoneme feature analysis based on the frame-level feature information to obtain phoneme feature information; and performing voiceprint feature analysis based on the phoneme feature information to obtain voiceprint feature information.

[0036] Specifically, based on the audio information, a Discrete Fourier Transform is performed using a preset window function to encode the audio information, obtaining encoded audio information. The encoded audio information is then processed into frames to obtain frame-processed audio information. The frame-processed audio information is then windowed using a preset window function to obtain windowed audio information. Finally, a Discrete Fourier Transform is performed on the windowed audio information to obtain the spectrum corresponding to the audio information in each time window. A matrix is ​​then constructed based on the spectrum corresponding to the audio information in each time window to obtain the time-frequency matrix.

[0037] The time-frequency matrix is ​​decomposed based on nonnegative matrix factorization (NMF), that is, the video matrix is ​​decomposed using a nonnegative matrix factorization algorithm. NMF is a matrix factorization technique that can decompose a nonnegative matrix into the product of two nonnegative matrices to obtain a basis matrix and an activation coefficient matrix. Based on the basis matrix and activation coefficient matrix, a classification algorithm is used to initially separate the audio information. Basis vectors and activation coefficient vectors are extracted from the basis matrix and activation coefficient matrix. Based on the basis vectors and activation coefficient vectors, the audio information is initially separated into environmental sound category information and human voice category information, that is, the initially separated audio information is obtained.

[0038] Based on the initially separated audio information and combined with an audio analysis model, human voice audio information is determined. The initially separated audio information is input into the audio analysis model to determine the final audio information belonging to human voice, which is the human voice audio information. Spectrogram extraction is performed on the human voice audio information to obtain spectrogram information. Frame-level feature information is then extracted based on the spectrogram information, i.e., spectrogram feature extraction is performed on the spectrogram information to obtain several spectrogram feature information. These several spectrogram feature information are then integrated to obtain frame-level feature information.

[0039] Phoneme feature analysis is performed based on the frame-level feature information to obtain the attention weights of the frame-level feature information. The feature mean and feature variance are determined based on the frame-level feature information and its attention weights. Phoneme features are then determined based on the feature mean and feature variance, thus obtaining the phoneme feature information. Voiceprint feature analysis is then performed based on the phoneme feature information. The phoneme feature information is input into a neural network for voiceprint feature recognition and extraction, thereby obtaining voiceprint feature information and improving the reliability of voiceprint feature extraction.

[0040] S209: Perform user identity detection based on the voiceprint feature information to obtain user identity information; In a specific implementation of this invention, the step of detecting user identity based on the voiceprint feature information to obtain user identity information includes: acquiring historical voiceprint feature information and user personal information corresponding to the historical voiceprint feature information, and constructing a voiceprint information database based on the historical voiceprint feature information and user personal information; calculating the cosine value of the angle between each historical voiceprint feature information and the voiceprint feature information in the voiceprint information database, and calculating the cosine distance between each historical voiceprint feature information and the voiceprint feature information based on the cosine value; determining the voiceprint authentication data of each historical voiceprint feature information and the voiceprint feature information based on the cosine distance, and determining the user identity information based on the voiceprint authentication data.

[0041] Specifically, historical voiceprint feature information and user personal information corresponding to the historical voiceprint feature information are obtained, and a voiceprint information database is constructed based on the historical voiceprint feature information and user personal information. That is, the historical voiceprint feature information and user personal information are input into a preset database to form a voiceprint information database.

[0042] Calculate the cosine value of the angle between each historical voiceprint feature and the voiceprint feature in the voiceprint information database. Vectorize each historical voiceprint feature and the voiceprint feature to obtain vectorized historical voiceprint feature and vectorized voiceprint feature. Calculate the cosine value of the angle between the vectorized historical voiceprint feature and the vectorized voiceprint feature. Calculate the cosine distance between each historical voiceprint feature and the voiceprint feature based on the cosine value of the angle. The cosine distance can be calculated using the cosine distance calculation formula.

[0043] Based on the cosine distance, the voiceprint authentication data of each historical voiceprint feature information is determined. The corresponding weight coefficient is determined according to the cosine distance. The voiceprint authentication value of each is calculated according to the cosine distance and its corresponding weight coefficient, which is the voiceprint authentication data. The user identity information is determined based on the voiceprint authentication data. The user personal information corresponding to the voiceprint feature information with the highest voiceprint authentication value is used as the user identity information, which ensures the reliability of user identity detection and can provide sufficient analysis data for subsequent screen control.

[0044] S210: Based on the user identity information and user action prediction information, perform screen scaling analysis and screen switching analysis on the display screen to obtain screen scaling information and screen switching information, and adjust the screen display based on the screen scaling information and screen switching information.

[0045] In a specific implementation of this invention, the step of performing screen scaling analysis and screen switching analysis based on the user identity information and user action prediction information to obtain screen scaling information and screen switching information includes: generating a pointing screen number based on the user identity information and user action prediction information, and determining screen switching information based on the pointing screen number; determining the screen focus object based on the user action prediction information and user identity information, and determining the screen adjustment amount of the screen focus object based on an approximation algorithm, wherein the screen adjustment amount includes vertical tilt adjustment information, horizontal displacement adjustment information, and vertical displacement adjustment information, and determining screen scaling information based on the screen adjustment amount.

[0046] Specifically, a pointing screen number is generated based on the user identity information and user action prediction information. This means that the speaker's identity is determined based on the user identity information, and the screen number the speaker will point to in the future is determined based on the user action prediction information. For example, if the speaker is a host, the screen number must include both the host and the invited user. Screen switching information is determined based on the pointing screen number, and screen switching parameters, such as the switching time and camera angle, are determined based on the pointing screen code. The screen focus object is determined based on the user action prediction information and user identity information, and the screen adjustment amount for the focus object is determined based on an approximation algorithm. The approximation algorithm is used to progressively approximate the screen adjustment amount, making the determined adjustment amount more accurate. The screen adjustment amount includes vertical tilt adjustment information, horizontal displacement adjustment information, and vertical displacement adjustment information. Screen scaling information is determined based on the screen scaling information and screen switching information. The screen display is adjusted based on the screen scaling information and screen switching information. The camera movement and scaling are determined using the screen scaling information and screen switching information, thereby adjusting and controlling the displayed screen.

[0047] In this embodiment of the invention, depth feature maps and tensor features are extracted based on image sequences to obtain depth feature map and tensor feature information for gait recognition. User action prediction is then performed based on target gait data, effectively improving the accuracy of user action prediction. Voiceprint feature analysis is performed based on audio information combined with non-negative matrix factorization, and user identity detection is performed based on voiceprint feature information, improving the accuracy of voiceprint feature extraction and ensuring the reliability of user identity detection. Screen scaling and switching analysis are performed based on user identity information and user action prediction information to adjust the screen display, enabling real-time tracking of the user's screen, avoiding omissions of user screen views, and achieving a more ideal screen control effect.

[0048] Example 3 Please see Figure 3 , Figure 3This is a schematic diagram of the structural composition of the intelligent screen control device in an embodiment of the present invention. The device includes: Gait recognition module 31: used to collect image sequences and audio information of users based on sensor combination, and extract depth feature maps and tensor features based on the image sequences to obtain depth feature map and tensor feature information, and perform gait recognition based on the depth feature map and tensor feature information combined with the center of gravity trajectory to obtain target gait data; Action prediction module 32: used to predict user actions based on the target gait data and brain-like semantic feature extraction, and obtain user action prediction information; Voiceprint feature analysis module 33: used to perform human voice audio separation based on the audio information combined with non-negative matrix decomposition to obtain human voice audio information, and to perform voiceprint feature analysis based on the human voice audio information to obtain voiceprint feature information; User identity detection module 34: used to perform user identity detection based on the voiceprint feature information and obtain user identity information; The screen control module 35 is used to perform screen scaling analysis and screen switching analysis on the display screen based on the user identity information and user action prediction information, obtain screen scaling information and screen switching information on the display screen, and adjust the screen display screen based on the screen scaling information and screen switching information.

[0049] In the specific implementation of this invention, the specific implementation of the device item can be referred to the implementation of the method item above, and will not be repeated here.

[0050] In this embodiment of the invention, depth feature maps and tensor features are extracted based on image sequences to obtain depth feature map and tensor feature information for gait recognition. User action prediction is then performed based on target gait data, effectively improving the accuracy of user action prediction. Voiceprint feature analysis is performed based on audio information combined with non-negative matrix factorization, and user identity detection is performed based on voiceprint feature information, improving the accuracy of voiceprint feature extraction and ensuring the reliability of user identity detection. Screen scaling and switching analysis are performed based on user identity information and user action prediction information to adjust the screen display, enabling real-time tracking of the user's screen, avoiding omissions of user screen views, and achieving a more ideal screen control effect.

[0051] This invention provides a computer-readable storage medium storing a computer program. When executed by a processor, this program implements the intelligent screen control method of any of the above embodiments. The computer-readable storage medium includes, but is not limited to, any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic cards, or optical cards. In other words, the storage device includes any medium that stores or transmits information in a readable form by a device (e.g., a computer, a mobile phone), and can be a read-only memory, a disk, or an optical disk, etc.

[0052] Example 4 Please see Figure 4 , Figure 4 This is a schematic diagram of the structural composition of the electronic device in an embodiment of the present invention.

[0053] This invention also provides an electronic device, such as... Figure 4 As shown, the electronic device includes a memory 41, a processor 43, and a computer program 42 stored in the memory 41 and executable on the processor 43. Those skilled in the art will understand that... Figure 4The illustrated electronic device does not constitute a limitation on all devices and may include more or fewer components than illustrated, or combine certain components. Memory 41 can be used to store computer program 42 and various functional modules. Processor 43 runs the computer program 42 stored in memory 41, thereby performing various functional applications and data processing of the device. Memory can be internal memory or external memory, or both. Internal memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, or random access memory. External memory may include hard disks, floppy disks, ZIP disks, USB flash drives, magnetic tapes, etc. Processor 43 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, a single-chip microcomputer, or a processor 43, or any conventional processor, etc. The processors and memories disclosed in this invention include, but are not limited to, these types of processors and memories. The processors and memories disclosed in this invention are merely examples and not intended to be limiting.

[0054] As one embodiment, the electronic device includes: one or more processors 43, a memory 41, and one or more computer programs 42, wherein the one or more computer programs 42 are stored in the memory 41 and configured to be executed by the one or more processors 43, and the one or more computer programs 42 are configured to execute the intelligent screen control method in any of the above embodiments. For specific implementation processes, please refer to the above embodiments, which will not be repeated here.

[0055] In this embodiment of the invention, depth feature maps and tensor features are extracted based on image sequences to obtain depth feature map and tensor feature information for gait recognition. User action prediction is then performed based on target gait data, effectively improving the accuracy of user action prediction. Voiceprint feature analysis is performed based on audio information combined with non-negative matrix factorization, and user identity detection is performed based on voiceprint feature information, improving the accuracy of voiceprint feature extraction and ensuring the reliability of user identity detection. Screen scaling and switching analysis are performed based on user identity information and user action prediction information to adjust the screen display, enabling real-time tracking of the user's screen, avoiding omissions of user screen views, and achieving a more ideal screen control effect.

[0056] Furthermore, the above provides a detailed description of the intelligent screen control method and related devices provided by the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A picture intelligent control method, characterized in that, The method comprises: Based on the sensor combination, the image sequence and the audio information of the user are collected, the deep feature map and the tensor feature extraction are carried out based on the image sequence, the deep feature map and the tensor feature information are obtained, the gait recognition is carried out based on the deep feature map and the tensor feature information combined with the center of gravity trajectory, and the target gait data is obtained; Based on the target gait data combined with the brain-like semantic feature extraction, the user action prediction is carried out, and the user action prediction information is obtained; Based on the audio information combined with the non-negative matrix decomposition, the human voice audio separation is carried out, the human voice audio information is obtained, and the voiceprint feature analysis is carried out based on the human voice audio information, and the voiceprint feature information is obtained; Based on the voiceprint feature information, the user identity detection is carried out, and the user identity information is obtained; Based on the user identity information and the user action prediction information, the picture zoom analysis and the picture switching analysis of the display screen are carried out, the picture zoom information and the picture switching information of the display screen are obtained, and the picture of the display screen is adjusted based on the picture zoom information and the picture switching information.

2. The picture intelligent control method of claim 1, wherein, The deep feature map and the tensor feature extraction based on the image sequence are carried out, the deep feature map and the tensor feature information are obtained, and the gait recognition based on the deep feature map and the tensor feature information combined with the center of gravity trajectory is carried out, and the target gait data is obtained, comprising: Based on the image sequence, the human body image sequence is extracted, and the tensor feature extraction is carried out based on the human body image sequence by using the genetic algorithm and the Tucker decomposition, and the tensor feature information is obtained; Based on the image sequence, the depth map sequence extraction is carried out, the target depth map sequence is obtained, and the target depth map sequence after normalization processing is obtained by carrying out the cutting and normalization processing on the target depth map sequence; The feature map extraction is carried out on the target depth map sequence after normalization processing, and the deep feature map is obtained; Based on the key point recognition model, the key point recognition is carried out on the human body image sequence, and the key point information is obtained; Based on the key point information, the center of gravity trajectory analysis is carried out, the center of gravity trajectory information is obtained, and the gait energy map is generated based on the human body image sequence; Based on the gait energy map, the center of gravity trajectory information, the deep feature map and the tensor feature information, the gait recognition is carried out, and the target gait data is obtained.

3. The picture intelligent control method of claim 2, wherein, The gait energy map generated based on the human body image sequence comprises: Based on the human body image sequence combined with the preset recursive algorithm, the gait sequence image analysis is carried out, and the target gait image sequence is obtained; The time matching data of the target gait image sequence is determined; Based on the target gait image sequence, the time matching data combined with the minimum cumulative distance algorithm is used to generate the gait energy map.

4. The method of claim 1, wherein, The user action prediction based on the target gait data combined with the brain-like semantic feature extraction is carried out, and the user action prediction information is obtained, comprising: Based on the target gait data, the brain-like semantic feature extraction of the gait behavior is carried out, and the brain-like semantic feature information is obtained; Based on the target gait data combined with the image sequence, the connection line analysis of the skeletal key points is carried out, the corresponding connection line data is obtained, and the user action direction analysis is carried out based on the connection line data, and the target user action direction is obtained; The user action prediction information is obtained based on the brain-like semantic feature information and a target user action direction.

5. The method of claim 1, wherein, The voice audio information is obtained based on the audio information and a preset window function, and the voice audio information is used for voiceprint feature analysis to obtain voiceprint feature information, including: The time-frequency matrix is obtained based on the audio information and a preset window function. The time-frequency matrix is decomposed based on the non-negative matrix factorization to obtain a basis matrix and an activation coefficient matrix, and the audio information is initially separated based on the basis matrix and the activation coefficient matrix and a classification algorithm to obtain the initially separated audio information. The voice audio information is determined based on the initially separated audio information and an audio analysis model. The spectrogram information is obtained by performing spectrogram extraction on the voice audio information, and frame-level feature information is extracted based on the spectrogram information. The phoneme feature information is obtained based on the frame-level feature information. The voiceprint feature information is obtained based on the phoneme feature information.

6. The method of claim 1, wherein, The user identity information is obtained based on the voiceprint feature information, including: The historical voiceprint feature information and user personal information corresponding to the historical voiceprint feature information are obtained, and a voiceprint information database is constructed based on the historical voiceprint feature information and the user personal information. The cosine values of the angles between the historical voiceprint feature information in the voiceprint information database and the voiceprint feature information are calculated, and the cosine distances between the historical voiceprint feature information and the voiceprint feature information are calculated based on the cosine values. The voiceprint authentication data of the historical voiceprint feature information and the voiceprint feature information is determined based on the cosine distances, and the user identity information is determined based on the voiceprint authentication data.

7. The method of claim 1, wherein, The picture scaling information and the picture switching information of the display screen are obtained based on the user identity information and the user action prediction information, including: The picture number is generated based on the user identity information and the user action prediction information, and the picture switching information is determined based on the picture number. The picture attention object is determined based on the user action prediction information and the user identity information, and the picture adjustment amount of the picture attention object is determined based on an approximation algorithm, the picture adjustment amount including vertical slope adjustment information, horizontal displacement adjustment information and vertical displacement adjustment information, and the picture scaling information is determined based on the picture adjustment amount.

8. An intelligent picture control device, characterized by The device includes: The gait recognition module is configured to acquire an image sequence and audio information of a user based on a sensor combination, extract a depth feature map and tensor feature based on the image sequence to obtain depth feature map and tensor feature information, and perform gait recognition based on the depth feature map and the tensor feature information and a center of gravity trajectory to obtain target gait data. The action prediction module is configured to perform user action prediction based on the target gait data and brain-like semantic feature extraction to obtain user action prediction information. The voiceprint feature analysis module is configured to perform human voice separation based on the audio information and non-negative matrix decomposition, obtain human voice information, and perform voiceprint feature analysis based on the human voice information to obtain voiceprint feature information. The user identity detection module is configured to perform user identity detection based on the voiceprint feature information to obtain user identity information. The picture control module is configured to perform picture zooming analysis and picture switching analysis of the display screen based on the user identity information and user action prediction information, to obtain picture zooming information and picture switching information of the display screen, and to adjust the picture of the display screen based on the picture zooming information and the picture switching information. 9.An electronic device comprising a processor and a memory, wherein The memory is configured to store instructions, and the processor is configured to invoke the instructions in the memory, so that the electronic device performs the picture intelligent control method in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, which, when executed on an electronic device, cause the electronic device to perform the picture intelligent control method in any one of claims 1 to 7.

Images