Digital human driving method and system for museum interaction in combination with large model technology

Abstract

The invention relates to the technical field of man-machine interaction, in particular to a digital human-driven method and system for museum interaction in combination with a large model technology, and the method comprises the following steps: analyzing a face video stream of a visitor, extracting image frames, and carrying out the graying processing and normalization of each frame of image, and obtaining a standardized face image; and inputting the standardized facial image into a convolutional neural network, analyzing image features, identifying expression changes, and generating a basic emotional state index. According to the invention, by analyzing the face video stream of the visitor and extracting the image frame, the expression change is captured, and the digital person is allowed to adjust the interaction mode according to the instant emotion of the visitor. The capability of adjusting intonation and speed in real time improves the naturalness and effectiveness of interaction, and brings more personalized experience to visitors. And by continuously monitoring the load of the server and the use condition of the memory, the resources can be dynamically adjusted, the visitor request can be efficiently processed, and the overload of the server can be prevented.

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction technology, and in particular to a digital human driving method and system for museum interaction combined with large model technology. Background Art

[0002] The digital human-driven approach for museum interaction is a technology that uses virtual agents or digital characters to interact with visitors in a museum environment. This approach uses animated characters or virtual assistants to deliver information, guide explanations, and provide educational content, thereby enhancing visitors' sense of participation and learning experience.

[0003] However, existing technologies lack real-time sentiment analysis capabilities, and digital humans cannot adjust their behavior to better match visitors' actual feelings, which may lead to a disconnect in visitor experience and affect engagement and satisfaction. In addition, fixed resource allocation strategies are not adaptable enough when faced with unforeseen visitor traffic, and are prone to resource waste or shortages, affecting performance and visitor experience. Therefore, improvements are needed. Summary of the invention

[0004] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a digital human driving method and system for museum interaction combined with large model technology.

[0005] In order to achieve the above-mentioned purpose, the present invention adopts the following technical results, and the digital human driving method for museum interaction combined with large model technology includes the following steps: Analyze the visitor's facial video stream, extract image frames, and grayscale and normalize each frame to obtain a standardized facial image; input the standardized facial image into a convolutional neural network, analyze image features and identify expression changes, and generate basic emotional state indicators; Based on the basic emotional state indicator, the tone and speed of the digital human voice response are adjusted to generate adjusted voice features; and the adjusted voice features are applied to generate optimized interactive content; Continuously monitor server load and visitor interaction events, collect processor usage and memory usage, and generate system performance monitoring data; based on the system performance monitoring data, evaluate the current resource usage of the server, calculate the number of CPU cores and memory that need to be increased or decreased, and generate resource adjustment results; According to the resource adjustment result, the resource allocation of the server is adjusted, and the processor usage and memory usage are continuously monitored, and it is determined whether to adjust the resources again according to the processor usage and memory usage.

[0006] Preferably, the steps of acquiring the standardized facial image are: Analyze the visitor's facial video stream, extract continuous image frames, and generate an original image frame set; Based on the original image frame set, each frame of the image is grayed out, and the brightness value after graying is calculated. The calculation formula is: ; in, Indicates that the image is at position The red, green, and blue components of is the grayscale brightness value of the position; Based on the grayscale brightness value, normalization processing is performed on each image frame to generate a standardized facial image.

[0007] Preferably, the steps of obtaining the basic emotional state indicator are: Training a convolutional neural network model, inputting the standardized facial image into the convolutional neural network model, and the convolutional neural network model detects key facial features through a multi-layer filter to obtain facial feature data; Based on the facial feature data, the convolutional neural network model continues to analyze the expression changes in the image, including the fine adjustment of eyebrows and the arc of the smile, to obtain the expression recognition result; Based on the expression recognition results, a classifier is used to correspond different expressions to emotional states of happiness, sadness and surprise to obtain a basic emotional state index.

[0008] Preferably, the steps of acquiring the adjusted voice features are: Extracting emotional values ​​from the basic emotional state indicators, including happiness and sadness, to obtain emotional analysis data; Based on the sentiment analysis data, the intonation of the digital human voice response is calculated, and the intonation calculation formula is: ; Where E represents the pleasure extracted from the sentiment analysis data, are the coefficients for adjusting intonation, and P is the adjusted intonation; The emotion analysis data is used to calculate the speed of the digital human's voice response, and the intonation and speed are summarized to obtain the adjusted voice features. The speed calculation formula is: ; Here, E also represents the pleasure extracted from the sentiment analysis data. is the coefficient for adjusting the speed, and V is the adjusted speed.

[0009] Preferably, the steps of acquiring the system performance monitoring data are: Continuously monitor the server, track server load and visitor interaction events, regularly collect server CPU and memory usage data, and obtain real-time monitoring data; Based on the real-time monitoring data, analyze the processor usage and memory usage, identify performance bottlenecks or abnormal usage patterns, and obtain performance analysis results; Based on the performance analysis results, the data is integrated and formatted to obtain system performance monitoring data.

[0010] Preferably, the steps of obtaining the resource adjustment result are: Extracting the real-time utilization rate of CPU and memory, as well as the peak utilization rate and idle rate from the system performance monitoring data to obtain comprehensive resource usage status data; Based on the comprehensive resource usage data, the number of CPU cores and memory required to be adjusted are calculated using the formula: ; and ; in, and They are the current CPU and memory usage, and is the target utilization rate, is the standard deviation of CPU usage, and They are the number of CPU cores and amount of memory adjusted respectively; Based on the adjusted number of CPU cores and memory amount, a resource adjustment result is formed.

[0011] Preferably, according to the resource adjustment result, the resource allocation of the server is adjusted, and the processor usage and memory usage are continuously monitored. The steps of determining whether to adjust the resources again according to the processor usage and memory usage are as follows: According to the resource adjustment result, the number of CPU cores and memory are increased or decreased to obtain an updated server configuration; Based on the updated server configuration, continuously track processor usage and memory usage to obtain real-time monitoring results; Based on the real-time monitoring results, the current processor utilization and memory usage are analyzed to evaluate whether the predetermined performance goals are met and whether resources need to be adjusted again.

[0012] The present invention provides a digital human driving system, comprising: The expression recognition module captures the visitor's facial video stream, selects key frames, grayscales and normalizes the image, and generates a standardized facial image; The emotion analysis module inputs the standardized facial images into the convolutional neural network, analyzes the image features, identifies expression changes, and generates basic emotional state indicators; The voice adjustment module adjusts the tone and speed of the digital human voice according to the basic emotional state indicators and generates the adjusted voice features; The content generation module applies the adjusted voice features to create interactive content that matches the visitor's emotions and generates optimized interactive content; Resource management module, monitors server load and visitor interaction data, collects processor usage and memory data, generates system performance monitoring data, evaluates resource usage, and calculates CPU and memory adjustment requirements for adjustment.

[0013] Compared with the prior art, the advantages and positive effects of the present invention are: The present invention captures changes in facial expressions by analyzing the visitor's facial video stream and extracting image frames, allowing the digital human to adjust the interaction mode according to the visitor's immediate emotions. This ability to adjust the tone and speed in real time improves the naturalness and effectiveness of the interaction and brings a more personalized experience to the visitor. By continuously monitoring the server load and memory usage, resources can be dynamically adjusted to ensure efficient processing of visitor requests and prevent server overload. Real-time optimization of resources not only ensures the stable operation of the server, but also improves energy and cost efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram of the steps of the present invention. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical results and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0016] See also Figure 1 The present invention provides a technical result, a digital human driving method for museum interaction combined with large model technology, comprising the following steps: Analyze the visitor's facial video stream, extract image frames, and grayscale and normalize each frame to obtain a standardized facial image; input the standardized facial image into a convolutional neural network, analyze image features and identify expression changes to generate basic emotional state indicators; Based on the basic emotional state indicators, the tone and speed of the digital human's voice response are adjusted to generate adjusted voice features; the adjusted voice features are applied to generate optimized interactive content; Continuously monitor server load and visitor interaction events, collect processor utilization and memory usage, and generate system performance monitoring data; based on the system performance monitoring data, evaluate the current resource usage of the server, calculate the number of CPU cores and memory that need to be increased or decreased, and generate resource adjustment results.

[0017] According to the resource adjustment results, adjust the resource allocation of the server, and continuously monitor the processor usage and memory usage. Determine whether to adjust the resources again based on the processor usage and memory usage.

[0018] The steps to obtain a standardized facial image are: Analyze the visitor's facial video stream, extract continuous image frames, and generate an original image frame set; Based on the original image frame set, each frame of the image is grayed out and the brightness value after graying is calculated. The calculation formula is: ; in, Indicates that the image is at position The red, green, and blue components of is the grayscale brightness value of the position; Based on the grayscale brightness value, normalization is performed on each image frame to generate a standardized facial image.

[0019] Specifically, a digital camera with a CCD image sensor and a fixed focal length lens is placed in the museum room to obtain continuous video data of visitors' faces. The analog video signal output by the camera is connected to the data acquisition module for digital conversion and cache processing. The RGB format image frame is extracted frame by frame from the data stream using the H.264 decoder, and the pixel data of each frame is stored in a random access memory in row and column order. The image frame rate is determined to be 30 frames per second by counting the number of decoded frames and referring to the internal recorded timestamp. The pixel value array is read frame by frame from the decoded data, and the clarity and brightness of each frame of the image are quantitatively evaluated. The brightness parameters are given by a contrast threshold of 100 and an average gradient threshold of 5 obtained in advance through laboratory measurements. These thresholds are determined by statistically analyzing facial images of multiple visitors under the same illumination conditions and calculating the pixel intensity contrast distribution and the local edge gradient mean. Frames with a contrast lower than 100 or an average gradient lower than 5 are directly discarded. For frames with obviously overexposed brightness, an overexposure threshold of 220 is obtained through actual measured data of an on-site illuminance meter as a judgment standard. Frames exceeding 220 are removed from the queue, and all frames in the acquired video stream are processed continuously. Image frames that meet the above quality standards are collected as a set of continuous original image frames and stored in the memory.

[0020] The benefit of the formula is that by weighting the squares of the RGB components and using a specific proportional coefficient, the calculated brightness value is closer to the human eye's perception of brightness changes, so that when subsequent image processing (such as normalization) is performed based on brightness, a more balanced and stable brightness distribution result can be obtained; The steps for obtaining R(i, j) are to capture the visitor's facial video frame using a high-sensitivity CCD camera under the indoor lighting condition of about 300 lux in the museum exhibition hall, read the red component of the pixel (i=200, j=300) from the corresponding frame, and obtain R(i=200, j=300)=185 after digital processing and multiple statistics; The acquisition step of G(i, j) is to read the green component of the corresponding pixel in the same frame under the same conditions. After multiple measurements, the stable value is G(i=200, j=300)=140; The steps for obtaining B(i, j) are similar. The blue component is extracted from the same pixel position, and B(i=200, j=300)=120 is measured. 0.299, 0.587, 0.114 are fixed values ​​from RGB to brightness conversion parameters of ITU-R BT.601 standard; Calculation process: First calculate the square value: , , ; Molecular computing: , , ; Add the parts of the numerator: ; Denominator calculation: ; Final calculation ; The result shows that the pixel is in the medium-low value area within the brightness range of 0 to 255. If the result value exceeds 128, the pixel is brighter, and below 50 it is darker. The result of 52.54 is close to the lower-middle brightness level, providing reliable brightness reference data for the subsequent image normalization step.

[0021] The brightness data of all pixels in each image frame are read from the grayscale brightness value obtained in the previous step, and these brightness values ​​are stored in a one-dimensional array structure. The brightness values ​​of all pixels in the array are compared in a linear traversal manner, and the maximum brightness value is recorded as Lm and the minimum brightness value is recorded as Ln. These values ​​are obtained by statistically analyzing multiple frame data under the same lighting conditions, and are generally distributed in the range of 0 to 255. Brightness values ​​exceeding 255 are truncated to 255, and brightness values ​​less than 0 are directly set to 0. The obtained Lm and Ln are normalized by the calculation formula The brightness of each pixel in the frame is calculated, and the calculated normalized brightness value is directly written into the data block of the corresponding pixel position. If individual abnormal brightness values ​​exceed the reference upper and lower limits, a limiting operation is required. The upper and lower limit values ​​are determined by statistically analyzing the facial images of multiple visitors and calculating the average and standard deviation of the brightness of each image. For example, if the upper and lower limits are 5 and 250 respectively, the brightness value exceeding 250 is reset to 250, and that less than 5 is reset to 5. After all pixels are processed, the normalized and evenly distributed brightness array data is obtained, and the array data is stored in the memory in the form of a standardized facial image.

[0022] The steps to obtain the basic emotional state indicators are: Training the convolutional neural network model, inputting the standardized facial image into the convolutional neural network model, and the convolutional neural network model detects the key features of the face through multi-layer filters to obtain facial feature data; Based on the facial feature data, the convolutional neural network model continues to analyze the expression changes in the image, including the fine adjustment of eyebrows and the arc of the smile, to obtain the expression recognition results; Based on the expression recognition results, the classifier is used to correspond different expressions to the emotional states of happiness, sadness and surprise to obtain the basic emotional state indicators.

[0023] Specifically, image samples are collected as training inputs. First, these image data are divided into training set and validation set according to a fixed ratio. The learning rate of 0.001 and a total of 30,000 iterations, which have been determined in the experimental stage according to the distribution characteristics of multiple groups of facial image features, are used for training. The images are converted into tensor form with a fixed resolution and bit depth and loaded into the video memory. The determined convolution kernel size of 3×3 and the number of convolution kernels of 64 are used as basic parameters. During the training process, forward propagation calculations are performed on each batch of data, and feature maps are recorded in each intermediate layer to identify local facial texture information. If the accuracy obtained on the validation set is low, Based on the benchmark value of 0.85 obtained in the previous statistics, the learning rate will be adjusted to 0.0005 in the next training cycle, and the loss value will be monitored in 1000 consecutive iterations. If the loss value continues to be higher than the threshold value of 0.2 selected in multiple preliminary experiments, the batch size of each iteration will be reduced from 64 to 32 to increase the parameter update intensity. The window size and step size of the pooling operation have been determined through diversified tests. After multiple rounds of training, the final convolutional layer output is recorded and flattened, and then the fully connected layer is connected to calculate the numerical feature vector that can be used for subsequent analysis. All feature vectors are aggregated to form a feature data set to obtain facial feature data.

[0024] Based on the facial feature data obtained in the previous paragraph, firstly, specific dimensions are selected from these feature data to characterize the eyebrow area and the smile arc area. These dimensions have been fixed when the statistics of multiple groups of annotated images were previously performed. The selected feature dimensions are recorded in time series and the difference between adjacent frames is calculated. If the change amplitude of a feature value in multiple consecutive frames exceeds 0.05, and this 0.05 is determined by the average value of the expression change amplitude of more than 100 subjects, then the feature is regarded as a sensitive feature and is further differentiated to obtain the change rate. The average feature value in the expressionless state is used as the reference base. The rate of change is compared with the benchmark difference. If the difference is greater than 0.03 (this 0.03 value is determined by analyzing the range of expression changes of multiple subjects in multiple laboratory environments), then it is recorded that the expression feature has changed significantly in the current frame. When it is determined that the change comes from the eyebrow feature dimension, it is marked as eyebrow fine-tuning; when it comes from the smile arc dimension, it is marked as smile feature change. Finally, the time change characteristics of all marked expression features are counted and matched with the pre-constructed feature change and expression category correspondence table. When the match is successful, the corresponding expression category is recorded to obtain the expression recognition result.

[0025] Based on the obtained expression recognition results, the recognition results are matched according to a pre-constructed mapping list, which is established by collecting expression change data of multiple subjects in different situations. The corresponding emotion labels are searched in the list for each type of expression in the recognition results. If it corresponds to a happy expression, it is marked as happy, if it corresponds to a sad expression, it is marked as sad, and if it corresponds to a surprised expression, it is marked as surprised. The proportion of each emotion label in all labeled results is counted. When the appearance ratio of a certain emotion label exceeds 0.5, this 0.5 value is determined by selecting the median proportion in the statistics of multiple groups of emotion distributions, then the emotion is regarded as the dominant emotion and further quantified. The emotional state is converted into a digital indicator by counting the frequency of occurrence of each expression and the intensity of change of its characteristic value. Finally, these digital indicators are output to obtain the basic emotional state indicator.

[0026] The steps for obtaining the adjusted speech features are: Extract emotional values ​​from basic emotional state indicators, including happiness and sadness, to obtain emotional analysis data; Based on the sentiment analysis data, the intonation of the digital human’s voice response is calculated. The intonation calculation formula is: ; Where E represents the pleasure extracted from the sentiment analysis data, are the coefficients for adjusting intonation, and P is the adjusted intonation; Using the sentiment analysis data, calculate the speed of the digital human's voice response, and summarize the tone and speed to obtain the adjusted voice features. The speed calculation formula is: ; Here, E also represents the pleasure extracted from the sentiment analysis data. is the coefficient for adjusting the speed, and V is the adjusted speed.

[0027] Specifically, the emotion value corresponding to the expression is read from the basic emotion state index obtained in the previous step, and the emotion value is recorded in the form of an array containing a set of pleasure and sadness. First, the pleasure and sadness stored in the array are checked item by item, and the reference range of pleasure of 0.0 to 1.0 and the reference range of sadness of 0.0 to 1.0 obtained by continuous observation of the emotional states of multiple volunteers in the early stage are used as benchmarks. Each emotion value is compared with these reference ranges. If the pleasure value is higher than 0.8, the point is recorded and marked as a high pleasure interval in the subsequent steps. The 0.8 threshold is determined by collecting facial expression data of no less than 30 resident visitors when viewing the exhibition and calculating the correlation between their expression parameters and the scores of the self-assessment questionnaire. If the sadness value is between 0.4 and 0.7, it is marked as medium in the subsequent steps. The sadness interval is equal. The values ​​of 0.4 and 0.7 are calculated by extracting the features of the volunteers who continuously show mild depression in the same batch. The high happiness and medium sadness data points marked in all the records are summarized, and then each data point is arranged in chronological order. These data points are read in sequence using a clear sequence index method, and they are matched with the expression change curve in the corresponding time period. The linear difference calculation of adjacent data points is performed at a fixed time interval of 5 to obtain the happiness change rate and sadness change rate. The time interval required for the calculation of these change rates is 5, which is selected after statistics on the fluctuation frequency of happiness changes in multiple samples to ensure that the change rate can represent the emotional fluctuation amount in a shorter period of time. All the change rate data are recorded to form a set of emotion analysis data for subsequent analysis. The emotion analysis data is the output of the final step.

[0028] The benefit of the formula is that it uses multiple parameters to participate in the operation, linking the quantified value of pleasure with each adjustment coefficient, thereby achieving dynamic adjustment of the emotional state in the process of speech feature generation; The steps to obtain the E parameter are as follows: Based on the emotion analysis data obtained above, continuous facial expression collection and emotion self-assessment questionnaire statistics are conducted on 200 visitors in a museum environment, and the characteristics such as skin light reflectivity and facial muscle stretch are quantified into a pleasure range of 0 to 1, from which a specific sample is selected with a pleasure level of ; The steps for obtaining the parameters are as follows: in the previously extracted voice data of multiple visitors, the acoustic features of the voice samples with different pleasantness marks are measured, and the relationship between the voice frequency change and pleasantness is fitted by linear regression to obtain a set of adjustment intonation coefficients suitable for the current scenario, where , , , These values ​​were obtained by collecting more than 100 pleasant speech samples in the laboratory, statistically analyzing the sound wave frequency and the corresponding pleasantness, and then regressing and solving them; Calculation process: calculate : ; calculate : ; calculate : ; Square root calculation ; calculate : ; calculate : ; calculate ; Final Result ; The results show that when the pleasantness is 0.65, the calculated adjusted intonation is 1.1652. When the result exceeds 1.0, it means that the voice intonation is more inclined to an ascending state than the neutral intonation. The higher the value, the more high-pitched the voice intonation is. When the result is lower than 1.0, it means that the intonation tends to be more flat or descending.

[0029] The benefit of the formula is that through multiple operations on pleasantness and speed adjustment coefficients, the speech rate can be precisely adjusted in the emotional context; The steps for obtaining the E parameter are the same as above, and the same pleasure level of 0.65 is used in the previous formula; The steps for obtaining the parameters are as follows: measuring the speech speed of multiple visitors under the same experimental conditions, establishing a mapping relationship between the speech speed of each speech sample and the corresponding pleasure, and determining the coefficient value through multiple linear regression analysis, where , , ; Calculation process: calculate : ; calculate ; calculate : ; calculate : ; calculate ; Final Result ; The results show that when the pleasantness is 0.65, the calculated adjusted speech speed is 1.5202. A value greater than 1.0 indicates that the speaking speed is faster than the baseline rate. A larger value indicates a higher speaking speed. A value lower than 1.0 indicates a slower speaking speed.

[0030] The steps to obtain system performance monitoring data are as follows: Continuously monitor the server, track server load and visitor interaction events, regularly collect server CPU and memory usage data, and obtain real-time monitoring data; Based on real-time monitoring data, analyze processor utilization and memory usage, identify performance bottlenecks or abnormal usage patterns, and obtain performance analysis results; Based on the performance analysis results, the data is integrated and formatted to obtain system performance monitoring data.

[0031] Specifically, in the process of continuous monitoring of the server, the deployed monitoring program is selected to periodically read the CPU activity frequency and memory usage of the target server, and the status information interface provided by the server is used as the data source. The CPU usage ratio is obtained by calling the kernel-level status query instruction at fixed intervals in the server operating system, and the memory usage is derived by using the amount of allocated memory page data recorded in the memory mapping table that stores the process running parameters. The query interval for CPU usage and memory usage is set to 10 seconds. The 10-second interval value is obtained by continuously observing multiple servers with the same configuration under high load and idle conditions for at least 72 hours, collecting CPU and memory data points, and calculating the average interval response speed. The time length that can ensure that the processor load data is synchronized with changes in visitor interaction events is selected to determine the CPU usage and memory usage obtained from each query. The values ​​are stored in the memory space in the form of an ordered list. Each data in the list is timestamped according to the chronological order. Data points with CPU usage exceeding 85% are marked. The 85% is determined by statistics on historical records provided by multiple system maintenance personnel, and the average of critical CPU usage at which the server has obviously experienced performance degradation in past operations is selected. If the memory usage remains above 70% in multiple consecutive queries, it is also marked. The 70% is determined by statistics on memory consumption curves under various usage scenarios of the application system and the average of obvious signs of slow response is selected. All marked data and unmarked data are placed in the same data structure, and the time series change trends of these data points are read and compared one by one through index pointers. When the recording is completed, real-time data containing CPU and memory usage information can be sorted out to obtain real-time monitoring data.

[0032] Based on the real-time monitoring data obtained, the CPU usage and memory occupancy values ​​in each group of data points are compared. First, during the execution process, the CPU usage is extracted in sequence and arranged in chronological order to form a set of sequences. The memory usage values ​​are also arranged in the same way to form another set of sequences. The corresponding values ​​in the same time period are selected from the two sets of sequences for corresponding comparison. If there are three or more consecutive values ​​in the CPU usage sequence that are higher than the 85% mark value determined above, the memory usage values ​​corresponding to these moments are extracted. If the memory usage value also exceeds 70% at these moments, the time period is marked as a potential performance bottleneck point in the analysis. This operation is completed through simple conditional judgment. If an abnormal fluctuation below the predetermined range is observed in the CPU or memory data sequence, the predetermined range is obtained by selecting the median with a fluctuation of 5% above and below after performing variance and standard deviation analysis on historical data, then the data point is recorded and classified as an abnormal usage mode. All marked potential performance bottleneck points and abnormal usage mode points are listed together, and finally these mark values ​​are output to clarify the performance characteristics of these time periods to obtain performance analysis results.

[0033] Based on the performance analysis results obtained, the CPU and memory data values ​​involved in the performance bottleneck points and abnormal usage pattern points marked above are selected for retrieval, and these data values ​​are sorted in ascending time order. These data points are assigned serial numbers in sequence using integer sequence numbers, and the corresponding CPU and memory values ​​are read for each data point. During the processing, the CPU value is output in percentage form, and the memory value is presented in MB. Points with CPU usage exceeding 85% are marked as "high load status", and points with memory usage exceeding 70% are marked as "high memory status". If the data point meets both high load and high memory status, a mark is added to the record, and all these sorted data are aggregated into a set of normalized structures to obtain system performance monitoring data.

[0034] The steps to obtain resource adjustment results are as follows: Extract the real-time CPU and memory usage, peak usage and idle rate from the system performance monitoring data to obtain comprehensive resource usage data; Based on the comprehensive resource usage data, calculate the number of CPU cores and memory that need to be adjusted. The formula is: ; and ; in, and They are the current CPU and memory usage, and is the target utilization rate, is the standard deviation of CPU usage, and They are the number of CPU cores and amount of memory adjusted respectively; Based on the adjusted number of CPU cores and memory amount, a resource adjustment result is formed.

[0035] Specifically, select the information corresponding to the CPU and memory status from the system performance monitoring data obtained in the previous step as input, arrange this information in the form of a time series, read the CPU usage at each time point, and record the value in floating-point format. Similarly, read and record the memory usage, obtain all CPU and memory usage data points in this period by performing multiple periodic queries within a continuous period of time, attach a timestamp to each data point to maintain the consistency of the time sequence, use a clear digital label to identify each record, correspond these identifiers to the CPU usage and memory usage at the corresponding time point, then select a time window from these data, compare each data point in the window in turn, identify the peak value and idle rate of the CPU and memory by calculating the maximum and minimum values, and select the peak usage rate by finding all data points within the time window. The highest CPU and memory values, and the idle rate are obtained by finding the lowest utilization data point. To ensure the credibility of the peak and idle values, it is necessary to determine the length and interval of the time window. Here, the time window length is set to 1 hour, which is obtained by referring to the CPU and memory change characteristics of multiple sets of system logs and based on the experience of operation and maintenance personnel in dozens of monitoring. The interval time is set to 5 minutes, which is determined by the distribution of CPU and memory change rates recorded by multiple servers with the same configuration under different load scenarios. The calculated peak and idle values ​​are arranged and combined with the real-time utilization in the corresponding time series and associated with the same structure. A set of integer indexes are used to number each utilization value to form an ordered data sequence. The real-time utilization, peak utilization, and idle rate contained in the sequence are uniformly extracted and integrated into comprehensive resource usage status data that can be directly read.

[0036] The formula is useful in that it uses parameters such as current CPU and memory usage, target usage, and standard deviation to perform operations, and can quantitatively adjust the number of CPU cores and the amount of memory; The steps of obtaining the parameters are as follows: reading the current CPU usage rate from the previously obtained comprehensive resource usage status data, the measured CPU usage rate at this time is 0.78, and this value is obtained by continuously collecting and counting the CPU usage rate in the on-site monitoring data; The steps for obtaining the parameters are as follows: the operation and maintenance personnel determine the target CPU usage rate to be 0.6 based on the actual operation requirements and the statistics of historical operation data. This value is determined by analyzing the average CPU usage values ​​of the system under high concurrency and normal load for dozens of times; The steps for obtaining the parameters are as follows: Calculate the standard deviation of CPU usage in the same historical monitoring data set. After recording and statistics for a week, the standard deviation of CPU usage is about 0.1. This value is obtained by calculating the standard deviation of the collected CPU usage sample data and taking the square root; The steps for obtaining the parameters are as follows: extract the current memory usage rate from the comprehensive resource usage status data obtained previously. After recording and analyzing, the memory usage rate at this time is 0.72. This value is the memory usage ratio at the current moment selected after multiple rounds of actual measurements; The steps for obtaining the parameters are as follows: After conducting statistics and control experiments on the memory usage of the operating environment for several days, the operation and maintenance personnel determined that the target memory usage rate was 0.5, which was set by selecting the average value of memory usage during the period with better performance; Calculation process: right formula: Calculating molecules : 0.78-0.6=0.18; Calculate the denominator : is 0.1; ; ; ; right formula: calculate : 0.72-0.5=0.22; ; ; ; The result shows that when the current CPU usage is 0.78, the target is 0.6, and the standard deviation is 0.1, the , this value is greater than 1, indicating that the number of CPU cores needs to be increased by about 1.62 cores, which is usually rounded up to 2 cores according to actual hardware conditions; when the current memory usage is 0.72 and the target is 0.5, the calculated A smaller value means that the amount of memory needs to be increased by about 0.0344 units. The amount of memory can be increased by taking the minimum step size based on the actual memory increment unit.

[0037] Based on the calculated adjusted number of CPU cores and memory, select the target value for actual resource adjustment from the parameters summarized in the previous step and use the value just obtained. and The values ​​are processed by the integerization rules. According to the minimum adjustment unit of hardware operation, 1 CPU core is adjusted to 2. The minimum memory capacity increment of 256MB is rounded up to 256MB. The values ​​involved in the execution process are obtained by associating with the statistical hardware device parameters. These hardware parameters are recorded after actual measurement of CPUs and memories of different brands and models in the computer room. The processed CPU core numbers and memory increments are summarized and compiled into an integrated data set. The final adjustment parameters of the CPU and memory are marked with clear index numbers. They are compared and verified with the target utilization rate calculated previously to check whether the difference between the two is lower than the pre-selected stability threshold. The threshold of 0.05 is determined by the average difference in the overall performance of the system after multiple adjustments. When it is confirmed that the difference is lower than 0.05, the final resource adjustment data can be recorded to obtain the resource adjustment result.

[0038] Adjust the resource allocation of the server based on the resource adjustment results, and continuously monitor the processor usage and memory usage. The steps to determine whether to adjust the resources again based on the processor usage and memory usage are as follows: According to the resource adjustment results, the number of CPU cores and memory are increased or decreased to obtain the updated server configuration; Based on the updated server configuration, the processor utilization and memory usage are continuously tracked to obtain real-time monitoring results; Based on the real-time monitoring results, analyze the current processor utilization and memory usage, evaluate whether the predetermined performance goals are met, and whether resources need to be adjusted again.

[0039] Specifically, according to the resource adjustment result obtained previously, the parameters of the number of CPU cores and memory capacity that need to be increased or decreased are read therefrom, and these parameters are used as the resource configuration information to be executed. The target virtual machine is modified by accessing the configuration interface provided in the virtualization management platform. First, log in to the virtualization management console and retrieve the unique identifier of the target virtual machine. The corresponding configuration option is found in the virtualization management interface through the identifier. The integer increase or decrease value obtained previously is input into the CPU core allocation parameter. For example, if it is necessary to increase from the existing 2 CPU cores to 4 CPU cores, the CPU core value is modified to 4 in the setting interface and the configuration is applied. The memory capacity is also adjusted with reference to the memory increase or decrease value obtained previously. If 256MB of memory needs to be increased, the allocation value of 256MB is added to the memory parameter column and the change is submitted. After submission, the virtualization management platform will dynamically allocate the underlying virtualization layer resources in the background, and update the CPU and memory parameters of the virtual machine to the new values. The virtual machine does not need to be offline and restarted after the update operation. After the configuration modification is completed, the updated number of CPU cores and memory capacity are recorded and compared and checked. If the setting value has been successfully effective, the adjustment operation is recorded as completed, and the updated server configuration is finally obtained.

[0040] Based on the updated server configuration, the processor usage and memory usage data of the virtual machines with the newly added CPU cores and memory capacity are queried regularly. The query command is sent to the virtual machine through the status interface provided by the virtualization management platform. The query command is repeatedly executed within an interval of 5 seconds. Each query can obtain the current CPU usage and memory usage values. The CPU and memory occupancy rates obtained are recorded in the form of floating point numbers, and a timestamp is attached to each record. These data points are arranged in order to form a time series. When the CPU usage rate exceeds the 80% reference value obtained by historical data statistics for many times, the corresponding time point is marked as a CPU high load interval. The 80% reference value is determined by selecting the most frequently occurring high value segment by comparing the CPU occupancy in multiple batches of running samples. When the memory usage rate exceeds the preset 70% threshold for many times, the corresponding time point is marked as a memory high usage interval in the record. The 70% threshold is obtained by empirical statistics of multiple operation and maintenance personnel under different load scenarios. The marking information is associated with the original data point and stored. The number of occurrences of high load and high usage intervals is counted by traversing the entire time series and the results are recorded, and finally a real-time monitoring result containing the latest usage rate and marking information is generated.

[0041] Based on the real-time monitoring results, the CPU and memory usage rates in the current period are selected from the recorded data sequence for comparison, and the CPU and memory values ​​at each time point are compared with the previously set target usage ranges. These target ranges are obtained by statistics of the operating data of the past two weeks. For example, the target CPU usage range is between 0.6 and 0.7, and the target memory usage range is between 0.4 and 0.5. If the CPU usage rate is found to be continuously higher than 0.7 in multiple consecutive queries, the information is recorded. If the CPU usage rate is high in a certain number of repeated queries, the situation is identified as a potential readjustment signal. Similarly, if the memory usage rate is higher than 0.5 in multiple queries, it is marked as excessive consumption of memory resources. The number of deviations is calculated and compared with the critical number determined in the statistics. If the critical number is exceeded, it indicates that resource allocation may need to be adjusted again. If all recorded points are within the target range, it means that the current resource allocation state can be maintained without reallocating resources. According to the final statistical results, it is determined whether the current system meets the predetermined performance goals and whether further resource adjustments are needed in the future.

[0042] The present invention provides a digital human driving system, comprising: The expression recognition module captures the visitor's facial video stream, selects key frames, grayscales and normalizes the image, and generates a standardized facial image; The emotion analysis module inputs the standardized facial images into the convolutional neural network, analyzes the image features, identifies expression changes, and generates basic emotional state indicators; The voice adjustment module adjusts the tone and speed of the digital human voice according to the basic emotional state indicators and generates the adjusted voice features; The content generation module applies the adjusted voice features to create interactive content that matches the visitor's emotions and generates optimized interactive content; Resource management module, monitors server load and visitor interaction data, collects processor usage and memory data, generates system performance monitoring data, evaluates resource usage, and calculates CPU and memory adjustment requirements for adjustment.

[0043] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical results of the present invention still falls within the protection scope of the technical results of the present invention.

Claims

1. A digital human driving method for museum interaction combined with large model technology, characterized in that: The following steps are involved: Analyze the visitor's facial video stream, extract image frames, and grayscale and normalize each frame to obtain a standardized facial image; Inputting the standardized facial image into a convolutional neural network, analyzing image features and identifying expression changes, and generating basic emotional state indicators; Based on the basic emotional state indicator, adjusting the tone and speed of the digital human's voice response to generate adjusted voice features; Applying the adjusted voice features to generate optimized interactive content; Continuously monitor server load and visitor interaction events, collect processor usage and memory usage, and generate system performance monitoring data; Based on the system performance monitoring data, evaluate the current resource usage of the server, calculate the number of CPU cores and memory that need to be increased or decreased, and generate resource adjustment results; According to the resource adjustment result, the resource allocation of the server is adjusted, and the processor usage and memory usage are continuously monitored, and it is determined whether to adjust the resources again according to the processor usage and memory usage.

2. The digital human driving method for museum interaction combined with large model technology according to claim 1 is characterized in that: The steps of obtaining the standardized facial image are: Analyze the visitor's facial video stream, extract continuous image frames, and generate an original image frame set; Based on the original image frame set, each frame of the image is grayed out, and the brightness value after graying is calculated. The calculation formula is: ； in, Indicates that the image is at position The red, green, and blue components of is the grayscale brightness value of the position; Based on the grayscale brightness value, normalization processing is performed on each image frame to generate a standardized facial image.

3. The digital human driving method for museum interaction combined with large model technology according to claim 1 is characterized in that: The steps for obtaining the basic emotional state index are: Training a convolutional neural network model, inputting the standardized facial image into the convolutional neural network model, and the convolutional neural network model detects key facial features through a multi-layer filter to obtain facial feature data; Based on the facial feature data, the convolutional neural network model continues to analyze the expression changes in the image, including the fine adjustment of eyebrows and the arc of the smile, to obtain the expression recognition result; Based on the expression recognition results, a classifier is used to correspond different expressions to emotional states of happiness, sadness and surprise to obtain a basic emotional state index.

4. The digital human driving method for museum interaction combined with large model technology according to claim 1 is characterized in that: The steps of obtaining the adjusted speech features are: Extracting emotional values ​​from the basic emotional state indicators, including happiness and sadness, to obtain emotional analysis data; Based on the sentiment analysis data, the intonation of the digital human voice response is calculated, and the intonation calculation formula is: ； Where E represents the pleasure extracted from the sentiment analysis data, are the coefficients for adjusting intonation, and P is the adjusted intonation; The emotion analysis data is used to calculate the speed of the digital human's voice response, and the intonation and speed are summarized to obtain the adjusted voice features. The speed calculation formula is: ； Here, E also represents the pleasure extracted from the sentiment analysis data. is the coefficient for adjusting the speed, and V is the adjusted speed.

5. The digital human driving method for museum interaction combined with large model technology according to claim 1 is characterized in that: The steps for obtaining the system performance monitoring data are as follows: Continuously monitor the server, track server load and visitor interaction events, regularly collect server CPU and memory usage data, and obtain real-time monitoring data; Based on the real-time monitoring data, analyze the processor usage and memory usage, identify performance bottlenecks or abnormal usage patterns, and obtain performance analysis results; Based on the performance analysis results, the data is integrated and formatted to obtain system performance monitoring data.

6. The digital human driving method for museum interaction combined with large model technology according to claim 1 is characterized in that: The steps for obtaining the resource adjustment result are: Extracting the real-time utilization rate of CPU and memory, as well as the peak utilization rate and idle rate from the system performance monitoring data to obtain comprehensive resource usage status data; Based on the comprehensive resource usage data, the number of CPU cores and memory required to be adjusted are calculated using the formula: ； and ； in, and They are the current CPU and memory usage, and is the target utilization rate, is the standard deviation of CPU usage, and They are the number of CPU cores and amount of memory adjusted respectively; Based on the adjusted number of CPU cores and memory amount, a resource adjustment result is formed.

7. The digital human driving method for museum interaction combined with large model technology according to claim 1 is characterized in that: According to the resource adjustment result, the resource allocation of the server is adjusted, and the processor usage and memory usage are continuously monitored. The steps of determining whether to adjust the resources again according to the processor usage and memory usage are as follows: According to the resource adjustment result, the number of CPU cores and memory are increased or decreased to obtain an updated server configuration; Based on the updated server configuration, continuously track processor usage and memory usage to obtain real-time monitoring results; Based on the real-time monitoring results, the current processor utilization and memory usage are analyzed to evaluate whether the predetermined performance goals are met and whether resources need to be adjusted again.

8. The digital human driving system according to any one of claims 1 to 7, characterized in that: include: The expression recognition module captures the visitor's facial video stream, selects key frames, grayscales and normalizes the image, and generates a standardized facial image; The emotion analysis module inputs the standardized facial images into the convolutional neural network, analyzes the image features, identifies expression changes, and generates basic emotional state indicators; The voice adjustment module adjusts the tone and speed of the digital human voice according to the basic emotional state indicators and generates the adjusted voice features; The content generation module applies the adjusted voice features to create interactive content that matches the visitor's emotions and generates optimized interactive content; Resource management module, monitors server load and visitor interaction data, collects processor usage and memory data, generates system performance monitoring data, evaluates resource usage, and calculates CPU and memory adjustment requirements for adjustment.

Images