Behavior guidance data generation method, system, device and computer equipment

Abstract

The application relates to a behavior guidance data generation method, system, device and computer equipment. The method comprises the following steps: collecting a multi-modal physiological perception signal of a target object; analyzing the multi-modal physiological perception signal based on a behavior recognition model to obtain effective behavior data in a target time period; processing the effective behavior data based on a state recognition model to obtain a behavior execution state; determining a behavior control strategy corresponding to the behavior execution state in a corresponding relationship between execution states and control strategies; and generating guidance information based on the behavior control strategy and outputting the guidance information. The method can improve the accuracy of the prompt for user diet management.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, system, apparatus and computer device for generating behavior-guided data. Background Technology

[0002] With the accelerated pace of modern life and changes in dietary structure, obesity and the chronic metabolic diseases it causes (such as diabetes and hypertension) have become a global public health problem. Scientific dietary management is a core means of preventing and controlling such health problems.

[0003] Automated eating behavior monitoring technologies typically rely on a single sensor (such as a microphone to collect chewing sounds or an inertial sensor to collect hand movements). However, single sensors are highly susceptible to interference in real-life scenarios. For example, user speech, swallowing, ambient noise, or non-eating head / hand movements (i.e., artifacts) can all lead to misjudgments of eating actions. The high false alarm rate of eating actions severely reduces the accuracy of guiding users' eating status. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, system, device, and computer equipment for generating behavioral guidance data that can improve the accuracy of prompts for users' dietary management, in order to address the aforementioned technical problems.

[0005] Firstly, this application provides a method for generating behavior guidance data, including:

[0006] Collect multimodal physiological sensing signals from the target object;

[0007] Based on the behavior recognition model, the multimodal physiological perception signals are analyzed to obtain effective behavior data within the target time period;

[0008] Based on the state recognition model, the effective behavior data is processed to obtain the behavior execution state; in the correspondence between the execution state and the control strategy, the behavior control strategy corresponding to the behavior execution state is determined; and guidance information is generated based on the behavior control strategy and output.

[0009] In one embodiment, the multimodal physiological sensing signal includes a first physiological sensing signal and a second physiological sensing signal. The step of analyzing the multimodal physiological sensing signal based on a behavior recognition model to obtain effective behavioral data within a target time period includes:

[0010] Based on the burst feature extraction model, feature extraction is performed on the first physiological sensing signal to obtain the first physiological sensing feature data;

[0011] Based on the energy feature extraction model, feature extraction is performed on the second physiological sensing signal to obtain the second physiological sensing feature data.

[0012] Based on the first physiological perception feature data and the second physiological perception feature data, the number of valid times that meet the preset valid conditions is determined, and based on the number of valid times, the valid behavioral data in the target time period is calculated.

[0013] In one embodiment, the step of extracting features from the first physiological sensing signal based on the burst feature extraction model to obtain first physiological sensing feature data includes:

[0014] The first physiological sensing signal is filtered using an interference filter to obtain a filtered electrical signal.

[0015] Based on the full-wave rectification algorithm, the filtered electrical signal is processed to obtain a unipolar signal;

[0016] The envelope is obtained by processing the unipolar signal based on the envelope extraction algorithm.

[0017] The envelope is divided into multiple sub-envelopes according to a preset calculation period; the first physiological perception feature data corresponding to the sub-envelopes is determined.

[0018] In one embodiment, the step of extracting features from the second physiological sensing signal based on the energy feature extraction model to obtain second physiological sensing feature data includes:

[0019] Based on a preset frequency band filter, the second physiological sensing signal is extracted and processed to obtain a filtered vibration signal;

[0020] Based on a preset frame length, the filtered vibration signal is processed into multiple filtered sub-vibration signals, and the second physiological perception feature data of the signal amplitude of each filtered sub-vibration signal is calculated.

[0021] In one embodiment, determining the number of valid times that satisfy the preset valid conditions based on the first physiological perception feature data and the second physiological perception feature data includes:

[0022] From the first physiological perception feature data and the second physiological perception feature data, each first effective feature value and each second effective feature value that meet the effective threshold are selected respectively;

[0023] Obtain the first time point corresponding to each of the first valid feature values, and the second time point corresponding to each of the second valid feature values;

[0024] The difference between each first time point and each second time point is calculated, and the number of times the difference is less than a preset time difference threshold is determined as the effective number of times the valid condition is met.

[0025] In one embodiment, the behavior execution state includes an abnormal behavior state, and the generation and output of guidance information based on the behavior control strategy includes:

[0026] Based on the behavior control strategy and abnormal display format corresponding to the abnormal behavior state, the image rendering parameters of the environmental image and the state area of ​​the virtual object are adjusted to obtain a first adjusted environmental image, and the first adjusted environmental image is displayed; and / or,

[0027] Based on the abnormal behavior control strategy corresponding to the abnormal behavior state, alarm information is generated and output.

[0028] In one embodiment, the behavior execution state further includes a normal behavior state, and the step of generating and outputting guidance information based on the behavior control strategy includes:

[0029] Based on the behavior control strategy and normal display format corresponding to the normal behavior state, the state area of ​​the virtual object in the environment image is adjusted to obtain a second adjusted environment image, and the second adjusted environment image is displayed.

[0030] Secondly, this application also provides a behavior guidance data generation system, including: a controller, a display unit, an alarm unit, and a multimodal physiological perception acquisition unit, wherein:

[0031] The multimodal physiological sensing acquisition unit is used to acquire multimodal physiological sensing signals of the target object;

[0032] The controller is configured to analyze the multimodal physiological sensing signals based on a behavior recognition model to obtain effective behavior data within a target time period; process the effective behavior data based on a state recognition model to obtain a behavior execution state; determine the behavior control strategy corresponding to the behavior execution state in the correspondence between the execution state and the control strategy; and generate guidance information based on the behavior control strategy and output the guidance information.

[0033] The display unit and the alarm unit are used to receive the guidance information and to display the guidance information to the target object.

[0034] Thirdly, this application also provides a behavior guidance data generation device, comprising:

[0035] The acquisition module is used to acquire multimodal physiological sensing signals of the target object;

[0036] The analysis module is used to analyze the multimodal physiological perception signals based on the behavior recognition model to obtain effective behavioral data within the target time period.

[0037] The generation module is used to process the effective behavior data based on the state recognition model to obtain the behavior execution state; determine the behavior control strategy corresponding to the behavior execution state in the correspondence between the execution state and the control strategy; and generate guidance information based on the behavior control strategy and output the guidance information.

[0038] Fourthly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0039] Collect multimodal physiological sensing signals from the target object;

[0040] Based on the behavior recognition model, the multimodal physiological perception signals are analyzed to obtain effective behavior data within the target time period;

[0041] Based on the state recognition model, the effective behavior data is processed to obtain the behavior execution state; in the correspondence between the execution state and the control strategy, the behavior control strategy corresponding to the behavior execution state is determined; and guidance information is generated based on the behavior control strategy and the guidance information is output.

[0042] Fifthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0043] Collect multimodal physiological sensing signals from the target object;

[0044] Based on the behavior recognition model, the multimodal physiological perception signals are analyzed to obtain effective behavior data within the target time period;

[0045] Based on the state recognition model, the effective behavior data is processed to obtain the behavior execution state; in the correspondence between the execution state and the control strategy, the behavior control strategy corresponding to the behavior execution state is determined; and guidance information is generated based on the behavior control strategy and the guidance information is output.

[0046] Sixthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0047] Collect multimodal physiological sensing signals from the target object;

[0048] Based on the behavior recognition model, the multimodal physiological perception signals are analyzed to obtain effective behavior data within the target time period;

[0049] Based on the state recognition model, the effective behavior data is processed to obtain the behavior execution state; in the correspondence between the execution state and the control strategy, the behavior control strategy corresponding to the behavior execution state is determined; and guidance information is generated based on the behavior control strategy and the guidance information is output.

[0050] The aforementioned behavior guidance data generation method, system, device, and computer equipment, by collecting multimodal physiological perception signals of the target object and extracting effective behavior data based on a behavior recognition model, realize the effective pre-set behavior execution of the target object from multiple modalities, achieve the recognition of effective behavior of pre-set actions, reduce interference from other artifact actions, and improve the comprehensiveness and reliability of effective behavior perception of the target object; and, based on the state recognition model and effective data, determine the behavior execution state, and determine the behavior control strategy corresponding to the behavior execution state in the correspondence between the execution state and the control strategy; and generate and output guidance information based on the behavior control strategy, improving the accuracy of identifying the current behavior execution state, generating corresponding behavior control strategies, and generating guidance information based on behavior control strategies, realizes positive guidance of the target object's behavior, improves the management of users' execution of pre-set behaviors, and improves the accuracy and timeliness of behavior prompts. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a schematic diagram of a behavior guidance data generation system in one embodiment;

[0053] Figure 2 This is a flowchart illustrating a behavior guidance data generation method in one embodiment;

[0054] Figure 3 This is a schematic diagram of a dietary behavior guidance system based on multimodal perception and augmented reality gamification in one embodiment;

[0055] Figure 4 This is a flowchart illustrating a dietary behavior guidance method in one embodiment;

[0056] Figure 5 This is a schematic diagram of the field of view of the AR glasses' display interface when it is in a preset reminder state in one embodiment;

[0057] Figure 6 This is a structural block diagram of a behavior guidance data generation device in one embodiment;

[0058] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0060] The behavior guidance data generation method provided in this application embodiment can be applied to, for example... Figure 1 The behavior guidance data generation system described above includes a behavior detection system that may comprise a controller, a display unit, an alarm unit, and a multimodal physiological perception acquisition unit. The multimodal physiological perception acquisition unit is positioned at multiple target locations within the behavior acquisition system to collect multimodal physiological perception signals from the target object. The controller transmits data with the multimodal physiological perception acquisition unit, processing the multimodal physiological perception signals to obtain the behavior execution state of the target object and generate corresponding guidance information. Both the display unit and the alarm unit can transmit data with the controller, receiving guidance information and outputting corresponding guidance actions to the target object. The behavior detection system may be a head-mounted device, such as a virtual reality (VR) device, an augmented reality (AR) device, or smart glasses. The display unit may be an AR rendering unit and a gamification interaction module, and may be a projection waveguide or a screen. Optionally, the controller can be a processing unit, and the display unit, alarm unit, and multimodal physiological sensing acquisition unit can be integrated into a wearable display device. This wearable display device can interact with the processing unit to complete multimodal sensing and augmented reality gamified dietary behavior guidance. It should be understood that... Figure 1 The behavior detection system in the text is only used to display the data transmission relationship between units, and cannot characterize the specific positional relationship between units.

[0061] In one exemplary embodiment, such as Figure 2 As shown, a method for generating behavior-guided data is provided, which can be applied to... Figure 1 Taking the behavior guidance data generation system in [the context of the original text] as an example, the following steps are included: [The steps are described in the original text.]

[0062] Step 201: Collect multimodal physiological sensing signals of the target object.

[0063] The target object is the object whose preset behavior needs to be managed and controlled, i.e., the object wearing the behavior detection system; the multimodal physiological sensing signal can include physiological sensing signals of multiple modalities, which are collected by the multimodal physiological sensing acquisition unit at each target location of the target object; the physiological sensing signal refers to the sensing signal collected when the user performs the preset behavior, which triggers the physiological behavior. For example, the preset behavior can be the target object's eating behavior or chewing behavior.

[0064] Specifically, the behavior detection system can collect multimodal physiological sensing signals of each target location of the target object in real time or according to a preset collection cycle through a multimodal physiological sensing acquisition unit.

[0065] Step 202: Based on the behavior recognition model, analyze the multimodal physiological perception signals to obtain effective behavior data within the target time period.

[0066] The behavior recognition model is used to identify valid behavioral data of a target object. Valid behavioral data represents the data on the target object performing preset behaviors. This valid behavior refers to the frequency or rate at which the target object performs the preset behaviors. The target time period can include any preset time window.

[0067] Specifically, the controller of the behavior detection system can receive multimodal physiological sensing signals and analyze the multimodal physiological sensing signals through a behavior recognition model to obtain the number of effective behaviors of the target object in performing preset behaviors within the target time period, and determine the effective behavior data within the target time period based on the number of effective behaviors.

[0068] Step 203: Based on the state recognition model, process the valid behavior data to obtain the behavior execution state; determine the behavior control strategy corresponding to the behavior execution state in the correspondence between the execution state and the control strategy; and generate guidance information based on the behavior control strategy and output the guidance information.

[0069] Among them, the state recognition model is a model used to identify the behavior execution state of a target object when performing a preset behavior within a target time period; the behavior execution state refers to the specific form in which the target object is when performing the preset behavior. For example, the behavior execution state can include the opposite sex behavior state and the normal behavior state; the behavior control strategy is a strategy used to guide the state of the preset behavior; the guidance information is data used to guide the preset behavior of the target user.

[0070] Specifically, the controller can process valid behavioral data based on the state recognition model to obtain the behavior execution state, and determine the behavior control strategy corresponding to the behavior execution state in the correspondence between the execution state and the control strategy; the controller can generate guidance information based on the behavior control strategy, and output the guidance information to the alarm unit and / or display unit, and the alarm unit and / or display unit execute the guidance information.

[0071] Optionally, the state recognition model can be a correspondence between valid behavioral data and behavioral execution states. The controller can determine the behavioral execution state corresponding to the valid behavioral data from the correspondence between valid behavioral data and behavioral execution states.

[0072] The aforementioned behavior guidance data generation method, by collecting multimodal physiological perception signals of the target object and extracting effective behavior data based on a behavior recognition model, enables the execution of effective preset behaviors of the target object from multiple modalities. This achieves the identification of effective behaviors for preset actions, reduces interference from other artifact actions, and improves the comprehensiveness and reliability of the effective behavior perception of the target object. Furthermore, based on the state recognition model and effective data, it determines the behavior execution state, identifies the corresponding behavior control strategy in the correspondence between the execution state and the control strategy, and generates and outputs guidance information based on the behavior control strategy. This improves the accuracy of identifying the current behavior execution state and generates corresponding behavior control strategies. By generating guidance information based on these strategies, it achieves positive guidance of the target object's behavior, improves the management of users' execution of preset behaviors, and enhances the accuracy and timeliness of behavior prompts.

[0073] In an exemplary embodiment, the multimodal physiological sensing signal includes a first physiological sensing signal and a second physiological sensing signal. The specific implementation process of step 202, "analyzing the multimodal physiological sensing signal based on the behavior recognition model to obtain effective behavioral data in the target time period," may include:

[0074] Based on the burst feature extraction model, features are extracted from the first physiological perception signal to obtain the first physiological perception feature data; based on the energy feature extraction model, features are extracted from the second physiological perception signal to obtain the second physiological perception feature data; based on the first physiological perception feature data and the second physiological perception feature data, the number of valid times that meet the preset valid conditions is determined, and based on the number of valid times, the valid behavioral data in the target time period is calculated.

[0075] The first and second physiological sensing signals are acquired by physiological sensing acquisition units of different modalities. They are used to acquire sensing signals from different locations. The first physiological sensing signal characterizes the masticatory muscle exertion data of the target object during eating, while the second physiological sensing signal characterizes acoustic or vibrational energy data during eating, such as chewing, swallowing, and teeth tapping. The burst feature extraction model extracts the burst features of the first physiological sensing signal. The energy feature extraction model extracts the energy features of the second physiological sensing signal at specific frequency bands. Burst features refer to the quantitative characteristics of rapid and intense contraction within a short period, reflecting the degree of instantaneous change or short-term energy accumulation level of the physiological part to be detected at the target location. The preset effective condition refers to both the first and second physiological sensing signals meeting a preset physiological sensing threshold, and the temporal matching of the first and second physiological sensing signals meeting the preset physiological sensing threshold. The effective number of times refers to the number of times the preset behavior is performed within the target time period.

[0076] Specifically, the controller can extract features from the first physiological sensing signal based on the burst feature extraction model to obtain first physiological sensing feature data; extract features from the second physiological sensing signal based on the energy feature extraction model to obtain second physiological sensing feature data; select each first effective feature value and each second effective feature value that meet the effective threshold from the first physiological sensing feature data and the second physiological sensing feature data respectively; obtain the first time corresponding to each first effective feature value and the second time corresponding to each second effective feature value; calculate the difference between each first time value and each second time value, and determine the number of differences less than the preset time difference threshold as the effective number of times that meet the effective conditions; determine the quotient of the effective number of times and the target time period as the effective behavior rate in the target time period, and determine the effective behavior rate as effective behavior data.

[0077] In this embodiment, features are extracted from physiological perception signals using a corresponding feature recognition model, and effective behaviors are identified using first and second physiological perception feature data, thereby improving the accuracy of effective behavior identification.

[0078] In an exemplary embodiment, the specific implementation process of the step "based on the burst feature extraction model, extracting features from the first physiological sensing signal to obtain the first physiological sensing feature data" may include:

[0079] The first physiological sensing signal is filtered using an interference filter to obtain a filtered electrical signal; the filtered electrical signal is then processed using a full-wave rectification algorithm to obtain a unipolar signal; the unipolar signal is then processed using an envelope extraction algorithm to obtain an envelope; the envelope is then divided according to a preset calculation period to obtain multiple sub-envelopes; and the first physiological sensing feature data corresponding to the sub-envelopes are determined.

[0080] Among them, the interference filter is used to filter out power frequency interference and low-frequency motion artifacts. The interference filter can be a bandpass filter in a preset frequency band, for example, the preset frequency band can be 20Hz-500Hz; the filtered electrical signal is the effective electrical signal after filtering out the interference signal; the full-wave rectification algorithm is an algorithm for converting the effective electrical signal into a unipolar signal; the envelope extraction algorithm can be a moving average filtering algorithm or a low-pass filter, which can be a Butterworth filter with a cutoff frequency of 10Hz; the preset calculation period is the smallest unit for dividing the target time period.

[0081] Specifically, the controller can filter the first physiological sensing signal based on an interference filter to obtain a filtered electrical signal; and perform absolute value processing on the filtered electrical signal to obtain a unipolar signal, i.e., the filtered electrical signal is a bipolar AC signal; extract the envelope of the unipolar signal based on an envelope extraction algorithm; divide the envelope according to a preset calculation period to obtain multiple sub-envelopes; and determine the maximum amplitude or integral area of ​​each sub-envelope, and determine the maximum amplitude or integral area of ​​the sub-envelope as the burst characteristic value, which is the first physiological sensing characteristic.

[0082] For example, the first physiological sensing signal can be a surface electromyography (sEMG) signal and / or a mechanical deformation signal. The sEMG signal is acquired by a sEMG sensor, which can be an Ag / AgCl electrode. This sEMG sensor can be attached to or in contact with the surface of the masticatory muscles of the target object. The surface of the masticatory muscles can be the skin surface of the masseter or temporalis muscle. It acquires the bioelectrical activity during muscle contraction, which is the sEMG signal. The mechanical deformation signal is acquired by a flexible strain sensor, a piezoelectric film sensor, or a pressure sensor. This sensor is attached to or against the surface of the masseter or temporalis muscle of the target object. When the target object performs a chewing action, the muscle bulge causes the sensor to undergo physical deformation, which is converted into an electrical signal. This electrical signal is the mechanical deformation signal. The behavior detection system can be a head-mounted glasses. The sEMG sensor, flexible strain sensor, piezoelectric film sensor, or pressure sensor can all be embedded in the inner wall of the temple and fit tightly against the temporalis muscle area of ​​the wearer using an elastic contact structure. The target object can be the wearer of the behavior detection system, i.e., the wearer of the head-mounted glasses. It should be understood that the process of feature extraction for different first physiological sensory signals is consistent. Their time-domain waveforms all show paroxysmal oscillations generated with chewing movements, and the extracted envelopes can accurately reflect the degree of instantaneous mutation or short-term energy accumulation level of the masticatory muscles (i.e., burst characteristics).

[0083] In this embodiment, by extracting features from bioelectric signals, first physiological perception feature data is obtained, thereby realizing the extraction of burst features of bioelectric signals, so as to facilitate the identification of preset behaviors and the filtering of invalid actions.

[0084] In an exemplary embodiment, the specific implementation process of the step "based on the energy feature extraction model, performing feature extraction on the second physiological sensing signal to obtain second physiological sensing feature data" may include:

[0085] Based on a preset frequency band filter, the second physiological perception signal is extracted and processed to obtain a filtered vibration signal; based on a preset frame length, the filtered vibration signal is divided into frames to obtain multiple filtered sub-vibration signals, and the second physiological perception feature data of the signal amplitude of each filtered sub-vibration signal is calculated.

[0086] Among them, the preset frequency band filter can be a high-frequency bandpass filter; the filtered vibration signal is the bone conduction signal of the preset frequency obtained by screening; the second physiological perception feature data refers to the energy feature value of a specific frequency band.

[0087] Specifically, the controller can extract filtered vibration signals of a preset frequency band from the bone conduction signal based on a preset frequency band filter; based on a preset frame length, it can perform frame segmentation processing on the filtered vibration signals to obtain multiple filtered sub-vibration signals; and calculate the energy of each preset frame length's filtered sub-vibration signal to obtain a specific frequency band energy characteristic value for that preset frame length. Optionally, the specific frequency band energy characteristic value can be the sum of squares or the root mean square value of the energy for that frame length. For example, the preset frame length can be 20 milliseconds. It should be understood that this limitation is for illustrative purposes only and does not constitute a specific limitation.

[0088] Optionally, the second physiological sensing signal may include at least one of bone conduction acoustic signals, air conduction acoustic signals, and inertial vibration signals.

[0089] Bone conduction acoustic signals are acquired by a bone conduction microphone or a piezoelectric vibration sensor attached to or against a bone node in the head of the target object. For example, the bone node could be the mastoid process behind the ear, the mandible, or the temporal bone. The sensor is used to capture chewing and swallowing vibrations transmitted through the skull to obtain bone conduction acoustic signals.

[0090] Airborne acoustic signals are acquired using miniature airborne microphones, which can be deployed near the mouth or cheek of the target, for example, integrated into the front of the temple of smart glasses, the earpiece of smart headphones, or a facial patch, to capture acoustic signals emitted from the mouth into the air during eating.

[0091] Inertial vibration signals are acquired by an inertial measurement unit (IMU). For example, an IMU may contain an accelerometer or a gyroscope. The IMU may be worn around the jaw, ear, or integrated into a wearable head device (such as smart glasses or headphones) to capture minute high-frequency inertial vibrations of the face or head caused by chewing and swallowing.

[0092] It should be understood that in the process of feature extraction for different second physiological sensing signals, only the cutoff frequency of the preset frequency band filter is different, while the other feature extraction processes are the same. Different physiological sensing signals can select the corresponding preset frequency band filter based on the frequency band characteristics of the signal, without making specific limitations here.

[0093] In this embodiment, by extracting features from the bone conduction signal, second physiological perception feature data is obtained, thereby realizing the extraction of specific frequency band energy feature values ​​of the bone conduction signal, so as to facilitate the subsequent identification of preset behaviors and the filtering of invalid actions.

[0094] In an exemplary embodiment, the specific implementation process of the step "determining the number of valid times that meet the preset valid conditions based on the first physiological perception feature data and the second physiological perception feature data" may include:

[0095] From the first physiological perception feature data and the second physiological perception feature data, each first effective feature value and each second effective feature value that meet the effective threshold are selected respectively; the first time corresponding to each first effective feature value and the second time corresponding to each second effective feature value are obtained; the difference between each first time value and each second time value is calculated, and the number of differences less than the preset time difference threshold is determined as the effective number of times that meet the effective conditions.

[0096] The effective thresholds for different physiological sensory feature data are different. Optionally, the effective threshold for the first physiological sensory feature data can be a preset electromyographic resting threshold; the effective threshold for the second physiological sensory feature data can be a preset vibration threshold. The preset time difference threshold is the maximum allowable physiological delay, for example, the preset time difference threshold can be 100 milliseconds.

[0097] Specifically, the controller can filter out feature values ​​greater than a preset electromyographic resting threshold from the first physiological sensing feature data and determine them as first effective feature values; it can also filter out feature values ​​greater than a preset vibration threshold from the second physiological sensing feature data and determine them as second effective feature values; it determines the time at a specified position within a preset calculation period corresponding to the first effective feature value as the corresponding first time moment, and the time at a specified position within a preset frame length corresponding to the second effective feature value as the corresponding second time moment; it calculates the difference between each first time moment and each second time moment, and determines the number of differences less than a preset time difference threshold as the effective number of times the effective condition is met. It should be understood that the specified position can refer to the middle or end of a preset calculation period or preset frame length. Optionally, the first effective feature value represents the occurrence of significant biting force, and the second effective feature value represents the occurrence of physical collision or brittle food breakage.

[0098] In addition, the controller can output a Boolean value True each time it determines that the difference is less than the preset time difference threshold, and determine that a valid preset behavior has been executed, that is, a valid chewing action has been sent.

[0099] In one example, the controller receives the number of "effective chewing actions" over a target time period and calculates the eating rate, which can be measured in times per minute. In another example, the terminal can calculate the effective chewing actions at various points in a continuous time axis, using a sliding window to calculate the real-time eating rate within that sliding window.

[0100] In this embodiment, a dual-channel hardware architecture combining electromyography (EMG) and bone conduction, along with collaborative verification logic, addresses the issue of single-sensor susceptibility to motion artifacts at the physical source. Based on a logic verification mechanism that matches temporal synchronization and frequency domain features, valid chewing events are identified, and the eating rate parameter is calculated accordingly. This effectively filters out artifacts such as speaking (which possesses acoustic vibration signals, but whose EMG signal's time-frequency characteristics or burst intensity do not match the preset chewing force characteristics), teeth clenching (which only has EMG without vibration frequency band characteristics), and swallowing saliva (which has no obvious teeth-colliding sound), thus quantifying the eating rate and improving the accuracy of valid behavior execution.

[0101] In an exemplary embodiment, the behavior execution state includes an abnormal behavior state, and the specific implementation process of step 203, "generating guidance information based on the behavior control strategy and outputting the guidance information," may include at least one of the following two implementation methods:

[0102] The first implementation method is to adjust the image rendering parameters of the environment image and the state area of ​​the virtual object based on the behavior control strategy and abnormal display form corresponding to the abnormal behavior state, so as to obtain the first adjusted environment image and display the first adjusted environment image.

[0103] The abnormal behavior state can refer to valid behavior data being greater than or equal to a preset execution threshold, i.e., the eating rate being greater than or equal to a preset rate threshold. Optionally, the abnormal behavior state can include a preset reminder state, i.e., the preset reminder state is valid behavior data greater than the preset execution threshold, and the preset reminder state can be a rate over-limit state. The environmental image refers to image data of the scene environment in which the target object is located, acquired by the visual acquisition unit; the environmental image can be a first-person perspective image of the target object. The image acquisition unit can be a miniature camera used to capture environmental images in front of the target object's line of sight. The environmental image includes food images; optionally, the environmental image can only contain food images. The virtual object refers to a virtual interactive object displayed on the display unit. The status area can represent the virtual object's vital signs or behavioral state, which are related to the detected behavioral state of the target object. Optionally, the status area can be a progress bar, and the display format is a way of adjusting the status area. An abnormal display format is to increase the progress bar according to set parameters. The first adjustment of the environmental image refers to the image after adjusting the image rendering parameters and the status area of ​​the virtual object. The image rendering parameters can be image color parameters. The environmental image can be displayed on the augmented reality display interface.

[0104] Specifically, the controller can adjust the image rendering parameters of the environment image based on the behavior control strategy corresponding to the abnormal behavior state, and adjust the state area of ​​the virtual object through the behavior control strategy and abnormal display form corresponding to the abnormal behavior state to obtain a first adjusted environment image, and display the first adjusted environment image on the display unit.

[0105] Optionally, the controller can pre-establish a negative correlation mapping model between the feeding rate and the image rendering parameters, and a pre-established mapping model between the feeding rate and the display format. That is, the controller can determine the corresponding current image rendering parameters based on the current feeding rate, and adjust the current image rendering parameters based on the current environmental image; and determine the corresponding abnormal display format based on the current feeding rate, and adjust the abnormal display format of the display area to obtain a first adjusted environmental image.

[0106] In another example, the controller can also adjust the pixel values ​​of the environment image. Specifically, it can calculate the average pixel value of the pixels near each pixel and replace the pixel value of that pixel with that average value to obtain an adjusted environment image. Alternatively, an adjusted environment image can be obtained by adding noise to the environment image.

[0107] The second implementation method is to generate and output alarm information based on the abnormal behavior control strategy corresponding to the abnormal behavior state.

[0108] The alarm information includes voice message reminders and / or activation of the bone conduction brake.

[0109] Specifically, the controller can generate alarm information based on the abnormal behavior control strategy corresponding to the abnormal behavior state, and then provide the alarm information via a voice prompt unit or activate a bone conduction actuator. The bone conduction actuator outputs vibration feedback to the target object, which can be intermittent or continuous. For example, the bone conduction actuator can be located at the nose pad position. Optionally, the alarm information, such as "eating too fast," can be displayed on the display interface of the display unit.

[0110] In one example, the visual acquisition unit is used to capture images of food in front of the wearer's line of sight. The visual acquisition unit is a miniature camera located at the front of the frame, with its optical axis tilted downwards.

[0111] In one example, the guidance information may include one or more of the following: an adjusted environment image after adjusting image rendering parameters and alarm information. The guidance information may include an adjusted environment image (displayed during abnormal behavior); or alarm information (sent to the target object during abnormal behavior); or both an adjusted environment image and alarm information (displayed and sent to the target object during abnormal behavior). When the guidance information includes an adjusted environment image, the first implementation method can be used; when it includes alarm information, the second implementation method can be used; and when it includes both, both methods can be used simultaneously.

[0112] In one example, image rendering parameters can be color attributes or optical overlay attributes. Color attributes can include saturation and hue. The controller can adjust the color attributes (such as saturation and hue) or optical overlay attributes of the target food area (environmental image) in real time, and the displayed content dynamically changes with the eating state. Virtual interactive objects (virtual objects) are generated and rendered in the augmented reality display interface. The state and state area of ​​these virtual objects can change with the eating rate. A dynamic correlation is established between the vital signs or behavioral state of the virtual interactive object and the eating behavior state, enabling the virtual interactive object to provide positive or negative empathetic feedback in real time based on the wearer's eating rate.

[0113] In another example, the abnormal behavior state may also include a critical alert state, where the valid behavior data equals a preset execution threshold. In the critical alert state, the controller can generate and output an alarm message, which can be delivered via voice prompts from a voice prompt module.

[0114] In this embodiment, visual parameter modulation (changing the color of food) is used to directly affect the visual senses, thereby influencing the user's eating behavior. Furthermore, the gamified interaction module transforms the user's self-discipline into the protection of virtual objects, which can enhance the user's willingness to use the system. The entire process from perception to guidance is automated, eliminating the need for manual recording by the user, lowering the barrier to entry, improving guidance efficiency, and providing adjustable visual and interactive feedback through augmented reality, thereby guiding users to form healthy eating behaviors.

[0115] In an exemplary embodiment, the behavior execution state further includes a normal behavior state, and the specific implementation process of the step "generating guidance information based on the behavior control strategy and outputting the guidance information" may include:

[0116] Based on the behavior control strategy and normal display form corresponding to the normal behavior state, the state area of ​​the virtual object in the environment image is adjusted to obtain the second adjusted environment image, and the second adjusted environment image is displayed.

[0117] The guidance information may include a second adjusted environment image. A normal behavior state can refer to effective behavior data being less than a preset execution threshold, i.e., the eating rate being less than a preset rate threshold; a normal display format is to increase the progress bar according to the set threshold; the second adjusted environment image refers to the image after adjusting the state area of ​​the virtual object.

[0118] Specifically, the controller can adjust the state area of ​​the virtual object based on the behavior control strategy and normal display format corresponding to the normal behavior state to obtain a second adjusted environment image, and display the second adjusted environment image on the display unit. Optionally, the state area can be a progress bar, and the display format is a way of adjusting the state area; the normal display format is to increase the progress bar according to set parameters. The image rendering parameters can be image color parameters.

[0119] Optionally, the normal behavior state can be a health reward state, and reward effects can be displayed on the display unit, or the display data of the virtual object can be adjusted to a dynamic virtual image.

[0120] In this embodiment, the displayed screen is adjusted during normal behavior to provide positive feedback to the user, thereby completing positive feedback interaction and cultivating behavioral habits, and improving the user's adaptability.

[0121] In one embodiment, the method further includes:

[0122] Based on a preset behavior recognition model, the first and second physiological perception signals are processed to obtain effective behavior data.

[0123] The preset behavior recognition model can be a pre-trained neural network model, such as a convolutional neural network, a recurrent neural network, or a spatiotemporal graph convolutional network. It should be understood that the model directly outputs the classification results of valid behavior data within the target time period.

[0124] In one instance, such as Figure 3 As shown, Figure 3This is a schematic diagram of a dietary behavior guidance system based on multimodal perception and augmented reality gamification. This system is a specific system schematic diagram of a behavior detection system and can be applied to the embodiments of the behavior guidance data generation method described above. The system may include a wearable display device and a processing unit (controller). The processing unit may be a central processing unit (MCU). The wearable display device may include a multimodal acquisition module (multimodal physiological perception acquisition unit), an augmented reality display interface (display unit), and an alarm unit. The processing unit may include a behavior recognition module, a state control module, an AR rendering module, and a gamified interaction module.

[0125] The multimodal acquisition module is used to simultaneously acquire the wearer's multimodal physiological perception signals and environmental signals (environmental images). This module includes: an electromyography sensing unit for acquiring the bioelectrical activity of the temporalis muscle, a bone conduction sensing unit for acquiring the physical vibration of the jawbone, and a visual acquisition unit for capturing first-view environmental images.

[0126] The behavior recognition module is used to receive data from the multimodal acquisition module and perform signal fusion processing. The module analyzes the muscle contraction characteristics of electromyography signals (first physiological perception feature data) and the vibration spectrum characteristics of bone conduction signals (second physiological perception feature data). Based on the time-domain synchronization and frequency-domain feature matching and logical verification mechanism, it identifies valid chewing events (chewing behavior) and calculates the eating rate based on the number of valid chewing events.

[0127] The state control module includes a preset eating behavior state model (state recognition model) and maps the eating rate parameter to the model in real time. The module is used to determine whether the current eating behavior is in a health reward state (normal behavior state), a critical warning state, or a preset reminder state, and generates corresponding control instructions (behavior control strategy).

[0128] The AR rendering module is used to modulate visual parameters (image rendering parameters) of the target food area in the augmented reality display interface in response to the control commands of the state control module. When in a preset reminder state, the module is configured to adjust the color attributes (such as saturation and hue) or optical overlay attributes of the target food area in real time, so that the displayed content changes dynamically with the eating state.

[0129] Adjusting the image rendering parameters of the environment image also includes, but is not limited to: blurring the target food area (such as Gaussian blur), mosaic occlusion, visually reducing the volume, or overlaying a warning virtual layer (such as rotting effect, warning icon) on the target food.

[0130] The gamified interaction module is used to generate and render a virtual interactive object (virtual object) in the augmented reality display interface. This module establishes a dynamic relationship between the vital signs or behavioral state of the virtual interactive object and the eating behavior state. The virtual interactive object can provide positive incentive feedback or negative empathy feedback in real time according to the wearer's eating rate.

[0131] The alarm unit may include a bone drive actuator and / or a voice prompter, wherein the bone drive actuator is used to output intermittent or continuous vibration feedback, and the voice prompter is used to provide voice prompts.

[0132] It should be understood that Figure 3 The arrows in the text only indicate the direction of data flow.

[0133] In one embodiment, such as Figure 4 As shown, methods for guiding dietary behavior may include the following steps:

[0134] Step 401, Image Acquisition and Perception Data Acquisition: Acquire muscle electrical signals and skeletal conduction signals of the target object through the multimodal physiological perception acquisition unit, and acquire environmental images through the vision acquisition unit.

[0135] Step 402, Feeding Behavior Detection: Extract muscle contraction features from electromyography (EMG) signals and vibration spectrum features of specific frequency bands from skeletal conduction signals; when the muscle contraction features and vibration spectrum features meet preset co-triggered conditions within the same time window, it is determined as a valid feeding action, and the current feeding rate is quantified accordingly; compare the real-time calculated feeding rate with a preset state interval to determine the current feeding behavior level; the preset state interval is a pre-determined correspondence between states and feeding rates, and the feeding behavior level refers to the behavior execution state.

[0136] Step 403: Determine whether the eating behavior level is a normal behavior state.

[0137] Step 404: When the eating behavior level corresponds to the normal behavior state, enter the game reward mode to adjust the virtual interactive object's own state and state area, so that the virtual interactive object's state enters a healthy state. The game reward mode can increase the eating state score, render reward effects, and evolve the growth stage of the virtual interactive object. The normal behavior state can be the state of chewing slowly.

[0138] Step 405: When the eating behavior level corresponds to the preset reminder state, enter the visual interference model. The AR rendering module dynamically modifies the image rendering parameters of the target food area in the display interface to obtain the adjusted food area image. It also adjusts the state and state area of ​​the virtual interactive object. Furthermore, the image rendering parameters cause the image on the display interface to deviate from its original visual attributes. The preset reminder state can be the "wolfing down food" state.

[0139] Step 406: Display the adjusted food area image and the adjusted virtual object on the display interface, and activate the bone conduction actuator and / or voice prompter to output intermittent or continuous vibration feedback or voice prompts to change the user's behavior and actively slow down chewing.

[0140] In this embodiment, due to the hardware configuration of the multimodal acquisition module, the behavior recognition module can automatically distinguish between "speaking" and "eating" interference without relying on the wearer's active input. When the wearer actively adjusts their eating rate (i.e., changes their behavior) due to visual or gamified feedback, the behavior recognition module can capture this change in real time and trigger the state control module to automatically remove negative feedback, thereby completing the cultivation of behavioral habits and positive feedback interaction without manual guidance. Through the dual-channel hardware architecture of "electromyography + bone conduction" and collaborative verification logic, the problem of single sensors being susceptible to motion artifact interference is solved from the physical source. Visual parameter modulation (changing food color) directly affects the visual senses, thereby influencing the user's eating behavior. The gamified interaction module transforms tedious self-discipline into the protection of a virtual avatar, which can improve the user's long-term willingness to use the device. The fully automated closed-loop management realizes the automation of the entire process from perception to guidance, without the need for manual recording by the user, reducing the usage threshold and improving guidance efficiency.

[0141] In one example, such as Figure 5 As shown, Figure 5 The diagram illustrates the field of view of the AR glasses' display interface when in a preset alert state. The upper left corner displays the warning text "Warning: Eating too fast!", the center displays a color-adjusted food image (which may be an image overlaid with a cool-toned filter), and the lower right corner displays a virtual interactive object in an abnormal state animation. It should be understood that this example is for illustrative purposes only and does not constitute a specific limitation.

[0142] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0143] Based on the same inventive concept, this application also provides a behavior guidance data generation apparatus for implementing the behavior guidance data generation method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more behavior guidance data generation apparatus embodiments provided below can be found in the limitations of the behavior guidance data generation method described above, and will not be repeated here.

[0144] In one exemplary embodiment, such as Figure 6 As shown, a behavior guidance data generation device 60 is provided, including: a data acquisition module 61, an analysis module 62, and a generation module 63, wherein:

[0145] Acquisition module 61 is used to acquire multimodal physiological sensing signals of the target object;

[0146] Analysis module 62 is used to analyze the multimodal physiological perception signals based on the behavior recognition model to obtain effective behavior data within the target time period;

[0147] The generation module 63 is used to process the effective behavior data based on the state recognition model to obtain the behavior execution state; determine the behavior control strategy corresponding to the behavior execution state in the correspondence between the execution state and the control strategy; and generate guidance information based on the behavior control strategy and output the guidance information.

[0148] In one embodiment, the multimodal physiological sensing signal includes a first physiological sensing signal and a second physiological sensing signal. The analysis module 62 is used to extract features from the first physiological sensing signal based on the burst feature extraction model to obtain first physiological sensing feature data.

[0149] Based on the energy feature extraction model, feature extraction is performed on the second physiological sensing signal to obtain the second physiological sensing feature data.

[0150] Based on the first physiological perception feature data and the second physiological perception feature data, the number of valid times that meet the preset valid conditions is determined, and based on the number of valid times, the valid behavioral data in the target time period is calculated.

[0151] In one embodiment, the first physiological sensing signal includes a bioelectric signal, and the analysis module 62 is used to filter the bioelectric signal based on an interference filter to obtain a filtered electrical signal.

[0152] Based on the full-wave rectification algorithm, the filtered electrical signal is processed to obtain a unipolar signal;

[0153] The envelope is obtained by processing the unipolar signal based on the envelope extraction algorithm.

[0154] The envelope is divided into multiple sub-envelopes according to a preset calculation period; the first physiological perception feature data corresponding to the sub-envelopes is determined.

[0155] In one embodiment, the second physiological sensing signal includes a skeletal conduction signal, and the analysis module 62 is used to extract and process the skeletal conduction signal based on a preset frequency band filter to obtain a filtered vibration signal.

[0156] Based on a preset frame length, the filtered vibration signal is processed into multiple filtered sub-vibration signals, and the second physiological perception feature data of the signal amplitude of each filtered sub-vibration signal is calculated.

[0157] In one embodiment, determining the number of valid times that satisfy the preset valid conditions based on the first physiological perception feature data and the second physiological perception feature data includes:

[0158] From the first physiological perception feature data and the second physiological perception feature data, each first effective feature value and each second effective feature value that meet the effective threshold are selected respectively;

[0159] Obtain the first time point corresponding to each of the first valid feature values, and the second time point corresponding to each of the second valid feature values;

[0160] The difference between each first time point and each second time point is calculated, and the number of times the difference is less than a preset time difference threshold is determined as the effective number of times the valid condition is met.

[0161] In one embodiment, the behavior execution state includes an abnormal behavior state. The generation module 63 is used to adjust the image rendering parameters of the environmental image and the state area of ​​the virtual object based on the behavior control strategy and abnormal display format corresponding to the abnormal behavior state, to obtain a first adjusted environmental image, and to display the first adjusted environmental image; and / or,

[0162] Based on the abnormal behavior control strategy corresponding to the abnormal behavior state, alarm information is generated and output.

[0163] In one embodiment, the behavior execution state further includes a normal behavior state. The generation module 63 is used to adjust the state area of ​​the virtual object in the environment image based on the behavior control strategy and normal display form corresponding to the normal behavior state to obtain a second adjusted environment image, and to display the second adjusted environment image.

[0164] Each module in the aforementioned behavior guidance data generation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0165] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When executed by the processor, the computer program implements a behavior-guided data generation method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0166] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0167] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0168] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0169] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0170] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0171] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0172] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0173] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for generating behavior-guided data, characterized in that, The method includes: Collect multimodal physiological sensing signals from the target object; Based on the behavior recognition model, the multimodal physiological perception signals are analyzed to obtain effective behavior data within the target time period; Based on the state recognition model, the effective behavior data is processed to obtain the behavior execution state; in the correspondence between the execution state and the control strategy, the behavior control strategy corresponding to the behavior execution state is determined; and guidance information is generated based on the behavior control strategy and output.

2. The method according to claim 1, characterized in that, The multimodal physiological sensing signals include a first physiological sensing signal and a second physiological sensing signal. The analysis of the multimodal physiological sensing signals based on the behavior recognition model yields effective behavioral data within the target time period, including: Based on the burst feature extraction model, feature extraction is performed on the first physiological sensing signal to obtain the first physiological sensing feature data; Based on the energy feature extraction model, feature extraction is performed on the second physiological sensing signal to obtain the second physiological sensing feature data. Based on the first physiological perception feature data and the second physiological perception feature data, the number of valid times that meet the preset valid conditions is determined, and based on the number of valid times, the valid behavioral data in the target time period is calculated.

3. The method according to claim 2, characterized in that, The feature extraction model based on the burst is used to extract features from the first physiological sensing signal to obtain first physiological sensing feature data, including: The first physiological sensing signal is filtered using an interference filter to obtain a filtered electrical signal. Based on the full-wave rectification algorithm, the filtered electrical signal is processed to obtain a unipolar signal; The envelope is obtained by processing the unipolar signal based on the envelope extraction algorithm. The envelope is divided into multiple sub-envelopes according to a preset calculation period; the first physiological perception feature data corresponding to the sub-envelopes is determined.

4. The method according to claim 2, characterized in that, The energy feature extraction model is used to extract features from the second physiological sensing signal to obtain second physiological sensing feature data, including: Based on a preset frequency band filter, the second physiological sensing signal is extracted and processed to obtain a filtered vibration signal; Based on a preset frame length, the filtered vibration signal is processed into multiple filtered sub-vibration signals, and the second physiological perception feature data of the signal amplitude of each filtered sub-vibration signal is calculated.

5. The method according to claim 2, characterized in that, The step of determining the number of valid times that meet the preset valid conditions based on the first physiological perception feature data and the second physiological perception feature data includes: From the first physiological perception feature data and the second physiological perception feature data, each first effective feature value and each second effective feature value that meet the effective threshold are selected respectively; Obtain the first time point corresponding to each of the first valid feature values, and the second time point corresponding to each of the second valid feature values; The difference between each first time point and each second time point is calculated, and the number of times the difference is less than a preset time difference threshold is determined as the effective number of times the valid condition is met.

6. The method according to claim 1, characterized in that, The behavior execution state includes abnormal behavior states. The generation and output of guidance information based on the behavior control strategy includes: Based on the behavior control strategy and abnormal display format corresponding to the abnormal behavior state, the image rendering parameters of the environmental image and the state area of ​​the virtual object are adjusted to obtain a first adjusted environmental image, and the first adjusted environmental image is displayed; and / or, Based on the abnormal behavior control strategy corresponding to the abnormal behavior state, alarm information is generated and output.

7. The method according to claim 1, characterized in that, The behavior execution state also includes a normal behavior state. The generation and output of guidance information based on the behavior control strategy includes: Based on the behavior control strategy and normal display format corresponding to the normal behavior state, the state area of ​​the virtual object in the environment image is adjusted to obtain a second adjusted environment image, and the second adjusted environment image is displayed.

8. A behavior guidance data generation system, characterized in that, The system is applied in the method as described in any one of claims 1 to 7, the system comprising: a controller, a display unit, an alarm unit, and a multimodal physiological sensing and acquisition unit, wherein: The multimodal physiological sensing acquisition unit is used to acquire multimodal physiological sensing signals of the target object; The controller is configured to analyze the multimodal physiological sensing signals based on a behavior recognition model to obtain effective behavior data within a target time period; process the effective behavior data based on a state recognition model to obtain a behavior execution state; determine the behavior control strategy corresponding to the behavior execution state in the correspondence between the execution state and the control strategy; and generate guidance information based on the behavior control strategy and output the guidance information. The display unit and the alarm unit are used to receive the guidance information and to display the guidance information to the target object.

9. A behavior guidance data generation device, characterized in that, The device includes: The acquisition module is used to acquire multimodal physiological sensing signals of the target object; The analysis module is used to analyze the multimodal physiological perception signals based on the behavior recognition model to obtain effective behavioral data within the target time period. The generation module is used to process the effective behavior data based on the state recognition model to obtain the behavior execution state; determine the behavior control strategy corresponding to the behavior execution state in the correspondence between the execution state and the control strategy; and generate guidance information based on the behavior control strategy and output the guidance information.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

Images