A screen control method and system

Abstract

The application provides a screen control method and system, the method comprises collecting target action behaviors, and analyzing target instructions and establishing an interactive mapping table; constructing an action interaction model based on the interactive mapping table and dividing; obtaining user action instructions and collecting user voice information, and identifying and distinguishing user action instructions to obtain different target action instructions; identifying different target action instructions based on a visual interaction area and a behavior interaction area to generate corresponding time and picture indication content, displaying the picture indication content on the screen, and uploading to a terminal; identifying user voice information based on an optimized voice recognition model to generate indication words corresponding to the user voice information, displaying the indication words on the screen, generating trigger signals corresponding to the indication words, and uploading the trigger signals to the terminal. The application can judge the needs of the elderly according to the language and body movements of the elderly, and avoid the situation that real-time monitoring and communication are difficult.

Description

Technical Field

[0001] This invention relates to the field of screen control technology, and in particular to a screen control method and system. Background Technology

[0002] The elderly population is large, growing rapidly, and increasingly exhibiting trends of aging and empty-nesting. A large number of "empty-nest families," "4-2-1 families" (families with four parents, two parents, and one child), and "families where parents and children live separately" are emerging. Family sizes are becoming smaller, more nuclear, and increasingly empty-nest, weakening the family's role in elder care. The demand for professional services in areas such as daily living assistance, rehabilitation, healthcare, and spiritual and cultural enrichment is becoming increasingly prominent. The number of disabled and semi-disabled elderly people requiring care is surging, highlighting the growing need for care and nursing, and consequently increasing the demand for socialized elder care.

[0003] When caring for elderly people in senior living rooms, surveillance cameras are used to monitor them and prevent accidents from happening.

[0004] In existing technologies, when monitoring elderly people in nursing homes using surveillance cameras, it is difficult to provide real-time warnings, which can easily lead to unexpected situations. Furthermore, when elderly people in the room have needs, they can only communicate with the surveillance camera. However, due to the poor communication function of the surveillance camera, it is difficult to effectively convey the specific needs of the elderly people, making it difficult to achieve the purpose of real-time monitoring and communication. Summary of the Invention

[0005] Therefore, the purpose of this invention is to provide a screen control method and system to overcome the shortcomings of the prior art.

[0006] In a first aspect, the present invention provides a screen control method, the method comprising:

[0007] Collect target action behaviors acquired by millimeter-wave radar, analyze target commands based on the target action behaviors, and establish an interaction mapping table between the target action behaviors and the target commands based on the target commands;

[0008] An action interaction model is constructed based on an interaction mapping table, and the interaction areas in the action interaction model are divided to obtain the visual interaction area and the behavioral interaction area.

[0009] A speech recognition model is constructed based on a lightweight ASR model, and the speech recognition model is trained and optimized to obtain an optimized speech recognition model.

[0010] Acquire user action commands and collect user voice information, and identify and distinguish the user action commands to obtain different target action commands;

[0011] Based on the visual interaction area and the behavioral interaction area, the different target action commands are identified to generate corresponding time and image instruction content, which is then displayed on the screen and uploaded to the terminal.

[0012] The optimized speech recognition model is used to recognize the user's speech information to generate an indicator word corresponding to the user's speech information. The indicator word is displayed on the screen, and a trigger signal corresponding to the indicator word is generated and uploaded to the terminal.

[0013] Compared with existing technologies, the beneficial effects of this invention are as follows: By constructing an action interaction model through an interaction mapping table and dividing the action interaction model, the visual interaction area and the behavioral interaction area can recognize the elderly's limb movements. Furthermore, by recognizing non-target action commands, it avoids the waste of resources caused by recognizing and judging the commands of visitors and caregivers. The interaction mapping table can determine the needs of the elderly based on limb movement recognition, thereby enabling real-time early warning of the elderly's physical condition. By optimizing the speech recognition model to recognize the speech information of the elderly in the room, it is possible to promptly understand the needs of the elderly based on their speech information and make corresponding responses in a timely manner. In addition, it is possible to determine the needs of the elderly based on their language and limb movements, avoiding situations where real-time monitoring and communication are difficult.

[0014] Furthermore, the steps of collecting target action behaviors, analyzing target instructions based on the target action behaviors, and establishing an interaction mapping table between the target action behaviors and the target instructions include:

[0015] The target's actions are captured by millimeter-wave radar, and the target's instructions are analyzed according to different time periods.

[0016] Based on the dynamic adjustment strategy, an interaction mapping table between the target action behavior and the target instruction is established.

[0017] Furthermore, the step of constructing an action interaction model based on an interaction mapping table and dividing the interaction regions in the action interaction model to obtain visual interaction regions and behavioral interaction regions includes:

[0018] Extract action behavior feature parameters from the interaction mapping table, and construct an action interaction model based on the action behavior feature parameters. The expression of the action interaction model is as follows:

[0019] ;

[0020] In the formula, Indicates in The overall action state of the target at any given moment. Represents the spatial motion feature vector. Represents the time-dynamic feature matrix. Represents the interaction feature tensor. , , This represents the weight coefficients of three different matrices. Indicates the error correction term;

[0021] The interaction area of ​​the action interaction model is divided into a visual interaction area and a behavioral interaction area according to the area division, and the visual interaction area and the behavioral interaction area are fed back to the screen control system.

[0022] The interaction expression of the visual interaction area is as follows:

[0023] ;

[0024] In the formula, Represents the visual interaction intensity function. Represents the sensitivity coefficient. Indicates the distance to the target. Indicates the visual attenuation coefficient. Indicates the angle of deviation from the target. Indicates the maximum field of view. Represent target features;

[0025] The interaction expression for the behavior interaction area is:

[0026] ;

[0027] In the formula, This represents the behavioral interaction strength function. Indicates the radius of the interaction area. , These represent the velocity influence coefficient and the acceleration influence coefficient, respectively. , These represent the target in time. Instantaneous speed of motion, target in time Instantaneous acceleration.

[0028] Furthermore, the step of training and optimizing the speech recognition model to obtain an optimized speech recognition model includes:

[0029] A target ambiguous pronunciation dataset and a local pronunciation dataset were collected, and the RNNoise algorithm was used to denoise the target ambiguous pronunciation dataset and the local pronunciation dataset.

[0030] The speech recognition model is trained based on the denoised target ambiguous pronunciation dataset and the local pronunciation dataset.

[0031] Furthermore, the steps of acquiring user action commands and collecting user voice information, and identifying and distinguishing the user action commands to obtain different target action commands include:

[0032] The user's action commands are acquired using millimeter-wave radar, and the user's voice information is collected through a microphone array.

[0033] Identify and distinguish the initiators of the user action commands in order to obtain different target action commands from different initiators.

[0034] Furthermore, the step of identifying different target action commands based on the visual interaction area and the behavioral interaction area to generate corresponding time and image instruction content, and displaying the image instruction content on the screen includes:

[0035] The time of issuance of different target action commands is identified based on the visual interaction area, and the different target action commands are identified based on the behavior interaction recognition area and the issuance time to obtain valid target action commands.

[0036] Based on the valid target action command, generate image instruction content and display the image instruction content on the screen.

[0037] Furthermore, the step of displaying the indicator word on the screen and generating a trigger signal corresponding to the indicator word includes:

[0038] A guiding animation is generated on the screen based on the indicated words, and a corresponding trigger signal is generated based on the indicated words;

[0039] Multiple prompts are given based on the trigger signal.

[0040] Secondly, the present invention also provides a screen control system, the system comprising:

[0041] The data collection and analysis module is used to collect target action behaviors acquired by millimeter-wave radar, analyze target commands based on the target action behaviors, and establish an interaction mapping table between the target action behaviors and the target commands based on the target commands.

[0042] A partitioning module is constructed to build an action interaction model based on an interaction mapping table and to partition the interaction areas in the action interaction model to obtain a visual interaction area and a behavioral interaction area.

[0043] A training module is constructed to build a speech recognition model based on a lightweight ASR model and to train and optimize the speech recognition model to obtain an optimized speech recognition model.

[0044] The acquisition and recognition module is used to acquire user action commands and collect user voice information, and to identify and distinguish the user action commands in order to obtain different target action commands;

[0045] The recognition and generation module is used to recognize the different target action commands based on the visual interaction area and the behavioral interaction area, so as to generate corresponding time and image instruction content, display the image instruction content on the screen, and upload it to the terminal;

[0046] The generation and display module is used to recognize the user's voice information based on the optimized speech recognition model, generate an indicator word corresponding to the user's voice information, display the indicator word on the screen, generate a trigger signal corresponding to the indicator word, and upload the trigger signal to the terminal.

[0047] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the screen control method described above.

[0048] Fourthly, the present invention also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the screen control method described above. Attached Figure Description

[0049] Figure 1 This is a flowchart of the screen control method in the first embodiment of the present invention;

[0050] Figure 2 This is a structural block diagram of the screen control system in the second embodiment of the present invention;

[0051] Figure 3 This is a structural block diagram of the electronic device in the third embodiment of the present invention.

[0052] Explanation of key component symbols:

[0053] 10. Data collection and analysis module; 20. Data segmentation module; 30. Data training module; 40. Data acquisition and recognition module; 50. Data generation module; 60. Data generation and display module;

[0054] 70. Bus; 71. Processor; 72. Memory; 73. Communication interface.

[0055] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0056] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0057] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0059] Example 1

[0060] Please see Figure 1 The image shows a screen control method according to a first embodiment of the present invention, the method comprising steps S1 to S6:

[0061] S1, collect target action behavior acquired by millimeter-wave radar, analyze target commands based on the target action behavior, and establish an interaction mapping table between the target action behavior and the target commands based on the target commands;

[0062] Specifically, step S1 includes steps S11 to S12:

[0063] S11, capture the target's actions and behaviors using millimeter-wave radar, and analyze the target's actions and behaviors according to different time periods to obtain target instructions;

[0064] S12, Based on the dynamic adjustment strategy and the target instruction, establish an interaction mapping table between the target action behavior and the target instruction;

[0065] It should be noted that when using millimeter-wave radar to capture the movements of elderly people or other targets in a room, it can not only penetrate blankets but also protect the privacy of the elderly. By analyzing the target's movement behavior and the corresponding commands at different times, the same waving gesture may be a command to change the channel or greet other people in the same room during the day, but may be an emergency call at night. Therefore, by analyzing the target's movement behavior and the corresponding commands at different times, the needs of the elderly's physical movements can be more accurately determined. By dynamically adjusting the different physical commands at different times to correspond to different physical needs, the interaction mapping table can better represent the needs indicated by the target's movement commands.

[0066] S2, construct an action interaction model based on the interaction mapping table, and divide the interaction area in the action interaction model to obtain the visual interaction area and the behavioral interaction area.

[0067] Specifically, step S2 includes steps S21 to S22:

[0068] S21, extract the action behavior feature parameters from the interaction mapping table, and construct an action interaction model based on the action behavior feature parameters. The expression of the action interaction model is:

[0069] ;

[0070] In the formula, Indicates in The overall action state of the target at any given moment. Represents the spatial motion feature vector. Represents the time-dynamic feature matrix. Represents the interaction feature tensor. , , This represents the weight coefficients of three different matrices. Indicates the error correction term;

[0071] S22, the interaction area of ​​the action interaction model is divided into a visual interaction area and a behavioral interaction area according to the area division, and the visual interaction area and the behavioral interaction area are fed back to the screen control system.

[0072] It should be explained that the interaction expression of the visual interaction area is:

[0073] ;

[0074] In the formula, Represents the visual interaction intensity function. Represents the sensitivity coefficient. Indicates the distance to the target. Indicates the visual attenuation coefficient. Indicates the angle of deviation from the target. Indicates the maximum field of view. Represent target features;

[0075] The interaction expression for the behavior interaction area is:

[0076] ;

[0077] In the formula, This represents the behavioral interaction strength function. Indicates the radius of the interaction area. , These represent the velocity influence coefficient and the acceleration influence coefficient, respectively. , These represent the target in time. Instantaneous speed of motion, target in time Instantaneous acceleration;

[0078] Understandably, by capturing the movements of elderly people in the monitoring room and parameterizing some characteristics of their movements, a motion interaction model is constructed based on these parameterized features. It is worth noting that the visual interaction area in the divided motion interaction model is used to monitor the current time in real time and follow and collect video image information of the elderly people in real time, while the behavioral interaction area is used to monitor the behaviors and actions of the elderly people in the video image information in real time and feed back the elderly people's video image information, as well as the corresponding actions in the predetermined time and interaction mapping table, to the screen for display.

[0079] S3. Construct a speech recognition model based on a lightweight ASR model, and train and optimize the speech recognition model to obtain an optimized speech recognition model;

[0080] Specifically, step S3 includes steps S31 to S32:

[0081] S31, Collect the target ambiguous pronunciation dataset and the local pronunciation dataset, and use the RNNoise algorithm to denoise the target ambiguous pronunciation dataset and the local pronunciation dataset;

[0082] S32, The speech recognition model is trained based on the denoised target ambiguous pronunciation dataset and the local pronunciation dataset;

[0083] Understandably, the target ambiguous pronunciation dataset consists of the speech of elderly people with unclear pronunciation, while the local pronunciation dataset includes the local dialect. By using the RNNoise algorithm for noise reduction, the information of ambiguous pronunciation and dialect pronunciation can be more clearly obtained. By training the speech recognition model with the data of both pronunciations, an optimized speech recognition model can be obtained, which can effectively recognize the speech information spoken by local elderly people.

[0084] S4, acquire user action commands and collect user voice information, and identify and distinguish the user action commands to obtain different target action commands;

[0085] Specifically, step S4 includes steps S41 to S42:

[0086] S41, acquire the user's action commands based on millimeter-wave radar, and collect the user's voice information through a microphone array;

[0087] S42, Identify and distinguish the initiator of the user action command in order to obtain different target action commands from different initiators;

[0088] Understandably, when millimeter-wave radar captures the movements of elderly people or other targets in a room, it can not only penetrate blankets but also protect the privacy of the elderly. Furthermore, the array of microphones can collect the elderly's voice information more comprehensively and clearly. By identifying the identity of the person initiating the action command, it can detect different target action commands from different initiators, such as caregivers or visitors, in the room. This avoids identifying visitors' commands and allows the radar to identify only the action commands of the elderly people in the room who need monitoring.

[0089] S5, based on the visual interaction area and the behavioral interaction area, identify the different target action commands to generate corresponding time and image instruction content, display the image instruction content on the screen, and upload it to the terminal;

[0090] Specifically, step S5 includes steps S51 to S52:

[0091] S51, based on the visual interaction area, identify the issuance time of the different target action commands, and based on the behavior interaction recognition area and the issuance time, identify the different target action commands to obtain valid target action commands;

[0092] S52, Generate image instruction content according to the effective target action command, and feed the image instruction content back to the screen;

[0093] Understandably, the system identifies the current room brightness through the visual interaction area and determines whether the current time period is within a rest period based on the real-time network time. Then, it identifies different target action commands during this time period through the behavior interaction recognition area, thereby obtaining the target, namely the elderly person's action commands. Based on the interaction mapping table, it determines the needs contained in the action commands, then feeds the needs back on the screen and uploads them to the monitoring terminal, such as the monitoring room or the monitor's mobile phone, so that timely measures can be taken.

[0094] S6, based on the optimized speech recognition model, the user's speech information is recognized to generate an indicator word corresponding to the user's speech information, the indicator word is displayed on the screen, a trigger signal corresponding to the indicator word is generated, and the trigger signal is uploaded to the terminal;

[0095] Specifically, step S6 includes steps S61 to S62:

[0096] S61, generate a guide animation on the screen according to the instruction, and generate a corresponding trigger signal according to the instruction;

[0097] S62, provide multiple prompts based on the trigger signal;

[0098] Understandably, by optimizing the speech recognition model to identify the user's voice information, the user's needs can be identified. When the needs can be temporarily alleviated, such as feeling hot or cold at the current time, prompts such as "turn on the air conditioner" can be displayed on the screen, and a guiding animation and an animation of operating the remote control can be generated. The animation of operating the remote control can be photographed in advance and played on the screen at a predetermined number of frames, while simultaneously issuing an indication signal sound to provide a prompt. The triggered indication signal is then uploaded to the terminal to remind the monitoring personnel.

[0099] In summary, the screen control method in the above embodiments of the present invention constructs an action interaction model through an interaction mapping table and divides the action interaction model, enabling the visual interaction area and the behavioral interaction area to recognize the elderly's limb movements. By recognizing non-target action commands, it avoids the waste of resources caused by recognizing and judging the commands of visitors and caregivers. The interaction mapping table can determine the needs of the elderly based on limb movements, thereby enabling real-time warnings of the elderly's physical condition. By optimizing the speech recognition model to recognize the speech information of the elderly in the room, it can promptly understand the needs of the elderly based on their speech information and make corresponding responses in a timely manner. Furthermore, it can determine the needs of the elderly based on their language and limb movements, avoiding situations where real-time monitoring and communication are difficult.

[0100] Example 2

[0101] This invention also provides a screen control system; please refer to [link / reference]. Figure 2 The image shows a screen control system according to a second embodiment of the present invention, the system comprising:

[0102] The collection and analysis module 10 is used to collect target action behaviors acquired by millimeter-wave radar, analyze target commands based on the target action behaviors, and establish an interaction mapping table between the target action behaviors and the target commands based on the target commands.

[0103] A partitioning module 20 is constructed to build an action interaction model based on an interaction mapping table and to partition the interaction areas in the action interaction model to obtain a visual interaction area and a behavioral interaction area.

[0104] A training module 30 is constructed to build a speech recognition model based on a lightweight ASR model and to train and optimize the speech recognition model to obtain an optimized speech recognition model.

[0105] The acquisition and recognition module 40 is used to acquire user action commands and collect user voice information, and to identify and distinguish the user action commands in order to obtain different target action commands;

[0106] The recognition and generation module 50 is used to recognize the different target action commands based on the visual interaction area and the behavioral interaction area, so as to generate corresponding time and image instruction content, display the image instruction content on the screen, and upload it to the terminal.

[0107] The generation and display module 60 is used to recognize the user's voice information based on the optimized speech recognition model, generate an indicator word corresponding to the user's voice information, display the indicator word on the screen, generate a trigger signal corresponding to the indicator word, and upload the trigger signal to the terminal.

[0108] In some alternative embodiments, the collection and analysis module 10 includes:

[0109] The analysis unit is used to capture target actions and behaviors through millimeter-wave radar and analyze the target instructions of the target actions and behaviors according to different time periods.

[0110] The establishment unit is used to establish an interaction mapping table between the target action behavior and the target instruction based on the dynamic adjustment strategy and the target instruction.

[0111] In some alternative embodiments, the construction partitioning module 20 includes:

[0112] An extraction unit is used to extract action behavior feature parameters from the interaction mapping table and construct an action interaction model based on the action behavior feature parameters. The expression of the action interaction model is:

[0113] ;

[0114] In the formula, Indicates in The overall action state of the target at any given moment. Represents the spatial motion feature vector. Represents the time-dynamic feature matrix. Represents the interaction feature tensor. , , This represents the weight coefficients of three different matrices. Indicates the error correction term;

[0115] The partitioning unit is used to divide the interaction area of ​​the action interaction model into a visual interaction area and a behavioral interaction area according to the region partitioning, and to feed back the visual interaction area and the behavioral interaction area to the screen control system.

[0116] The interaction expression of the visual interaction area is as follows:

[0117] ;

[0118] In the formula, Represents the visual interaction intensity function. Represents the sensitivity coefficient. Indicates the distance to the target. Indicates the visual attenuation coefficient. Indicates the angle of deviation from the target. Indicates the maximum field of view. Represent target features;

[0119] The interaction expression for the behavior interaction area is:

[0120] ;

[0121] In the formula, This represents the behavioral interaction strength function. Indicates the radius of the interaction area. , These represent the velocity influence coefficient and the acceleration influence coefficient, respectively. , These represent the target in time. Instantaneous speed of motion, target in time Instantaneous acceleration.

[0122] In some alternative embodiments, the construction training module 30 includes:

[0123] The acquisition unit is used to acquire the target ambiguous pronunciation dataset and the local pronunciation dataset, and to denoise the target ambiguous pronunciation dataset and the local pronunciation dataset using the RNNoise algorithm;

[0124] The training unit is used to train the speech recognition model based on the denoised target ambiguous pronunciation dataset and the local pronunciation dataset.

[0125] In some optional embodiments, the acquisition and identification module 40 includes:

[0126] A collection unit is used to acquire the user's action commands based on millimeter-wave radar and to collect the user's voice information through a microphone array.

[0127] The differentiation unit is used to identify and differentiate the initiator of the user action command in order to obtain different target action commands from different initiators.

[0128] In some alternative embodiments, the identification generation module 50 includes:

[0129] The recognition unit is used to recognize the issuance time of the different target action commands based on the visual interaction area, and to recognize the different target action commands based on the behavior interaction recognition area and the issuance time, so as to obtain valid target action commands.

[0130] The first generation unit is used to generate image instruction content according to the effective target action instruction, and to feed the image instruction content back to the screen.

[0131] In some alternative embodiments, the display generation module 60 includes:

[0132] The second generation unit is used to generate a guide animation on the screen according to the instruction word, and to generate a corresponding trigger signal according to the instruction word;

[0133] The prompting unit is used to provide multiple prompts based on the trigger signal.

[0134] The functions or operation steps implemented by the above modules and units are largely the same as those in the above method embodiments, and will not be repeated here.

[0135] The screen control system provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the system embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0136] Example 3

[0137] The present invention also proposes an electronic device, please refer to [link to relevant documentation]. Figure 3The image shows an electronic device according to a third embodiment of the present invention.

[0138] The electronic device may include a processor 71 and a memory 72 storing computer program instructions.

[0139] Specifically, the processor 71 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement this application.

[0140] The memory 72 may include a mass storage device for data or instructions. For example, and not limitingly, the memory 72 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 72 may include removable or non-removable (or fixed) media. Where appropriate, the memory 72 may be internal or external to a data processing device. In a particular embodiment, the memory 72 is non-volatile memory. In a particular embodiment, the memory 72 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), Extended Data Out Dynamic Random-Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0141] The memory 72 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 71.

[0142] The processor 71 implements the screen control method of Embodiment 1 by reading and executing computer program instructions stored in the memory 72.

[0143] In some embodiments, the electronic device may further include a communication interface 73 and a bus 70. For example, Figure 3 As shown, the processor 71, memory 72, and communication interface 73 are connected through bus 70 and complete communication with each other.

[0144] The communication interface 73 is used to enable communication between the various modules, devices, units, and / or equipment in this application. The communication interface 73 can also enable data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations.

[0145] Bus 70 includes hardware, software, or both, that couples the components of a device together. Bus 70 includes, but is not limited to, at least one of the following: data bus, address bus, control bus, expansion bus, and local bus. For example, and not as a limitation, bus 70 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 70 may include one or more buses. Although this application describes and illustrates a specific bus, this application considers any suitable bus or interconnection.

[0146] The electronic device can access the screen control system and execute the screen control method of this embodiment.

[0147] In addition, in conjunction with the screen control method in Embodiment 1 above, this application can provide a storage medium for implementation. The storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement the screen control method of Embodiment 1 above.

[0148] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0149] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A screen control method characterized by, The method comprises: collecting target action behaviors collected by a millimeter wave radar, analyzing target instructions according to the target action behaviors, and establishing an interactive mapping table of the target action behaviors and the target instructions according to the target instructions; constructing an action interaction model based on the interactive mapping table, and dividing an interactive region in the action interaction model to obtain a visual interaction region and a behavior interaction region, which specifically comprises: extracting action behavior characteristic parameters in the interactive mapping table, and constructing an action interaction model according to the action behavior characteristic parameters, an expression of the action interaction model being: ； In the formula, represents the target comprehensive motion state at the time , represents a spatial motion feature vector, represents a time dynamic feature matrix, represents an interaction feature tensor, , , represents three different matrix weight coefficients, represents an error correction term; dividing the interactive region of the action interaction model into the visual interaction region and the behavior interaction region according to region division, and feeding back the visual interaction region and the behavior interaction region to a screen control system; wherein an interactive expression of the visual interaction region is: ； wherein, represents a visual interaction intensity function, represents a sensitivity coefficient, represents a target distance, represents a visual attenuation coefficient, represents a target deviation angle, represents a maximum field of view angle, represents a target feature; an interactive expression of the behavior interaction region is: ； wherein, represents a behavior interaction intensity function, represents a radius of the interaction zone, , respectively represent a speed influence coefficient, an acceleration influence coefficient, , respectively represent an instantaneous motion speed of the target at time , an instantaneous acceleration of the target at time ; constructing a speech recognition model based on a lightweight ASR model, and training and optimizing the speech recognition model to obtain an optimized speech recognition model; obtaining user action instructions and collecting user speech information, and identifying and distinguishing the user action instructions to obtain different target action instructions; identifying the different target action instructions based on the visual interaction region and the behavior interaction region to generate corresponding time and picture instruction contents, displaying the picture instruction contents on a screen, and uploading the picture instruction contents to a terminal; identifying the user speech information based on the optimized speech recognition model to generate instruction words corresponding to the user speech information, displaying the instruction words on the screen, generating trigger signals corresponding to the instruction words, and uploading the trigger signals to the terminal.

2. The screen control method according to claim 1, characterized by, The step of collecting target action behaviors collected by a millimeter wave radar, analyzing target instructions according to the target action behaviors, and establishing an interactive mapping table of the target action behaviors and the target instructions according to the target instructions comprises: capturing target action behaviors by a millimeter wave radar, and analyzing target instructions of the target action behaviors according to different time periods; establishing an interactive mapping table of the target action behaviors and the target instructions according to a dynamic adjustment strategy and based on the target instructions.

3. The screen control method according to claim 1, wherein The step of training and optimizing the speech recognition model to obtain an optimized speech recognition model comprises: collecting target ambiguous pronunciation data sets and local pronunciation data sets, and denoising the target ambiguous pronunciation data sets and the local pronunciation data sets by using an RNNoise algorithm; training the speech recognition model based on the denoised target ambiguous pronunciation data sets and the local pronunciation data sets.

4. The screen control method of claim 1, wherein, The step of obtaining user action instructions and collecting user speech information, and identifying and distinguishing the user action instructions to obtain different target action instructions comprises: obtaining the user action instructions based on a millimeter wave radar, and collecting the user speech information by using a microphone array; identifying and distinguishing initiators of the user action instructions to obtain different target action instructions of different initiators.

5. The screen control method of claim 1, wherein, The step of identifying the different target action instructions based on the visual interaction area and the behavioral interaction area to generate corresponding time and picture indication content and displaying the picture indication content on the screen comprises: Identifying the time of issuing the different target action instructions based on the visual interaction area and identifying the different target action instructions based on the behavioral interaction area and the time of issuing to obtain effective target action instructions; Generating picture indication content according to the effective target action instructions and feeding back the picture indication content to the screen.

6. The screen control method of claim 1, wherein, The step of displaying the indication word on the screen and generating a trigger signal corresponding to the indication word comprises: Generating a guide animation on the screen according to the indication word and generating a corresponding trigger signal according to the indication word; Multiple prompts are performed according to the trigger signal.

7. A screen control system characterized by comprising: The system comprises: A collection and analysis module for collecting target action behaviors collected by a millimeter wave radar, analyzing target instructions according to the target action behaviors, and establishing an interaction mapping table of the target action behaviors and the target instructions according to the target instructions; A construction and division module for constructing an action interaction model based on the interaction mapping table and dividing the interaction areas in the action interaction model to obtain a visual interaction area and a behavioral interaction area; The construction and division module comprises: An extraction unit for extracting action behavior characteristic parameters in the interaction mapping table and constructing an action interaction model according to the action behavior characteristic parameters, the expression of the action interaction model being: ； In the formula, represents the target comprehensive motion state at the time , represents a spatial motion feature vector, represents a time dynamic feature matrix, represents an interaction feature tensor, , , represents three different matrix weight coefficients, represents an error correction term; A division unit for dividing the interaction areas of the action interaction model into a visual interaction area and a behavioral interaction area according to area division, and feeding back the visual interaction area and the behavioral interaction area to a screen control system; The interaction expression of the visual interaction area is: ； wherein, represents a visual interaction intensity function, represents a sensitivity coefficient, represents a target distance, represents a visual attenuation coefficient, represents a target deviation angle, represents a maximum field of view angle, represents a target feature; The interaction expression of the behavioral interaction area is: ； wherein represents a behavior interaction intensity function, represents a radius of the interaction zone, , respectively represent a speed influence coefficient, an acceleration influence coefficient, , respectively represent an instantaneous motion speed of the target at time , an instantaneous acceleration of the target at time . A construction and training module for constructing a speech recognition model based on a lightweight ASR model and training and optimizing the speech recognition model to obtain an optimized speech recognition model; An acquisition and identification module for acquiring user action instructions and collecting user speech information and identifying and distinguishing the user action instructions to obtain different target action instructions; An identification and generation module for identifying the different target action instructions based on the visual interaction area and the behavioral interaction area to generate corresponding time and picture indication content, displaying the picture indication content on the screen, and uploading to a terminal; A generation and display module for identifying the user speech information based on the optimized speech recognition model to generate an indication word corresponding to the user speech information, displaying the indication word on the screen, generating a trigger signal corresponding to the indication word, and uploading the trigger signal to a terminal.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the screen control method of any one of claims 1-6.

9. A storage medium having stored thereon a computer program, characterized in that The program is executed by the processor to implement the screen control method of any one of claims 1-6.

Images