Millimeter wave scene interaction method, device and terminal based on human sensing recognition

Abstract

The invention discloses a millimeter wave scene interaction method and device based on human sensing recognition and a terminal. The method comprises the steps that a first signal is received, a millimeter wave human sensing recognition model is initialized based on the first signal, an initial human sensing interaction model is obtained, and the initial human sensing interaction model comprises recognition parameters corresponding to a plurality of scene modes; receiving a second signal, and adjusting identification parameters corresponding to a plurality of scene modes in the initial human-sensing interaction model based on the second signal to obtain a target human-sensing interaction model; and receiving a signal for opening a target scene, and performing human sensing recognition interaction based on recognition parameters corresponding to the target scene in the target human sensing interaction model. According to the millimeter wave scene interaction method based on human sensing recognition provided by the invention, the user can accurately interact with the television through the millimeter wave module, and more comfortable experience is brought to the user.

Description

Technical Field

[0001] The present invention relates to the field of millimeter wave interaction, and in particular to a millimeter wave scene interaction method, device and terminal based on human sensor recognition. Background Art

[0002] In the prior art, millimeter waves have begun to be used as a human body sensing medium in smart TVs. However, although millimeter waves can provide some basic services and scene interactions through data collection and analysis, their application scenario construction and interaction models in smart TVs still lack accuracy and humanity. As long as it is within the recognition range of the millimeter wave, changes in the recognition data of any object may trigger a scene of interaction with a "person". But sometimes, the target of these data changes is not the human itself, or the user is not in the TV viewing area, which leads to a high probability of false triggering scenes. Therefore, there is an urgent need for a method based on human body sensing that can accurately identify the data changes generated by a specific actor corresponding to the scene according to the current scene and interact with it.

[0003] Therefore, the prior art still needs to be improved and enhanced. Summary of the invention

[0004] The present invention provides a millimeter wave scene interaction method, device and terminal based on human sensor recognition, aiming to solve the problem that the prior art smart TV has a high false touch rate when interacting based on a millimeter wave module and cannot accurately identify the specific actor corresponding to the scene.

[0005] A first aspect of the present invention provides a millimeter wave scene interaction method based on human sensor recognition, comprising:

[0006] receiving a first signal, and initializing a millimeter wave human sensing recognition model based on the first signal to obtain an initial human sensing interaction model, wherein the initial human sensing interaction model includes recognition parameters corresponding to a plurality of scene modes;

[0007] receiving a second signal, and adjusting recognition parameters corresponding to a plurality of the scene modes in the initial human-sensing interaction model based on the second signal to obtain a target human-sensing interaction model;

[0008] A signal for opening a target scene is received, and human-sensing recognition interaction is performed based on recognition parameters corresponding to the target scene in the target human-sensing interaction model.

[0009] In one implementation, the initializing the millimeter wave human sense recognition model based on the first signal to obtain an initial human sense interaction model includes:

[0010] Acquire initial accuracy and initial features of the millimeter wave human perception recognition model based on the first signal;

[0011] Collect target data samples based on the initial accuracy, and construct a target information map based on the target data samples, wherein the target information map includes type information of the identified target and location information of the target, wherein the type information is an object or a person;

[0012] The initial human-sensing interaction model is obtained based on the initial features and the target information map.

[0013] In one implementation, the initial accuracy includes an initial recognition distance, an initial recognition range, and an initial recognition area;

[0014] The initial features include initial user events, initial information map useful features, initial information map useless features and initial specified features.

[0015] In one implementation, the collecting target data samples based on the initial accuracy and constructing a target information map based on the target data samples include:

[0016] Collecting millimeter wave information within the initial identification range, and constructing an initial information map based on the millimeter wave information;

[0017] Identify all objects in the initial information map, and record the name and location of each object in the initial information map;

[0018] A third signal is received, and the initial information map is adjusted based on the third signal to obtain a target information map.

[0019] In one implementation, adjusting recognition parameters corresponding to a plurality of scene modes in the initial human-sensing interaction model based on the second signal includes:

[0020] Setting a target recognition area so that human sensory interactions outside the target recognition area are not recognized;

[0021] Set the target child's height so that the human interaction of children of other heights will not be recognized in children's mode.

[0022] In one implementation, the performing human sense recognition interaction based on the recognition parameters corresponding to the target scene in the target human sense interaction model includes:

[0023] When the target human-sensing interaction information is identified, it is determined whether the target human-sensing interaction information meets the human-sensing recognition parameters carried by the target scene. If it meets the human-sensing recognition parameters carried by the target scene, the millimeter wave application scenario corresponding to the target human-sensing interaction information is triggered; if it does not meet the human-sensing recognition parameters carried by the target scene, the millimeter wave application scenario corresponding to the target human-sensing interaction is not triggered.

[0024] In one implementation, the determining whether the target human-sensing interaction information meets the human-sensing recognition parameters of the target scene, and if the human-sensing recognition parameters of the target scene are met, triggering the millimeter wave application scenario corresponding to the target human-sensing interaction includes:

[0025] If the target person is detected to have left the target recognition area or entered a sleeping state, the current display content will be paused;

[0026] If it is detected that a person other than the target person is in a sleeping state, turn down the volume of the TV;

[0027] If it is detected that a child is closer than the target distance to the TV, the current display content will be paused and a prompt box will pop up indicating that the viewing distance is too close.

[0028] A second aspect of the present invention is a millimeter wave scene interaction device based on human sensor recognition, comprising:

[0029] An initialization module, configured to receive a first signal, initialize a millimeter wave human sense recognition model based on the first signal, and obtain an initial human sense interaction model, wherein the initial human sense interaction model includes recognition parameters corresponding to a plurality of scene modes;

[0030] an adjustment module, configured to receive a second signal, and adjust recognition parameters corresponding to a plurality of the scene modes in the initial human-sensing interaction model based on the second signal to obtain a target human-sensing interaction model;

[0031] The interaction module is used to receive a signal for opening a target scene and perform human sense recognition interaction based on recognition parameters corresponding to the target scene in the target human sense interaction model.

[0032] According to a third aspect of the present invention, a terminal is provided, wherein the terminal includes: a memory, a processor, and a millimeter wave scene interaction program based on human sense recognition stored in the memory and executable on the processor, wherein the millimeter wave scene interaction program based on human sense recognition, when executed by the processor, implements the steps of the millimeter wave scene interaction method based on human sense recognition as described above.

[0033] A fourth aspect of the present invention provides a storage medium, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any of the above-mentioned millimeter wave scene interaction methods based on human perception recognition.

[0034] Beneficial effect: Compared with the prior art, the present invention provides a millimeter wave scene interaction method, device and terminal based on human sense recognition. In the millimeter wave scene interaction method based on human sense recognition provided by the present invention, when a TV loaded with a millimeter wave module is turned on, a first signal is received, and a millimeter wave human sense recognition model is initialized based on the first signal to obtain an initial human sense interaction model, the initial human sense interaction model includes recognition parameters corresponding to multiple scene modes, and then a second signal is received, and the recognition parameters corresponding to multiple scene modes in the initial human sense interaction model are adjusted based on the second signal to obtain a target human sense interaction model. After obtaining the target human sense interaction model, a signal to open the target scene is received to perform human sense recognition interaction based on the recognition parameters corresponding to the target scene in the target human sense interaction model. The present invention provides users with a millimeter wave scene interaction method based on human sensor recognition, which solves the problems in the prior art that the smart TV has a high false touch rate when interacting based on the millimeter wave module, cannot accurately identify the specific actors corresponding to the scene, and is not accurate and humane enough. It provides a method that can accurately identify people and the interaction between specific actors and millimeter wave scenes, so that users can interact accurately through the millimeter wave module when watching TV, bringing a more comfortable experience to users. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A flowchart of an embodiment of a millimeter wave scene interaction method based on human sensor recognition provided by the present invention;

[0036] Figure 2 A user usage flow chart of an embodiment of the millimeter wave scene interaction method based on human sensor recognition provided by the present invention;

[0037] Figure 3 A structural principle diagram of an embodiment of a millimeter wave scene interaction device based on human sensor recognition provided by the present invention;

[0038] Figure 4 A schematic diagram of an operating environment of an embodiment of a terminal provided by the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0040] Embodiment 1

[0041] The millimeter wave scene interaction method based on human sensor recognition provided in this embodiment can be executed by a terminal, and the terminal can be but is not limited to a smart TV, a computer, etc., and a smart TV is used as an example for explanation below.

[0042] In the prior art, since the construction and interaction model of millimeter wave in smart TV application scenarios is not accurate and user-friendly enough, for example, as long as the recognition data of a target changes within the millimeter wave recognition range, it is possible to trigger the millimeter wave to recognize that there are people in the current scene and perform scene interaction based on actions. However, in many cases, the target of the data change is not a person, but other movable home devices, such as sweeping robots, etc., which will lead to the erroneous triggering of some interactive scenes due to incorrect recognition. Therefore, in this embodiment, a method based on human body sensing is provided to accurately identify people and specific actors interacting with millimeter wave scenes.

[0043] Specifically, Figure 1 As shown, the millimeter wave scene interaction method based on human sensor recognition provided in this embodiment includes the steps of:

[0044] S100, receiving a first signal, and initializing a millimeter-wave human-sensing recognition model based on the first signal to obtain an initial human-sensing interaction model, wherein the initial human-sensing interaction model includes recognition parameters corresponding to multiple scene modes.

[0045] Reference Figure 2 When a user uses the smart TV provided by this embodiment, the millimeter wave human perception recognition model will be initialized first.

[0046] Specifically, after the smart TV is installed, when the user uses the smart TV for the first time and turns on the smart TV, since the smart TV is equipped with a millimeter wave module, the initialization setting of the human-sensing interaction will be performed in the power-on wizard stage, and the millimeter wave human-sensing recognition model can be initialized in this initialization stage. In this embodiment, the number of millimeter waves is 1, and the installation position is located in the middle of the lower frame or the middle of the upper frame of the TV. In more embodiments, the installation position of the millimeter wave module can be other positions, and the number can also be multiple, without limitation. The millimeter wave module can provide environmental information collection capabilities, and the location and attributes of the target can be identified based on the millimeter wave module.

[0047] In this embodiment, the initializing the millimeter wave human sense recognition model based on the first signal to obtain an initial human sense interaction model includes:

[0048] S110, acquiring initial accuracy and initial features of the millimeter wave human perception recognition model based on the first signal;

[0049] S120, collecting target data samples based on the initial accuracy, and constructing a target information map based on the target data samples, wherein the target information map includes type information of the identified target and location information of the target, wherein the type information is an object or a person;

[0050] S130: Obtain the initial human-sensing interaction model based on the initial features and the target information map.

[0051] The initial accuracy includes the initial recognition distance, the initial recognition range and the initial recognition area;

[0052] The initial features include initial user events, initial information map useful features, initial information map useless features and initial specified features.

[0053] Specifically, when the millimeter wave human sensing recognition model is initialized, the parameters of the millimeter wave human sensing recognition service object may be set.

[0054] In this embodiment, the parameter setting of the millimeter wave human sensing recognition service object includes setting the initial accuracy and initial features of the millimeter wave human sensing recognition model. The initial accuracy includes the initial recognition distance, the initial recognition range and the initial recognition area. Further, the initial accuracy may also include the type of items that need to be recognized or not recognized; the initial features include initial user events, initial information map useful features, initial information map useless features and initial designated features. Among them, the initial information map has useful information for marking the features that need to be recognized in the information map, the initial information map useless features are used to mark the features that do not need to be recognized in the information map, and the initial designated features are used to specify the features that need to be recognized in the marked information in the information map. The parameters of the millimeter wave human sensing recognition service object are set so that when the user uses the smart TV, a scenario-based interactive service experience for the current scene can be provided based on the different application scenarios corresponding to the millimeter wave human sensing interaction function.

[0055] Furthermore, the initialization of the millimeter-wave human sensing recognition model can be reset. If the user wants to restore the initial state, or make adjustments based on the initial state, the user can enter the millimeter-wave human sensing recognition model through the reset entrance to make adjustments to reset the millimeter-wave human sensing recognition model.

[0056] In the initialization stage of the millimeter wave human perception recognition model, the recognition data samples are also collected. Specifically, the target data samples are collected, and the target information map is constructed based on the target data samples, including:

[0057] Collecting millimeter wave information within the initial identification range, and constructing an initial information map based on the millimeter wave information;

[0058] Identify all objects in the initial information map, and record the name and location of each object in the initial information map;

[0060] A third signal is received, and the initial information map is adjusted based on the third signal to obtain a target information map.

[0061] Specifically, the target data samples are collected during the millimeter wave initialization phase, and the target information map is constructed based on the target data samples. The target information map records the initial recognition distance and all targets within the initial recognition range. In this embodiment, the smart TV is equipped with a large human sensing model, so the attributes of all targets can also be identified based on the large human sensing model, for example, whether the identified target is an object or a person. Furthermore, the specific object name or person type under each attribute can also be identified based on the large human sensing model.

[0062] That is to say, in this embodiment, the information recognized by the millimeter wave can be annotated and classified based on the human sensing large model. For example, the collected people will be classified and annotated, and the number and category of people in the map will be annotated. In this embodiment, the categories of people can be divided into three types, such as adults, the elderly or children; at the same time, the collected objects are also classified and annotated, such as furniture, home appliances and small objects, among which furniture includes sofas, tables, stools, etc., home appliances include refrigerators, fans, sweeping robots, etc., and small objects include green plants, ornaments, etc. The targets classified and annotated based on the human sensing large model will be dynamically presented in the information map.

[0063] Specifically, by collecting the millimeter wave information within the initial recognition range based on the initial accuracy, the initial information map can be constructed.

[0064] However, in this embodiment, it is not enough to construct the initial information map based on the human-sensing large model, because the human-sensing large model cannot ensure that the type of person or item identified is completely correct. Moreover, the items and characteristics of people purchased by each family are different. For example, if a human-shaped sculpture is placed in the living room, it may be recognized as a person by the millimeter wave large model, or a sweeping robot may pass by, which may make the millimeter wave large model think that it is a child walking. Furthermore, some elderly people may look younger, which will make the millimeter wave not recognize them as elderly people. Therefore, the user needs to query the initial information map, correct it, and define the information annotation of the identification content.

[0065] Specifically, the initial information map constructed in real time will pop up during the initialization phase. When the user queries the initial information map, if an identification error is found, the error information in the initial information map can be adjusted by sending the third signal. In this way, a more correct target information map can be obtained based on the user's correction. The target information map is more suitable for the user's home situation than the initial information map. However, many millimeter wave recognitions in the prior art do not have a user error correction step, resulting in frequent identification errors and execution of incorrect interactive instructions.

[0066] After obtaining the target information map after manual error correction, the method further includes:

[0067] The initial human-sensing interaction model is obtained based on the initial features and the target information map.

[0068] Specifically, the millimeter wave human perception recognition model includes different scene modes by default, such as elderly mode, children mode, dynamic mode, relaxation mode, etc. In different scene modes, each target in the initial information map is marked with the recognition type based on the initial features corresponding to different scene modes, and the recognition type includes recognition required, recognition not required, and specified recognition, so as to obtain the initial human perception interaction model. For example, in the children's mode, the human perception interaction signal of the designated recognition character type is a child; such as furniture, household appliances, and small items, which are not included in the parameter settings of the millimeter wave human perception recognition service object.

[0069] S200, receiving a second signal, and adjusting recognition parameters corresponding to the plurality of scene modes in the initial human-sensing interaction model based on the second signal to obtain a target human-sensing interaction model.

[0070] The adjusting, based on the second signal, the recognition parameters corresponding to the plurality of scene modes in the initial human-sensing interaction model includes:

[0071] Setting a target recognition area so that human sensory interactions outside the target recognition area are not recognized;

[0072] Set the target child's height so that the human interaction of children of other heights will not be recognized in children's mode.

[0073] Constructing a millimeter wave human perception recognition model through the information collection capability of millimeter waves and the information of information annotation maps

[0074] Specifically, after obtaining the target information map and constructing the initial human-sensing interaction model based on the target information map, the user can also adjust the recognition parameters corresponding to each of the scene modes in the initial human-sensing interaction model by sending a second signal.

[0075] Specifically, a target recognition distance can be defined, and the initial recognition distance in the initial human-sensing interaction model can be changed to the target recognition distance to adjust the recognition range. Specifically, in this embodiment, the default initial recognition distance is the maximum distance that the millimeter wave module can collect, but some families may have an integrated living room and dining room. In this way, the smart TV can easily recognize the status of the people in the restaurant to adjust the TV playback status according to the status of the people in the restaurant. This is equivalent to recognizing the wrong adjustment instruction and executing the wrong action. Therefore, the user can define the target recognition distance by shortening the distance so that the recognition distance is only within the range of the user watching TV, avoiding the problem of incorrectly recognizing the status of distant people and executing the wrong action.

[0076] Furthermore, different recognition distances can be customized according to different scene modes. For example, in children's mode, the target recognition distance can be adjusted to a smaller distance to suit children's viewing experience. For example, the millimeter wave module defaults to the maximum recognition distance that can be provided. If the millimeter wave module hardware specification supports 5 meters, it is 5 meters. However, in children's mode, the recognition distance of the millimeter wave module can be defined as 1.5 meters to suit children's viewing experience.

[0077] In this embodiment, the recognition area can also be defined, and the target recognition area is defined as the recognition area of ​​the human-sensing interaction model. For example, the default is the entire range of millimeter wave data collection, such as 5 meters * 3 meters. However, in a certain millimeter wave application scenario mode, for example, when there are many people at home, there are also many people walking back and forth. At this time, a sofa viewing mode can be defined, and the sofa area can be designated as the recognition area of ​​the millimeter wave module. In this way, only the status of the people in the sofa area can be identified, so as to perform corresponding actions according to one or more task states of the sofa area.

[0078] Furthermore, in this embodiment, target-specified features can also be set. In the default scene mode, the millimeter wave module can recognize the feature values ​​of all people within the recognition range by default, but in the scene mode of a specific millimeter wave application, the target-specified features can be customized. For example, in the children's mode, only the feature values ​​marked as children will trigger the corresponding instructions. In the elderly mode, only the feature values ​​marked as elderly will trigger the corresponding instructions.

[0079] Furthermore, in this embodiment, special user characteristics can also be defined. Specifically, for the millimeter wave module, the default support specification is to be able to identify all characteristic values ​​of the corresponding characters in the current scene. However, the user can further define the instructions corresponding to the user time in the millimeter wave application scenario. For example, in relaxation mode, when a characteristic event of the user sleeping is detected, the TV is turned off; if a characteristic event of the corresponding user walking back and forth is detected, the TV playback picture is paused; if a characteristic event of the elderly and children watching TV together is detected, the TV volume is turned down and some TV letters are played.

[0080] Furthermore, the recognition events can also be defined. Specifically, the default recognition event is given by the feature collection of millimeter waves and the calculation of the large model. Usually, the human-sensing interaction application defines a group of event groups that can be interacted through millimeter waves. The event group is a standard event group. Selecting the event group can obtain the default millimeter-wave human-sensing recognition model parameters: recognition distance, recognition area, recognition features, human-sensing interaction instructions and other expandable parameter definitions. Based on the defined parameters and the millimeter-wave collection information map, the millimeter-wave human-sensing recognition parameters are constructed and wait for the millimeter-wave application scenario to call. In this embodiment, the application scenario of the target human-sensing interaction model defaults to the human-sensing interaction constructed by the human-sensing large model according to the service scope and experience design of the application. By sending the second signal, which is a signal for adjusting the recognition parameters corresponding to the scene mode in the human-sensing interaction model, it can be personalized and customized to make the target human-sensing interaction model more suitable for the use of current family members.

[0081] After obtaining the target human-sensing interaction model, millimeter-wave scene interaction can be performed based on the target human-sensing interaction model when using a smart TV.

[0082] S300: Receive a signal for opening a target scene, and perform human-sensing recognition interaction based on recognition parameters corresponding to the target scene in the target human-sensing interaction model.

[0083] Specifically, when switching to an application that supports millimeter-wave human-sensing interaction scenarios, you can start the human-sensing interaction scenario through the human-sensing service switch control, obtain the parameter calling interface provided by the target human-sensing recognition model, and recommend the best human-sensing recognition parameters for the application scenario built based on the large model based on the calling interface, or customize the human-sensing recognition model parameters for the application interaction scenario based on the parameters.

[0084] The performing human sense recognition interaction based on the recognition parameters corresponding to the target scene in the target human sense interaction model includes:

[0085] When the target human-sensing interaction information is identified, it is determined whether the target human-sensing interaction information meets the human-sensing recognition parameters carried by the target scene. If it meets the human-sensing recognition parameters carried by the target scene, the millimeter wave application scenario corresponding to the target human-sensing interaction information is triggered; if it does not meet the human-sensing recognition parameters carried by the target scene, the millimeter wave application scenario corresponding to the target human-sensing interaction is not triggered.

[0086] The determining whether the target human-sensing interaction information meets the human-sensing recognition parameters of the target scene, and if it meets the human-sensing recognition parameters of the target scene, triggering the millimeter wave application scenario corresponding to the target human-sensing interaction includes:

[0087] If the target person is detected to have left the target recognition area or entered a sleeping state, the current display content will be paused;

[0088] If it is detected that a person other than the target person is in a sleeping state, turn down the volume of the TV;

[0089] If it is detected that a child is closer than the target distance to the TV, the current display content will be paused and a prompt box will pop up indicating that the viewing distance is too close.

[0090] Specifically, when the target human sensing recognition model obtains the application running status of the human sensing interaction scene, and the service location and millimeter wave human sensing interaction switch of the current application are in the on state, the human sensing interaction information within the target recognition range will be identified in real time based on the human sensing recognition parameters carried by the target human sensing large model. When the human sensing interaction information within the target recognition range meets the millimeter wave human sensing recognition parameters corresponding to the target scene, the application instructions of the human sensing interaction scene are triggered. If it does not meet the requirements, no scene will be triggered.

[0091] For example, in the children's mode: in the children's mode, through the application of the children's mode, enter the setting of the human sensing scene. The human sensing scene setting obtains the millimeter wave human sensing recognition model parameters that are open to the children's mode application by default in the human sensing large model, such as the recognition distance is 1.5 meters; the recognition area is the fan-shaped area 11-22-33-44 directly in front of the TV; the recognition feature is the feature value marked as a child. In this embodiment, the user can define which feature value is a child, such as child A-feature value A001, child B-feature value B001; the recognition event is that the viewing distance is less than 1.5 meters, and the first recognition area such as 10-20-30-40 is for watching TV at a close distance. At this time, the TV screen needs to be paused, and the first recognition area is a certain area circled in the identifiable area as the recognition event response range.

[0092] Furthermore, the user can also create a user-defined recognition event, that is, define the event name, recognition distance, recognition area, recognition features, human interaction instructions, etc. For example, the human interaction instruction is to pause TV playback, play a reminder sound, or switch to the eye protection interface.

[0093] Based on this, the user changes the recognition parameters of the initial human-sensing interaction model through the human-sensing scene settings in the initialization stage, such as changing the recognition distance to 1 meter, changing the recognition area to two areas or expanding or reducing the range of the recognition area, specifying that only the feature of child A watching a movie is triggered, etc. In the target human-sensing interaction model obtained in this way, in its corresponding children's mode, the millimeter-wave human-sensing recognition model parameters do not obtain the millimeter-wave acquisition parameters involved in non-parameters such as furniture, home appliances, and small items in the millimeter-wave acquisition information map, eliminating the accuracy differences of the millimeter-wave service caused by these parameters, and only taking the parameters that the user needs to use in this scene after the millimeter-wave information map is marked.

[0094] In this way, the system obtains the human sensing settings related to the children's mode. When the user turns on and uses the children's mode, the millimeter wave module will continue to collect the data sensed by the millimeter wave recognition. In this embodiment, it is assumed that the millimeter wave recognizes that the user's TV usage environment control is 4 meters * 5 meters, and the coordinates of the recognition area centered on the TV are AA:BB:CC:DD; the recognized features include: home-sofa-coordinates are AA-EE-FF-GG, home-chair-coordinates are BB-EE-FF-GG, home N-home type N-coordinates are N, people-elderly (L001)-coordinates are AA-EE-FF-GG, people-child A (A001)-coordinates are BB-EE-FF-GG, etc.; the behavioral events to be recognized include: people-elderly (L001)-lying on the sofa to sleep, people-child A (A001) is watching TV, people-child A (A001)-coordinates (BB-EE-FF-GG) are within 1.5 meters, which belongs to close-range viewing. In this way, through the identified event, such as when a person-child A (A001) watches TV at a close distance, the instructions are automatically executed: pause the TV, play a reminder voice "the viewing distance is too close", and open a prompt box including the words "the viewing distance is too close". In other words, when the target human-sensing interaction model obtains the data sensed by the millimeter wave module and matches the identification event (such as the above-mentioned behavioral event triggers the child to watch a movie at a close distance, the event execution instruction is paused, the reminder voice XXX is played, and the interface XX is opened), it is determined whether it meets the adjustment of the triggering identification event. If it meets the requirements, it is triggered. If it does not meet the requirements, the millimeter wave continues to collect data and the human-sensing interaction module continues to determine whether the conditions meet the requirements.

[0095] This embodiment provides a millimeter wave scene interaction method based on human perception recognition, which collects recognition data samples through the millimeter wave initialization stage, and constructs an initial millimeter wave collection information map based on the initial recognition distance, recognition range, recognized people / objects, etc. - the initial information map. The initial information map is based on the human perception large model carried by the smart TV to mark and classify the recognized information, such as people can be divided into adults, children, and the elderly; furniture can be divided into sofas, tables, stools, etc.; home appliances can be divided into refrigerators, fans, etc.; small items can be divided into green plants, ornaments, etc. Then, the user can query, correct, and define the information annotation of the recognition content through the information of the map to obtain the target information map. Then, based on the initial features and the target information map, the initial human perception interaction model can be obtained.

[0096] The initial human-sensing interaction model constructed through the information collection capability of millimeter waves and the information of the information annotation map can perform recognition distance definition, recognition area definition, behavior event definition, recognition feature definition, recognition event definition and other expandable parameter definitions. Based on the defined parameters and the millimeter wave collection information map, the millimeter wave human-sensing recognition parameters suitable for the current user's home are constructed to obtain the call of the target human-sensing interaction model for the millimeter wave application scenario.

[0097] When a user uses an application that supports millimeter-wave human-sensing interaction scenarios, by obtaining the parameter calling interface provided by the target human-sensing interaction model, such as controlling whether to start the application of the human-sensing interaction scenario through the human-sensing service switch, the optimal human-sensing recognition parameters recommended by the application scenario constructed by the target human-sensing interaction model can be called, and the user can customize the human-sensing recognition model parameters of the application interaction scenario based on the parameters. In this way, based on the target human-sensing interaction model, the application running status of the human-sensing interaction scenario is obtained. When the service of the application and the millimeter-wave human-sensing interaction switch are in the on state, the identification data information of the millimeter-wave human-sensing large model is obtained. When the information meets the millimeter-wave human-sensing recognition parameters, the application instructions of the human-sensing interaction scenario are triggered. It brings a better millimeter-wave application experience to users.

[0098] In summary, this embodiment provides a millimeter wave scene interaction method based on human perception recognition. When a TV loaded with a millimeter wave module is turned on, by receiving a first signal, the millimeter wave human perception recognition model is initialized based on the first signal to obtain an initial human perception interaction model, the initial human perception interaction model includes recognition parameters corresponding to multiple scene modes, and then a second signal is received. Based on the second signal, the recognition parameters corresponding to multiple scene modes in the initial human perception interaction model are adjusted to obtain a target human perception interaction model. After obtaining the target human perception interaction model, by receiving a signal to open the target scene, human perception recognition interaction can be performed based on the recognition parameters corresponding to the target scene in the target human perception interaction model. This embodiment provides a millimeter wave scene interaction method based on human perception recognition for users, which solves the problems in the prior art that the false touch rate of smart TVs based on millimeter wave modules is high, the specific actor corresponding to the scene cannot be accurately identified, and the specific actor is not accurate and humanized. It provides a method that can accurately identify people and specific actors interacting with millimeter wave scenes, so that users can accurately interact through millimeter wave modules when watching TV, bringing users a more comfortable experience.

[0099] It should be understood that, although the various steps in the flowcharts given in the accompanying drawings of the present invention are displayed in sequence according to the indications of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, the execution of the steps in the present invention is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps of the present invention may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0100] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by signaling related hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0101] Embodiment 2

[0102] Based on the above embodiments, the present invention also provides a millimeter wave scene interaction device based on human sensor recognition, and its functional module schematic diagram is as follows: Figure 3 As shown, the millimeter wave scene interaction device based on human sensor recognition includes:

[0103] An initialization module, configured to receive a first signal, initialize a millimeter wave human sensing recognition model based on the first signal, and obtain an initial human sensing interaction model, wherein the initial human sensing interaction model includes recognition parameters corresponding to a plurality of scene modes, as specifically described in Embodiment 1;

[0104] an adjustment module, configured to receive a second signal, and adjust recognition parameters corresponding to a plurality of the scene modes in the initial human-sensing interaction model based on the second signal to obtain a target human-sensing interaction model, as specifically described in Embodiment 1;

[0105] The interaction module is used to receive a signal for opening a target scene, and perform human-sensing recognition interaction based on the recognition parameters corresponding to the target scene in the target human-sensing interaction model, as specifically described in Embodiment 1.

[0106] Embodiment 3

[0107] like Figure 4As shown, based on the above-mentioned millimeter wave scene interaction method based on human sensor recognition, the present invention also provides a terminal accordingly, and the terminal includes a processor 10, a memory 20 and a display 30. Figure 4 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0108] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a millimeter wave scene interaction program 40 based on human perception recognition is stored on the memory 20, and the millimeter wave scene interaction program 40 based on human perception recognition can be executed by the processor 10, thereby realizing the millimeter wave scene interaction method based on human perception recognition in the present application.

[0109] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the millimeter wave scene interaction method based on human sensor recognition.

[0110] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0111] In one embodiment, when the processor 10 executes the millimeter wave scene interaction program 40 based on human sensor recognition in the memory 20, the following steps are implemented:

[0112] receiving a first signal, and initializing a millimeter wave human sensing recognition model based on the first signal to obtain an initial human sensing interaction model, wherein the initial human sensing interaction model includes recognition parameters corresponding to a plurality of scene modes;

[0113] receiving a second signal, and adjusting recognition parameters corresponding to a plurality of the scene modes in the initial human-sensing interaction model based on the second signal to obtain a target human-sensing interaction model;

[0114] A signal for opening a target scene is received, and human-sensing recognition interaction is performed based on recognition parameters corresponding to the target scene in the target human-sensing interaction model.

[0115] The initializing the millimeter wave human sensing recognition model based on the first signal to obtain an initial human sensing interaction model includes:

[0116] Acquire initial accuracy and initial features of the millimeter wave human perception recognition model based on the first signal;

[0117] Collect target data samples based on the initial accuracy, and construct a target information map based on the target data samples, wherein the target information map includes type information of the identified target and location information of the target, wherein the type information is an object or a person;

[0118] The initial human-sensing interaction model is obtained based on the initial features and the target information map.

[0119] Wherein, the initial accuracy includes the initial recognition distance, the initial recognition range and the initial recognition area;

[0120] The initial features include initial user events, initial information map useful features, initial information map useless features and initial specified features.

[0121] Wherein, said collecting target data samples based on the initial accuracy and constructing a target information map based on the target data samples include:

[0122] Collecting millimeter wave information within the initial identification range, and constructing an initial information map based on the millimeter wave information;

[0123] Identify all objects in the initial information map, and record the name and location of each object in the initial information map;

[0124] A third signal is received, and the initial information map is adjusted based on the third signal to obtain a target information map.

[0125] The adjusting, based on the second signal, the recognition parameters corresponding to the multiple scene modes in the initial human-sensing interaction model includes:

[0126] Setting a target recognition area so that human sensory interactions outside the target recognition area are not recognized;

[0127] Set the target child's height so that the human interaction of children of other heights will not be recognized in children's mode.

[0128] The performing human sense recognition interaction based on the recognition parameters corresponding to the target scene in the target human sense interaction model includes:

[0129] When the target human-sensing interaction information is identified, it is determined whether the target human-sensing interaction information meets the human-sensing recognition parameters carried by the target scene. If it meets the human-sensing recognition parameters carried by the target scene, the millimeter wave application scenario corresponding to the target human-sensing interaction information is triggered; if it does not meet the human-sensing recognition parameters carried by the target scene, the millimeter wave application scenario corresponding to the target human-sensing interaction is not triggered.

[0130] The step of judging whether the target human-sensing interaction information meets the human-sensing recognition parameters of the target scene, and if the human-sensing recognition parameters of the target scene are met, triggering the millimeter wave application scenario corresponding to the target human-sensing interaction includes:

[0131] If the target person is detected to have left the target recognition area or entered a sleeping state, the current display content will be paused;

[0132] If it is detected that a person other than the target person is in a sleeping state, turn down the volume of the TV;

[0133] If it is detected that a child is closer than the target distance to the TV, the current display content will be paused and a prompt box will pop up indicating that the viewing distance is too close.

[0134] Embodiment 4

[0135] The present invention also provides a storage medium storing one or more programs, which can be executed by one or more processors to implement the steps of the millimeter wave scene interaction method based on human perception recognition described in the above embodiment.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A millimeter wave scene interaction method based on human sensor recognition, characterized in that: include: receiving a first signal, and initializing a millimeter wave human sensing recognition model based on the first signal to obtain an initial human sensing interaction model, wherein the initial human sensing interaction model includes recognition parameters corresponding to a plurality of scene modes; receiving a second signal, and adjusting recognition parameters corresponding to a plurality of the scene modes in the initial human-sensing interaction model based on the second signal to obtain a target human-sensing interaction model; A signal for opening a target scene is received, and human-sensing recognition interaction is performed based on recognition parameters corresponding to the target scene in the target human-sensing interaction model.

2. The millimeter wave scene interaction method based on human sensory recognition according to claim 1 is characterized in that: Initializing the millimeter wave human perception recognition model based on the first signal to obtain an initial human perception interaction model includes: Acquire initial accuracy and initial features of the millimeter wave human perception recognition model based on the first signal; Collect target data samples based on the initial accuracy, and construct a target information map based on the target data samples, wherein the target information map includes type information of the identified target and location information of the target, wherein the type information is an object or a person; The initial human-sensing interaction model is obtained based on the initial features and the target information map.

3. The millimeter wave scene interaction method based on human sensory recognition according to claim 2 is characterized in that: The initial accuracy includes the initial recognition distance, the initial recognition range and the initial recognition area; The initial features include initial user events, initial information map useful features, initial information map useless features and initial specified features.

4. The millimeter wave scene interaction method based on human sensory recognition according to claim 3 is characterized in that: The collecting target data samples based on the initial accuracy and constructing a target information map based on the target data samples include: Collecting millimeter wave information within the initial identification range, and constructing an initial information map based on the millimeter wave information; Identify all objects in the initial information map, and record the name and location of each object in the initial information map; A third signal is received, and the initial information map is adjusted based on the third signal to obtain a target information map.

5. The millimeter wave scene interaction method based on human sensory recognition according to claim 1 is characterized in that: The adjusting, based on the second signal, the recognition parameters corresponding to the plurality of scene modes in the initial human-sensing interaction model includes: Setting a target recognition area so that human sensory interactions outside the target recognition area are not recognized; Set the target child's height so that the human interaction of children of other heights will not be recognized in children's mode.

6. The millimeter wave scene interaction method based on human sensory recognition according to claim 1 is characterized in that: The performing human sense recognition interaction based on the recognition parameters corresponding to the target scene in the target human sense interaction model includes: When the target human-sensing interaction information is identified, it is determined whether the target human-sensing interaction information meets the human-sensing recognition parameters carried by the target scene. If it meets the human-sensing recognition parameters carried by the target scene, the millimeter wave application scenario corresponding to the target human-sensing interaction information is triggered; if it does not meet the human-sensing recognition parameters carried by the target scene, the millimeter wave application scenario corresponding to the target human-sensing interaction is not triggered.

7. The millimeter wave scene interaction method based on human sensory recognition according to claim 6 is characterized in that: The determining whether the target human-sensing interaction information meets the human-sensing recognition parameters of the target scene, and if it meets the human-sensing recognition parameters of the target scene, triggering the millimeter wave application scenario corresponding to the target human-sensing interaction includes: If the target person is detected to have left the target recognition area or entered a sleeping state, the current display content will be paused; If it is detected that a person other than the target person is in a sleeping state, turn down the volume of the TV; If it is detected that a child is closer than the target distance to the TV, the current display content will be paused and a prompt box will pop up indicating that the viewing distance is too close.

8. A millimeter wave scene interaction device based on human sensor recognition, characterized in that: The device comprises: An initialization module, configured to receive a first signal, initialize a millimeter wave human sense recognition model based on the first signal, and obtain an initial human sense interaction model, wherein the initial human sense interaction model includes recognition parameters corresponding to a plurality of scene modes; an adjustment module, configured to receive a second signal, and adjust recognition parameters corresponding to a plurality of the scene modes in the initial human-sensing interaction model based on the second signal to obtain a target human-sensing interaction model; The interaction module is used to receive a signal for opening a target scene and perform human-sensing recognition interaction based on recognition parameters corresponding to the target scene in the target human-sensing interaction model.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a millimeter wave scene interaction program based on human sense recognition stored in the memory and executable on the processor. When the millimeter wave scene interaction program based on human sense recognition is executed by the processor, the steps of the millimeter wave scene interaction method based on human sense recognition as described in any one of claims 1 to 7 are implemented.

10. A storage medium, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the millimeter wave scene interaction method based on human sense recognition as described in any one of claims 1-7.

Images