Perception Processing Method, Device and Electronic Device for Human-Computer Interaction

Abstract

The present application provides a perception processing method, device, electronic device and computer-readable storage medium for human-computer interaction; the method includes: displaying an interaction process in a human-computer interaction interface; detecting a scene that appears during the interaction process; when the scene that appears during the interaction process is a marked scene for which perception information is configured by a custom operation, obtaining a perception information configuration file corresponding to the marked scene; and outputting the perception information corresponding to the perception information configuration file during the progress of the scene. Through the present application, it is possible to make full use of the peripherals of the electronic device to flexibly customize the interaction effect.

Description

Technical Field

[0001] This application relates to computer human-computer interaction technology, and in particular, to a perception processing method, apparatus, electronic device, and computer-readable storage medium for human-computer interaction. Background Art

[0002] The electronic devices in the related art have rich peripherals, which expand the presentation channels of the perception environment. For example, when there is an incoming call reminder, it can be prompted by the ringing output by the sound peripheral, can be prompted by the light output by the visual peripheral, and can be prompted by the vibration output by the tactile peripheral. For the application program client running on the electronic device, it will be authorized to call the peripheral interface of the electronic device during its operation, so that during the operation of the application program client, a rich human-computer interaction experience can be provided for users through diverse perception information presentation methods.

[0003] However, in the related art, the presentation of perception information is controlled by a small number of developers, and the presented methods have a high degree of uniformity and limited number, resulting in difficulty in meeting the diverse needs of users and difficulty in fully and effectively utilizing the hardware resources of the electronic device. Summary of the Invention

[0004] The embodiments of this application provide a perception processing method, apparatus, electronic device, and computer-readable storage medium for human-computer interaction, which can make full use of the peripherals of the electronic device to realize the customization of perception information.

[0005] The technical solution of the embodiments of this application is implemented as follows:

[0006] The embodiments of this application provide a perception processing method for human-computer interaction, including:

[0007] Displaying an interaction process in a human-computer interaction interface;

[0008] Detecting a scene that appears during the interaction process;

[0009] When the scene that appears during the interaction process is a marked scene for which perception information is configured by a custom operation, obtaining a perception information configuration file corresponding to the marked scene;

[0010] Outputting the perception information corresponding to the perception information configuration file during the progress of the scene.

[0011] The embodiments of this application provide a perception processing apparatus for human-computer interaction, including:

[0012] A display module, configured to display an interaction process in a human-computer interaction interface;

[0013] A detection module, configured to detect a scene that appears during the interaction process;

[0014] An acquisition module, configured to acquire a perception information configuration file corresponding to the marked scene when the scene that appears during the interaction process is a marked scene configured with perception information for a custom operation;

[0015] An output module, configured to output the perception information corresponding to the perception information configuration file during the progress of the scene.

[0016] In the above solution, the types of perception information configured by the perception information configuration file include tactile feedback information, and the tactile feedback information includes at least one of the following characteristics: frequency, intensity, duration, orientation; the output module is further configured to: output tactile feedback information configured to be associated with the type of the event according to the type of the event included in the scene.

[0017] In the above solution, the output module is further configured to: perform at least one of the following operations: in a marked scene including a user operation event, synchronously output tactile feedback information synchronized with the user operation event; in a marked scene including sound, output at least one of the following: tactile feedback information adapted to the type of the sound, tactile feedback information consistent with the intensity of the sound, tactile feedback information consistent with the orientation of the sound source of the sound; in a marked scene including a collision event, output at least one of the following: tactile feedback information synchronized with the collision event, tactile feedback information consistent with the intensity of the collision event; in a marked scene including a notification event, output tactile feedback information synchronized with the notification event.

[0018] In the above solution, the types of perception information configured by the perception information configuration file include visual feedback information, and the visual feedback information includes at least one of the following characteristics: state, frequency, color, brightness; the output module is further configured to: output visual feedback information configured to be associated with the type of the event according to the type of the event included in the scene.

[0019] In the above solution, the output module is further configured to: perform at least one of the following operations: in a marked scene including a notification event, output visual feedback information synchronized with the notification event; in a marked scene including sound, output at least one of the following: visual feedback information adapted to the type of the sound, visual feedback information adapted to the rhythm of the sound; in a marked scene including a progress event, output visual feedback information representing the progress of the progress event; in a marked scene including a result event, output visual feedback information representing the result of the result event.

[0020] In the above solution, before obtaining the perception information configuration file corresponding to the marked scenario, the device further includes: a configuration module, configured to: determine the marked scenario; display the types of perception information that can be configured for the marked scenario, and the characteristics that can be configured in each type; wherein, the types of perception information include: tactile feedback information; visual feedback information; olfactory feedback information; auditory feedback information; in response to a custom operation for the marked scenario, obtain the parameters configured by the custom operation for the characteristics in at least one of the types, and generate a perception information configuration file corresponding to the marked scenario based on the obtained parameters of the characteristics in at least one of the types.

[0021] In the above solution, the configuration module is further configured to: display the historical interaction process in the human-computer interaction interface, where the historical interaction process is earlier than the interaction process; in response to a marking operation for a target scenario in the historical interaction process, record the target scenario as the marked scenario.

[0022] In the above solution, the configuration module is further configured to: before responding to a custom operation for the marked scenario, play the recorded screen file of the historical interaction process; when the marked scenario appears during the playback of the recorded screen file, display an entry for custom perception information for the marked scenario; in response to a trigger operation for the entry, display a custom interface for receiving the custom operation and including the types of perception information that can be configured for the marked scenario, and the characteristics that can be configured in each type.

[0023] In the above solution, the configuration module is further configured to: in response to a marking operation for a target scenario during the interaction process, record the target scenario as the marked scenario.

[0024] In the above solution, the configuration module is further configured to: after determining the marked scenario and before responding to a custom operation for the marked scenario, display an entry for custom perception information for the marked scenario; in response to a trigger operation for the entry, display a custom interface for receiving the custom operation and including the types of perception information that can be configured for the marked scenario, and the characteristics that can be configured in each type.

[0025] In the above solution, each type of characteristic has pre-configured default parameters; the configuration module is further configured to: in response to the modification of the default parameters of the characteristics in at least one of the types by the custom operation, update the default parameters to the parameters set by the custom operation.

[0026] In the above solution, the configuration module is further configured to: display multiple pre-configured modes of the marked scenario; where each of the modes includes at least one type of perception information associated with the marked scenario, and default parameters pre-configured for the characteristics in the type; obtain at least one type of perception information associated with the marked scenario included in the target mode, and default parameters pre-configured for the characteristics in the type; where the target mode is the mode selected by the custom operation from the multiple pre-configured modes.

[0027] In the above solution, the configuration module is further configured to: obtain candidate parameters for the characteristics in at least one of the types; perform a scoring prediction process on the candidate parameters through a first neural network model to obtain a prediction score for each candidate parameter; use the candidate parameter with the highest prediction score as the parameter for the characteristics in at least one of the types; where the first neural network model is trained based on candidate parameter samples and the true scores corresponding to the candidate parameter samples.

[0028] In the above solution, the configuration module is further configured to: obtain operation behavior data in the human-computer interaction interface; perform a preference prediction process on the operation behavior data through a second neural network model to obtain the operation preference represented by the operation data; obtain parameters for the characteristics in at least one of the types pre-associated with the operation preference; where the second neural network model is trained based on operation data samples and the operation preferences represented by the operation data samples.

[0029] In the above solution, the output module is further configured to: before outputting the perception information corresponding to the perception information profile, query the types of perception information supported for customization by the electronic device and the characteristics supported for customization in the corresponding types; according to the query result, filter out the types of perception information not supported for customization and the characteristics in the corresponding types in the perception information profile to update the perception information profile.

[0030] In the above solution, the output module is further configured to: obtain from the perception information profile the types of perception information configured by the custom operation and the parameters of the characteristics in the corresponding types; send a call request through an application programming interface opened to the human-computer interaction interface to call a peripheral corresponding to the type to output the perception information of the corresponding type, so that the output perception information of the corresponding type conforms to the parameters of the characteristics in the corresponding type.

[0031] An embodiment of the present application provides a perception processing device for human-computer interaction, including:

[0032] A configuration module, configured to receive a custom operation for a marked scenario in a human-computer interaction interface;

[0033] The configuration module is further configured to, in response to the custom operation, generate a perception information configuration file corresponding to the marked scenario according to the type of the perception information configured by the custom operation and the parameters of the features in the corresponding type.

[0034] The display module is configured to display the interaction process on the human-computer interaction interface.

[0035] The output module is configured to, when the scenario that appears again during the interaction process is the marked scenario, output the perception information of the type configured by the custom operation during the progress of the scenario, and the output perception information of the corresponding type conforms to the parameters of the features in the corresponding type.

[0036] An embodiment of the present application provides an electronic device, including:

[0037] A memory for storing executable instructions;

[0038] A processor, configured to implement the perception processing method for human-computer interaction provided by the embodiment of the present application when executing the executable instructions stored in the memory.

[0039] An embodiment of the present application provides a computer-readable storage medium, storing executable instructions, which are configured to implement the perception processing method for human-computer interaction provided by the embodiment of the present application when being executed by a processor.

[0040] The embodiment of the present application has the following beneficial perception effects:

[0041] By flexibly configuring the perception information of the marked scenario through custom operations, the corresponding perception information configuration file can be reused when the same marked scenario reappears, so as to call the hardware resources in the electronic device to output perception information, presenting an atmosphere adapted to the scenario and creating an immersive personalized interaction experience, ultimately realizing the effective utilization of the hardware resources of the electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1A-1B is a schematic structural diagram of a perception processing system for human-computer interaction provided by an embodiment of the present application;

[0043] Figure 2 is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

[0044] Figure 3A-3D is a schematic flowchart of a perception processing method for human-computer interaction provided by an embodiment of the present application;

[0045] Figure 4 is a schematic flowchart of a perception processing method for human-computer interaction provided by an embodiment of the present application;

[0046] Figure 5 It is an application schematic diagram of the perception processing method for human-computer interaction provided by an embodiment of the present application;

[0047] Figure 6 It is an application schematic diagram of the perception processing method for human-computer interaction provided by an embodiment of the present application;

[0048] Figure 7 It is an application schematic diagram of the perception processing method for human-computer interaction provided by an embodiment of the present application;

[0049] Figure 8 It is an application schematic diagram of the perception processing method for human-computer interaction provided by an embodiment of the present application;

[0050] Figure 9 It is a protocol schematic diagram of the perception processing method for human-computer interaction provided by an embodiment of the present application;

[0051] Figure 10A-10B It is an interface schematic diagram of the perception processing method for the human-computer interaction interface provided by an embodiment of the present application. Detailed implementation manners

[0052] In order to make the objectives, technical solutions, and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be construed as limitations on the present application. All other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present application.

[0053] In the following description, "some embodiments" are involved, which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0054] In the following description, the terms "first / second" involved are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first / second" can be interchanged with a specific order or sequence when allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0056] Before further elaborating on the embodiments of the present application, the nouns and terms involved in the embodiments of the present application are described, and the nouns and terms involved in the embodiments of the present application are applicable to the following explanations.

[0057] 1) Intelligent terminal: Also known as a mobile intelligent terminal, it refers to an electronic device with rich human-computer interaction methods, the ability to access the Internet, usually equipped with various operating systems, and having strong processing capabilities. Mobile intelligent terminals include smartphones, tablets, in-vehicle terminals, handheld game consoles, etc.

[0058] 2) Tactile: Tactile is the sensation generated when the skin tactile receptors contact mechanical stimuli. In an intelligent terminal, vibration can be generated through devices such as motors to produce tactile sensations.

[0059] 3) Lighting effect: Some light strips can be set on the front, side, and back of the intelligent terminal, and different lighting effect atmospheres can be generated through the on / off state, brightness, color, and other characteristics of these light strips.

[0060] 4) Responsive to, used to represent the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more executed operations can be real-time or have a set delay; without special instructions, there is no restriction on the execution order of multiple executed operations.

[0061] 5) Client, an application program running on the terminal for providing various services, such as game clients, military exercise simulation clients, etc.

[0062] Intelligent terminals in the related art have very rich peripherals. For example, vibration motors, ambient lights, etc. The peripherals can bring a personalized experience to users. There are also rich interaction scenarios in the client (such as game clients, video clients, etc.). Usually, the intelligent terminal is pre-set with output effects (such as vibration or lighting effects). The client calls the pre-set output effects of the intelligent terminal in specific scenarios. The intelligent terminal will pre-define several types of output effects and open interfaces to upper-layer clients such as games. During the interaction process of the client (for example, during the game interaction process), when the client enters a specific scenario, it calls the pre-set interface of the intelligent terminal to trigger the corresponding vibration or lighting effects, thereby outputting the corresponding tactile feedback information (vibration) and visual feedback information (lighting effect). However, due to the limited number of pre-defined output effects, it is impossible to fully customize the corresponding tactile feedback information (vibration) and visual feedback information (lighting effect) according to the personalized scenarios of the client. When implementing the embodiments of the present application, the applicant found that there is a lack of a complete solution in the related art to combine rich interaction scenarios (clients) with personalized experiences (peripherals can provide personalized experiences to users). Therefore, it is impossible to provide users with a custom function that combines the above two, it is difficult to make full use of the peripherals of the intelligent terminal running the client, and it is difficult to present an atmosphere adapted to the interaction scenarios in the client, and it is impossible to create an immersive interaction experience.

[0063] The embodiments of the present application provide a perception processing method, device, electronic device, and computer-readable storage medium for human-computer interaction, which can make full use of the peripherals of the electronic device to flexibly customize the interaction effect. The following describes the exemplary applications of the electronic device provided by the embodiments of the present application. The electronic device provided by the embodiments of the present application can be implemented as various types of user terminals such as laptop computers, tablet computers, desktop computers, set-top boxes, mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable game devices), or can be implemented as a server.

[0064] In an implementation scenario, refer to Figure 1A , Figure 1A which is a schematic structural diagram of the perception processing system for human-computer interaction provided by the embodiments of the present application. To support a game client 410-1, the terminal 400-1 is connected to the server 200 through the network 300. The network 300 can be a wide area network, a local area network, or a combination of the two. In response to the user's custom operation, the terminal 400-1 configures the association between the marked scene and the peripheral output effect (perception information) in the game client 410-1. The perception information configuration file can be locally stored in the terminal 400-1 and a prompt of successful configuration is presented. When the user uses the game client 410-1 running in the terminal 400-1 (sending application interaction operations and receiving application interaction results), the interaction process between the user and the client 410-1 can be presented in the terminal 400-1. If the marked scene is reproduced in the game client 410-1, the perception information configuration file is obtained from the local of the terminal 400-1, and the corresponding peripheral output effect (perception information) is output. For example, if the marked scene custom-configured by the user is the scene of a grenade hitting the ground and exploding in the game, and the associated peripheral output effect (perception information) is that the blue light stays on for 5 seconds and vibrates, then during the process of the user using the game client 410-1 (while presenting the interaction process), if the scene of the grenade hitting the ground and exploding appears in the game client 410-1, the associated peripheral output effect (perception information) corresponding to the "grenade hitting the ground and exploding scene" is output, that is, the terminal 400-1 used by the user can control the output effect of "the blue light stays on for 5 seconds and vibrates".

[0065] In some embodiments, the perception information configuration file generated based on the configured association can be sent to the server 200 (the server corresponding to the client), that is, the perception information configuration file is stored in the server 200 corresponding to the client 410-1.

[0066] In an implementation scenario, refer to Figure 1B , Figure 1BFIG. 0 is a schematic structural diagram of a perception processing system for human-computer interaction provided by an embodiment of the present application. To support a video client, terminal 400-2 and terminal 400-3 are connected to server 200 through network 300. Network 300 can be a wide area network, a local area network, or a combination of the two. In response to a user's custom operation, terminal 400-2 configures the association between the marked scene and the peripheral output effect (perception information) in video client 410-2, generates a perception information configuration file based on the configured association, and sends it to server 200 (the server corresponding to the client). Terminal 400-2 that receives the user's custom operation can be different from terminal 400-3 when the subsequent user uses video client 410-2. That is, after receiving the user's custom operation, terminal 400-2 uploads the generated perception information configuration file to server 200 and returns a prompt indicating successful configuration. When the subsequent user uses video client 410-3 running on terminal 400-3, the user will obtain the perception information configuration file from server 200 and output the corresponding associated peripheral output effect (perception information). In the case where the user uses video client 410-3 in terminal 400-3, the interaction process between the user and client 410-3 (sending application interaction operations and receiving application interaction results) can be presented in terminal 400-3. If the marked scene is reproduced in video client 410-3, the perception information configuration file is obtained from server 200 and the corresponding peripheral output effect (perception information) is output. For example, if the marked scene for the association configuration is that the video playback is completed, and the associated peripheral output effect (perception information) is that the red light stays on and vibrates continuously for 2 seconds, then during the user's use of video client 410-3, if the scene of "video playback completed" appears in video client 410-3, the peripheral output effect (perception information) associated with the "video playback completed" scene is output, that is, the terminal 400-3 used by the user can control the output of the effect of "red light stays on and vibrates continuously for 2 seconds".

[0067] In some embodiments, terminal 400-1, terminal 400-2, and terminal 400-3 can implement the perception processing method for human-computer interaction provided by the embodiments of the present application by running a computer program. For example, the computer program can be a native program or software module in the operating system; it can be a local (Native) application (APP, Application), that is, a program that needs to be installed in the operating system to run, such as a game APP (i.e., the above-mentioned game client 410-1, video client 410-2, and video client 410-3); it can also be a small program, that is, a program that only needs to be downloaded to the browser environment to run; it can also be a small program that can be embedded in any APP. In short, the above computer program can be any form of application program, module, or plug-in.

[0068] Embodiments of this application can be implemented with the help of cloud technology. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or a local area network to achieve data calculation, storage, processing, and sharing. Cloud technology is the general term for network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form a resource pool, be used on demand, and is flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources.

[0069] As an example, the server 200 can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminals 400-1, 400-2, and 400-3 can be smart phones, tablet computers, laptop computers, desktop computers, smart speakers, smart watches, etc., but are not limited thereto. The terminals 400-1, 400-2, 400-3, and the server 200 can be directly or indirectly connected through wired or wireless communication methods, and there is no limitation in the embodiments of this application.

[0070] See Figure 2 , Figure 2 is a schematic structural diagram of an electronic device provided by the embodiments of this application. Figure 2 The terminal 400-1 shown includes: at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. Each component in the terminal 400-1 is coupled together through a bus system 440. It can be understood that the bus system 440 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 2 all kinds of buses are labeled as the bus system 440.

[0071] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0072] The user interface 430 includes one or more output devices 431 enabling the presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components facilitating user input, such as a keyboard, a mouse, a microphone, a touch screen display, a camera, and other input buttons and controls.

[0073] The memory 450 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid state memory, hard disk drives, optical disk drives, etc. The memory 450 optionally includes one or more storage devices physically located remote from the processor 410.

[0074] The memory 450 includes volatile memory or non-volatile memory, and may also include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), and the volatile memory can be random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0075] In some embodiments, the memory 450 is capable of storing data to support various operations. Examples of such data include programs, modules, and data structures, or subsets or supersets thereof, which are illustrated below.

[0076] The operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and handling hardware-based tasks;

[0077] The network communication module 452 is used to reach other computing devices via one or more (wired or wireless) network interfaces 420. Exemplary network interfaces 420 include: Bluetooth, Wi-Fi (Wireless Fidelity), and USB (Universal Serial Bus), etc.;

[0078] The presentation module 453 is used to enable the presentation of information (such as a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 associated with the user interface 430 (such as a display screen, a speaker, etc.);

[0079] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more of the one or more input devices 432.

[0080] In some embodiments, the perception processing device 455 for human-computer interaction provided in the embodiments of the present application may be implemented in software. Figure 2 Shown in the memory 450 is the perception processing device 455 for human-computer interaction, which may be software in the form of a program, a plug-in, etc., and includes the following software modules: a display module 4551, a detection module 4552, an acquisition module 4553, an output module 4554, a configuration module 4555, a configuration module 4556, a display module 4557, and an output module 4558. These modules are logical, so they can be combined or further split arbitrarily according to the functions to be implemented, and the functions of each module will be described below.

[0081] Next, taking the execution of the human-computer interaction perception processing method provided in the embodiments of the present application by the Figure 1A terminal 400-1 as an example for illustration. Refer to Figure 3A , Figure 3A which is a schematic flowchart of the human-computer interaction perception processing method provided in the embodiments of the present application, and will be described in combination with the Figure 3A steps shown.

[0082] In step 101, display the interaction process in the human-computer interaction interface.

[0083] As an example, the interaction process displayed in the human-computer interaction interface may be an interaction process for any client, for example, an interaction process of a game client, an interaction process of a video client, an interaction process of an e-commerce client, an interaction process of a social client, and so on.

[0084] As an example, the interaction process in step 101 may be a human-computer interaction process in a single-user case. In response to any operation of the user on the human-computer interaction interface, the human-computer interaction interface will present an interaction result for the any operation. For example, if the any operation is a click operation on a certain task menu, the interaction result includes displaying the task menu, and the interaction process is the process from receiving the click operation until the interaction result is displayed in the human-computer interaction interface. The interaction process in step 101 may be a human-computer interaction process in a multi-user interaction case. The interaction process may include: an attack operation, a defense operation, and a process of obtaining corresponding operation response results issued by multiple objects controlled by multiple users in a game or military simulation during a battle process. The interaction process is presented in the form of a battle process.

[0085] In step 102, detect the scenes that appear during the interaction process.

[0086] As an example, a scenario can be any situation that occurs during an interaction, including but not limited to the following situations: a scenario where an instant notification pops up within a client, a scenario where a user clicks on a menu or button within a client, a scenario accompanied by sound, a scenario with physical collisions, a scenario with progress changes, for example, downloading of a resource package, a scenario of a special event within a client, for example, winning a game, reaching a certain level, etc.

[0087] In step 103, when the scenario that occurs during the interaction is a marked scenario for which perception information is configured with a custom operation, obtain the perception information configuration file corresponding to the marked scenario.

[0088] As an example, before performing step 103, a custom configuration process for the scenario needs to be carried out. During the custom configuration process, in response to a user's custom operation for any marked scenario, obtain the parameters of the perception information configured for the marked scenario in the custom operation, and generate a perception information configuration file based on the parameters. Thus, when reproducing the marked scenario, the corresponding perception information configuration file can be obtained to output the corresponding perception information.

[0089] In step 104, during the progress of the scenario, output the perception information corresponding to the perception information configuration file.

[0090] As an example, the output of the perception information corresponding to the perception information configuration file in step 104 can be implemented through an interaction control protocol. Specifically, from the perception information configuration file, obtain the type of the perception information configured by the custom operation and the parameters of the characteristics in the corresponding type; send a call request through the application programming interface open to the human - machine interaction interface to call the peripheral corresponding to the type to output the perception information of the corresponding type, so that the output perception information of the corresponding type conforms to the parameters of the characteristics of the corresponding type.

[0091] The following combines Figure 3B to detail the perception information customization process involved in the perception processing method for human - machine interaction provided in the embodiments of the present application.

[0092] Refer to Figure 3B , Figure 3B which is a schematic flowchart of the perception processing method for human - machine interaction provided in the embodiments of the present application. Before obtaining the perception information configuration file corresponding to the marked scenario in step 103, the following steps 105 to 107 can also be performed. Figure 3B The implementation of steps 105 to 107 shown in

[0093] is after step 102, but in the actual implementation process, there is no strict limitation on this and it can be adjusted adaptively.

[0094] As an example, a marked scenario is obtained by recording the target scenario of a marking operation in response to the user's marking operation. The obtained marked scenario is used to associate the parameters of each feature of various types of perception information in subsequent custom operations, so as to form different perception information files adapted to different marked scenarios.

[0095] In step 106, display the types of perception information that can be configured for the marked scenario, as well as the features that can be configured in each type.

[0096] As an example, the types of perception information include: tactile feedback information; visual feedback information; olfactory feedback information; auditory feedback information.

[0097] As an example, the custom configuration process mainly includes two stages. In the first stage, determine the scenario that needs to be customized, that is, determine the marked scenario that needs to be customized from multiple scenarios in the interaction process. In the second stage, obtain the configured parameters for the marked scenario with customization requirements. The implementation of the first stage and the second stage can be carried out immediately in succession or separated in terms of time dimension.

[0098] In some embodiments, the determination of the marked scenario in step 105 can be achieved through the following technical solution: display the historical interaction process in the human-computer interaction interface, where the historical interaction process is earlier than the interaction process; in response to the marking operation on the target scenario in the historical interaction process, record the target scenario as the marked scenario.

[0099] In some embodiments, the display of the types of perception information that can be configured for the marked scenario and the features that can be configured in each type in step 106 can be achieved through the following technical solution: play the screencast file of the historical interaction process; when the marked scenario appears during the playback of the screencast file, display the entry for custom perception information for the marked scenario; in response to the trigger operation on the entry, display a custom interface, where the custom interface is used to receive custom operations and the custom interface includes a custom interface for displaying the types of perception information that can be configured for the marked scenario and the features that can be configured in each type.

[0100] As an example, if the implementation of the first stage (the implementation manner of step 105 above) and the second stage (the implementation manner of step 106 above) are separated in terms of time dimension, the process of determining the marked scenario is to mark during any historical interaction process, and perform screencast processing on the historical interaction process. After the historical interaction process is completed, the screencast file obtained by screencast can be played, and the second stage is implemented for the marked scenario in the played screencast file.

[0101] As an example, the historical interaction process is displayed in the human-computer interaction interface of the battle game client. The historical interaction process is earlier than the interaction process. The historical interaction process can be a certain historical battle process, which includes various operation inputs and the presentation of response results of operations, etc. The occurrence time of the historical interaction process is earlier than the interaction process. Regarding the historical occurrence time point of the historical interaction process, the historical interaction process is also carried out in real time. In response to the marking operation on the target scene in the historical interaction process, the target scene is recorded as the marked scene. The target scene is the scene selected by the marking operation. For example, in the historical interaction process carried out in real time at the historical occurrence time point, two objects collide. If the user hopes to customize this collision scene, then this collision scene is the target scene of the marking operation and is recorded as the marked scene. When this collision scene appears in real time at the historical occurrence time point, the user's marking operation on this collision scene is received, and this collision scene is recorded as the marked scene. After the historical interaction process is completed, that is, after the historical battle ends, the screen recording file of the historical interaction process can be played at any time; it is equivalent to the user performing screen recording during the historical battle and being able to play the screen recording file after the historical battle ends to perform specific parameter customization. When the marked scene appears during the playback of the screen recording file, an entry for the customized perception information of the marked scene is displayed. The entry for the customized perception information can be presented in the form of a prompt message, that is, a trigger control is displayed, and information for prompting the user that the marked scene can be customized is displayed on the control. In response to the trigger operation on the entry, a customized interface for receiving customized operations and including the types of perception information that can be configured for the marked scene and the characteristics that can be configured in each type is displayed. After the trigger operation on the entry, the customized interface can be switched to. The customized interface includes the types of perception information that can be configured and at least one characteristic of each corresponding type. The types include at least one of the two types of visual feedback information or tactile feedback information. Each type has different characteristics. For example, the visual feedback information includes intensity characteristics, time characteristics, etc. The customized interface is also used to receive customized operations, and the parameters input by the customized operations are for the types of perception information that can be configured and at least one characteristic of each corresponding type.

[0102] Through the above embodiments, it is possible to mark the scenarios that need to be customized when the user is using the client in real time, and record them as marked scenarios, so that the user can accurately record the target scenarios that need to be customized. However, in the interaction process with high real-time requirements, it is difficult for the user to customize the marked scenario immediately after marking the scenario. For example, in the process of a game battle, marking the target scenario does not take much time, so it can be marked at any time during the battle. However, the customization process takes a lot of time, which affects the player's battle progress. Therefore, during the subsequent playback of the screen recording, it is possible to stay in the marked scenario for customization, thus achieving an effective balance between the real-time interactive experience of the game and the refined adjustment of customization.

[0103] In some embodiments, to determine the marked scenario in step 105, the following technical solution can be adopted: in response to the marking operation on the target scenario during the interaction process, record the target scenario as the marked scenario.

[0104] In some embodiments, to display the types of perceivable information that can be configured for the marked scenario and the characteristics that can be configured in each type in step 106, the following technical solution can be adopted: display the entry for custom perceivable information for the marked scenario; in response to the triggering operation on the entry, display a custom interface, where the custom interface is used to receive custom operations and the custom interface includes the types of perceivable information that can be configured for the marked scenario and the custom interface for the characteristics that can be configured in each type.

[0105] As an example, the implementation of the first stage (the implementation method of the above step 105) and the second stage (the implementation method of the above step 106) is not forced to be separated in the time dimension, that is, the two stages can be carried out consecutively.

[0106] In some embodiments, refer to Figure 10A , Figure 10AIt is a schematic diagram of the interface of the perception processing method for the human-computer interaction interface provided by the embodiments of the present application. In the human-computer interaction interface 1001A, in response to a marking operation on a target scene during the interaction process of the cultivation game client, the target scene is recorded as a marked scene. For example, when the scene of "the virtual object becomes the heroine" appears during the interaction process of the cultivation game, the scene 1002A of "the virtual object becomes the heroine" is displayed in the human-computer interaction interface 1001A. In response to the marking operation on this scene 1002A, it is recorded as a marked scene, and an entry 1003A for custom perception information of the marked scene is displayed; in response to the triggering operation on the entry 1003A, a custom interface 1004A for receiving custom operations is displayed. The entry for custom perception information can be presented in the form of a prompt message, that is, a trigger control is displayed, and the control displays information for prompting the user that the marked scene can be customized. In response to the triggering operation on the entry, a custom interface for receiving custom operations and including the types of perception information that can be configured for the marked scene and the characteristics that can be configured in each type is displayed. After the triggering operation on the entry, it can be switched to the custom interface. The custom interface includes the types of perception information that can be configured and at least one characteristic of each corresponding type. The types include at least one of the two types of visual feedback information or tactile feedback information. Each type has different characteristics. For example, the visual feedback information includes intensity characteristics, time characteristics, etc. The custom interface is also used to receive custom operations, and the parameters input by the custom operations are for the types of perception information that can be configured and at least one characteristic of each corresponding type.

[0107] Through the above implementation manner, when the user is using the client in real time, the scenes that need to be customized can be marked and recorded as marked scenes, so that the user can accurately record the target scenes that need to be customized. For the interaction process with relatively low real-time requirements, the user can immediately customize the marked scenes after marking the scenes. For example, during the cultivation game process, marking the target scene does not take much time, so it can be marked at any time during the cultivation game process. Although the customization process takes more time, the real-time requirement of the interaction of the cultivation game is relatively low. Therefore, it can stay on the marked scene during the interaction process for customization, effectively ensuring the real-time nature of the user's customization operation and preventing the user from forgetting to customize the marked scene in the subsequent process, resulting in low utilization rate of this function.

[0108] In some embodiments, the real-time performance can be measured by marking the client. For example, the battle game client can be marked as a client with high real-time requirements, and the cultivation game client can be marked as a client with low real-time requirements. For clients with different real-time markings, the above two different implementation methods can be adopted. The process of marking the client can be completed during the development stage, and can be queried during subsequent application processes. The corresponding applicable implementation method can be used as the default implementation method of the client. In response to the user's adjustment operation for the implementation method, the default implementation method can be adjusted to the implementation method selected by the user.

[0109] In some embodiments, step 106 of displaying the types of perception information that can be configured for the marked scenario and the characteristics that can be configured in each type can be implemented through the following technical solution: display multiple pre-configured modes of the marked scenario; where each mode includes at least one type of perception information associated with the marked scenario and the default parameters for which the characteristics in the type are pre-configured.

[0110] In step 107, in response to a custom operation for the marked scenario, obtain the parameters configured by the custom operation for the characteristics in at least one type, and generate a perception information configuration file for the corresponding marked scenario based on the obtained parameters of the characteristics in at least one type.

[0111] In some embodiments, step 107 of obtaining the parameters configured by the custom operation for the characteristics in at least one type can be implemented through the following technical solution: obtain at least one type of perception information associated with the marked scenario included in the target mode and the default parameters for which the characteristics in the type are pre-configured; where the target mode is the mode selected by the custom operation from multiple pre-configured modes.

[0112] As an example, see Figure 10B , Figure 10B is a schematic diagram of the interface of the perception processing method for the human-computer interaction interface provided by the embodiments of the present application. In the human-computer interaction interface 1001B, multiple pre-configured modes 1002B of the marked scenario are displayed. The multiple pre-configured modes 1002B are presented in the form of controls. Each mode includes at least one type of perception information associated with the marked scenario and the default parameters for which the characteristics in the type are pre-configured. If the custom operation in step 107 does not involve modifying the parameters but only involves selecting an operation for the mode 1002B, that is, obtain the at least one type of perception information associated with the marked scenario and the default parameters for which the characteristics in the type are pre-configured under the selected mode as the parameters configured for the characteristics in at least one type.

[0113] Next, in combination with Figure 3CDescribe in detail the process of modifying custom parameters involved in the perception processing method of human-computer interaction provided in the embodiments of the present application.

[0114] See Figure 3C , Figure 3C is a schematic flowchart of the perception processing method of human-computer interaction provided in the embodiments of the present application. Each characteristic in each type has a pre-configured default parameter; obtaining the parameter configured by the custom operation for the characteristic in at least one type in step 107 can be achieved through the following steps.

[0115] In step 1071, in response to the modification of the default parameter of the characteristic in at least one type by the custom operation, update the default parameter to the parameter set by the custom operation.

[0116] As an example, there is a default parameter for a pre-configured mode. There is a default mode in the pre-configured mode, and there is also a default parameter in the default mode. Different from directly obtaining the default parameter as the parameter configured for the characteristic in at least one type, obtain the parameter set by the custom operation (the set parameter obtained by modifying based on the default parameter) as the parameter configured for the characteristic in at least one type.

[0117] In some embodiments, obtaining the parameter configured by the custom operation for the characteristic in at least one type in step 107 can be achieved through the following technical solution: obtaining candidate parameters for the characteristic in at least one type; performing a scoring prediction process on the candidate parameters through a first neural network model to obtain a prediction score for each candidate parameter; taking the candidate parameter with the highest prediction score as the parameter for the characteristic in at least one type; wherein, the first neural network model is trained based on candidate parameter samples and the true scores corresponding to the candidate parameter samples.

[0118] In some embodiments, obtaining the parameter configured by the custom operation for the characteristic in at least one type in step 107 can be achieved through the following technical solution: obtaining operation behavior data in the human-computer interaction interface; performing a preference prediction process on the operation behavior data through a second neural network model to obtain the operation preference represented by the operation data; obtaining the parameter of the characteristic in at least one type that is pre-associated with the operation preference; wherein, the second neural network model is trained based on operation data samples and the operation preferences represented by the operation data samples.

[0119] As an example, both of the above two implementation manners apply the idea of machine learning to predict the parameters of the characteristics in at least one type by training a model. The custom operation may be an operation that triggers the operation of the first neural network model and / or the second neural network model. The first neural network model uses the user score as the prediction basis, and the trained first neural model can predict the parameters of the characteristics in at least one type of the perceptual information that meets the user's expectations. The second neural network model uses the operation behavior data as the prediction basis, and the trained second neural model can predict the parameters of the characteristics in at least one type of the perceptual information that is pre-associated with the operation preference. Through the above implementation manners, the parameters configured by the custom operation for the characteristics in at least one type can be quickly obtained.

[0120] In some embodiments, the types of perceptual information configured by the perceptual information profile include tactile feedback information, and the tactile feedback information includes at least one of the following characteristics: frequency, intensity, duration, orientation. The output of the perceptual information corresponding to the perceptual information profile in step 104 can be implemented by the following technical solution: according to the type of event included in the scenario, output the tactile feedback information configured to be associated with the type of event.

[0121] In some embodiments, the above output of the tactile feedback information configured to be associated with the type of event according to the type of event included in the scenario can be implemented by the following technical solution: perform at least one of the following operations: in a marked scenario including a user operation event, synchronously output the tactile feedback information synchronized with the user operation event; in a marked scenario including sound, output at least one of the following: tactile feedback information adapted to the type of sound, tactile feedback information consistent with the intensity of the sound, tactile feedback information consistent with the orientation of the sound source of the sound; in a marked scenario including a collision event, output at least one of the following: tactile feedback information synchronized with the collision event, tactile feedback information consistent with the intensity of the collision event; in a marked scenario including a notification event, output the tactile feedback information synchronized with the notification event.

[0122] As an example, the event types associated with the configured haptic feedback information in the scenario include at least one of the following: user operation event, when the user clicks on a menu or button in the game, haptic feedback information can be output, and a single click operation received during the game can also cause the output of haptic feedback information; sound event, for scenarios with sound, haptic feedback information can be output. There are many sound effects in the game client to create an atmosphere, and vibrations can be made according to the rhythm of the sound effects, so as to provide users with a more intuitive implementation of interactive information feedback; collision event, there are many scenarios of physical collision events in the game client. For example, the event of colliding with an obstacle during driving, etc. By outputting haptic feedback information to feedback these collision information to the user, not only can interactive information be more intuitively feedback, but also the user can be more focused on the game content itself; notification event, for scenarios including instant notification events, haptic feedback information can be output. The notification event is not limited to ordinary pop-up notifications, but can be the output of any information. For example, the output of the enemy's position information can not only display the enemy's position information, but also output the enemy's position information through haptic feedback information. Assuming the enemy is in the front left, the motor arranged in the front left of the terminal vibrates to inform the player that the enemy is in the front left.

[0123] As an example, the characteristics of haptic feedback information (vibration) include at least one of the following: intensity, the intensity is the intensity of haptic feedback information, such as vibration intensity. Usually, the intensity is divided into grades. For example, the intensity of haptic feedback information can be marked by numbers from 0 to 100. Different intensities can have different vibration performances. For scenarios with differences in intensity, haptic feedback information can be output. The intensity of vibration (haptic) can be used to feedback the intensity of collision scenarios and sound scenarios; frequency, the frequency is the frequency of haptic feedback information, such as vibration frequency, with the unit of Hertz (number of vibrations per second). Different frequencies correspond to different vibration effects. Different vibration effects can be shown through changes in frequency; duration, the duration is the duration of haptic feedback information, such as vibration duration. The length of the vibration time will also produce different vibration effects; orientation, the vibration orientation is used to simulate the direction and source of sound or collision. Haptic feedback can be generated for the source and direction of objects in the scenario. There are many scenarios in the game that require distinguishing the source and direction of sound (such as the footsteps and explosions of enemies, etc.). The direction and source can be clearly feedback through haptic. For example, if the explosion sound comes from the front left of the player, only the upper left corner of the terminal can be vibrated to identify the source of the explosion sound, so that the user's visual perception can be more focused on the game screen.

[0124] In some embodiments, the types of perception information configured by the perception information profile include visual feedback information, and the visual feedback information includes at least one of the following characteristics: status, frequency, color, brightness; the output of the perception information corresponding to the perception information profile in step 104 can be achieved through the following technical solutions: according to the type of event included in the scene, output visual feedback information configured to be associated with the type of event.

[0125] In some embodiments, the above-mentioned output of visual feedback information configured to be associated with the type of event according to the type of event included in the scene can be achieved through the following technical solutions: perform at least one of the following operations: in a marked scene including a notification event, output visual feedback information synchronized with the notification event; in a marked scene including sound, output at least one of the following: visual feedback information adapted to the type of sound, visual feedback information adapted to the rhythm of the sound; in a marked scene including a progress event, output visual feedback information for characterizing the progress of the progress event; in a marked scene including a result event, output visual feedback information for characterizing the result of the result event.

[0126] As an example, the types of events associated with the configured visual feedback information in the scene include at least one of the following: a notification event. When there is a notification event (such as a notification in the game, etc.), the user can be notified and reminded through visual feedback information such as custom light flashing, that is, output visual feedback information synchronized with the notification event; a sound event. The music rhythm can be made explicit through visual feedback information. In-game sound effects and music rhythms can be visually displayed through lights, especially in music games. That is, output at least one of visual feedback information adapted to the type of sound and visual feedback information adapted to the rhythm of the sound; a progress event. There are scenes in the game client that include events that require a long wait, such as the download of a resource pack, etc. A progress bar is simulated through visual feedback information (light strip), so that the user can judge the current download progress through the light, and output visual feedback information for characterizing the progress of the progress event without repeatedly entering the game client to check; a result event. Special events in the game can be displayed through lights, such as winning a game, reaching a certain level, etc., and output visual feedback information for characterizing the result of the result event.

[0127] As an example, the characteristics of the visual feedback information (light effects) include at least one of the following: state (the light on or off), different shapes and effects can be displayed by controlling the light on and off at different positions; blinking frequency, the blinking effect can be achieved by quickly switching between on and off, and the number of switches per second is the blinking frequency. Different blinking frequencies can achieve different effects, and different effects can be achieved by switching frequencies at different positions and at different times; color, various different color special effects such as gradient and transformation can be achieved by controlling the color of each display unit; brightness, brightness can not only display light and dark effects, but also combine with color, blinking frequency and state to complete richer visual effects.

[0128] The following will describe in detail Figure 3D the authorization query process involved in the perception processing method for human-computer interaction provided by the embodiments of the present application.

[0129] Refer to Figure 3D , Figure 3D which is a schematic flowchart of the perception processing method for human-computer interaction provided by the embodiments of the present application. Before step 104 outputs the perception information corresponding to the perception information configuration file, the following steps 108-109 can also be executed.

[0130] In step 108, query the types of perception information supported for customization by the electronic device, and the characteristics supported for customization in the corresponding types.

[0131] In step 109, according to the query result, filter out the types of perception information that do not support customization and the characteristics in the corresponding types in the perception information configuration file, so as to update the perception information configuration file.

[0132] As an example, before the client controls the electronic device to output corresponding perception information, it is necessary to query the authorization of the electronic device through a communication protocol. First, the client initiates a connection request, and the operating system of the electronic device (hereinafter referred to as the electronic device for short) responds to the connection request to establish a connection. The client queries the types of perception information supported by the electronic device for customization and the customizable features in the corresponding types. The client queries the supported methods of the electronic device (visual feedback information and tactile feedback information). The electronic device returns a list of supported effects (whether visual feedback information is supported and whether tactile feedback information is supported). The client queries the features supported by the visual feedback information (this query is only initiated when the previous query result is that visual feedback information is supported, otherwise this query is not initiated). The electronic device returns the features supported by the authorization control of the visual feedback information. The client queries the features supported by the tactile feedback information (this query is only initiated when the previous query result is that tactile feedback information is supported, otherwise this query is not initiated). The electronic device returns the features supported by the authorization control of the tactile feedback information. Thus, according to the query results, the types of perception information that do not support customization and the features in the corresponding types can be filtered out in the perception information configuration file to update the perception information configuration file.

[0133] In some embodiments, the technical solutions described in the above steps 108 to 109 are implemented after the implementation configuration process and before the perception information is output. Considering the possibility of terminal replacement, that is, referring to Figure 1B the application scenarios described therein, if the terminal that receives the user-defined operation is different from the terminal that outputs the perception information in the reproduction scenario, it is difficult for different terminals to ensure the same types of perception information they support and the features of the corresponding types of the authorized call interfaces. Therefore, the technical solutions described in the above steps 108 to 109 are implemented after the implementation configuration process and before the perception information is output, which can save computing response resources and avoid meaningless queries from occupying limited computing response resources.

[0134] In some embodiments, the technical solutions described in the above steps 108 to 109 can be implemented before the implementation configuration process. Considering the possibility of long-term use of the terminal, that is, referring to Figure 1A the application scenarios described therein, if the terminal that receives the user-defined operation is the same as the terminal that outputs the perception information in the reproduction scenario, it can be ensured that the types of perception information supported and the features of the corresponding types of the authorized call interfaces obtained during configuration are the same as those during control output. Since the technical solutions described in the above steps 108 to 109 are implemented before the implementation configuration process, it can enable the user to obtain a consistent usage experience and avoid the situation where the configured effects during the customization process cannot be output on the terminal.

[0135] See Figure 4 , Figure 4It is a schematic flowchart of the perception processing method for human-computer interaction provided by an embodiment of the present application, and will be described in conjunction with Figure 4 the steps shown.

[0136] In step 201, a custom operation for a marked scene is received on the human-computer interaction interface.

[0137] In step 202, in response to the custom operation, according to the type of perception information configured by the custom operation and the parameters of the characteristics in the corresponding type, a perception information configuration file corresponding to the marked scene is generated.

[0138] The implementation processes of step 201 and step 202 can refer to the implementation processes of steps 105-107 in the foregoing embodiment.

[0139] In step 203, the interaction process is displayed on the human-computer interaction interface;

[0140] In step 204, when the scene that appears again during the interaction process is a marked scene, during the progress of the scene, perception information of the type configured by the custom operation is output according to the perception information configuration file, and the output perception information of the corresponding type conforms to the parameters of the characteristics in the corresponding type.

[0141] The implementation processes of step 203 and step 204 can refer to the implementation processes of steps 101-104 in the foregoing embodiment.

[0142] Next, an exemplary application of the embodiment of the present application in an actual application scenario will be described. Taking the client as a game client as an example, the processing method for human-computer interaction provided by the embodiment of the present application allows users to perform scene marking during the game, and customize perception information of types such as haptics (haptic feedback information) and lighting effects (visual feedback information) by playing back the game recording file after the game ends, so that users can flexibly adjust the parameters of each characteristic item of perception information of types such as haptics and lighting effects, thereby creating a user-specific game atmosphere. And through an agreed protocol between the game client and the electronic device (for example, the terminal), the game client can finely control the relevant haptic and lighting effect hardware in the terminal, so that the user-defined effects are perfectly presented on the terminal.

[0143] In some embodiments, the haptics of the terminal are mainly achieved through the vibration of the motor, which is used to remind the user through vibration or enable the user to more realistically experience various scenes in the game through haptics. For example, in a shooting game, the scene of hitting an enemy, the footsteps of the enemy, the bumpy scene of driving a vehicle, the collision scene, the scene with sound effects, etc. There are the following situations for the marked scenes of specific haptic feedback information (vibration): 1. Haptic feedback can be generated for instantaneous notification scenarios. The client controls the terminal to trigger instantaneous vibration (single or multiple times). Through the haptic perception brought by the vibration, the user is notified of the occurrence of events, button clicks, etc. For example, when the user clicks on a menu or button in the game, haptic feedback can be triggered, and single-click operations during the game can also trigger haptic feedback, etc.; 2. Haptic feedback can be generated for sound scenarios. There are many sound effects in the game to create an atmosphere, and vibration can be carried out according to the rhythm of the sound effects (music), so as to provide the user with a more intuitive implementation of interactive information feedback; 3. Haptic feedback can be generated for physical collision scenarios. There are many physical collision scenarios in the game. For example, the scenario of colliding with an obstacle during driving, the bumpy scenario of an uneven road surface, etc. By haptically feedbacking these collision information to the user, not only can the interactive information be more intuitively feedbacked, but also the user can be more focused on the game content itself; 4. Haptic feedback can be generated for the source and direction of objects in the scene. There are many scenes in the game that require distinguishing the source and direction of sounds (such as the footsteps of the enemy, the explosion sound, etc.). The direction and source can be clearly feedbacked through haptics. For example, if the explosion sound comes from the upper left front of the player, vibration can be carried out only in the upper left corner of the terminal to identify the source of the explosion sound, so that the user's visual perception is more focused on the game screen; 5. Haptic feedback can be generated for scenes with different intensities. The intensity of the vibration (haptics) can be used to feedback the intensity of the collision scene and the sound scene.

[0144] In some embodiments, the characteristics of the haptic feedback information (vibration) include the following: 1. Intensity, which is the intensity of the haptic feedback information, such as the vibration intensity. Usually, the intensity is divided into grades. For example, the intensity of the haptic feedback information can be marked by numbers from 0 to 100. Different intensities can have different vibration performances; 2. Frequency, which is the frequency of the haptic feedback information, such as the vibration frequency, with the unit of Hertz (number of vibrations per second). Different frequencies correspond to different vibration effects, and different vibration effects can be shown through the change of frequency; 3. Duration, which is the duration of the haptic feedback information, such as the vibration duration. The length of the vibration time will also produce different vibration effects; 4. Orientation, which simulates the direction and source of sound or collision through the vibration orientation.

[0145] In some embodiments, there are several situations in the marked scenarios of specific visual feedback information (lighting effects): 1. Visual feedback information can be generated for event notification scenarios. When there is a notification event, the user can be notified and reminded through visual feedback information such as breathing lighting effects or custom lighting flashes. For example, notifications in the game, etc.; 2. For scenarios with music rhythms, the music rhythms can be made explicit through visual feedback information. The rhythms of in-game sound effects and music can be visually displayed through lighting effects, especially for the visualization of music games; 3. Visual feedback information can be generated for scenarios with progress changes. In the game, there are some scenarios that require long waiting times, such as the download of resource packs, etc. The visual feedback information (light strip) is used to simulate the progress bar, enabling the user to judge the current download progress through the lighting without repeatedly entering the game to check; 4. Visual feedback can be generated for scenarios with game events. Special events in the game can be displayed through lighting effects, such as winning a game, reaching a certain level, etc.

[0146] In some embodiments, the characteristics of the visual feedback information (lighting effects) include the following: 1. State (the lighting and extinguishing of the lights), different shapes and effects are shown by controlling the lighting and extinguishing of lights at different positions; 2. Blinking frequency, the blinking effect can be achieved through the rapid switching of lighting and extinguishing. The number of switches per second is the blinking frequency. Different blinking frequencies can achieve different effects, and different effects can be achieved by switching frequencies at different positions and at different times; 3. Color, various different color special effects such as gradient and transformation can be achieved by controlling the color of each display unit; 4. Brightness, the brightness can not only display light and dark effects, but also combine with color, blinking frequency, and state to complete richer visual effects.

[0147] In some embodiments, refer to Figure 5 , Figure 5 FIG. is a schematic application diagram of the perception processing method for human-computer interaction provided by the embodiments of the present application. In the game, parameters of the characteristics of the default tactile feedback information and parameters of the characteristics of the visual feedback information are usually preset. In the setting interface 501, the trigger operation of the user for the screen recording control 502 can be received. In response to receiving the trigger operation for the screen recording control 502, the game process can be recorded, and then the corresponding scenario can be located by playing back the screen recording file, so as to customize these effects.

[0148] Refer to Figure 6 , Figure 6It is an application schematic diagram of the perception processing method for human-computer interaction provided by an embodiment of the present application. During the game replay displayed on the human-computer interaction interface 601, when a marked scene appears, a customizable prompt message 602 is displayed on the human-computer interaction interface 601. The prompt message can be used to prompt that the scene can be customized, and further can also be used to prompt the type of perception information that can be customized. At the same time, the terminal can also output the type of perception information that can be customized in a prompt manner. For example, if the scene can be customized for tactile feedback information, then at this time, tactile feedback information (vibration) can also be output to remind the user that the tactile feedback information can be customized for this scene. Or, the terminal can also output visual feedback information and / or tactile feedback information to prompt the user that the perception information can be customized for this scene, rather than specifically prompting the type of perception information that can be specifically configured. In response to a trigger operation (click operation) on the prompt message 602, a customization interface can be displayed for the user to customize the perception information.

[0149] In some embodiments, for each scene, the game will have a preset default effect (default perception information output). The user can fine-tune on the basis of the default perception information to obtain a personalized effect. For the customization of tactile feedback information, the content that the user needs to customize is the parameters of each characteristic defined in the tactile feedback information in the above embodiment. Refer to Figure 7 , Figure 7 It is an application schematic diagram of the perception processing method for human-computer interaction provided by an embodiment of the present application. Specifically, it is an example of a tactile effect editor. The instantaneous state at each time point can be adjusted according to the timeline. The user can adjust the parameters (orientation, frequency, and intensity) of the tactile feedback information as a whole by operating on the waveform, or can define the tactile parameters at each time point through the three sub-items of orientation, frequency, and intensity respectively. For the customization of visual feedback information, the content that the user needs to customize is the parameters of each characteristic defined in the visual feedback information in the above embodiment. Refer to Figure 8 , Figure 8 It is an application schematic diagram of the perception processing method for human-computer interaction provided by an embodiment of the present application. Specifically, it is an example of a tactile effect editing interface 801 (customization interface). The instantaneous state at each time point can be adjusted according to the timeline, similar to the parameter adjustment of tactile feedback information. The user can adjust the parameters of visual feedback information as a whole through the waveform, or can define the visual parameters at each time point through the four sub-items of frequency, brightness, color, and state respectively. There is a corresponding control 802 for each sub-item, that is, the user can control the display effect at each time point. If there are multiple light strips, each light strip can be independently controlled, and the on / off state, flashing frequency, brightness, and color (gradient color) of the light strip can be freely adjusted.

[0150] In some embodiments, refer to Figure 9 , Figure 9 which is a protocol schematic diagram of the perception processing method for human-computer interaction provided by the embodiments of the present application. Before the client controls the terminal peripherals, a connection and authorization query need to be established based on the following communication protocol: 1. The game initiates a connection request. 2. The terminal responds to the connection request to establish a connection. 3. The game queries the supported methods of the terminal (visual feedback information and tactile feedback information). 4. The terminal returns a list of supported effects (whether visual feedback information is supported, whether tactile feedback information is supported). 5. The game queries the characteristics supported by the visual feedback information (this query is only initiated when the previous query result is that visual feedback information is supported, otherwise this query is not initiated). 6. The terminal returns the characteristics supported by the authorized control of the visual feedback information. 7. The game queries the characteristics supported by the tactile feedback information (this query is only initiated when the previous query result is that tactile feedback information is supported, otherwise this query is not initiated). 8. The terminal returns the characteristics supported by the authorized control of the tactile feedback information.

[0151] In some embodiments, the tactile effect file (vibration) in the perception information configuration file is defined as follows: The file is defined as a sequence of time, and the time interval is in milliseconds. Each time point has the following data: The intensity of the tactile feedback information (vibration intensity), which is an integer ranging from 0 to 100, and the larger the number, the greater the intensity. Other integer ranges can also be taken according to actual needs; the frequency of the tactile feedback information (vibration frequency), with the unit of "times / second"; the duration of the tactile feedback information (vibration duration), with the unit of milliseconds, indicating the duration of a certain vibration; the orientation of the tactile feedback information (vibration orientation), with the upper left corner of the terminal as the origin of the coordinate system, and a region is marked by four coordinate points as the vibration region, representing the vibration orientation.

[0152] In some embodiments, the visual effect file (light effect) in the perception information configuration file is defined as follows: The file is defined as a sequence of time, and the time interval is in milliseconds. The data for each time point is a list. Among them, the smallest light-emitting unit is a piece of data, and the content is as follows: The duration of the smallest light-emitting unit: in milliseconds; the state of the light strip (the light strip can be the smallest light-emitting unit) is displayed through a list, and each element in the list marks the state of each light-emitting point on the light strip. The content is as follows: Coordinates, the mark of each point on the light strip, numbered from 0, and different numbers are used to mark different light-emitting points; the state of each light-emitting point, the on state, the off state, and the flashing state; color, encoded in RGB format (other encodings can also be used); flashing frequency: with the unit of "times / second", this field needs to be set when the state is flashing; brightness, which is represented by integers from 0 to 100 to indicate different brightness levels, and other integer ranges can also be taken according to actual needs.

[0153] In some embodiments, the process of sending and playing the effect file is as follows. The input of the PlayHaptics function is the haptic effect file defined in the above embodiments, and the output is two states: success or failure. The function it implements is to play the haptic effect (vibration) defined in the haptic effect file. The input of the PlayLampEffect function is the visual effect file defined in the above embodiments, and the output is two states: success or failure. The function it implements is to play the visual effect (lamp effect) defined in the visual effect file.

[0154] In some embodiments, the custom functions of haptics and lamp effects can also be opened to development and design users, so that the game has rich and realistic default effects, providing a unified default interaction experience for ordinary users (the aforementioned default mode).

[0155] This application defines a complete set of solutions, enabling users to independently select in-game scenarios and customize peripheral effects through supporting tools, so that users can customize exclusive experience effects and give full play to the optimal performance of the terminal hardware. It can achieve: allowing users to customize haptic and lighting effects in the game, so that the user-defined effects can be triggered every time in the same scenario, bringing a personalized gaming experience to users; through the interaction control protocol between the game and the terminal (the operating system of the terminal), the game controls the haptic (vibration) and visual (lamp effect) components of the terminal, enabling the user-defined effects to be realized on the terminal. Since users' requirements for personalized gaming experiences are increasing, the human-computer interaction perception processing method provided by the embodiments of this application can make full use of the terminal hardware to create a personalized gaming atmosphere. Since the custom function is provided to users, they can participate more deeply in the customization of the game experience effect, improving the human-computer interaction efficiency. The interaction protocol established between the game and the terminal can achieve a deeper combination between the haptic and lamp effects of the terminal and the game, enabling users to customize richer display effects.

[0156] Next, continue to describe the exemplary structure of the software module implementation of the human-computer interaction perception processing device 455 provided by the embodiments of this application. In some embodiments, as Figure 2 shown, the software module in the human-computer interaction perception processing device 455 stored in the memory 450 may include: a display module 4551 for displaying the interaction process in the human-computer interaction interface; a detection module 4552 for detecting the scenarios that occur during the interaction process; an acquisition module 4553 for acquiring the perception information configuration file corresponding to the marked scenario when the scenario that occurs during the interaction process is a marked scenario configured with perception information for a custom operation; and an output module 4554 for outputting the perception information corresponding to the perception information configuration file during the progress of the scenario.

[0157] In some embodiments, the types of sensing information configured by the sensing information profile include haptic feedback information, and the haptic feedback information includes at least one of the following characteristics: frequency, intensity, duration, orientation; the output module 4554 is further configured to: according to the type of event included in the scene, output haptic feedback information configured to be associated with the type of event.

[0158] In some embodiments, the output module 4554 is further configured to: perform at least one of the following operations: in a marked scene including a user operation event, synchronously output haptic feedback information synchronized with the user operation event; in a marked scene including sound, output at least one of the following: haptic feedback information adapted to the type of sound, haptic feedback information consistent with the intensity of the sound, haptic feedback information consistent with the orientation of the sound source of the sound; in a marked scene including a collision event, output at least one of the following: haptic feedback information synchronized with the collision event, haptic feedback information consistent with the intensity of the collision event; in a marked scene including a notification event, output haptic feedback information synchronized with the notification event.

[0159] In some embodiments, the types of sensing information configured by the sensing information profile include visual feedback information, and the visual feedback information includes at least one of the following characteristics: state, frequency, color, brightness; the output module 4554 is further configured to: according to the type of event included in the scene, output visual feedback information configured to be associated with the type of event.

[0160] In some embodiments, the output module 4554 is further configured to: perform at least one of the following operations: in a marked scene including a notification event, output visual feedback information synchronized with the notification event; in a marked scene including sound, output at least one of the following: visual feedback information adapted to the type of sound, visual feedback information adapted to the rhythm of the sound; in a marked scene including a progress event, output visual feedback information for characterizing the progress of the progress event; in a marked scene including a result event, output visual feedback information for characterizing the result of the result event.

[0161] Before obtaining the sensing information profile corresponding to the marked scene, the device further includes: a configuration module 4555, configured to: determine the marked scene; display the types of sensing information that can be configured for the marked scene, and the characteristics that can be configured in each type; wherein the types of sensing information include: haptic feedback information; visual feedback information; olfactory feedback information; auditory feedback information; in response to a custom operation for the marked scene, obtain the parameters configured by the custom operation for the characteristics in at least one type, and generate a sensing information profile corresponding to the marked scene based on the obtained parameters of the characteristics in at least one type.

[0162] In some embodiments, the configuration module 4555 is further configured to: display a historical interaction process in a human-computer interaction interface, where the historical interaction process is earlier than the interaction process; in response to a marking operation on a target scenario during the historical interaction process, record the target scenario as a marked scenario.

[0163] In some embodiments, the configuration module 4555 is further configured to: play a screencast file of the historical interaction process before a custom operation on the marked scenario; when the marked scenario appears during the playback of the screencast file, display an entry for custom perception information for the marked scenario; in response to a trigger operation on the entry, display a custom interface for receiving a custom operation, including the types of perception information that can be configured for the marked scenario and the characteristics that can be configured in each type.

[0164] In some embodiments, the configuration module 4555 is further configured to: in response to a marking operation on a target scenario during the interaction process, record the target scenario as a marked scenario.

[0165] In some embodiments, the configuration module 4555 is further configured to: after determining the marked scenario and before a custom operation on the marked scenario, display an entry for custom perception information for the marked scenario; in response to a trigger operation on the entry, display a custom interface for receiving a custom operation, including the types of perception information that can be configured for the marked scenario and the characteristics that can be configured in each type.

[0166] In some embodiments, the characteristics in each type have pre-configured default parameters; the configuration module 4555 is further configured to: in response to a modification of the default parameters of the characteristics in at least one type by a custom operation, update the default parameters to the parameters set by the custom operation.

[0167] In some embodiments, the configuration module 4555 is further configured to: display multiple pre-configured modes of the marked scenario; where each mode includes at least one type of perception information associated with the marked scenario and the default parameters of the characteristics in the type that are pre-configured; obtain at least one type of perception information associated with the marked scenario included in the target mode and the default parameters of the characteristics in the type that are pre-configured; where the target mode is the mode selected by the custom operation from the multiple pre-configured modes.

[0168] In some embodiments, the configuration module 4555 is further configured to: obtain candidate parameters for features in at least one type; perform a scoring prediction process on the candidate parameters through a first neural network model to obtain a prediction score for each candidate parameter; use the candidate parameter with the highest prediction score as the parameter for the feature in at least one type; wherein, the first neural network model is trained based on a candidate parameter sample and the true score corresponding to the candidate parameter sample.

[0169] In some embodiments, the configuration module 4555 is further configured to: obtain operation behavior data in the human-computer interaction interface; perform a preference prediction process on the operation behavior data through a second neural network model to obtain the operation preference represented by the operation data; obtain parameters for features in at least one type that are pre-associated with the operation preference; wherein, the second neural network model is trained based on an operation data sample and the operation preference represented by the operation data sample.

[0170] In some embodiments, the output module 4554 is further configured to: before outputting the perception information corresponding to the perception information profile, query the types of perception information supported for customization by the electronic device and the features supported for customization in the corresponding types; according to the query result, filter out the types of perception information not supported for customization and the features in the corresponding types in the perception information profile to update the perception information profile.

[0171] In some embodiments, the output module 4554 is further configured to: obtain the types of perception information configured for the custom operation and the parameters of the features in the corresponding types from the perception information profile; send a call request through the application programming interface open to the human-computer interaction interface to call the peripheral device corresponding to the type to output the perception information of the corresponding type, so that the output perception information of the corresponding type conforms to the parameters of the features in the corresponding type.

[0172] In some embodiments, as Figure 2 shown, the software module in the perception processing device 455 for human-computer interaction stored in the memory 450 may include: a configuration module 4556, configured to receive a custom operation for a marked scene in the human-computer interaction interface; the configuration module 4556 is further configured to, in response to the custom operation, generate a perception information profile corresponding to the marked scene according to the type of perception information configured by the custom operation and the parameters of the features in the corresponding type; a display module 4557, configured to display the interaction process in the human-computer interaction interface; an output module 4558, configured to, when the scene that appears again during the interaction is the marked scene, during the progress of the scene, output the perception information of the type configured by the custom operation according to the perception information profile, and the output perception information of the corresponding type conforms to the parameters of the features in the corresponding type.

[0173] An embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the perception processing method of human-computer interaction described above in the embodiment of the present application.

[0174] An embodiment of the present application provides a computer-readable storage medium storing executable instructions, where the executable instructions, when executed by a processor, will cause the processor to execute the perception processing method of human-computer interaction provided by the embodiment of the present application. For example, as Figure 3A-3D and Figure 4 the perception processing method of human-computer interaction shown.

[0175] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disc, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0176] In some embodiments, the executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including being deployed as an independent program or being deployed as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0177] As an example, the executable instructions may or may not correspond to a file in the file system, and may be stored as part of a file that stores other programs or data. For example, they may be stored in one or more scripts in a Hyper Text Markup Language (HTML) document, stored in a single file dedicated to the program in question, or stored in multiple cooperating files (for example, files that store one or more modules, subroutines, or code portions).

[0178] As an example, the executable instructions may be deployed to execute on one computing device, or on multiple computing devices located at one location, or on multiple computing devices distributed at multiple locations and interconnected by a communication network.

[0179] In summary, by flexibly configuring the perception information of the marked scene through the custom operation of the embodiments of the present application, the corresponding perception information configuration file can be reused when the same marked scene is reproduced, so as to call the hardware resources in the electronic device to output the perception information, enabling the atmosphere adapted to the scene to be presented and creating an immersive personalized interaction experience, ultimately realizing the effective utilization of the hardware resources of the electronic device.

[0180] As described above, the above are only embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and scope of the present application are all included in the protection scope of the present application.

Claims

1. A perception processing method for human-computer interaction, characterized in that Including: Display an interaction process in a human - machine interaction interface; Detect scenes that occur during the interaction process; When the scene that occurs during the interaction process is a marked scene with perception information configured by a custom operation, obtain a perception information configuration file corresponding to the marked scene; During the progress of the scene, output the perception information corresponding to the perception information configuration file; Before obtaining the perception information configuration file corresponding to the marked scene, it further includes: determining the marked scene; Play a screencast file of a historical interaction process; When the marked scene appears during the playback of the screencast file, display an entry for custom perception information for the marked scene; In response to a trigger operation for the entry, display a custom interface for receiving the custom operation, including the types of perception information that can be configured for the marked scene, and the characteristics that can be configured for each type; Wherein, the types of the perception information include: tactile feedback information; visual feedback information; olfactory feedback information; auditory feedback information; In response to the custom operation for the marked scene, obtain the parameters configured by the custom operation for the characteristics in at least one of the types, and generate a perception information configuration file corresponding to the marked scene based on the obtained parameters of the characteristics in at least one of the types.

2. The method according to claim 1, wherein: The tactile feedback information includes at least one of the following characteristics: frequency, intensity, duration, orientation; The output of the perception information corresponding to the perception information configuration file includes: According to the type of event included in the scene, output tactile feedback information configured to be associated with the type of the event.

3. The method according to claim 2, characterized in that, The output of the tactile feedback information configured to be associated with the type of the event according to the type of event included in the scene includes: Perform at least one of the following operations: In a marked scene including a user operation event, synchronously output tactile feedback information synchronized with the user operation event; In a marked scene including sound, output at least one of the following: tactile feedback information adapted to the type of the sound, tactile feedback information consistent with the intensity of the sound, tactile feedback information consistent with the orientation of the sound source of the sound; In a marked scene including a collision event, output at least one of the following: tactile feedback information synchronized with the collision event, tactile feedback information consistent with the intensity of the collision event; In a marked scene including a notification event, output tactile feedback information synchronized with the notification event.

4. The method according to claim 1, wherein: The visual feedback information includes at least one of the following characteristics: state, frequency, color, brightness; The output of the perception information corresponding to the perception information configuration file includes: According to the type of event included in the scene, output visual feedback information configured to be associated with the type of the event.

5. The method according to claim 4, characterized in that, The output of the visual feedback information configured to be associated with the type of the event according to the type of event included in the scene includes: Perform at least one of the following operations: In a marking scenario including a notification event, output visual feedback information synchronized with the notification event; In a marking scenario including sound, output at least one of the following: visual feedback information adapted to the type of the sound, visual feedback information adapted to the rhythm of the sound; In a marking scenario including a progress event, output visual feedback information for characterizing the progress of the progress event; In a marking scenario including a result event, output visual feedback information for characterizing the result of the result event.

6. The method according to claim 1, wherein The determining of the marking scenario includes: Display the historical interaction process in the human-computer interaction interface, where the historical interaction process is earlier than the interaction process; In response to a marking operation on a target scenario in the historical interaction process, record the target scenario as a marking scenario.

7. The method according to claim 1, characterized in that, The determining of the marking scenario includes: In response to a marking operation on a target scenario in the interaction process, record the target scenario as a marking scenario.

8. The method according to claim 1, characterized in that, The method further includes: Display an entry for custom perception information for the marking scenario; In response to a trigger operation on the entry, display a custom interface for receiving the custom operation, including the types of perception information that can be configured for the marking scenario, and the characteristics that can be configured in each type.

9. The method according to claim 1, wherein The characteristics in each type have pre-configured default parameters; The obtaining of the parameters configured by the custom operation for the characteristics in at least one of the types includes: In response to the modification of the default parameters of the characteristics in at least one of the types by the custom operation, update the default parameters to the parameters set by the custom operation.

10. The method according to claim 1, wherein The method further includes: Display multiple pre-configured modes of the marking scenario; Wherein each mode includes at least one type of perception information associated with the marking scenario, and the default parameters pre-configured for the characteristics in the type; The obtaining of the parameters configured by the custom operation for the characteristics in at least one of the types includes: Obtain at least one type of perception information associated with the marking scenario included in the target mode, and the default parameters pre-configured for the characteristics in the type; Wherein the target mode is the mode selected by the custom operation from the multiple pre-configured modes.

11. The method according to claim 1, characterized in that, The obtaining of the parameters configured by the custom operation for the characteristics in at least one of the types includes: Obtain candidate parameters for the characteristics in at least one of the types; Perform a scoring prediction process on the candidate parameters through a first neural network model to obtain a prediction score for each candidate parameter; Use the candidate parameter with the highest prediction score as the parameter for the characteristics in at least one of the types; Wherein the first neural network model is trained based on candidate parameter samples and the true scores corresponding to the candidate parameter samples.

12. The method according to claim 1, characterized in that, The obtaining of the parameters configured by the custom operation for the characteristics in at least one of the types includes: Obtain operation behavior data in the human-computer interaction interface; Perform preference prediction processing on the operation behavior data through a second neural network model to obtain the operation preference represented by the operation data; Obtain parameters of characteristics in at least one of the types pre-associated with the operation preference; Wherein, the second neural network model is trained based on operation data samples and the operation preferences represented by the operation data samples.

13. The method according to claim 1, characterized in that, Before outputting the perception information corresponding to the perception information profile, the method further includes: Query the types of perception information supported for customization by the electronic device and the characteristics supported for customization in the corresponding types; According to the query result, filter out the types of perception information that do not support customization and the characteristics in the corresponding types from the perception information profile to update the perception information profile.

14. The method according to any one of claims 1 to 13, characterized in that, The output of the perception information corresponding to the perception information profile includes: Obtain from the perception information profile the types of perception information configured by the custom operation and the parameters of the characteristics in the corresponding types; Send a call request through the application programming interface open to the human-computer interaction interface to call the peripheral corresponding to the type to output the perception information of the corresponding type, so that the output perception information of the corresponding type conforms to the parameter of the characteristic in the corresponding type.

15. A perception processing method for human-computer interaction, characterized in that, Includes: Receive a custom operation for a marked scene in the human-computer interaction interface; In response to the custom operation, generate a perception information profile corresponding to the marked scene according to the type of perception information configured by the custom operation and the parameters of the characteristics in the corresponding type; Display the interaction process in the human-computer interaction interface; When the scene that appears again during the interaction process is the marked scene, during the progress of the scene, output the perception information of the type configured by the custom operation according to the perception information profile, and the output perception information of the corresponding type conforms to the parameter of the characteristic in the corresponding type; The receiving a custom operation for a marked scene in the human-computer interaction interface; in response to the custom operation, generating a perception information profile corresponding to the marked scene according to the type of perception information configured by the custom operation and the parameters of the characteristics in the corresponding type includes: Determine the marked scene; Play the screencast file of the historical interaction process; When the marked scene appears during the playback of the screencast file, display the entry for the custom perception information for the marked scene; In response to the trigger operation for the entry, display a custom interface for receiving the custom operation and including the types of perception information that can be configured for the marked scene and the characteristics that can be configured in each type; Wherein, the types of perception information include: tactile feedback information; visual feedback information; olfactory feedback information; auditory feedback information; In response to the custom operation for the marked scenario, obtain the parameters configured by the custom operation for the features in at least one of the types, and generate the perception information configuration file corresponding to the marked scenario based on the obtained parameters of the features in at least one of the types.

16. A perception processing device for human-computer interaction, characterized in that, Comprising: A display module for displaying the interaction process in a human-computer interaction interface; A detection module for detecting the scenarios that occur during the interaction process; An acquisition module for, when the scenario that occurs during the interaction process is a marked scenario configured with perception information by a custom operation, obtaining the perception information configuration file corresponding to the marked scenario; An output module for outputting the perception information corresponding to the perception information configuration file during the progress of the scenario; A configuration module, before obtaining the perception information configuration file corresponding to the marked scenario, further comprising: determining the marked scenario; playing a screen recording file of the historical interaction process; when the marked scenario appears during the playback of the screen recording file, displaying an entry for custom perception information for the marked scenario; in response to a trigger operation for the entry, displaying a custom interface for receiving the custom operation, including the types of perception information that can be configured for the marked scenario and the features that can be configured in each type; wherein the types of perception information include: tactile feedback information; visual feedback information; olfactory feedback information; auditory feedback information; in response to the custom operation for the marked scenario, obtaining the parameters configured by the custom operation for the features in at least one of the types, and generating the perception information configuration file corresponding to the marked scenario based on the obtained parameters of the features in at least one of the types.

17. A perception processing device for human-computer interaction, characterized in that, Comprising: A configuration module for receiving a custom operation for a marked scenario in a human-computer interaction interface; The configuration module is further configured to, in response to the custom operation, generate a perception information configuration file corresponding to the marked scenario according to the type of perception information configured by the custom operation and the parameters of the features in the corresponding type; A display module for displaying the interaction process in the human-computer interaction interface; An output module for, when the scenario that appears again during the interaction process is the marked scenario, outputting, during the progress of the scenario, the perception information of the type configured by the custom operation according to the perception information configuration file, and the output perception information of the corresponding type conforms to the parameters of the features in the corresponding type; The configuration module is further configured to receive, on the human-computer interaction interface, a custom operation for a marked scenario; in response to the custom operation, generate a perception information configuration file corresponding to the marked scenario according to the type of perception information configured by the custom operation and the parameters of the features in the corresponding type, including: determining the marked scenario; playing a screen recording file of a historical interaction process; when the marked scenario appears during the playback of the screen recording file, displaying an entry for custom perception information for the marked scenario; in response to a trigger operation for the entry, displaying a custom interface for receiving the custom operation and including the type of perception information that can be configured for the marked scenario and the features that can be configured in each type; wherein the type of perception information includes: tactile feedback information; visual feedback information; olfactory feedback information; auditory feedback information; in response to the custom operation for the marked scenario, obtaining the parameters configured by the custom operation for the features in at least one of the types, and generating the perception information configuration file corresponding to the marked scenario based on the obtained parameters of the features in at least one of the types.

18. An electronic device, characterized in that, Including: a memory for storing executable instructions; a processor for implementing the perception processing method of human-computer interaction according to any one of claims 1 to 14 or claim 15 when executing the executable instructions stored in the memory.

19. A computer-readable storage medium, characterized in that, Stored with executable instructions for implementing the perception processing method of human-computer interaction according to any one of claims 1 to 14 or claim 15 when being executed by a processor.

20. A computer program product comprising computer instructions, characterized in that, The computer instructions, when executed by a processor, implement the method according to any one of claims 1 to 14 or claim 15.

Images