Method, system and related device for generating haptic feedback effects

Abstract

The application relates to the field of artificial intelligence technology application, in particular to a haptic feedback effect generation method and system and related equipment, the method comprising: acquiring a training data set containing video and audio information; performing data cutting on the training data set to obtain cutting data; using a preset artificial intelligence to map the cutting data into haptic feedback information according to network coefficients; and outputting a haptic feedback effect according to the haptic feedback information. Compared with related technologies, the haptic feedback effect generation method of the application fuses the generation of haptic feedback information based on artificial intelligence, can reduce manual operation in the generation process of the haptic feedback effect, and optimizes the network coefficients on the basis of the existing artificial results in the early stage as a training set to obtain the expected haptic feedback effect, thereby improving the vibration feedback experience in actual application.

Description

[0001] The present application relates to the field of artificial intelligence technology application, and in particular to a haptic feedback effect generation method, system and related equipment.

[0002] With the progress of science and technology, artificial intelligence (AI for short) has gradually entered people's lives. Today, artificial intelligence is based on language, images, and text, establishes a large database, and conducts deep and autonomous learning to calculate results through this mode. Artificial intelligence identifies environmental information through sensors and filters known information, and feeds back to practical applications.

[0003] In related technologies, haptic feedback systems using motors as carriers are widely used in mobile phones, smart watches, tablet computers, and car machines. How to drive the vibration motor to achieve the desired experience effect has become the key action of haptic feedback effect generation. Haptic feedback uses "intensity + frequency" to describe the desired effect. In the conventional method, designers give "amplitude + phase" information in different time periods based on an audio or video through manual operation, and control the motor through these two abstract parameters to achieve the desired vibration effect. However, this method has a high requirement for audio designers, and it takes a long time to manually convert audio and video into effect files, and different people may get different results.

[0004] Therefore, it is necessary to provide a new haptic feedback generation method to save the labor cost of designers and to produce different effects based on various sounds and videos in real life.

[0005] The technical problem to be solved by the present application is to provide a method for saving labor cost and producing different haptic feedback effects based on real life environment.

[0006] To solve the above technical problems, in a first aspect, the present application provides a haptic feedback effect generation method, which comprises the following steps:

[0007] Obtain a training data set containing video and audio information;

[0008] Cut the training data set to obtain cut data;

[0009] Map the cut data to haptic feedback information according to the network coefficient using a preset artificial intelligence;

[0010] ​​​Output a haptic feedback effect according to the haptic feedback information.

[0011] Preferably, the method of data cutting of the training data set is that according to the length of time of the training data set, the training data set is processed by frame according to a preset frame length.

[0012] Preferably, the haptic feedback information includes vibration intensity information and vibration frequency information.

[0013] Preferably, before the step of using the preset artificial intelligence to map the cutting data into the haptic feedback information, the method further comprises the following steps:

[0014] By means of artificial labeling, the cutting data is labeled with the haptic feedback information to obtain pre-training data;

[0015] According to the pre-training data, the preset artificial intelligence is trained, the parameters of the trained preset artificial intelligence are saved, and the network coefficients used by the preset artificial intelligence to generate the haptic feedback information are output.

[0016] Preferably, after the step of outputting a haptic feedback effect according to the haptic feedback information, the method further comprises the following steps:

[0017] Determine whether the haptic feedback effect meets a preset haptic feedback requirement, wherein:

[0018] If yes, map the next piece of cutting data by using the preset artificial intelligence according to the current network coefficients;

[0019] If no, use artificial calibration and update the network coefficients synchronously.

[0020] In a second aspect, the application further provides a haptic feedback effect generation system, comprising:

[0021] A data acquisition module is configured to acquire a training data set containing audio information.

[0022] A data cutting module is configured to cut the training data set to obtain cutting data.

[0023] A data mapping module is configured to map the cutting data into haptic feedback information according to network coefficients by using a preset artificial intelligence.

[0024] A haptic feedback output module is configured to output a haptic feedback effect according to the haptic feedback information.

[0025] In a third aspect, the present application also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in the method for generating a haptic feedback effect according to any one of the preceding aspects.

[0026] In a fourth aspect, the present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executable on a processor to implement the steps in the method for generating a haptic feedback effect according to any one of the preceding aspects.

[0027] Compared with the related art, in the method for generating a haptic feedback effect, the generation of haptic feedback information based on artificial intelligence is combined, the audio data composed of a certain number of videos or audios is cut and calibrated to complete the training process, which can reduce manual operation in the generation of a haptic feedback effect, and when the sample data and the number of iterations are sufficient, the optimized network coefficient can be used to obtain the expected haptic feedback effect, thereby improving the vibration feedback experience in actual application. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor, wherein:

[0029] Figure 1 is a step flowchart of the method for generating a haptic feedback effect provided by the embodiments of the present application;

[0030] Figure 2 is a schematic diagram of haptic feedback information provided by the embodiments of the present application;

[0031] Figure 3 is a structural schematic diagram of the system 200 for generating a haptic feedback effect provided by the embodiments of the present application;

[0032] Figure 4 is a structural schematic diagram of the computer device provided by the embodiments of the present application.

DETAILED DESCRIPTION

[0033] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts are within the scope of the present application.

[0034] Please refer to Figure 1 , Figure 1 is a step flow diagram of a method for generating a haptic feedback effect provided by an embodiment of the present application. The method comprises the following steps:

[0035] S1, obtaining a training data set containing video or audio information.

[0036] Specifically, the data containing information can be video data or audio data. The audio information is continuous with the increase of time in the training data set and has acoustic characteristics such as frequency. In the embodiments of the present application, the way to obtain the training data set can be to cut from existing audio data, or to obtain by real-time collection such as recording and shooting.

[0037] S2, data cutting is performed on the training data set to obtain cutting data.

[0038] Preferably, the method of data cutting on the training data set is to perform frame processing on the training data set according to a preset frame length according to the length of the training data set.

[0039] The preset frame length can be set according to actual needs. For different types of audio data, different preset frame lengths can be set according to the speed of rhythm, the way of audio data entry, etc.

[0040] Preferably, the haptic feedback information includes vibration intensity information and vibration frequency information.

[0041] Preferably, before the step of using the preset artificial intelligence to map the cutting data into haptic feedback information, the following steps are further included:

[0042] The cutting data is labeled with the haptic feedback information by manual labeling to obtain pre-training data;

[0043] The preset artificial intelligence is trained according to the pre-training data. The parameters of the trained preset artificial intelligence are saved, and the network coefficients used by the preset artificial intelligence to generate the haptic feedback information are output.

[0044] Specifically, please refer to Figure 2 ,Figure 2 is a schematic diagram of the haptic feedback information provided by the embodiment of the present application, for the haptic feedback information, in the embodiment of the present application, is represented in the form of two-dimensional data, so as to Figure 2 Taking the two-dimensional data of I in the embodiment as an example, a piece of audio data can be represented by a frequency, in order to obtain the haptic feedback information to be represented by the piece of audio data, in the spatial coordinates, the abscissa is used to represent the vibration intensity information of the audio data, and the ordinate is used to represent the vibration frequency information of the audio data, and finally the corresponding haptic feedback information is marked as the low-intensity and low-frequency effect shown by I; and for a second piece of audio data with a higher frequency, the corresponding vibration frequency information can be adjusted to be higher during marking, so as to reflect the characteristics of different audio. The artificial marking manner used to obtain the preliminary training of the preset artificial intelligence is to control the directionality of the preset artificial intelligence for generating the haptic feedback information by human intervention, so as to obtain a feedback effect meeting the user experience.

[0045] S3, mapping the cutting data into haptic feedback information according to the network coefficient by using the preset artificial intelligence.

[0046] The preset artificial intelligence can be implemented based on a neural network model, or can be an automatic program with parameter updating and iteration. The network coefficient is equivalent to a model parameter in the neural network model, or is a control parameter in the automatic program. When one training is completed, the network coefficient is output, so that the process of generating the haptic feedback information by the preset artificial intelligence is fixed, and the generation ability of the preset artificial intelligence is gradually improved in continuous iteration.

[0047] S4, outputting a haptic feedback effect according to the haptic feedback information.

[0048] The haptic feedback effect is generated according to the vibration intensity information and the vibration frequency information in the haptic feedback information, and the haptic feedback effect has a unique corresponding relationship with the audio data when the haptic feedback information is generated. In the embodiment of the present application, the haptic feedback effect needs to be realized through a motor-based vibration feedback system.

[0049] Preferably, after the step of outputting the haptic feedback effect according to the haptic feedback information, the following step is further included:

[0050] determining whether the haptic feedback effect meets a preset haptic feedback requirement, wherein:

[0051] if yes, mapping the next piece of cutting data by using the preset artificial intelligence according to the current network coefficient;

[0052] If not, the artificial calibration mode is adopted, and the network coefficient is updated synchronously.

[0053] Specifically, the preset haptic feedback requirement is a feedback mechanism for indicating whether the haptic feedback effect has a good corresponding relationship with the corresponding audio data. When the haptic feedback effect does not meet the preset haptic feedback requirement, the haptic feedback information can be processed in an artificial calibration mode, and the existing network coefficient is updated, so that the finally mapped haptic feedback information can be closer to the effect corresponding to the audio data.

[0054] Compared with the related art, in the haptic feedback effect generation method of the present application, the generation of the haptic feedback information is combined with artificial intelligence, the audio data composed of a certain number of videos or audios is cut and calibrated to complete the training process, which can reduce the artificial operation in the generation process of the haptic feedback effect, and when the sample data and the iteration number are sufficient, the expected haptic feedback effect can be obtained through the optimized network coefficient based on the existing artificial results as the training set, thereby improving the vibration feedback experience in actual application.

[0055] The present application also provides a haptic feedback effect generation system, please refer to Figure 3 , Figure 3 is a structure diagram of the haptic feedback effect generation system 200 provided by the present application, which comprises:

[0056] The data acquisition module 201 is configured to acquire a training data set containing video and audio information.

[0057] The data cutting module 202 is configured to cut the training data set to obtain cutting data.

[0058] The data mapping module 203 is configured to map the cutting data into haptic feedback information according to the network coefficient using a preset artificial intelligence.

[0059] The haptic feedback output module 204 is configured to output a haptic feedback effect according to the haptic feedback information.

[0060] The haptic feedback effect generation system 200 provided by the present application can realize the steps in the haptic feedback effect generation method of the above-mentioned embodiment, and can realize the same technical effects. Please refer to the description in the above-mentioned embodiment, which will not be repeated here.

[0061] The present application also provides a computer device, please refer to Figure 4 as shown, Figure 4is a structural schematic diagram of a computer device provided by an embodiment of the present application. The computer device 300 comprises a processor 301, a memory 302, and a computer program stored in the memory 302 and capable of running on the processor 301.

[0062] Please refer to Figure 1 The processor 301 invokes the computer program stored in the memory 302, and implements the steps in the generation method of the haptic feedback effect in the above-mentioned embodiment when the computer program is executed, comprising:

[0063] Obtain a training data set containing video and audio information;

[0064] Data cutting is performed on the training data set to obtain cutting data;

[0065] The cutting data is mapped into haptic feedback information according to network coefficients using a preset artificial intelligence;

[0066] Output a haptic feedback effect according to the haptic feedback information.

[0067] Preferably, the method for data cutting on the training data set is that, according to the length of the training data set, the training data set is frame-processed according to a preset frame length.

[0068] Preferably, the haptic feedback information comprises vibration intensity information and vibration frequency information.

[0069] Preferably, before the step of mapping the cutting data into haptic feedback information using a preset artificial intelligence, the following steps are further included:

[0070] The cutting data is labeled with the haptic feedback information by manual labeling to obtain pre-training data;

[0071] The preset artificial intelligence is trained according to the pre-training data, the parameters of the trained preset artificial intelligence are saved, and the network coefficients used by the preset artificial intelligence to generate the haptic feedback information are output.

[0072] Preferably, after the step of outputting a haptic feedback effect according to the haptic feedback information, the following steps are further included:

[0073] Determine whether the haptic feedback effect meets a preset haptic feedback requirement, wherein:

[0074] If yes, map the next piece of cutting data using the preset artificial intelligence according to the current network coefficients;

[0075] If no, use manual calibration and update the network coefficients synchronously.

[0076] The computer device 300 provided by the embodiment of the present application can realize the steps in the method for generating the haptic feedback effect in the above embodiment, and can realize the same technical effects. For details, refer to the description in the above embodiment, which will not be repeated here.

[0077] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to realize each process and step in the method for generating the haptic feedback effect provided by the embodiment of the present application, and can realize the same technical effects. To avoid repetition, details will not be repeated here.

[0078] The above only describes the embodiments of the present application. It should be pointed out that, for those skilled in the art, improvements can be made without departing from the concept of the present application, but these all belong to the protection scope of the present application.

Claims

1. A method of generating a haptic feedback effect, characterized by, The generation method comprises the following steps: obtaining a training data set containing video and audio information; data cutting is performed on the training data set to obtain cut data; using a preset artificial intelligence to map the cut data into haptic feedback information according to network coefficients; outputting a haptic feedback effect according to the haptic feedback information; Before the step of using the preset artificial intelligence to map the cut data into haptic feedback information, the following steps are further included: annotating the cut data with the haptic feedback information in an artificial annotation manner to obtain pre-training data; training the preset artificial intelligence according to the pre-training data, saving the parameters of the trained preset artificial intelligence, and outputting the network coefficients used by the preset artificial intelligence to generate the haptic feedback information; After the step of outputting a haptic feedback effect according to the haptic feedback information, the following steps are further included: determining whether the haptic feedback effect meets preset haptic feedback requirements, wherein: if yes, mapping the next piece of cut data using the preset artificial intelligence according to the current network coefficients; if no, using an artificial calibration manner and synchronously updating the network coefficients.

2. The method for generating a tactile feedback effect according to claim 1, wherein The method for data cutting on the training data set is to perform frame processing on the training data set according to a preset frame length based on the length of the training data set.

3. The method for generating a tactile feedback effect according to claim 1, wherein The haptic feedback information includes vibration intensity information and vibration frequency information.

4. A system for generating haptic feedback effects, characterized by It comprises: a data acquisition module for obtaining a training data set containing audio information; a data cutting module for performing data cutting on the training data set to obtain cut data; a data mapping module for using a preset artificial intelligence to map the cut data into haptic feedback information according to network coefficients; a haptic feedback output module for outputting a haptic feedback effect according to the haptic feedback information; The data mapping module is further used for: annotating the cut data with the haptic feedback information in an artificial annotation manner to obtain pre-training data; training the preset artificial intelligence according to the pre-training data, saving the parameters of the trained preset artificial intelligence, and outputting the network coefficients used by the preset artificial intelligence to generate the haptic feedback information; The haptic feedback output module is further used for: determining whether the haptic feedback effect meets preset haptic feedback requirements, wherein: if yes, mapping the next piece of cut data using the preset artificial intelligence according to the current network coefficients; if no, using an artificial calibration manner and synchronously updating the network coefficients.

5. A computer device, comprising: It comprises: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps in the haptic feedback effect generation method according to any one of claims 1 to 3 when executing the computer program.

6. A computer readable storage medium characterized by, A computer program is stored on the computer readable storage medium, and the computer program is executed by the processor to implement the steps in the haptic feedback effect generation method according to any one of claims 1 to 3.

Images