Method, system and related device for generating haptic feedback effects

Abstract

The present invention relates to the application field of artificial intelligence technology, and in particular to a method, system and related equipment for generating haptic feedback effects. The method includes the steps of obtaining a training dataset including video or audio information, performing data partitioning on the training dataset to obtain partitioned data, mapping the partitioned data as haptic feedback information based on a network coefficient using a preset artificial intelligence, and outputting a haptic feedback effect based on the haptic feedback information.Compared with the prior art, the method for generating a haptic feedback effect according to the present invention combines the generation of haptic feedback information based on artificial intelligence, thereby reducing manual operations in the process of generating a haptic feedback effect, and using the existing artificial results as a training set to obtain a desired haptic feedback result through an optimized network coefficient, thereby improving the vibration feedback experience in practical application.

Description

[Technical field]

[0001] The present invention relates to the application field of artificial intelligence technology, and in particular to a method, system and related equipment for generating haptic feedback effects. [Background technology]

[0002] With the advancement of science and technology, artificial intelligence (AI) has gradually entered into people's lives. At present, AI builds huge databases based on language, images, and characters, learns deeply and independently, and uses this model to perform deep learning and predict the results. AI recognizes and selects environmental information and input known information through sensors, and feeds it back to practical applications.

[0003] In the past, haptic feedback systems using motors as carriers widely use vibration motors in applications such as mobile phones, smart watches, tablets, and in-vehicle devices, and the key operation of generating haptic feedback effects is how to drive the vibration motor to achieve the desired experience effect. Haptic feedback uses the "intensity + frequency" method to describe the desired effect. Usually, designers manually provide "amplitude + phase" information for different time periods based on the audio and video of a certain segment, and control the motor through these two abstracted parameters to achieve the desired vibration effect. However, this method places relatively high demands on audio designers, and audio and video need to be manually converted into effect files, which is time-consuming and may result in significant differences in the results obtained by different people.

[0004] Therefore, it is necessary to provide a new haptic feedback generating method to save the labor costs of designers and generate different effects of haptic feedback based on different audio and video in real life environment. Summary of the Invention [Problem to be solved by the invention]

[0005] The problem to be solved by the present invention is to provide a method for saving labor costs and generating different haptic feedback effects based on real life environments. [Means for solving the problem]

[0006] In order to achieve the above object, in a first aspect, the present invention provides a method for generating a haptic feedback effect, the method including the steps of obtaining a training dataset including video or audio information, performing data division on the training dataset to obtain divided data, utilizing a preset artificial intelligence to map the divided data as haptic feedback information based on a network coefficient, and outputting a haptic feedback effect based on the haptic feedback information.

[0007] Preferably, the method of data division for the training data set comprises dividing the training data set into frames with a preset frame length according to a time length of the training data set.

[0008] Preferably, the haptic feedback information includes vibration intensity information and vibration frequency information.

[0009] Preferably, before the step of mapping the divided data as haptic feedback information using the preset artificial intelligence, the method further includes the steps of annotating the haptic feedback information for the divided data in a manual annotation manner to obtain pre-training data, training the preset artificial intelligence based on the pre-training data, saving parameters of the preset artificial intelligence after training is completed, and outputting the network coefficients for the preset artificial intelligence to generate the haptic feedback information.

[0010] Preferably, after the step of outputting a haptic feedback effect based on the haptic feedback information, the method further includes a step of judging whether the haptic feedback effect meets a preset haptic feedback requirement, and if so, using the preset artificial intelligence to map the divided data of the next segment based on the current network coefficient; if not, adopting a manual calibration method and synchronously updating the network coefficient.

[0011] In a second aspect, the present invention further provides a haptic feedback effect generation system, comprising: a data acquisition module for acquiring a training dataset including video and audio information; a data partitioning module for performing data partitioning on the training dataset to obtain partitioned data; a data mapping module for mapping the partitioned data as haptic feedback information based on a network coefficient using a pre-configured artificial intelligence; and a haptic feedback output module for outputting a haptic feedback effect based on the haptic feedback information.

[0012] In a third aspect, the present invention further provides a computer apparatus comprising a processor, a memory and a computer program stored in the memory and executable within the processor, the processor, when executing the computer program, realising the steps of the method for generating a haptic feedback effect as described in any one of the above claims.

[0013] In a fourth aspect, the present invention further provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method for generating a haptic feedback effect according to any one of the above claims. Effect of the Invention

[0014] Compared with the prior art, the haptic feedback effect generating method of the present invention combines the generation of haptic feedback information based on artificial intelligence, and divides audio data consisting of a certain amount of video or audio to calibrate the haptic feedback information and complete the training process, thereby reducing manual operations in the haptic feedback effect generating process. With the existing artificial results as the training set, when the sample data and the number of iterations are sufficient, the desired haptic feedback result can be obtained through the optimized network coefficients, and the vibration feedback experience in practical application can be improved. [Brief description of the drawings]

[0015] In order to more clearly describe the technical solutions in the embodiments of the present invention, the drawings necessary for describing the embodiments will be briefly described below. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can obtain other drawings based on these drawings without any inventive work, among which: [Figure 1] FIG. 1 is a flowchart of a method for generating a haptic feedback effect according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a schematic diagram of haptic feedback information provided by an embodiment of the present invention. [Diagram 3] FIG. 3 is a diagram showing a configuration of a haptic feedback effect generating system 200 according to an embodiment of the present invention. [Figure 4] FIG. 4 is a diagram illustrating the configuration of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] The technical solutions in the embodiments of the present invention will be described below clearly and completely with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by those skilled in the art without making inventive efforts based on the embodiments of the present invention shall be included in the protection scope of the present invention.

[0017] FIG. 1 is a flowchart of a method for generating a haptic feedback effect according to an embodiment of the present invention. As shown in FIG. 1, the method includes the following steps S1 to S4.

[0018] S1: Obtain a training dataset containing video or audio information.

[0019] Specifically, the information-containing data may be video data or audio data, and the audio information is continuous with time increase in the training data set and has acoustic characteristics such as frequency, etc. In an embodiment of the present invention, the manner for obtaining the training data set may be intercepting existing audio data, or collecting in real time by recording, filming, etc.

[0020] S2: Perform data division on the training data set to obtain divided data.

[0021] Preferably, the method of data division for the training data set comprises dividing the training data set into frames with a preset frame length according to a time length of the training data set.

[0022] The preset frame length can be set according to actual needs, and different preset frame lengths can be set for different types of audio data according to the tempo speed, input method of audio data, etc.

[0023] Preferably, the haptic feedback information includes vibration intensity information and vibration frequency information.

[0024] Preferably, before the step of mapping the divided data as haptic feedback information using the preset artificial intelligence, the method further includes the steps of annotating the haptic feedback information for the divided data in a manual annotation manner to obtain pre-training data, training the preset artificial intelligence based on the pre-training data, saving parameters of the preset artificial intelligence after training is completed, and outputting the network coefficients for the preset artificial intelligence to generate the haptic feedback information.

[0025] Specifically, as shown in FIG. 2, FIG. 2 is a schematic diagram of haptic feedback information according to an embodiment of the present invention, and the haptic feedback information is shown in a two-dimensional format in the embodiment of the present invention. Taking the two-dimensional data I in FIG. 2 as an example, the audio data of the first segment can be shown by the frequency of one segment. In order to obtain the haptic feedback information represented by the audio data of this segment, in the spatial coordinate, the vibration intensity information of the audio data is represented by the abscissa, and the vibration frequency information of the audio data is represented by the ordinate, and finally, the corresponding haptic feedback information is annotated as a low intensity, low frequency effect represented by I. Meanwhile, for the audio data of the second segment having a relatively high frequency, the corresponding vibration frequency information can be increased when annotating to show different audio characteristics. Obtaining the initial training for the preset artificial intelligence by a manual annotation method is to control the directionality of the generation of the haptic feedback information by the preset artificial intelligence by a human intervention method, and obtain a feedback effect that matches the user's experience.

[0026] S3: Using a preset artificial intelligence, map the divided data as haptic feedback information based on a network coefficient.

[0027] The preset artificial intelligence may be realized based on a neural network model, or may be an automation program with parameter updating and iteration. The network coefficient corresponds to a model parameter in a neural network model, or a control parameter in an automation program, and when one training is completed, the network coefficient is output to fix the flow of generating the haptic feedback information by the preset artificial intelligence, and the generation ability of the preset artificial intelligence can be gradually improved through repeated iterations.

[0028] S4: Output a haptic feedback effect based on the haptic feedback information.

[0029] The haptic feedback effect is specifically generated based on the vibration intensity information and the vibration frequency information in the haptic feedback information, and one haptic feedback effect has a unique correspondence with the audio data when generating the haptic feedback information. In an embodiment of the present invention, the haptic feedback effect needs to be realized by a vibration feedback system mainly based on a motor.

[0030] Preferably, after the step of outputting a haptic feedback effect based on the haptic feedback information, the method further includes a step of judging whether the haptic feedback effect meets a preset haptic feedback requirement, and if so, using the preset artificial intelligence to map the divided data of the next segment based on the current network coefficient; if not, adopting a manual calibration method and synchronously updating the network coefficient.

[0031] Specifically, the preset haptic feedback requirement is a feedback mechanism for indicating whether the haptic feedback effect has a good correspondence with the corresponding audio data, and if the haptic feedback effect does not satisfy the preset haptic feedback requirement, the haptic feedback information is processed in a manner of manual calibration and the existing network coefficients are updated so that the finally mapped haptic feedback information is closer to the effect corresponding to the audio data.

[0032] Compared with the prior art, the haptic feedback effect generating method of the present invention combines the generation of haptic feedback information based on artificial intelligence, and divides audio data consisting of a certain amount of video or audio to calibrate the haptic feedback information and complete the training process, thereby reducing manual operations in the haptic feedback effect generating process. With the existing artificial results as the training set, when the sample data and the number of iterations are sufficient, the desired haptic feedback result can be obtained through the optimized network coefficients, and the vibration feedback experience in practical application can be improved.

[0033] FIG. 3 is a diagram showing the configuration of a haptic feedback effect generation system 200 according to an embodiment of the present invention. As shown in FIG. 3, the embodiment of the present invention further provides a haptic feedback effect generation system, which includes: a data acquisition module 201 for acquiring a training dataset including video and audio information; a data division module 202 for performing data division on the training dataset and obtaining divided data; a data mapping module 203 for mapping the divided data as haptic feedback information based on network coefficients using preset artificial intelligence; and a haptic feedback output module 204 for outputting a haptic feedback effect based on the haptic feedback information.

[0034] The haptic feedback effect generating system 200 according to the embodiment of the present invention can realize the steps in the haptic feedback effect generating method in the above embodiment, and achieve the same technical effects, which can be referred to in the description of the above embodiment, and thus will not be described again here.

[0035] 4 is a diagram showing the configuration of a computer device provided by an embodiment of the present invention, and the embodiment of the present invention further provides a computer device as shown in FIG 4. The computer device 300 includes a processor 301, a memory 302, and a computer program stored in the memory 302 and executable in the processor 301.

[0036] As shown in FIG. 1, the processor 301 calls a computer program stored in the memory 302, and when executing the computer program, realizes the steps of the haptic feedback effect generating method in the above embodiment, including: obtaining a training dataset including video and audio information; performing data partitioning on the training dataset to obtain partitioned data; using a preset artificial intelligence to map the partitioned data as haptic feedback information based on a network coefficient; and outputting a haptic feedback effect based on the haptic feedback information.

[0037] Preferably, the method of data division for the training data set is to divide the training data set into frames with a preset frame length based on the time length of the training data set.

[0038] Preferably, the haptic feedback information includes vibration intensity information and vibration frequency information.

[0039] Preferably, before the step of mapping the segmented data as haptic feedback information using a preset artificial intelligence, the method further includes the steps of annotating the haptic feedback information for the segmented data in a manual annotation manner to obtain pre-training data, training the preset artificial intelligence based on the pre-training data, saving parameters of the preset artificial intelligence after training is completed, and outputting the network coefficients for the preset artificial intelligence to generate the haptic feedback information.

[0040] Preferably, after the step of outputting a haptic feedback effect based on the haptic feedback information, the method further includes a step of judging whether the haptic feedback effect meets a preset haptic feedback requirement, and if so, using the preset artificial intelligence to map the divided data of the next segment based on the current network coefficient; if not, adopting a manual calibration method and synchronously updating the network coefficient.

[0041] The computer device 300 according to the embodiment of the present invention can implement the steps in the haptic feedback effect generating method in the above embodiment, and achieve the same technical effects, which can be referred to in the description of the above embodiment, and thus will not be described again here.

[0042] The embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can realize each process and step in the method for generating a haptic feedback effect according to the embodiment of the present invention, and achieve the same technical effect, which will not be described again here to avoid duplication.

[0043] The above is merely an embodiment of the present invention, and it should be noted here that those skilled in the art may make modifications without departing from the creative idea of ​​the present invention, and all such modifications are within the scope of protection of the present invention.

Claims

1. obtaining a training dataset comprising video or audio information; performing data partitioning on the training data set to obtain partitioned data; mapping the segmented data as haptic feedback information based on a network coefficient using a pre-configured artificial intelligence; and outputting a haptic feedback effect based on the haptic feedback information. the haptic feedback information includes vibration intensity information and vibration frequency information, the haptic feedback information is represented in a two-dimensional format, in which the abscissa indicates the vibration intensity information of the haptic feedback information and the ordinate indicates the vibration frequency information of the haptic feedback information in a spatial coordinate system; After the step of outputting a haptic feedback effect based on the haptic feedback information, the method further includes the step of determining whether the haptic feedback effect satisfies a preset haptic feedback requirement, wherein: If yes, map the segmented data of the next segment using the pre-configured artificial intelligence based on the current network coefficient; If not, adopt manual calibration method and update network coefficients at the same time; 13. A method for generating a haptic feedback effect, comprising:

2. The method of performing data division on the training data set includes dividing the training data set into frames with a preset frame length according to a time length of the training data set.

2. The method of claim 1, wherein the first and second haptic feedback effects are generated by the first and second haptic feedback effects.

3. before the step of mapping the segmented data as haptic feedback information using the preset artificial intelligence, annotating the haptic feedback information for the segmented data in a manual annotation manner to obtain pre-training data; training the pre-defined artificial intelligence based on the pre-training data, storing parameters of the pre-defined artificial intelligence after the training is completed, and outputting the network coefficients for the pre-defined artificial intelligence to generate the haptic feedback information.

2. The method of claim 1, wherein the first and second haptic feedback effects are generated by the first and second haptic feedback effects.

4. A haptic feedback effect generating system for implementing steps in the haptic feedback effect generating method according to any one of claims 1 to 3, comprising: a data acquisition module for acquiring a training dataset including video and audio information; a data partitioning module for performing data partitioning on the training data set to obtain partitioned data; a data mapping module for mapping the segmented data as haptic feedback information based on a network coefficient using a pre-configured artificial intelligence; a haptic feedback output module for outputting a haptic feedback effect based on the haptic feedback information. A system for generating a haptic feedback effect.

5. A method for generating a haptic feedback effect comprising: a processor; a memory; and a computer program stored in the memory and executable within the processor, the method implementing the steps of the method for generating a haptic feedback effect according to any one of claims 1 to 3 when the processor executes the computer program. A computer device comprising:

6. A computer program is stored, the computer program being adapted to implement the steps of the method for generating a haptic feedback effect according to any one of claims 1 to 3 when executed by a processor. A computer-readable storage medium comprising:

Images