Operation feedback method and display device

By detecting the target object in the display device and outputting multimodal physical signals, the problem of single touch feedback function in the prior art is solved, rich multimodal feedback is achieved, and the human-computer interaction experience is improved.

CN118939117APending Publication Date: 2024-11-12BEIJING BOE TECH DEV CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410978990.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-19
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The touch feedback function of existing display devices is relatively single, and cannot provide multimodal physical feedback, which limits the richness of human-computer interaction.

Method used

By detecting the target object on the display interface, based on its attributes and operations, the display device is driven to output at least one of a sound signal, an odor signal and a vibration signal to provide multimodal physical feedback.

Benefits of technology

In addition to tactile feedback, it can also provide olfactory and auditory feedback, enhance feedback function, and is suitable for all graphics that support the interaction between the touch body and the display device, enriching the human-computer interaction experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118939117A_ABST
    Figure CN118939117A_ABST
Patent Text Reader

Abstract

The invention provides an operation feedback method and a display device, and relates to the technical field of display devices.The method comprises the steps that an object displayed on a display interface is detected; when it is detected that a target object is displayed, in response to an operation on the target object, the display device is driven to output a physical signal of at least one mode based on the attribute of the target object and the operation, so that physical feedback is provided for the operation; wherein the target object comprises a graph used for supporting interaction between the display device and the touch control body, and the physical signal comprises at least one of a sound signal, a smell signal and a vibration signal.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the technical field of display devices, and in particular to operation feedback for display devices, and specifically to an operation feedback method and a display device. Background Art

[0002] Many current display screens have integrated touch functions, which can realize interactive functions such as human-computer interaction through touch. As display devices are used by more and more people, most users need to operate directly on the screen, for example, by touching controls and specific interactive areas on the touch screen to perform touch operations, writing operations, etc., to work, study and entertain.

[0003] In the related art, in addition to the touch function, the display device also provides touch feedback, but the function of the touch feedback is relatively simple. Summary of the invention

[0004] Based on the content of the background technology, the present disclosure proposes an operation feedback method and a display device.

[0005] In a first aspect of the present disclosure, an operation feedback method is provided, which is applied to a display device, and the method comprises:

[0006] Detecting objects displayed on the display interface;

[0007] When detecting that a target object is displayed, in response to an operation on the target object, based on the properties of the target object and the operation, driving the display device to output a physical signal of at least one mode to provide physical feedback for the operation;

[0008] The target object includes a graphic for supporting interaction between the display device and the touch-sensitive body, and the physical signal includes at least one of a sound signal, an odor signal, and a vibration signal.

[0009] Exemplarily, in response to the operation performed on the target object, based on the attribute of the target object and the operation, driving the display device to output a physical signal of at least one modality includes:

[0010] generating at least one stimulus signal based on the attribute of the target object and the operation;

[0011] The display device is driven to output a physical signal corresponding to at least one of the excitation signals.

[0012] Exemplarily, the display interface includes a plurality of the target objects, and in response to an operation performed on the target object, based on the attributes of the target object and the operation, driving the display device to output a physical signal of at least one modality includes:

[0013] In response to an operation performed on the target object, acquiring a touch object at a position where the operation is performed;

[0014] Based on the properties of the touch object and the properties of the operation, the display device is driven to generate a physical signal corresponding to the touch object; wherein different touch objects correspond to different physical signals.

[0015] Exemplarily, the target object includes a control graphic and a background pattern, and the driving the display device to generate a physical signal corresponding to the touch object based on the attribute of the touch object and the attribute of the operation includes:

[0016] Based on the properties of the touch object and the properties of the operation, driving the display device to output a vibration signal; and

[0017] Based on the attributes of the background pattern, the display device is driven to generate the odor signal and / or the sound signal.

[0018] Exemplarily, in the case where the target object includes a background pattern, the physical signal includes at least the odor signal;

[0019] In a case where the target object does not include the background pattern, the physical signal includes the vibration signal and / or the sound signal.

[0020] Exemplarily, before driving the display device to output at least one modal physical signal based on the attribute of the target object and the operation, the method further includes:

[0021] Acquiring the touch pressure generated by the operation on the display device;

[0022] The step of driving the display device to output at least one modal physical signal based on the attribute of the target object and the operation includes:

[0023] Determining a signal strength of each of the physical signals based on the touch pressure;

[0024] Based on the signal strength, the attribute of the target object and the operation, the display device is driven to output the physical signal.

[0025] Exemplarily, before detecting the object displayed on the display interface, the method further includes:

[0026] Display a preset icon, wherein the preset icon includes multiple flags;

[0027] In response to a target operation performed on the plurality of flag positions, acquiring an intensity level corresponding to the target operation, the target operation comprising a drag operation on a touch-sensitive body between the flag positions or a selection operation of touching the flag positions;

[0028] The step of driving the display device to output at least one modal physical signal based on the attribute of the target object and the operation includes:

[0029] Based on the attribute of the target object and the operation, the display device is driven to output the physical signal conforming to the intensity level.

[0030] Exemplarily, the physical signal includes the sound signal and the vibration signal; and the step of driving the display device to output at least one modal physical signal based on the attribute of the target object and the operation includes:

[0031] Obtaining the acceleration corresponding to the operation;

[0032] generating a first excitation signal based on the property of the target object and the operation, and generating a second excitation signal based on the property of the target object, the acceleration, and the operation;

[0033] Based on the first excitation signal, dispatching audio data, and based on the acceleration, playing the audio data to output the sound signal;

[0034] And, based on the second excitation signal, output the vibration signal.

[0035] Exemplarily, the physical signal includes the vibration signal, and the step of driving the display device to output at least one modal physical signal based on the attribute of the target object and the operation includes:

[0036] Determine a follow target based on the attribute of the target object, the follow target being a target followed by the vibration region generating the vibration signal, the follow target including the operation or the target object;

[0037] Based on the following target, determining a target area;

[0038] Based on the attribute of the target object and the operation, the vibration signal is output in the target generation area.

[0039] Exemplarily, the physical signal includes the sound signal, the smell signal, and the vibration signal, and the driving the display device to output at least one modal physical signal based on the attribute of the target object and the operation includes:

[0040] Based on the properties of the target object and the operation, determining the occurrence time, end time, and output frequency corresponding to each of the plurality of physical signals;

[0041] According to the occurrence time, the end time and the output frequency, a plurality of physical signals are output.

[0042] Exemplarily, when the target object includes a background pattern, the method further includes:

[0043] In response to a touch operation on the area where the background pattern is located, acquiring a texture type of the background pattern and a material type corresponding to the background pattern;

[0044] Based on the type of the touch operation, the texture type, and the material type, the background pattern is transformed within a range of a position where the touch operation is located.

[0045] Exemplarily, in response to the operation performed on the target object, based on the attribute of the target object and the operation, driving the display device to output a physical signal of at least one modality includes:

[0046] Detecting an excitation element configured for the display device;

[0047] Determining a target excitation element based on the detection result of the detection and the attribute of the target object, wherein the excitation element includes at least one of an actuating element, an odor generating element, and a sound generating element;

[0048] Based on the attribute of the target object and the operation, the target excitation element is driven to output a corresponding physical signal.

[0049] According to a second aspect of the present disclosure, a display device is provided, comprising: a display component, a main controller and a multimodal component, wherein the main controller is connected to the display component and the multimodal component respectively; the multimodal component comprises at least one of an actuating element, an odor generating element and a sound generating element;

[0050] Wherein, the display component is used to provide a display interface;

[0051] The main controller is used to execute the operation feedback method described in any exemplary embodiment of the first aspect;

[0052] The multimodal component is used to output at least one physical signal under the drive of the main controller.

[0053] Exemplarily, the display device further includes a pressure sensor, and the pressure sensor is connected to the main controller;

[0054] The pressure sensor is configured to detect the touch pressure borne by the display interface;

[0055] The main controller is specifically configured to determine the signal strength corresponding to the physical signal based on the touch pressure, and generate a physical signal that meets the signal strength.

[0056] Exemplarily, the display device further includes an acceleration sensing module, and the acceleration sensing module is connected to the main controller;

[0057] The acceleration sensing module is configured to detect the acceleration of the operation performed on the target object on the display interface;

[0058] The main controller is specifically configured to determine the vibration frequency of the vibration signal and / or the playback rate corresponding to the sound signal based on the acceleration.

[0059] The operation feedback method provided by the embodiment of the present disclosure can be applied to a display device. In the method, an object displayed on a display interface can be detected; then, when a target object is detected, in response to the operation of the touch body on the target object, based on the properties and operation of the target object, the display device is driven to output at least one modal physical signal to provide physical feedback for the operation; wherein the target object includes a graphic for supporting the interaction between the display device and the touch body, and the physical signal includes at least one of a sound signal, an odor signal, and a vibration signal. Since the target object for the operation feedback is a graphic that supports the interaction between the touch body and the display device, the operation feedback can be provided not only for the control graphic, but also for any graphic that supports the interaction between the touch body and the display device, thereby avoiding the problem that the touch feedback function is relatively single. In addition, since the physical signal for the operation feedback can include at least one of a sound signal, an odor signal, and a vibration signal, the touch feedback is not limited to tactile feedback, but can also provide olfactory feedback and auditory feedback, thereby forming a multi-modal physical signal feedback, further enhancing the feedback function.

[0060] The above description is only an overview of the technical solution of the present disclosure. In order to more clearly understand the technical means of the present disclosure, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present disclosure more obvious and easy to understand, the specific implementation methods of the present disclosure are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or related technologies, the following is a brief introduction to the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. It should be noted that the proportions in the drawings are only for illustration and do not represent the actual proportions.

[0062] Figure 1 A schematic diagram of a step flow of an operation feedback method in an embodiment of the present disclosure is shown;

[0063] Figure 2 A schematic diagram of an application scenario of this embodiment of the present disclosure is shown;

[0064] Figure 3 A schematic diagram of an interface in which the target object is a control in an embodiment of the present disclosure is shown;

[0065] Figure 4 A schematic diagram of an interface in which the target object is a background pattern in an embodiment of the present disclosure is shown;

[0066] Figure 5 and Figure 6 Schematic diagrams of two preset icons are shown respectively;

[0067] Figure 7 A schematic diagram showing a specific process of generating a physical signal in an embodiment of the present disclosure is shown;

[0068] Figure 8-Figure 10 Schematic diagrams of the flow of the operation feedback method in three exemplary embodiments are shown respectively;

[0069] Fig.11 A schematic diagram of a circuit connection framework of a display device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0070] In order to make the above-mentioned purposes, features and advantages of the present disclosure more obvious and understandable, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0071] In view of the problems in the related art, the inventor of the present application has proposed an operation feedback method, in which the types of physical signals fed back for touch operations are enriched, so that in addition to tactile feedback, auditory feedback and olfactory feedback can be provided to the user, and in subsequent embodiments, visual feedback can also be provided. At the same time, the objects of touch feedback can also be expanded, for example, all graphics with interactive functions can get corresponding signal feedback.

[0072] Reference Figure 1 and Figure 2 As shown, Figure 1 A schematic diagram showing the steps of an operation feedback method in an embodiment of the present disclosure is shown. Figure 2 A schematic diagram of an application scenario of this embodiment is shown. Figure 1 and Figure 2 As shown, the operation feedback method proposed in this embodiment can be applied to a display device, wherein the display device can have a touch function and can integrate a multimodal component, which can include at least one of an actuating element, an odor generating element and a sound generating element.

[0073] The actuator can be used to provide a vibration signal, such as vibrating in response to a touch operation when tactile feedback is required; in practice, the actuator can include at least one of a piezoelectric driver, a solenoid valve, a linear motor, and an eccentric rotor. Of course, it can also be made of piezoelectric material.

[0074] The odor generating element can be used to provide an odor signal. In practice, the odor generating element can include a shell, a cavity is formed in the shell, and the cavity can be divided into a plurality of sealed small cavities. The shell can be provided with a valve at the position of each small cavity. When the valve is opened, the fragrance placed in the corresponding small cavity will emit an odor, thereby generating a corresponding odor signal. In this embodiment, in response to a touch operation, the valve at the corresponding small cavity can be opened according to the properties of the target object targeted by the operation, thereby generating an odor signal corresponding to the properties of the target object.

[0075] Among them, the sound generating element can be a speaker configured in the display device. In practice, multiple audio files can be stored in the display device in advance. When a touch operation occurs, the corresponding audio file can be played from the multiple audio files according to the properties of the touch operation and the properties of the target object targeted by the touch operation, so that the speaker generates a corresponding sound signal.

[0076] The display device further integrates a display screen and a controller connected to the display screen, and the display screen is used to provide display under the drive of the controller.

[0077] like Figure 1As shown, the operation feedback method in this embodiment mainly includes the following steps:

[0078] Step S101: Detecting an object displayed on a display interface;

[0079] Step S102: when a target object is detected to be displayed, in response to the touch-sensitive body's operation on the target object, based on the properties and operation of the target object, driving the display device to output a physical signal of at least one mode to provide physical feedback for the operation.

[0080] The target object includes a graphic for supporting interaction between the display device and the touch body, and the physical signal includes at least one of a sound signal, an odor signal, and a vibration signal.

[0081] In this embodiment, the display interface refers to the interface when the display screen is in the display state. The objects displayed on the display interface can be graphic objects, video objects, text objects, etc. in the display page. For example, if the display interface displays a browser page, the text, controls, images, etc. in the browser page can all be called objects displayed on the display interface.

[0082] In practice, the object displayed on the display interface can be detected. Specifically, the property of the displayed object can be detected, and the property can be used to describe the category of the object. In this embodiment, the category of the object can be roughly divided into two types, one is an object that can be touched for human-computer interaction (hereinafter referred to as the target object), and the other is an object that cannot be touched for human-computer interaction. In this implementation, physical feedback can be given to the operation on the target object. As mentioned above, the target object is an object that can be touched to achieve human-computer interaction. In one example, the target object can include graphics that support the interaction between the display device and the touch body, such as a control graphic that can be touched and a background pattern that can be used as a writing background.

[0083] Please refer to Figure 3 and Figure 4 As shown, Figure 3 The schematic diagram of the interface in which the target object is a control is shown. Figure 4 A schematic diagram of an interface in which the target object is a background pattern is shown, such as Figure 3 As shown, it includes a variety of controls, such as a control 11 for adjusting temperature, which is in the shape of a knob, and the temperature can be adjusted by touching and dragging a round black button. For another example, it can also include a sliding bar 12, and the sliding icon on the sliding bar 12 can be touched to adjust the volume, wind speed, etc. For another example, it can also include a round control button 13, which can be called a switch button, which can be used to perform a closing or opening operation, etc.

[0084] like Figure 4As shown, schematic diagrams of several background patterns are shown. In this embodiment, the background pattern can be the background of a writing page, which can be understood as the "paper" used for writing. When providing operational feedback for the background pattern, different background patterns can correspond to "paper" of different textures, which can provide different tactile feedback, auditory feedback and olfactory feedback for "paper" of different textures.

[0085] In this embodiment, when detecting an object displayed on the display interface, the display interface may be captured, and then the object in the captured image may be identified, for example, by performing graphic recognition on the captured image to identify whether the image contains the target object. In other examples, each time the display interface is refreshed, the script file corresponding to the current display interface may be identified to determine whether it contains the target object.

[0086] In practice, when the target object is identified in the display interface, the operation performed on the target object can be detected, and the operation may include a touch operation. In some other scenarios, the operation may also include a voice control operation and a gesture control operation. For example, the voice control operation may include a voice control operation. The gesture control operation refers to operating the target object by making gestures. For example, when the target object is a background pattern, writing can be performed on the page where the background pattern is located by making gestures.

[0087] In this embodiment, the operation on the target object may be performed by a user's finger, a stylus pen, or an operating robot.

[0088] Wherein, when an operation on a target object is detected, the display device can be driven to output a physical signal of at least one mode based on the properties of the target object and the properties of the operation. Wherein, since the target object includes a graphic that supports the interaction between the display device and the touch body, the property of the target object can refer to the category of the graphic, such as the category of the graphic can include a slider category, a button category, a knob category, a background pattern category, etc. Wherein, for the physical signal of the same mode, target objects with different properties can correspond to physical signals with different properties, such as corresponding to a vibration signal, when the target object of the slider category is operated, the vibration signal generated by it can be different from the vibration signal generated when the target object of the button category is operated. For the physical signal of the same mode, the different properties can be reflected in the frequency, phase, waveform and intensity, such as for the vibration signal, the different properties can be reflected in the phase, waveform and intensity. For example, for the sound signal, the different properties can be reflected in the playback rate and audio frequency. For another example, for the odor signal, the different properties can be reflected in the intensity of the odor, and the intensity can be expressed as the concentration of aromatic substances in a unit volume.

[0089] In this embodiment, at least one modal physical signal may include any one of a sound signal, an odor signal, and a vibration signal, or may include at least two of the sound signal, the odor signal, and the vibration signal, such as a sound signal and an odor signal, or a vibration signal and a sound signal, or an odor signal and a vibration signal. In another example, a sound signal, an odor signal, and a vibration signal may be included, so that the operation on the target object may feedback physical signals of three modalities, thereby providing comprehensive feedback in terms of hearing, smell, and touch.

[0090] In combination with the above-mentioned embodiments, the display device may include a multi-modal component, such as an actuating element, an odor generating element, and a sound generating element, wherein, when outputting a physical signal of at least one mode, the display device may generate a first driving electrical signal for a vibration signal, and the driving signal may drive the actuating element to generate vibration. For an odor signal, the display device may generate a second driving electrical signal, thereby driving the valve at the position of the small cavity to open, so that the odor in the small cavity escapes. For a sound signal, the display device may output a call instruction, thereby calling an audio file in a corresponding storage address, and then playing the audio file.

[0091] By adopting the operation feedback method of this embodiment, during the human-computer interaction process, a physical signal of at least one modality can be output for the operation occurring on the target object to provide the user with feedback of at least one of a sound signal, an odor signal, and a vibration signal, thereby providing feedback in terms of smell, touch, and taste.

[0092] In some embodiments, when the display device outputs a physical signal of at least one mode, it can first generate at least one excitation signal based on the properties of the target object and the current operation, and then drive the display device to output physical signals corresponding to the at least one excitation signal. Wherein, each modal physical signal can be driven by an excitation signal. For example, for a vibration signal, it can be driven by a first driving electrical signal. The phase, waveform, frequency, and amplitude of the first driving electrical signal can be determined by the properties and operation of the target object. In practice, it can be an AC signal or a DC signal. Wherein, the first driving electrical signal can be generated by the control circuit board of the display device. After the control circuit board generates the corresponding first driving electrical signal, the first driving electrical signal acts on the actuator element, so that the actuator element generates a vibration corresponding to the first driving electrical signal.

[0093] In this embodiment, different types of excitation signals correspond to physical signals of different modes. For physical signals of the same mode, target objects of different attributes may correspond to excitation signals of different attributes, and operations of different attributes may also correspond to excitation signals of different attributes. The attributes may include the phase, waveform, frequency, and amplitude of the excitation signal. Different attributes may refer to at least one of the phase, waveform, frequency, and amplitude being different. For example, for vibration signals, target objects of different attributes or operations of different attributes may generate first drive electrical signals with different waveforms. For example, for target objects of the button type, the waveform of the first drive electrical signal may be a triangular wave, and for target objects of the slider type, the waveform of the first drive electrical signal may be a sine wave. For another example, for a background pattern of a texture, the frequency of the first drive electrical signal may be a first frequency, and for a background pattern of another texture, the frequency of the first drive electrical signal may be a second frequency. Thus, by setting the corresponding frequency, the generated vibration signal may bring the user the touch of "paper" represented by the background pattern.

[0094] In some embodiments, a target object may be displayed on the display interface, such as only a control graphic, for example, only a slider control, or only a key switch. In this case, a physical signal of a corresponding mode may be outputted only according to the operation received by the displayed target object. In other embodiments, two or more target objects may be displayed on the display interface, such as a control graphic and a background pattern. For example, in a writing page, not only a background pattern for the user to write is displayed, but also controls for the user to operate may be displayed. In this case, a matching physical signal may be provided according to the properties of the target object where the operation occurs.

[0095] In this embodiment, when the target object on the display interface includes two graphics with different attributes, such as a control graphic and a background pattern, the display device can be driven to generate a physical signal matching the touch object according to the attributes of the touch object where the operation occurs and the attributes of the operation. Different touch objects may have different matching physical signals.

[0096] For example, if the touch object is a control graphic, the display device can be driven to generate a physical signal that matches the control graphic, such as a vibration signal; if the touch object is a background pattern, the display device can be driven to generate a physical signal that matches the background pattern, such as an odor signal. For another example, if the touch object is a control graphic, the display device can be driven to generate a vibration signal that matches the control graphic; if the touch object is a background pattern, the display device can be driven to generate a vibration signal and an odor signal that match the background pattern.

[0097] In this embodiment, when multiple target objects are displayed on the display interface, different physical signals can be generated depending on the touch object where the operation occurs. As a result, the generated physical signals can change in the same display interface as the operation position changes.

[0098] In a specific example of this embodiment, when the target object includes a control graphic and a background pattern, when the display device is driven to output a physical signal of at least one modality based on the attributes of the target object and the operation, the display device can be driven to generate a vibration signal based on the attributes of the touch object and the attributes of the operation; and, based on the attributes of the background pattern, the display device can be driven to generate an odor signal and / or a sound signal.

[0099] In this embodiment, no matter the touch object is a control graphic or a background pattern, it can drive the display device to generate a vibration signal according to the properties of the touch object and the properties of the operation, and drive the display device to generate an odor signal or a sound signal, or generate an odor signal and a sound signal according to the properties of the background pattern. In this embodiment, when the display interface includes both control graphics and background patterns, the physical signal generated includes at least a vibration signal, and the vibration signal can be generated according to the properties of the touch object being touched and the properties of the operation, while other physical signals, such as sound signals and odor signals, can be generated according to the properties of the background pattern.

[0100] In this embodiment, different background patterns can represent different "papers". In addition to different touches, they can also indicate that the raw materials of the "papers" are different. The differences in raw materials can be reflected in different smells and different sounds produced when touched. Therefore, olfactory and auditory operational feedback can be formed based on the background pattern.

[0101] Among them, when the touch object is a control graphic, its vibration signal can drive the display device to output a vibration signal based on the shape of the control and the attributes of the operation. The attributes of the operation may include the position of the icon touched on the control graphic, the touch force, the moving track of the touched icon, etc. When the touch object is a background pattern, its vibration signal can drive the display device to output a vibration signal based on the texture type of the background pattern and the attributes of the operation. In this case, different texture types can generate different vibration signals, thereby providing users with different "paper" touches. The attributes of the operation may include the changing track of the operation position, the touch force, etc. It should be noted that different touch forces can correspond to vibration signals of different intensities.

[0102] Wherein, no matter the touch object is a control graphic or a background pattern, it can generate a sound signal and / or an odor signal while generating a vibration signal. The signals of the other two modes generated can be determined according to the properties of the background pattern. Specifically, for a physical signal of the same mode, such as an odor signal, background patterns with different textures can generate different odors. For example, Figure 4 As shown, including bark ( Figure 4 As shown in c), sand ( Figure 4 As shown in a), hair ( Figure 4 As shown in (d), grid ( Figure 4 In the case of a background pattern of different materials and textures (as shown in b), if the background pattern is tree bark, its smell can be a plant-like aroma. Furthermore, tree bark patterns with different textures represent different types of trees, and the aroma can be related to the corresponding trees. For example, if it is birch bark, it is a birch smell signal, and if it is cypress bark, it is a cypress smell signal. For another example, if the background pattern is sand, the smell can be the smell of soil.

[0103] For another example, for a sound signal, background patterns with different textures can produce different sounds. Figure 4 As shown, the background patterns include different materials and textures such as bark, sand, hair, and grids. When the background pattern is bark, the sound may be the sound of touching and sliding on the bark. For another example, when the background pattern is sand, the sound may be the corresponding sound of fingers writing or drawing on the sand.

[0104] Of course, in some examples, at least one of a sound signal and an odor signal may be generated according to the object category corresponding to the background pattern. This may be determined based on whether the object category has an odor attribute and a sound attribute. For example, for an object category with an odor attribute and a sound attribute, such as bark, a sound signal and an odor signal may be generated; for another example, for an object category with only a sound attribute, such as a grid, a sound signal may be generated.

[0105] In some embodiments, when the target object includes a background pattern, the physical signal may include at least an odor signal, that is, when the background pattern is touched, at least an odor signal corresponding to the background pattern is generated. In practice, in addition to the odor signal, a vibration signal and / or a sound signal may also be generated. Of course, in combination with the above embodiments, when the target object includes a background pattern, if a control graphic is also displayed on the display interface, the display interface includes two target objects. In practice, a vibration signal and / or a sound signal may be generated according to the touch object where the operation occurs.

[0106] In the case where the target object does not include a background pattern, the physical signal may include a vibration signal and / or a sound signal, but does not include an odor signal. In this case, the physical signal may be a vibration signal, a sound signal, or a vibration signal and a sound signal. In the case where the target object does not include a background pattern, the target object may be a control graphic, and the touch control graphic may generate vibration and / or sound, thereby providing the user with sound feedback and tactile feedback.

[0107] In some embodiments, the strength of the generated physical signal can be determined according to the touch pressure generated by the operation on the display device. For example, the greater the touch pressure, the stronger the strength of the physical signal. In practice, before outputting the physical signal, the touch pressure generated by the operation on the display device can be obtained, and then, based on the touch pressure, the signal strength of each physical signal can be determined, and based on the signal strength, the properties of the target object and the operation, the display device is driven to output the physical signal.

[0108] In this embodiment, the touch pressure can be detected by a pressure sensor configured by the display device, wherein for each physical signal, the signal strength corresponding to the physical signal can be determined according to the touch pressure. It should be noted that for the detected touch pressure, the signal strength of physical signals of different modes may be different, and the representation of the signal strength of different physical signals may be different. For example, for a vibration signal, the signal strength can be represented by the amplitude; for a sound signal, the signal strength can be represented by the volume; for an odor signal, the signal strength can be represented by the intensity of the odor.

[0109] Then, according to the properties and operation of the target object, the display device can be driven to output a physical signal of a corresponding signal strength. Specifically, for a vibration signal, a first driving electrical signal can be generated according to the properties of the target object, the properties of the operation, and the signal strength, wherein the waveform and frequency of the first driving electrical signal can be determined according to the properties and operation of the target object, and the amplitude of the first driving electrical signal can be determined according to the signal strength. For example, for a sound signal, a call instruction can be generated according to the properties of the target object and the properties of the operation, so as to call the corresponding audio file, and then according to the signal strength, the volume of the speaker can be controlled to generate a sound signal of the corresponding volume. For another example, for an odor signal, a second driving electrical signal can be generated according to the properties of the target object and the properties of the operation, and the second driving electrical signal can control the valve at the small cavity to open, and the signal strength can control the opening of the valve, and the larger the opening, the stronger the odor produced.

[0110] In some embodiments, in addition to controlling the strength of the physical signal according to the touch pressure, a preset icon is also pre-set, so that the user can set the signal strength of the physical signal according to their own needs. Specifically, a preset icon can be displayed in the display interface, and the preset icon can be displayed when the display interface is turned on. The preset icon can be displayed at the edge of the display interface so as not to affect the display of other display content. Among them, the preset icon may include multiple flags, and the corresponding signal strength can be selected by selecting a certain flag. Specifically, in response to a target operation performed on at least one flag, an intensity level corresponding to the target operation can be obtained, wherein the target operation includes a drag operation of a moving flag or a selection operation of a touch flag.

[0111] For example, please refer to Figure 5 and Figure 6 As shown, a schematic diagram of two preset icons is shown, such as Figure 5 and Figure 6 As shown, the preset icon includes multiple flags, and different flags correspond to different signal strengths. Figure 5 As shown, the preset icon may also include a touch body. Dragging the touch body to a certain mark indicates that the signal strength corresponding to the mark is selected. For example, dragging Figure 5 If the touch object in the touch screen reaches the flag 503, it means that the selected signal strength is a low level. Figure 6 As shown, the various flags in the preset icon can be button graphics, which can be selected, such as being selected by touching once, and not being selected by touching again, so that the corresponding signal strength can be selected by directly touching the flag. For example, touching the flag 501 once indicates that the signal strength is selected as a high level.

[0112] In practice, the display device can be driven to output a physical signal of a corresponding intensity level based on the properties and operation of the target object. Specifically, the process of generating a physical signal of a corresponding signal level can refer to the process of generating a physical signal based on touch pressure, which will not be described in detail here.

[0113] In some other embodiments, the preset icon may further include a switch button, which can turn off tactile feedback and turn on tactile feedback. For example, when the user turns on tactile feedback through the switch button, a vibration signal may be output in response to an operation on the target object; when the user turns off tactile feedback through the switch button, no vibration signal is output in response to the operation.

[0114] Accordingly, in some embodiments, the switch button in the preset icon can turn off operation feedback and turn on operation feedback. When the operation feedback is turned off, the display device will not output physical signals of any modality. When the operation feedback is turned on, the display device can output physical signals of the corresponding modality.

[0115] In some embodiments, when generating a physical signal, the frequency of the physical signal can also be controlled so that the frequency of the physical signal can match the operation speed of the operation. Figure 7 As shown, a schematic diagram of the specific process of generating physical signals is shown, such as Figure 7 As shown, the following steps may be included:

[0116] Step S61: obtaining the acceleration corresponding to the operation;

[0117] Step S62: generating a first excitation signal based on the properties and operation of the target object, and generating a second excitation signal based on the properties, acceleration and operation of the target object;

[0118] Step S63: dispatching audio data based on the first excitation signal, and playing the audio data based on the acceleration to output a sound signal;

[0119] Step S64: output a vibration signal based on the second excitation signal.

[0120] In this embodiment, the acceleration corresponding to the operation can be obtained, and the acceleration can characterize the moving speed of the operation on the display interface, the leaving speed of the operation from the touch display interface to the leaving display interface, and the touch speed of the operation from outside the display interface to the touch display interface. In the upgrade, the acceleration can characterize the changing speed of the operation. The greater the acceleration, the greater the changing speed of the operation, and the smaller the acceleration, the smaller the changing speed of the operation. In practice, the acceleration of the operation can be detected by a touch driver chip installed in the display device.

[0121] After obtaining the acceleration, a first excitation signal can be generated based on the properties and operations of the target object. The first excitation signal is used to retrieve the audio data. Specifically, the first excitation signal can be an instruction signal. After retrieving the audio data, the playback speed of the audio data can be adjusted based on the acceleration, so that the audio data is played to output a sound signal. In this way, the playback speed of the sound signal is easy to match the acceleration of the operation. For example, the greater the acceleration, the faster the playback speed, and the smaller the acceleration, the slower the playback speed.

[0122] Wherein, a second excitation signal can be generated based on the properties, acceleration and operation of the target object, and the second excitation signal can be a driving electrical signal, such as a first driving electrical signal that drives a vibration signal and a second driving electrical signal that drives an odor signal. Specifically, the acceleration can be used to adjust the frequency of the first driving electrical signal, which can be determined by the acceleration. For example, the greater the acceleration, the higher the frequency of the first driving electrical signal, and the smaller the acceleration, the lower the frequency of the first driving electrical signal. Wherein, the acceleration can be used to adjust the duration of the second driving electrical signal, and the duration can be used to adjust the duration of the odor. Specifically, the duration can adjust the opening duration of the valve at the small cavity. For example, the greater the acceleration, the shorter the opening duration can be, thereby accelerating the disappearance of the odor. The smaller the acceleration, the longer the opening duration can be, so that the odor can last for a longer time.

[0123] Therefore, by adopting the technical solution of this embodiment, since the physical signal can be output according to the acceleration, the feedback of the operation can be matched with the acceleration of the operation, thereby improving the matching degree between the operation feedback and the human behavior, making the feedback more suitable for user use and realizing more refined operation feedback.

[0124] In some embodiments, the vibration signal may generate vibration at the location where the operation occurs, or may generate vibration on the entire display device. Specifically, the following target to be followed by the vibration area may be determined based on the properties of the target object, and the target area may be determined based on the following target. Then, the vibration signal may be output in the target occurrence area based on the properties of the target object and the operation.

[0125] Among them, the following target can be the target followed by the vibration area that generates the vibration signal, and the following target includes the operation or target object; in practice, it can be determined based on the attributes of the target object whether it is necessary to follow the operation position to generate a vibration signal, or to follow the control graphic and generate a vibration signal at the control graphic.

[0126] Among them, target objects with different attributes can correspond to different follow targets. For example, if the target object is a control graphic, the follow target is the control graphic. If the target object is a background pattern, the follow target can be the operation position. In this way, the vibration occurrence areas of target objects with different attributes can be slightly different. For example, when the target object is a control graphic, the occurrence area can be the location of the control; when the target object is a background pattern, the occurrence area can be the operation position, such as the touch position.

[0127] Among them, based on the following target, the position of the following target can be determined as the target area. If the following target is an operation, the position of the operation is determined as the target area. If the following target is a target object, the position of the target object is the target area. Then, a vibration signal can be output in the target area. In this case, the display device can include a plurality of actuators, and the plurality of actuators can be distributed at different positions of the display device. The plurality of actuators can be driven independently, so that according to the target area, the actuators at the target area can be driven to generate a vibration signal, and then the vibration signal can be dynamically changed in position following the human operation.

[0128] In another example, it is also possible to determine whether the vibration is for the entire display area or for a portion of the display area based on the properties of the target object. Specifically, when the target object includes a control graphic, the target area where the vibration occurs can be the entire display area of ​​the display device, thereby feeding back the vibration of the entire display surface to the control graphic; when the target object includes a background pattern, the target area where the vibration occurs can be the interactive area where the background pattern is located, wherein the interactive area can be the entire display area or a portion of the display area. In the case of a portion of the display area, one or more actuators located in the interactive area can be called to generate vibrations in the interactive area of ​​the display device, thereby accurately providing tactile feedback.

[0129] In some embodiments, when outputting physical signals of multiple modes, physical signals of different modes are used to bring different feedback to the user. In order to make the physical signals of multiple modes output harmoniously, the occurrence time, end time and output frequency of each physical signal can be determined. The output frequency can be determined according to the required frequency difference between the physical signals. Then, the multiple physical signals are output according to the occurrence time, end time and output frequency.

[0130] The occurrence times of different types of physical signals may be the same or different, such as different, so that multiple physical signals can be generated in sequence; the end times of different types of physical signals may be the same or different, such as different, so that multiple physical signals can end in sequence. For example, when the occurrence times are different, the end times of different types of physical signals may be different; when the end times are the same, the occurrence times of different types of physical signals may be different.

[0131] Different physical signals may correspond to different frequencies. For a vibration signal, the frequency may be the vibration frequency of the vibration signal; for a sound signal, the frequency may be the playback rate of the sound signal; for an odor signal, the frequency may be the frequency of the valve opening change. In this way, when different physical signals correspond to different frequencies, they can adapt to the physical signal of the current mode, so that the physical signals of various modes can cooperate with each other and provide feedback to the operation together.

[0132] In some embodiments, in addition to providing olfactory, auditory and tactile feedback, visual feedback can also be provided. Specifically, for the background pattern, when the background pattern receives a touch operation, the background pattern at the location where the touch operation is located can be changed. Exemplarily, in response to the touch operation in the area where the background pattern is located, the texture type of the background pattern and the material type corresponding to the background pattern can be obtained; and based on the type of touch operation, the texture type and the material type, the background pattern can be changed in the location range where the touch operation is located.

[0133] Among them, texture may refer to the type of pattern on the background pattern, such as bark texture, sand texture, etc. Material type may refer to the category of the object corresponding to the pattern on the background pattern, such as bark, sand, grid, etc. In practice, the background pattern may be transformed according to the type of touch operation, texture type, and material type within the range of the location where the touch operation is located. Specifically, the transformation method of the background group may be determined according to the type of touch operation, texture type, and material type. The transformation method may include: magnification transformation, reduction transformation, pattern concave transformation, etc. For example, if the texture type is a grid type and the material display is wool, the transformation may be a concave transformation at the touch position, so that the grid at the touch position appears visually collapsed. For another example, if the texture type is a bark type and the material is birch bark, the transformation may be an enlargement transformation at the touch position.

[0134] By adopting this embodiment, the texture change of the background pattern after being touched can be visually fed back to the user, thereby bringing a more realistic writing experience.

[0135] In some embodiments, the display device includes a multimodal component, which may include multiple excitation elements. In practice, in response to operations performed on a target object, the display device may output corresponding physical signals based on the configured excitation elements. For example, the physical signal output by the type of excitation element that is configured drives the type of excitation element.

[0136] Exemplarily, when driving the display device to output a physical signal of at least one modality, the excitation element configured in the display device can be detected, and based on the detection result and the properties of the target object, the target excitation element can be obtained; then, based on the properties and operation of the target object, the target excitation element can be driven to output a corresponding physical signal.

[0137] In this embodiment, the target excitation element includes at least one of an actuating element, an odor generating element, and a sound generating element. The detection of the excitation element can be determined by a controller configured by the display device, such as the controller can detect the element connected to itself, thereby obtaining the configured excitation element. Among the detected excitation elements, some or all of the detected excitation elements can be screened out as target excitation elements according to the properties of the target object. For target objects with different properties, the determined target excitation elements may be slightly different. Exemplarily, when the property of the target object is a control graphic, it can screen out the target excitation element corresponding to the control graphic from the configured excitation elements, such as the actuating element and the sound generating element can be used as the target excitation element. Exemplarily again, when the property of the target object is a background pattern, it can screen out the target excitation element corresponding to the background pattern from the configured excitation elements, such as the actuating element, the sound generating element, and the odor generating element can be used as the target excitation element.

[0138] It should be noted that the target excitation element is the excitation element currently configured to the display device. In practice, the working state of the excitation element can also be detected to determine whether the excitation element is faulty. For excitation elements with faults, they can be removed from the list of target excitation elements, and for excitation elements that work normally, they can be used as elements of target excitation elements. In practice, in response to the operation on the target object, based on the properties and operations of the target object, the target excitation element can be driven to output a corresponding physical signal.

[0139] By adopting the technical solution of this embodiment, when feeding back a physical signal, the physical signal to be fed back can be jointly determined according to the properties of the target object and the configuration of the display device, thereby improving the efficiency of the operation feedback.

[0140] The above operation feedback method is described below with reference to an example.

[0141] Example 1

[0142] Reference Figure 8 As shown, a schematic diagram of the steps of this example is shown. Figure 8 As shown, the following processes may be specifically included:

[0143] First, turn on the display screen and start touch detection to determine whether an interaction in the component area occurs on the display interface; if so, determine which control graphic is the target object of the component interaction;

[0144] Next, the tactile switch state, that is, the flag of the preset icon mentioned above, is obtained, the signal strength is determined, and then an excitation signal (first driving electrical signal) of corresponding strength is output to output a vibration signal. If the tactile switch is in the off state, no vibration signal is output.

[0145] Example 2

[0146] Reference Fig. 9 As shown, the function is different from Example 1 in that it is also necessary to determine whether the sound effect switch is on. If it is on, a sound signal is output in response to the operation. Specifically, the sound effect excitation signal (call instruction) is triggered to call the corresponding audio file to play the audio.

[0147] Example 3

[0148] Reference Fig.10 As shown, different from Example 1, a pressure sensor can be used to detect touch pressure, such as pressure intensity detection, so as to output an excitation signal (first driving electrical signal) of corresponding signal intensity according to the detection result, and then output a vibration signal.

[0149] Based on the same inventive concept, the present disclosure also provides a display device, please continue to refer to Figure 2 and Fig.11 As shown, Fig.11 A schematic diagram of the circuit connection framework of the display device is shown. Figure 2 and Fig.11 As shown, the display device may include a display component, a main controller and a multimodal component, the main controller is connected to the display component and the multimodal component respectively; the multimodal component includes at least one of an actuating element, an odor generating element, and a sound generating element; wherein the display component is used to provide a display interface; the main controller is used to execute the operation feedback method in the above-mentioned first aspect embodiment; wherein the multimodal component is used to output at least one physical signal under the drive of the main controller.

[0150] In this embodiment, the main controller stores a main control program. The main controller mainly includes a CPU, memory, hard disk, graphics card, motherboard, etc., and may also include communication modules such as WiFi and Bluetooth, and may be an operating system platform such as Windows, Linux, Android, etc.

[0151] In this embodiment, the display device is connected to the main controller for communication through the display driver chip, the touch driver chip and the multimodal component. The main controller controls the display image on the screen through the display driver chip; the main controller detects the user's interactive information on the screen through the touch driver chip. The main controller generates tactile feedback signals, olfactory feedback signals and auditory feedback signals through the multimodal component.

[0152] like Fig.11 As shown, the main controller is connected to the display driver chip, the touch driver chip and the drive circuit respectively. The display driver chip is connected to the display component to drive the display of the display screen; the touch driver chip is connected to the touch component to detect the touch occurring on the display screen; the drive circuit is connected to the multimodal component to drive the multimodal component to output physical signals of multiple modes.

[0153] The display device in this embodiment can provide a variety of physical signals perceptible to the user for operations on target objects occurring on the display interface, thereby improving the functional integration of the display device in human-computer interaction scenarios.

[0154] In some examples of this embodiment, the display device may also be configured with a pressure sensor, which may be connected to the main controller. Specifically, the pressure sensor is configured to detect the touch pressure exerted on the display interface; the main controller is specifically configured to determine the signal strength corresponding to the physical signal based on the touch pressure, and generate a physical signal that matches the signal strength.

[0155] In this embodiment, the pressure sensor can be located at the touch screen, such as on the side of the touch screen close to the display screen, to detect the touch pressure on the touch screen. According to the touch pressure, the signal strength can be determined, and then a physical signal corresponding to the signal strength can be generated. The specific process can be described in detail in the embodiment of the above-mentioned operation feedback method.

[0156] In this embodiment, the actuating element may include at least one of a piezoelectric driver, a solenoid valve, a linear motor, and an eccentric rotor.

[0157] In some other examples of this embodiment, the display device may also include an acceleration sensor, and the acceleration sensing module is connected to the main controller; the acceleration sensing module is configured to detect the acceleration of the operation performed on the target object on the display interface; the main controller is specifically configured to determine the frequency of the excitation signal corresponding to the vibration signal, and / or the playback rate corresponding to the sound signal based on the acceleration.

[0158] Among them, the acceleration sensing module can detect the moving speed of the operation on the display interface, the leaving speed of the operation from the touch display interface to the leaving display interface, and the touch speed of the operation from outside the display interface to the touch display interface. Specifically, the acceleration sensing module may include a pressure sensor and a calculation unit connected to the pressure sensor, and the calculation unit is used to obtain a pressure change line according to the touch pressure collected by the pressure sensor, and an acceleration can be determined according to the curvature and slope of the pressure change line. The main controller can control the physical signal according to the acceleration.

[0159] For another example, the acceleration sensing module may be a sensor such as an accelerometer, a strain gauge or a pressure sensor, which is used to detect physical quantities such as vibration acceleration and pressure sensing values.

[0160] If the physical signal is a vibration signal, the vibration frequency of the vibration signal can be controlled according to the acceleration; if the physical signal is a sound signal, the playback rate of the audio file can be controlled according to the acceleration.

[0161] By using the display device and operation feedback method of this embodiment, operation feedback can be provided not only for control graphics, but also for all graphics used to support the interaction between the touch body and the display device, such as control graphics and background patterns, thereby avoiding the problem of a relatively single function of touch feedback. In addition, since the physical signal for operation feedback may include sound signals, odor signals, and vibration signals, touch feedback is not limited to tactile feedback, but can also provide olfactory feedback and auditory feedback, thereby forming multi-modal physical signal feedback, further enhancing the feedback function.

[0162] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0163] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0164] The above is a detailed introduction to an operation feedback method and a display device provided by the present disclosure. Specific examples are used in this article to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method of the present disclosure and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present disclosure, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present disclosure.

[0165] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0166] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0167] References herein to "one embodiment," "embodiment," or "one or more embodiments" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment of the present disclosure. In addition, please note that examples of the term "in one embodiment" do not necessarily all refer to the same embodiment.

[0168] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present disclosure can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0169] In the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present disclosure may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.

[0170] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. An operation feedback method, It is characterized in that Applied to a display device, the method comprises: Detecting objects displayed on the display interface; When detecting that a target object is displayed, in response to an operation on the target object, based on the properties of the target object and the operation, driving the display device to output a physical signal of at least one mode to provide physical feedback for the operation; The target object includes a graphic for supporting interaction between the display device and the touch-sensitive body, and the physical signal includes at least one of a sound signal, an odor signal, and a vibration signal.

2. The operation feedback method according to claim 1, It is characterized in that The step of driving the display device to output at least one modal physical signal in response to the operation performed on the target object based on the attribute of the target object and the operation comprises: generating at least one stimulus signal based on the attribute of the target object and the operation; The display device is driven to output a physical signal corresponding to at least one of the excitation signals.

3. The operation feedback method according to claim 1, It is characterized in that The display interface includes a plurality of target objects, and in response to an operation performed on the target object, based on the attributes of the target object and the operation, driving the display device to output a physical signal of at least one mode includes: In response to an operation performed on the target object, acquiring a touch object at a position where the operation is performed; Based on the properties of the touch object and the properties of the operation, the display device is driven to generate a physical signal corresponding to the touch object; wherein different touch objects correspond to different physical signals.

4. The operation feedback method according to claim 3, It is characterized in that The target object includes a control graphic and a background pattern, and the driving the display device to generate a physical signal corresponding to the touch object based on the attribute of the touch object and the attribute of the operation includes: Based on the properties of the touch object and the properties of the operation, driving the display device to output the vibration signal; and Based on the attributes of the background pattern, the display device is driven to generate the odor signal and / or the sound signal.

5. The operation feedback method according to claim 1, It is characterized in that In the case where the target object includes a background pattern, the physical signal includes at least the odor signal; In a case where the target object does not include the background pattern, the physical signal includes the vibration signal and / or the sound signal.

6. The operation feedback method according to claim 1, It is characterized in that Before driving the display device to output at least one modal physical signal based on the attribute of the target object and the operation, the method further includes: Acquiring the touch pressure generated by the operation on the display device; The step of driving the display device to output at least one modal physical signal based on the attribute of the target object and the operation includes: Determining a signal strength of each of the physical signals based on the touch pressure; Based on the signal strength, the attribute of the target object and the operation, the display device is driven to output the physical signal.

7. The operation feedback method according to claim 1, It is characterized in that Before detecting the object displayed on the display interface, the method further includes: Display a preset icon, wherein the preset icon includes multiple flags; In response to a target operation performed on the plurality of flag positions, acquiring an intensity level corresponding to the target operation, the target operation comprising a drag operation on a touch-sensitive body between the flag positions or a selection operation of touching the flag positions; The step of driving the display device to output at least one modal physical signal based on the attribute of the target object and the operation includes: Based on the attribute of the target object and the operation, the display device is driven to output the physical signal conforming to the intensity level.

8. The operation feedback method according to claim 1, It is characterized in that The physical signal includes the sound signal and the vibration signal; and the physical signal of driving the display device to output at least one modality based on the attribute of the target object and the operation includes: Obtaining the acceleration corresponding to the operation; generating a first excitation signal based on the property of the target object and the operation, and generating a second excitation signal based on the property of the target object, the acceleration, and the operation; Based on the first excitation signal, dispatching audio data, and based on the acceleration, playing the audio data to output the sound signal; And, based on the second excitation signal, output the vibration signal.

9. The operation feedback method according to claim 1, It is characterized in that The physical signal includes the vibration signal, and the physical signal of driving the display device to output at least one mode based on the attribute of the target object and the operation includes: Determine a follow target based on the attribute of the target object, the follow target being a target followed by the vibration region generating the vibration signal, the follow target including the operation or the target object; Based on the following target, determining a target area; Based on the attribute of the target object and the operation, the vibration signal is output in the target generation area.

10. The operation feedback method according to claim 1, It is characterized in that The physical signal includes the sound signal, the smell signal and the vibration signal, and the driving the display device to output at least one modal physical signal based on the attribute of the target object and the operation includes: Based on the properties of the target object and the operation, determining the occurrence time, end time, and output frequency corresponding to each of the plurality of physical signals; According to the occurrence time, the end time and the output frequency, a plurality of physical signals are output.

11. The operation feedback method according to claim 1, It is characterized in that When the target object includes a background pattern, the method further includes: In response to a touch operation on the area where the background pattern is located, acquiring a texture type of the background pattern and a material type corresponding to the background pattern; Based on the type of the touch operation, the texture type, and the material type, the background pattern is transformed within a range of a position where the touch operation is located.

12. The operation feedback method according to claim 1, It is characterized in that The step of driving the display device to output at least one modal physical signal in response to the operation performed on the target object based on the attribute of the target object and the operation comprises: Detecting an excitation element configured for the display device; Determining a target excitation element based on the detection result of the detection and the attribute of the target object, wherein the excitation element includes at least one of an actuating element, an odor generating element, and a sound generating element; Based on the attribute of the target object and the operation, the target excitation element is driven to output a corresponding physical signal.

13. A display device, It is characterized in that include: A display component, a main controller and a multimodal component, wherein the main controller is connected to the display component and the multimodal component respectively; The multimodal assembly includes at least one of an actuating element, a scent generating element, and a sound generating element; Wherein, the display component is used to provide a display interface; The main controller is used to execute the operation feedback method according to any one of claims 1 to 12; The multimodal component is used to output at least one physical signal under the drive of the main controller.

14. The display device according to claim 13, It is characterized in that The display device further comprises a pressure sensor, and the pressure sensor is connected to the main controller; The pressure sensor is configured to detect the touch pressure borne by the display interface; The main controller is specifically configured to determine the signal strength corresponding to the physical signal based on the touch pressure, and generate a physical signal that meets the signal strength.

15. The display device according to claim 12, It is characterized in that The display device further comprises an acceleration sensing module, and the acceleration sensing module is connected to the main controller; The acceleration sensing module is configured to detect the acceleration of the operation performed on the target object on the display interface; The main controller is specifically configured to determine the vibration frequency of the vibration signal and / or the playback rate corresponding to the sound signal based on the acceleration.