Control method for aerial imaging touch feedback, touch feedback subsystem and device

Abstract

The present invention discloses a control method for aerial imaging touch feedback, a touch feedback subsystem and an aerial imaging device, wherein the control method for aerial imaging touch feedback is characterized in that the aerial imaging device includes an imaging subsystem for providing an aerial interactive interface and a tactile feedback subsystem including an air jet unit, and the control method is used for the tactile feedback subsystem, and the control method includes: receiving coordinate information of a touch point in the aerial interactive interface; obtaining relative position information of the air jet unit and the touch point; obtaining an air jet control parameter according to the relative position information and the coordinate information of the touch point; and controlling the air jet unit to spray gas toward the touch point according to the air jet control parameter. Combining aerial imaging and gas jet touch feedback, contactless touch and feedback are realized, which is highly safe, provides users with actual touch support in the air, and improves touch interaction efficiency on the basis of visual feedback.

Description

Technical Field

[0001] The present invention relates to the field of aerial imaging, and in particular to a control method for aerial imaging touch feedback, a touch feedback subsystem and an aerial imaging device. Background Art

[0002] At present, the most common tactile feedback mechanism is contact tactile feedback. Contact tactile feedback technology is assisted by related equipment (such as screens, gloves, etc.) and uses principles and technologies such as vibration and electrostatic force to achieve tactile perception. Although the relevant equipment has been miniaturized and widely used in commercial applications, since users must contact the auxiliary equipment, there are the following problems: on the one hand, the cross-use of equipment will cause public health and safety problems, and users cannot get rid of the constraints of bulky equipment; on the other hand, after using the equipment, users will leave personal information such as fingerprints and palm prints on the equipment, which poses a risk of personal information leakage. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, one object of the present invention is to provide a control method for mid-air imaging touch feedback.

[0004] Another object of the present invention is to provide a tactile feedback subsystem.

[0005] Another object of the present invention is to provide an aerial imaging device.

[0006] The method of aerial imaging touch feedback of the first aspect embodiment of the present invention, wherein the aerial imaging device includes an imaging subsystem for providing an aerial interactive interface and a tactile feedback subsystem including a jet unit, and the control method is used for the tactile feedback subsystem, and the control method includes: receiving coordinate information of a touch point in the aerial interactive interface; obtaining relative position information between the jet unit and the touch point; obtaining a jet control parameter based on the relative position information and the coordinate information of the touch point; and controlling the jet unit to spray gas toward the touch point based on the jet control parameter.

[0007] According to the method of aerial imaging touch feedback of an embodiment of the present invention, based on the coordinate information of the touch point in the aerial interactive interface and the relative position information between the jet unit and the touch point, the jet control parameters are obtained to control the jet unit to spray gas to the touch point, so that when the user performs touch operation in the aerial interactive interface, the user can feel the impact of the gas, realize contactless tactile feedback, realize visual and tactile human-computer interaction, and will not leave behind personal information such as user fingerprints and palm prints, thereby reducing the risk of personal information leakage. In addition, the control method of the present invention sprays gas to the touch point of the aerial interactive interface to realize tactile feedback, realizes contactless tactile feedback, and thus will not cause public health and safety problems caused by cross-use of equipment, which is safer.

[0008] In some embodiments, the jet unit includes a movable jet component, and the jet control parameters are obtained according to the relative position information and the coordinate information of the touch point, including: obtaining the jet angle of the jet component according to the coordinate information of the touch point and the relative position of the jet unit and the touch point; and determining the target jet intensity of the jet component according to the relative distance between the touch point and the jet unit.

[0009] In some embodiments, the jet unit includes multiple jet components, and multiple air beams or air curtains jetted by the multiple jet components can cover the air interaction interface. The jet control parameters are obtained according to the relative position information and the coordinate information of the touch point, including: determining the existence of multiple touch points according to the coordinate information of the touch point; determining the corresponding target jet components according to the coordinate information of the multiple touch points; and determining the target jet intensity of the target jet component according to the relative distance between the touch point and the corresponding target jet component.

[0010] In some embodiments, the jet mode of the jet unit is a real-time opening type, and the control method also includes: receiving a touch feedback control instruction, controlling the jet unit to start and be in a jet state or a jet preparation state, and when receiving the coordinate information of the touch point in the air interaction interface, obtaining the jet control parameter to control the jet device to spray gas toward the touch point according to the jet control parameter.

[0011] In some embodiments, the jetting mode of the jet unit is normally open, and the control method further includes: upon receiving a touch feedback control instruction, controlling the plurality of jet components to be in a continuous jetting state until receiving a touch feedback closing instruction.

[0012] In some embodiments, the control method also includes: obtaining a target tactile perception gas intensity threshold value based on the relative distance between the touch point and the jet component and a human tactile perception model, and obtaining a gas intensity attenuation value based on the relative distance between the touch point and the jet component and a gas transmission attenuation model; obtaining the target jet intensity of the jet component based on the target tactile perception gas intensity threshold value and the gas intensity attenuation value.

[0013] In some embodiments, before controlling the jet unit to jet gas toward the touch point according to the jet control parameter, the control method further includes: acquiring ambient temperature information; and controlling the temperature of the gas jetted by the jet unit according to the ambient temperature information.

[0014] The second aspect of the present invention proposes a tactile feedback subsystem, including: a jet unit for jetting gas; a data processing module, the data processing unit is used to generate a control signal for controlling the jet unit according to the control method of the aerial imaging touch feedback; a driving module, the driving module is connected to the data processing module, and is used to generate a driving signal according to the control signal to control the jet unit to jet gas toward the touch point.

[0015] According to the tactile feedback subsystem of the embodiment of the present invention, the data touch module generates a control signal for controlling the jet unit according to the control method of the aerial imaging touch feedback of the above embodiment to control the jet unit to spray gas to the touch point, so that when the user performs touch operation on the aerial interactive interface, the user can feel the impact of the gas, realize contactless tactile feedback, and will not leave behind personal information such as user fingerprints and palm prints, thereby reducing the risk of personal information leakage. In addition, the jet unit is controlled to spray gas to the touch point of the aerial interactive interface to realize tactile feedback, and contactless tactile feedback is realized, so that public health and safety issues caused by cross-use of equipment will not be caused, which is safer.

[0016] The third aspect of the present invention provides an aerial imaging device, comprising: an imaging subsystem, which is used to image and display human-computer interaction information in an aerial target area to form an aerial interaction interface, and send touch feedback control instructions and coordinate information of touch points when an interaction signal between a user and the aerial interaction interface is detected; and a tactile feedback subsystem as described in the above embodiment, which is connected to the imaging subsystem and is used to spray gas to the touch points according to the touch feedback control instructions and the coordinate information of the touch points.

[0017] In some embodiments, the imaging subsystem includes: a casing, the casing having a display window and a accommodating cavity formed inside; a display, the display being disposed in the accommodating cavity and used to display human-computer interaction information; an optical component, the optical component being disposed in the accommodating cavity and used to converge light carrying the human-computer interaction information into an aerial target area to form an aerial interaction interface; wherein the display is disposed on the light source side of the optical component, and the display window is on the imaging side of the optical component; a detection module, used to detect interaction signals between a user and the aerial interaction interface; a main control module, connected to the imaging component and the detection module, and used to send touch point coordinate information in response to the interaction signal.

[0018] According to the aerial imaging device of the embodiment of the present invention, the imaging subsystem and the tactile feedback subsystem are combined. The imaging subsystem displays the human-computer interaction information in the aerial target area to form an aerial interaction interface, and uses the aerial interaction interface as the reference plane of the user's tactile perception to guide the user's touch control. When the imaging subsystem detects the interaction signal between the user and the aerial interaction interface, the touch feedback control instruction and the touch point coordinate information are sent to the tactile feedback subsystem. The tactile feedback subsystem sprays gas to the touch point according to the received touch feedback control instruction and the touch point coordinate information, and produces a tactile feedback effect between the human body and the touch point, so that the human body can truly feel the existence of the touch point, and the purpose of non-contact tactile feedback of the user is achieved. In the embodiment of the present invention, based on presenting the aerial interaction interface in the aerial target area, the touch feedback operation can be triggered by the user touching the aerial interaction interface, which satisfies the user's interactive experience in both vision and touch, and the operation mode is more natural and comfortable, without setting additional user operation restriction devices, avoiding the risk of the user touching the device during operation. At the same time, the information security problem caused by the leakage of personal information due to the residual user fingerprint information is avoided.

[0019] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0021] Figure 1 FIG. 4 is a schematic diagram of an aerial imaging device according to an embodiment of the present invention.

[0022] Figure 2 The figure is a flow chart of a method for controlling mid-air imaging touch feedback according to an embodiment of the present invention.

[0023] Figure 3 FIG. 4 is a schematic diagram of an injection control method according to an embodiment of the present invention.

[0024] Figure 4 The figure is a flow chart of a method for controlling mid-air imaging touch feedback according to another embodiment of the present invention.

[0025] Figure 5 FIG. 4 is a schematic diagram of an injection control method according to another embodiment of the present invention.

[0026] Figure 6 The figure is a flow chart of a method for controlling mid-air imaging touch feedback according to yet another embodiment of the present invention.

[0027] Figure 7The figure is a flow chart of a method for controlling mid-air imaging touch feedback according to yet another embodiment of the present invention.

[0028] Figure 8 is a functional block diagram of a data processing module according to an embodiment of the present invention.

[0029] Fig. 9 FIG. 4 is a functional block diagram of a driving module according to an embodiment of the present invention.

[0030] Fig.10 FIG. 4 is a schematic diagram of an aerial imaging device according to an embodiment of the present invention.

[0031] Fig.11 FIG. 4 is a functional block diagram of an aerial imaging device according to an embodiment of the present invention.

[0032] Fig.12 Schematic diagram of the working principle of an aerial imaging device according to an embodiment of the present invention.

[0033] Fig.13 Schematic diagram of the structure of the optical components of an aerial imaging device according to an embodiment of the present invention.

[0034] Fig.14 FIG. 4 is a schematic diagram of a first optical waveguide array and a second optical waveguide array of an aerial imaging device according to an embodiment of the present invention.

[0035] Reference numerals:

[0036] Aerial imaging device 100, aerial imaging system 200, tactile feedback system 300,

[0037] Casing 1, optical component 2, display 3, detection module 4, main control module 5, first optical waveguide array 6, second optical waveguide array 7, transparent substrate 8, reflection strip 9, shell 12, jet unit 13, jet component 131, data processing module 14, drive module 15, touch area 16, pan / tilt 19, and three-dimensional touch space D where the touch area is located. DETAILED DESCRIPTION

[0038] The embodiments of the present invention are described in detail below. The embodiments described with reference to the accompanying drawings are exemplary. Figures 1 to 12 The methods, systems and devices of the present invention are described in detail.

[0039] The first embodiment of the present invention provides a method for controlling touch feedback of aerial imaging, which can be used in an aerial imaging device. Figure 1 FIG. 1 is a schematic diagram of an aerial imaging device according to an embodiment of the present invention. Figure 1As shown, the aerial imaging device 100 includes an imaging subsystem 200 for providing an aerial interactive interface and a tactile feedback subsystem 300 including an air jet unit 13. The aerial imaging device may include an ATM, an ATM registration machine, and the like.

[0040] The imaging subsystem 200 is used to image and display the human-computer interaction information in the aerial target area to form an aerial interaction interface. For example, the imaging subsystem 200 can use interactive aerial imaging technology to form a floating real image, i.e., an aerial interaction interface, at a certain position in the air as a reference surface for the user's tactile perception, and the three-dimensional space covering the floating real image is the aerial target area. When the imaging subsystem 200 detects the interaction signal between the user and the aerial interaction interface, it controls the display content of the aerial interaction interface according to the interaction signal to realize contactless aerial human-computer interaction, and sends the touch point coordinate information to the tactile feedback subsystem 300. The tactile feedback subsystem 300 is connected to the imaging subsystem 200 and is used to spray gas to the touch point according to the touch point coordinate information.

[0041] The touch area of ​​the tactile feedback subsystem 300 covers the three-dimensional space where the air interaction interface is located, that is, the air target area. Specifically, based on the relatively fixed size and display position of the air interaction interface, the tactile feedback subsystem 300 generates a touch feedback plane of the same size and position as the known air interaction interface. That is, the imaging subsystem 200 provides the air interaction interface to guide the user to touch, and the tactile feedback subsystem 300 provides the touch feedback plane to feedback the user's perception of the touch object.

[0042] The control method of the mid-air imaging touch feedback according to the embodiment of the present invention is used in the tactile feedback subsystem. The control method of the mid-air imaging tactile feedback according to the embodiment of the present invention is described below.

[0043] Figure 2 is a flow chart of a method for controlling aerial imaging tactile feedback according to an embodiment of the present invention. Figure 2 As shown, the control method of aerial imaging tactile feedback in the embodiment of the present invention at least includes steps S1-S4, which are specifically as follows.

[0044] S1, receiving coordinate information of a touch point in the air interaction interface.

[0045] S2, obtaining the relative position information between the jet unit and the touch point.

[0046] S3, obtaining the jet control parameters according to the relative position information and the coordinate information of the touch point.

[0047] S4, controlling the jet unit to jet gas toward the touch point according to the jet control parameter.

[0048] Specifically, the imaging subsystem images and displays the human-computer interaction information in the air target area to form an air interaction interface. When the user performs a touch operation on the air interaction interface, the imaging subsystem detects the interaction signal between the user and the air interaction interface, determines the position of the touch point in the air interaction interface according to the interaction signal, generates the coordinate information of the touch point, and sends the coordinate information of the touch point to the tactile feedback subsystem. The tactile feedback subsystem receives the coordinate information of the touch point in the air interaction interface, and can obtain the relative position information of the jet unit and the touch point, such as the relative distance and relative offset angle between the two, through coordinate transformation and calculation according to the coordinates of the touch point and the position (spatial coordinates) of the jet unit, and obtains the jet control parameters according to the relative positions of the two and the coordinates of the touch point. The jet control parameters are control parameters that control the jet unit to spray gas to the touch point and can be perceived by the user, such as the jet angle of the jet unit and the intensity of the jet gas. In an embodiment, the jet control parameters can be obtained based on a pre-stored calculation model or by looking up a table. Therefore, when the user operates the air interaction interface, the jet unit is controlled to jet according to the jet control parameters, so that the gas can be sprayed to the touch point and the user can feel the gas impact, thereby achieving tactile feedback.

[0049] According to the control method of aerial imaging tactile feedback of an embodiment of the present invention, based on the coordinate information of the touch point in the aerial interactive interface and the relative position information between the jet unit and the touch point, the jet control parameter is obtained to control the jet unit to spray gas to the touch point, so that when the user performs touch operation in the aerial interactive interface, the user can feel the impact of the gas, realize contactless tactile feedback, realize visual and tactile human-computer interaction, and will not leave behind personal information such as user fingerprints and palm prints, thereby reducing the risk of personal information leakage. In addition, the control method of the present invention sprays gas to the touch point of the aerial interactive interface to realize tactile feedback, realizes contactless tactile feedback, and thus will not cause public health and safety problems caused by cross-use of equipment, which is safer.

[0050] In some embodiments of the present invention, for example Figure 3 As shown, the jet unit 13 includes a movable jet component 131, and the jet component 131 can be set on the pan-tilt platform 19, and the jet angle of the jet component can be adjusted by the pan-tilt platform 19. Alternatively, other driving mechanisms are used to drive the jet component 131 to move, so that the jet direction of the jet component 131 is adjustable. In an embodiment, the jet component 131 can be an air compressor or an air knife.

[0051] like Figure 4 The flowchart of the control method of aerial imaging touch feedback according to an embodiment of the present invention is shown. When performing tactile feedback control, step S3, obtaining the jet control parameters according to the relative position information and the coordinate information of the touch point, may include the following steps S31 and S32.

[0052] S31, obtaining the jet angle of the corresponding jet component according to the coordinate information of the touch point and the relative position between the jet unit and the touch point.

[0053] Specifically, in order to achieve the purpose of touch feedback, it is necessary to control the jet unit to spray air toward the touch point. The relative positions of the jet unit and the touch point are different, so the jet angle of the jet unit is different. In the embodiment, the pitch angle and the yaw angle of the jet unit can adjust the jet angle. For example, the pitch angle and the yaw angle of the jet unit are obtained according to the coordinate information of the touch point and the relative position of the jet unit and the touch point, so that the jet unit can spray air to the touch point.

[0054] S32, determining the jet intensity of the corresponding jet component according to the relative distance between the touch point and the jet unit.

[0055] Then, the pitch angle and yaw angle of the corresponding jet unit are adjusted according to the jet angle, and the jet unit is controlled to spray gas at the target jet intensity, so that the user can feel the gas impact at the touch point and achieve tactile feedback. This method is mainly used in scenes with high-precision touch requirements or in occasions with many and small control units. It can be applied to stereo touch objects such as Figure 3 The three-dimensional touch space D where the middle touch area is located has high real-time performance and fast response speed.

[0056] In other embodiments of the present invention, Figure 5 As shown, the jet unit 13 may include multiple jet components 131. For example, the jet component 131 may be an air compressor or an air knife. The multiple air beams or air curtains sprayed by the multiple jet components 131 may cover the air interaction interface. That is to say, the gas sprayed by a certain jet component 131 may act on a certain touch point or touch area in the air interaction interface, and the jet components 131 in different positions may correspond to different touch points or touch areas in the air interaction interface.

[0057] like Figure 6 As shown, when performing tactile feedback control, step S3, obtaining the jet control parameter according to the relative position information and the coordinate information of the touch point, may include the following steps S33-S35.

[0058] S33: Determine, according to the received coordinate information of the touch points, whether there are multiple touch points.

[0059] S34, determining the corresponding target jetting component according to the coordinate information of the plurality of touch points, that is, determining the jetting component that can jet gas to the corresponding coordinate point.

[0060] Specifically, when multi-touch is performed on the air interactive interface, the detection module of the imaging subsystem detects the coordinate information of multiple touch points and sends it to the tactile feedback subsystem. The gas ejected by the multiple jets of the tactile feedback subsystem can cover different touch areas. In order to make each touch point feel the impact of the airflow, the jet that the ejected gas or air curtain can cover, i.e., the target jet, is determined according to the coordinate information of each touch point. In the embodiment, the positions of the multiple jets are fixed, and the touch area that can be covered by the corresponding ejected gas or air curtain is determined, so that the corresponding target jet can be determined according to the position of the touch point.

[0061] S35, determining a target jet intensity of the target jet component according to a relative distance between the touch point and the corresponding target jet component.

[0062] Specifically, the jet intensity is related to the relative distance from the touch point to the jet component. A relationship table or a fitted mathematical relationship model between the two can be pre-stored. The target jet intensity of the target jet component can be determined based on the relative distance between the two by querying the relationship table or by satisfying the mathematical relationship model. This ensures that the gas sprayed by the jet component can accurately reach the corresponding touch point and can be felt by the human body, thereby achieving contactless tactile feedback.

[0063] Furthermore, in order to ensure that the gas sprayed from the jet unit to the touch point can be better perceived by the user, so that the gas impact force is not too weak to be perceived or too strong to make the user feel uncomfortable, in an embodiment of the present invention, the tactile feedback subsystem is fitted according to the relative distance between the touch point and the jet component and the threshold of the gas intensity perceived by the human body to establish a human tactile perception model, and the human tactile perception model is a corresponding relationship model between the relative distance between the touch point and the jet component and the threshold of the gas intensity perceived by the human body, wherein the threshold of the gas intensity perceived by the human body is the gas intensity threshold at which the gas can be comfortably perceived by the human body part such as the hand when the gas is sprayed to the touch point at a certain test distance, and the model can be pre-stored in the data processing module of the tactile feedback subsystem. Furthermore, the tactile feedback subsystem corrects the target jet intensity of the jet component according to the human tactile perception model combined with the gas transmission attenuation model, wherein the gas transmission attenuation model is a relationship curve model of the gas intensity attenuation degree with the increase of the transmission distance during the gas transmission process of the jet component, and the model can also be pre-stored in the data processing module of the tactile feedback subsystem.

[0064] Specifically, the tactile feedback subsystem obtains a target tactile perception gas intensity threshold value according to the relative distance between the touch point and the jet component and a human tactile perception model, and obtains a gas intensity attenuation value according to the relative distance between the touch point and the jet component and a gas transmission attenuation model, and obtains a target jet intensity of the jet component according to the target tactile perception gas intensity threshold value and the gas intensity attenuation value. For example, the difference between the target tactile perception gas intensity threshold value and the gas intensity attenuation value is used as the target jet intensity, or the difference between the target tactile perception gas intensity threshold value and the gas intensity attenuation value is obtained, and a smaller gas intensity correction value is added to the difference as the target jet intensity. The jet component is controlled to spray gas according to the target jet intensity, thereby ensuring that the jet gas can reach the corresponding touch point and making the user's tactile perception more comfortable.

[0065] In an embodiment of the present invention, the jetting mode of the jetting unit may include a real-time opening type and a normally open type. Among them, the real-time opening type jetting unit is used to jet when the user performs a touch operation on the air interactive interface or when the user clicks a touch point. As in the control method proposed above, the normally open type jetting unit is in a normally open state, that is, in a real-time jetting state.

[0066] Specifically, when the jetting mode of the jet unit is the real-time on type, the user inputs a tactile feedback control instruction through the air interactive interface or sets a special tactile feedback start-stop trigger unit on the imaging device. The tactile feedback subsystem receives the tactile feedback control instruction, controls the jet unit to start and be in the jetting state or jet preparation state, and when the coordinate information of the touch point in the air interactive interface is received, the jet control parameters are obtained as described above, and then the jet device is controlled to spray gas to the touch point according to the jet control parameters. This method can solve a large amount of compressed gas. For the jetting mode of the jet unit normally open, a tactile feedback control instruction is received, and multiple jet parts are controlled to be in a continuous jetting state, so that the user can feel the gas impact force when operating in the touch area, and tactile feedback is realized until the tactile feedback off instruction is received, and the jet parts are controlled to stop jetting.

[0067] For example, for the normally open air curtain / air beam spraying method, when the finger enters the touch area, the spray unit is triggered to be in the normally open state until the preset time period is exceeded without receiving any interactive instructions, that is, the tactile feedback closing instruction is received, and then the spraying stops. For the real-time open air curtain / air beam spraying method, the spray unit can be triggered to start up when the imaging subsystem detects that the finger enters the preset position range. At this time, the gas can be stopped from being sprayed, but the compressor is started and can spray gas at any time. When it is detected that the finger triggers the corresponding touch point, the spray unit is controlled to immediately spray gas towards the current touch point. In this way, there will be no delay in the spraying reaction, and the gas can be sprayed to the touch point that the finger wants to touch in time to achieve tactile feedback.

[0068] In some embodiments, when a finger enters the touch area, the jet unit is awakened. When the jet unit sprays an air curtain, the jet unit can be controlled to spray. When the jet device sprays an air beam, the jet unit is in a jet preparation state. When it is further detected that the finger performs a touch operation on the air interaction interface, the jet unit starts to spray, thereby achieving a rapid response of the jet.

[0069] Furthermore, in some embodiments of the present invention, the influence of the temperature of the gas sprayed by the spray unit on human perception is taken into consideration, such as Figure 7 As shown, before controlling the jet unit to jet gas toward the touch point according to the jet control parameter, the control method of the present invention further includes:

[0070] Step S5, obtaining ambient temperature information, and controlling the temperature of the gas sprayed by the spray unit according to the ambient temperature information.

[0071] Among them, adjusting the temperature of the jet gas of the jet unit and obtaining the jet control parameters can be performed simultaneously, or the jet gas temperature can be adjusted first and then the jet control parameters can be obtained, or the jet control parameters can be obtained first and then the jet gas temperature is adjusted. There is no specific restriction on the order of the two, and it only needs to be performed before controlling the jet unit to spray gas to the touch point to improve the comfort perceived by the user during tactile feedback.

[0072] Specifically, in a low temperature environment, the gas ejected by the jet unit is heated to provide warm airflow; in a high temperature environment, the gas ejected by the jet unit is cooled to provide cold airflow. The temperatures of the cold airflow and the warm airflow are both within the human body sensing comfort zone, thereby ensuring the comfort of the user when feeling the airflow.

[0073] When the jet unit is working normally, the air source provides air input, and then the jet unit sprays an air beam or air curtain to form an air flow impact at the touch area or touch point to achieve tactile feedback. By adjusting the temperature of the air beam or air curtain according to the ambient temperature, the user will not feel temperature discomfort when contacting the air beam or air curtain, thereby improving the user experience.

[0074] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the control method of the mid-air imaging touch feedback of the above embodiment is implemented.

[0075] Based on the control method of the aerial imaging touch feedback in the above embodiment, a tactile feedback subsystem according to an embodiment of the second aspect of the present invention is described below with reference to the accompanying drawings.

[0076] like Figure 1As shown, the tactile feedback subsystem 300 includes an air jet unit 13, a data processing module 14 and a driving module 15. The air jet unit 13 is used to spray gas. The data processing module 14 is used to generate a control signal for controlling the air jet unit according to the control method of the air imaging touch feedback in the above embodiment, wherein the control method of the air imaging touch feedback can refer to the description of the above embodiment and will not be repeated here. In some embodiments, the data processing module 14 is small in size and can be fixed on the driving board through a high-speed connector and connected to the driving module 15 on the driving board. The data processing module 14 can also be connected to the air jet unit 13 through a cable. The data processing module 14 calculates the jet angle and jet intensity of the corresponding touch point of the air jet unit 13 according to the coordinate information of the touch point and the position of the air jet unit 13. The driving module 15 is electrically connected to the data processing module 14 to receive the control signal of the data processing module 14, and generates a driving signal according to the control signal. The driving module 15 is electrically connected to the air jet unit 13 to control the air jet unit 13 to spray gas toward the touch area 16 or the current touch point.

[0077] The jet unit 13 is the core unit of the tactile feedback subsystem 300, which is used to generate the air beam or air curtain required for interaction. When the human body contacts the air beam or air curtain, it will sense the impact of the airflow on the skin, thereby achieving tactile feedback. In this system, the data processing module 14 completes the processing and calculation of the focus point coordinate data, and sends the formed control signal to the driving module 15. The driving module 15 amplifies the control signal and generates a driving signal to control the jet unit 13, so that the jet unit 13 sprays airflow at an appropriate time and acts on the coordinate position of the user operation of the interactive interface, so that the user can feel the airflow and achieve tactile feedback.

[0078] In the embodiment, the data processing module 14 is used for receiving coordinate data, calculating phase data and transmitting data in parallel. Its functional block diagram is as follows: Figure 8 As shown. Figure 8As shown, the data processing module 14 mainly includes a data processing unit, a first data storage unit, a first interface unit and a first power supply unit. The data processing unit mainly completes the reception of touch coordinate data and the phase calculation of each channel. In some embodiments, a floating point DSP (Digital Signal Process) chip can be used as the data processing unit. Preferably, the main frequency of the DSP chip is above 200MHz, such as 250MHz or 300MHz or 400MHz, to ensure the real-time requirements of the system. The first data storage unit is divided into a data storage subunit and a program storage subunit, wherein the data storage subunit can be DDR2 (Double Data Rate 2) or SDRAM (synchronous dynamic random-access memory), which is used for data storage during data processing; the program storage subunit uses NAND FLASH (NAND flash memory) memory, which is used for the storage of system boot programs and loader programs. The first interface unit is used to complete the data communication between the data processing unit and the external device, and is connected to the control module 5 through the serial port / USB port to realize the transmission of touch coordinate data and system control instructions; through the Upp parallel port integrated in the DSP, the phase data and feedback data are transmitted. The first power supply unit mainly completes power conversion, converting the input power into various stable and reliable power supplies required by each unit.

[0079] like Fig. 9 As shown, the drive module 15 includes a control unit, a second data storage unit, a second interface unit, a drive unit and a second power supply unit. Among them, the control unit completes the control parameter reception, processing and control signal output, and can select ARM (Advanced RISC Machines) with rich pin resources as the main control chip, which has rich interface resources and meets the system design requirements. The second data storage unit is expanded with an SPI FLASH as data storage, and can also be used for power-off storage of initialization parameters. The second interface unit communicates data with the data processing unit through the Upp parallel port. The drive unit is used to amplify the control signal output by ARM and output a drive signal that meets the requirements. The second power supply unit is used to provide a stable and reliable power supply for the entire system. The external power supply is input to the drive board, and the required power is provided to each power unit through DC-DC conversion and linear conversion. Due to the system design requirements, the distance between the drive unit and the jet unit 13 may be far, and the drive signal can be transmitted in the form of differential signals or 485 bus or industrial field bus, and the signal transmission cable uses multiple strands of single-core twisted pair to eliminate signal interference as much as possible.

[0080] According to the tactile feedback subsystem 300 of the embodiment of the present invention, the data touch module 14 generates a control signal for controlling the jet unit according to the control method of the aerial imaging touch feedback of the above embodiment to control the jet unit to spray gas to the touch point, so that when the user performs touch operation on the aerial interactive interface, the user can feel the impact of the gas, realize contactless tactile feedback, and will not leave behind personal information such as user fingerprints and palm prints, thereby reducing the risk of personal information leakage. In addition, the jet unit is controlled to spray gas to the touch point of the aerial interactive interface to realize tactile feedback, and contactless tactile feedback is realized, so that public health and safety problems caused by cross-use of equipment will not be caused, which is safer.

[0081] The aerial imaging device according to the third aspect of the embodiment of the present invention is described below.

[0082] like Figure 1 As shown, the aerial imaging device 100 of the embodiment of the present invention includes an imaging subsystem 200 and a tactile feedback subsystem 300 of the above second aspect embodiment, and the tactile feedback subsystem 300 is connected to the imaging subsystem 200 .

[0083] The imaging subsystem 200 is used to image and display the human-computer interaction information in the aerial target area to form an aerial interaction interface, and send the touch point coordinate information when the interaction signal between the user and the aerial interaction interface is detected. The tactile feedback subsystem 300 sprays gas to the touch point according to the touch feedback control instruction and the coordinate information of the touch point to achieve contactless tactile feedback. Among them, the process of the tactile feedback subsystem 300 controlling the jet unit 13 to spray gas to the touch point can refer to the description of the control method of aerial imaging touch feedback above, and will not be described in detail here.

[0084] According to the aerial imaging device 100 of the embodiment of the present invention, in combination with the imaging subsystem 200 and the tactile feedback subsystem 300, the imaging subsystem 200 displays the human-computer interaction information in the aerial target area to form an aerial interaction interface, and uses the aerial interaction interface as a reference plane for the user's tactile perception to guide the user's touch control, and when the imaging subsystem detects the interaction signal between the user and the aerial interaction interface, the touch feedback control instruction and the touch point coordinate information are sent to the tactile feedback subsystem. The tactile feedback subsystem 300 sprays gas to the touch point according to the received touch feedback control instruction and the touch point coordinate information, and generates a tactile feedback effect between the human body and the touch point, so that the human body can truly feel the existence of the touch point, and the purpose of non-contact tactile feedback of the user is achieved. In the embodiment of the present invention, based on presenting the aerial interaction interface in the aerial target area, the touch feedback operation can be triggered by the user touching the aerial interaction interface, which satisfies the user's interactive experience in both vision and touch, and the operation mode is more natural and comfortable, and there is no need to set an additional device to restrict the user's operation, so as to avoid the risk of the user contacting the device during operation. At the same time, it avoids information security issues caused by leakage of personal information due to residual user fingerprint information.

[0085] In some embodiments, Fig.10 As shown, the imaging subsystem 200 includes: a housing 1, an imaging component, a detection module 4, and a main control module 5, wherein the imaging component includes an optical component 2 and a display 3. The housing 1 is formed with a display window and a receiving cavity is formed inside; the imaging component is arranged in the receiving cavity, and is used to image and display the human-computer interaction information in the air target area to form an air interaction interface. Specifically, the display 3 and the optical element 2 are both arranged in the receiving cavity, the display 3 is connected to the main control module 5, the display 3 is arranged on the light source side of the optical component 1, and the display window is located on the imaging side of the optical component 2. The display 3 is used to display the human-computer interaction information, and the light carrying the human-computer interaction information emitted by the display 3 is projected to the optical component 2, and the optical component 2 converges the light carrying the human-computer interaction information to form an air interaction interface in the air target area. For example, the display 3 can display a guide screen, and the air interaction interface will present a real image of the guide screen, and the user touches the icon in the real image to send an interaction signal, and the main control module 5 receives the interaction signal and controls the display 3 to display the corresponding interaction information to provide it to the user, thereby realizing contactless human-computer interaction.

[0086] The detection module 4 is used to detect the interaction signal between the user and the air interaction interface. In the embodiment, the sensing form of the detection module 4 includes but is not limited to far-infrared, ultrasonic, laser interference, grating, encoder, fiber or CCD (Charge-coupled Device). The sensing area of ​​the detection module 4 is located in the same plane as the floating real image and covers the three-dimensional space where the floating real image is located. The best sensing form can be selected according to the installation space, viewing angle and use environment, so that the user can operate the floating real image in the best posture, thereby improving the sensitivity and convenience of the user's operation.

[0087] The main control module 5 is connected to the detection module 4, and the Fig.11 As shown, the main control module 5 and the detection module 4 can be connected by wire or wirelessly to transmit digital or analog signals, so that the volume of the aerial imaging device 100 can be flexibly controlled, and the electrical stability of the aerial imaging device 100 can be enhanced. The main control module 5 is used to send touch feedback control instructions and touch point coordinate information in response to the interactive signal. The tactile feedback subsystem 300 receives the touch feedback control instruction and sprays gas to the touch point coordinate position, so that the user can feel the touch feedback and improve the interactive experience.

[0088] The position of the floating real image generated by the imaging component is relatively fixed in the air, and is an excellent interactive guide. The spray area of ​​the tactile feedback subsystem 300 is set to cover the three-dimensional space where the floating real image is located. For example, the path of the tactile feedback subsystem 300 spraying the airflow can cover the area where the floating real image to be touched is located. When the user touches the floating real image, the tactile feedback subsystem 300 sprays an air curtain or air beam at the right time. On the airflow spray path, the human body will feel the impact of the airflow, and then the human body can actually feel the existence of the touch point, realize the perception of the touch object, and make the user more relaxed, natural, efficient, and flexible when performing non-contact interactive operations, and there is no risk of contacting the device body, truly achieving non-contact tactile feedback.

[0089] In some embodiments, Fig.12 As shown, the optical component 2 is mounted on the housing 1 , and the display 3 is located on one side of the optical component 2 and inside the housing 1 , so as to generate an operation interface showing a floating real image on the other side of the optical component 2 .

[0090] Preferably, the angle between the optical component 2 and the display 3 is set to be in the range of 45°±5°, which can make fuller use of the size of the optical component 2 while obtaining relatively good imaging quality and less residual image effect. However, if there are other requirements for the imaging position, other angles can be selected at the expense of some imaging quality. Also preferably, the size of the optical component 2 is set so that the user can see the entire floating real image presented by the display 3 at a glance, but if only part of the content of the display 3 needs to be seen in actual use, the size and position of the optical component 2 can also be freely adjusted according to the actual display screen.

[0091] The imaging mode of the display 3 may include RGB (red, green, blue) light-emitting diodes (LEDs), LCOS (Liquid Crystal On Silicon) devices, OLED (Organic Light-Emitting Diode) arrays, projections, lasers, laser diodes, or any other suitable displays or stereoscopic displays. The display 3 may provide a clear, bright, and high-contrast dynamic image light source. In some embodiments, the brightness of the display 3 is not less than 500 cd / m2, which can reduce the impact of brightness loss during light path propagation.

[0092] In the embodiment of the present invention, the optical component 2 may be an equivalent negative refractive index optical element, such as Fig.13 As shown, the optical component 2 includes two transparent substrates 8, and a first optical waveguide array 6 and a second optical waveguide array 7 disposed between the two transparent substrates 8. Fig.14 As described, the first optical waveguide array 6 and the second optical waveguide array 7 are composed of a plurality of reflective strips 9 with a rectangular cross-section, and a reflective film 10 is provided on one side or both sides along the arrangement direction of the reflective strips 9, and each reflective strip 9 of the first optical waveguide array 6 and each reflective strip 9 of the second optical waveguide array 7 are orthogonal to each other.

[0093] Based on the description of the aerial imaging device 100 of the present invention in the above embodiment, the aerial imaging device 100 of the embodiment of the present invention is combined with an imaging subsystem 200 and a tactile feedback subsystem 300, wherein the imaging subsystem 200 adopts floating imaging technology to present a real image in the air, realizes contactless touch, can avoid the spread of viruses or bacteria, and improves the safety of use. The tactile feedback subsystem 300 adopts gas injection and utilizes the impact of airflow to realize tactile feedback, so that the user can perform non-contact interactive operations more easily and naturally, and realize human tactile perception without the help of physical equipment.

[0094] Specifically, when the aerial imaging device 100 of the embodiment of the present invention is working, in the initial state, the main control module 5 controls the display 3 to display the guide picture, and the optical component 2 converges the light carrying the guide picture emitted by the display 3 and images it into the air on the other side of the optical component 2 to guide the user to touch in the correct area. At the same time, the main control module 5 transmits the touch point coordinate information in the interactive interface to the data processing module 14, and the data processing module 14 calculates the two-dimensional angle (such as the pitch angle and deflection angle of the jet head), the jet intensity, and the airflow temperature and other parameters corresponding to each touch point according to the coordinates of each touch point and the position of the jet head of the jet unit 13, wherein the jet intensity is related to the distance between the touch point and the jet unit 13, and the airflow temperature is related to the ambient temperature. The data processing module 14 sends the calculated parameters to the driving module 15 according to the control instruction of the main control module 5. The driving module 15 generates a control signal according to the control instruction and related parameters, and drives and amplifies the control signal to obtain the final required driving signal. The driving module 15 transmits the driving signal to the jet unit 13, and the jet unit 13 deflects to a suitable angle under the control of the driving signal and sprays a certain amount of constant temperature air beam. In some embodiments, if there are multiple touch points, touch feedback can be achieved through high-speed refresh of the system, combined with the real image pattern presented by the optical component 2, to guide the user to perform touch operations; in addition, multiple jet units 13 can also be used to work synchronously, and through combination and splicing, human-computer interaction of complex operating interfaces and even contour perception of three-dimensional display objects can be achieved.

[0095] Among them, in the embodiment, the jet unit 13 can adopt a low-power, miniaturized jet device. According to the distance between the touch object and the jet unit 13, the data processing module 14 and the driving module 15 control the intensity and duration of the airflow jet of the jet unit 13, and the temperature of the jet airflow can be adjusted according to the ambient temperature through the temperature adjustment module. Therefore, the tactile feedback subsystem 300 of the embodiment of the present invention will not cause any harm to the human body.

[0096] In addition, the tactile feedback subsystem 300 adopts a miniaturized and modular design, with simple setup, low cost, and ease of commercial use and integration. For application scenarios with larger touch areas or three-dimensional touch objects, the focus point data of the touch feedback is significantly increased, and multiple tactile feedback subsystems 300 can be combined and spliced ​​to suit more application scenarios and a wider range of applications.

[0097] In the description of the present invention, "first feature" and "second feature" may include one or more of the features. In the description of the present invention, "plurality" means two or more. In the description of the present invention, a first feature "above" or "below" a second feature may include the first and second features being in direct contact, or may include the first and second features not being in direct contact but being in contact through another feature between them. In the description of the present invention, a first feature "above", "above" and "above" a second feature may include the first feature being directly above and obliquely above the second feature, or may simply mean that the first feature is higher in level than the second feature.

[0098] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example.

[0099] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A control method for mid-air imaging touch feedback, characterized in that: The aerial imaging device includes an imaging subsystem for providing an aerial interactive interface and a tactile feedback subsystem including an air jet unit. The control method is used for the tactile feedback subsystem, and the control method includes: Receive the coordinate information of the touch point in the air interaction interface; Obtaining relative position information between the jet unit and the touch point; Obtaining jet control parameters according to the relative position information and the coordinate information of the touch point; controlling the jet unit to jet gas toward the touch point according to the jet control parameter; The jet unit includes a plurality of jet components, and a plurality of air beams or air curtains jetted by the plurality of jet components can cover the air interaction interface. The jet control parameters are obtained according to the relative position information and the coordinate information of the touch point, including: Determining the existence of multiple touch points according to the coordinate information of the touch points; Determine the corresponding target jetting part according to the coordinate information of the multiple touch points; and The target jet intensity of the target jet component is determined according to the relative distance between the touch point and the corresponding target jet component.

2. The control method of mid-air imaging touch feedback according to claim 1, characterized in that: The jet unit includes a movable jet component, and the jet control parameter is obtained according to the relative position information and the coordinate information of the touch point, including: Obtaining the jet angle of the jet member according to the coordinate information of the touch point and the relative position between the jet unit and the touch point; and The target jet intensity of the jet member is determined according to the relative distance between the touch point and the jet unit.

3. The control method of mid-air imaging touch feedback according to claim 1, characterized in that: The jetting mode of the jetting unit is a real-time opening type, and the control method further includes: When a touch feedback control instruction is received, the jet unit is controlled to start and be in a jet state or a jet preparation state. When the coordinate information of the touch point in the air interaction interface is received, the jet control parameter is obtained to control the jet unit to spray gas toward the touch point according to the jet control parameter.

4. The control method of mid-air imaging touch feedback according to claim 1, characterized in that: The injection mode of the injection unit is a normally open type, and the control method further includes: When a touch feedback control instruction is received, the plurality of jet components are controlled to be in a continuous jetting state until a touch feedback closing instruction is received.

5. The control method of mid-air imaging touch feedback according to claim 1 or 2, characterized in that: The control method further comprises: Obtaining a target tactile perception gas intensity threshold value according to the relative distance between the touch point and the jet component and a human tactile perception model, and obtaining a gas intensity attenuation value according to the relative distance between the touch point and the jet component and a gas transmission attenuation model; The target jet intensity of the jet component is obtained according to the target tactilely perceived gas intensity threshold and the gas intensity attenuation value.

6. The control method for mid-air imaging touch feedback according to any one of claims 1 to 4, characterized in that: Before controlling the jet unit to jet gas toward the touch point according to the jet control parameter, the control method further includes: Get ambient temperature information; The temperature of the gas sprayed by the spray unit is controlled according to the ambient temperature information.

7. A tactile feedback subsystem, characterized in that: include: An injection unit for injecting gas; A data processing module, the data processing module is used to generate a control signal for controlling the jet unit according to the control method for aerial imaging touch feedback according to any one of claims 1 to 6; A driving module is connected to the data processing module and is used to generate a driving signal according to the control signal to control the jet unit to jet gas toward the touch point.

8. An aerial imaging device, characterized in that: include: An imaging subsystem, the imaging subsystem being used to image and display the human-computer interaction information in an aerial target area to form an aerial interaction interface, and to send a touch feedback control instruction and coordinate information of a touch point when an interaction signal between a user and the aerial interaction interface is detected; as well as The tactile feedback subsystem of claim 7 is connected to the imaging subsystem and is used to spray gas to the touch point according to the touch feedback control instruction and the coordinate information of the touch point.

9. The aerial imaging device according to claim 8, characterized in that: The imaging subsystem comprises: A housing, wherein the housing is formed with a display window and a receiving cavity is formed inside the housing; An imaging component, the imaging component comprising a display, the display being arranged in the accommodating cavity and being used for displaying human-computer interaction information; The imaging component further includes an optical component, which is disposed in the accommodating cavity and is used to converge the light carrying the human-machine interaction information into an image on an aerial target area to form an aerial interaction interface; Wherein, the display is arranged on the light source side of the optical component, and the display window is on the imaging side of the optical component; A detection module, used to detect an interaction signal between a user and the air interaction interface; A main control module is connected to the imaging component and the detection module, and is used to send touch point coordinate information in response to the interactive signal.

Images