A medium-free holographic imaging device and a control method thereof

Abstract

The present disclosure provides a medium-free holographic imaging device and a control method thereof. The medium-free holographic imaging device comprises a medium-free holographic imaging assembly for generating a suspended medium-free holographic image in a preset imaging area, the medium-free holographic image comprising a plurality of user-operable interactive areas; and a sensor assembly comprising a plurality of first sensors for generating first perception data about a performing subject when the user performs a subject operation on the interactive area of the medium-free holographic image, wherein the emission beams of the plurality of first sensors are parallel to the medium-free holographic image and the distance between the emission beams and the medium-free holographic image is less than a threshold value. By configuring the first sensors, the position determination of the performing subject operating the interactive area in the three-dimensional space can be converted into the two-dimensional position determination of the interactive area in the medium-free holographic image, the position of the interactive area can be determined without a complex gesture recognition model, the computational complexity is reduced, and the interaction efficiency is improved.

Description

Technical Field

[0001] This manual relates to the field of human-computer interaction technology, and in particular to a medium-free holographic imaging device and its control method. Background Technology

[0002] Screen-to-screen interactive devices refer to devices that use a display screen or virtual display screen as an interactive interface, combining touch, gestures, buttons, and other methods to achieve two-way information transmission between the user and the device. Screen-to-screen interactive devices based on virtual display screens (e.g., media-free holographic imaging devices) typically require infrared sensors to recognize user gestures. Conventional infrared sensor-based gesture recognition usually requires building complex gesture recognition models, increasing costs and resulting in low gesture recognition efficiency.

[0003] Therefore, there is a need to provide a medium-free holographic imaging device and its control method for interaction, which can reduce the complexity of user operation and recognition, improve recognition efficiency, and reduce costs. Summary of the Invention

[0004] This specification provides one or more embodiments of a medium-free holographic imaging device. The medium-free holographic imaging device includes a medium-free holographic imaging component configured to generate a suspended medium-free holographic image in a preset imaging region, the medium-free holographic image including a plurality of user-operable interactive areas. The medium-free holographic imaging device also includes a sensor component. The sensor component includes a plurality of first sensors configured to generate first perceptual data about an executing subject when a user operates the interactive areas of the medium-free holographic image through an executing subject. The emission beams of the plurality of first sensors are parallel to the medium-free holographic image, and the distance between the emission beams and the medium-free holographic image is less than a threshold.

[0005] In some embodiments, the distance between the emitted beams and the medium-free holographic images is greater than 0 and less than a threshold, and the emitted beams of the multiple first sensors cover multiple interactive areas in the projection area of ​​the medium-free holographic images along a direction perpendicular to the medium-free holographic images.

[0006] In some embodiments, the positions of multiple interactive regions in a medium-free holographic image are represented as a first position in a first direction and a second position in a second direction of the medium-free holographic image, wherein the second direction is parallel to the transmission direction of the emitted light beam, the first direction is perpendicular to the second direction, and multiple first sensors are arranged along a direction parallel to the first direction.

[0007] In some embodiments, the medium-free holographic imaging device further includes a controller configured to acquire first perception data collected by a first sensor of a target, the first perception data being generated by the first sensor of the target when a user's executive body operates on a target interactive area; determine a first position of the target interactive area in a first direction of the medium-free holographic image based on the position of the first sensor of the target; determine a second position of the target interactive area in a second direction of the medium-free holographic image based on the first perception data collected by the first sensor of the target; and control the medium-free holographic imaging device to provide a response based on the first position and the second position of the target interactive area.

[0008] In some embodiments, the sensor assembly further includes a second sensor configured to acquire second sensing data, which includes the appearance of the user executive and / or the user's biometric features.

[0009] In some embodiments, the controller is further configured to acquire second sensing data; determine, based on the second sensing data, whether the user and / or user execution entity conforms to a preset object to obtain a determination result; and determine, based on the determination result, whether to control the medium-free holographic imaging device to provide a response.

[0010] This specification provides one or more embodiments of a control method for a medium-free holographic imaging device. The method includes: acquiring first sensing data collected by a first target sensor, the first sensing data being generated by the first target sensor when a user's execution entity operates on a target interactive area of ​​a medium-free holographic image provided by the medium-free holographic imaging device; determining a first position of the target interactive area in a first direction of the medium-free holographic image based on the first target sensor; determining a second position of the target interactive area in a second direction of the medium-free holographic image based on the first sensing data; and controlling the medium-free holographic imaging device to provide a response based on the first and second positions of the target interactive area.

[0011] In some embodiments, the medium-free holographic imaging device includes a sensor assembly comprising a plurality of first sensors, wherein the emitted beams of the plurality of first sensors are parallel to the medium-free holographic image, and the distance between the emitted beams and the medium-free holographic image is less than a threshold.

[0012] In some embodiments, determining the first position of the target interactive region in a first direction of the medium-free holographic image based on the target first sensor includes: determining the position of the emitted beam of the target first sensor in a first direction based on the position of the target first sensor; and designating the position of the emitted beam of the target first sensor in a first direction as the first position of the target interactive region in the first direction of the medium-free holographic image, wherein the emitted beam of the target first sensor covers the target interactive region.

[0013] In some embodiments, determining the second position of the target interactive region in a second direction of the mediumless holographic image based on the first perception data includes: determining the distance between the user's execution subject and the target first sensor based on the first perception data; and determining the second position of the target interactive region in the second direction based on the distance between the execution subject and the target first sensor and the distance between the target first sensor and the mediumless holographic image in the second direction.

[0014] The beneficial effects that the embodiments of this specification may bring include, but are not limited to: (1) by setting up the sensor components, the position of the target interaction area can be determined without a complex gesture recognition model, and further, the position of the target interaction area can be used to control the mediumless holographic imaging device to provide a response; it can reduce computational complexity, improve interaction efficiency, and reduce costs; (2) by using the second perception data, it can be determined that the user and / or the user's execution subject conforms to the preset object, which can reduce the probability of misoperation. It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects that may be produced can be any one or a combination of the above, or any other possible beneficial effects. Attached Figure Description

[0015] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. The same numbers in the drawings denote the same structures or steps.

[0016] Figure 1 It is a medium-free holographic imaging device according to some embodiments shown in this specification.

[0017] Figure 2 yes Figure 1 The diagram shows the viewing angle of the medium-free holographic imaging device in the x1 axis direction.

[0018] Figure 3 yes Figure 1 A schematic diagram of the viewing angle of the medium-free holographic imaging device in the y1 axis direction is shown.

[0019] Figure 4 This is a schematic flowchart illustrating the control of a medium-free holographic imaging device according to some embodiments of this specification.

[0020] Figure 5 This is a schematic flowchart illustrating the control of a medium-free holographic imaging device according to other embodiments of this specification.

[0021] Figure 6 This is a schematic diagram of a module for controlling a medium-free holographic imaging device, according to some embodiments of this specification.

[0022] Figure 7 This is a schematic diagram of another module of a control device for a medium-free holographic imaging apparatus, as shown in some embodiments of this specification. Detailed Implementation

[0023] To more clearly illustrate the technical solutions of the embodiments in this specification, the embodiments will be described in detail below with reference to the accompanying drawings. Obviously, the content described below are some examples or embodiments of this specification. For those skilled in the art, without creative effort, the technical solutions or means disclosed in this specification can be applied to other scenarios based on this technical content.

[0024] It should be understood that the terms "system," "device," "unit," and / or "module" used in this specification are a method of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0025] Unless otherwise specified, the technical terms used to describe components, elements, etc. in this specification are not singular but may include plural. Generally speaking, terms such as "comprising" or "including" only indicate that explicitly identified steps, elements, or components are included, and these steps, elements, and components do not constitute an exclusive list, as the described method or apparatus may also include other steps or components.

[0026] This specification uses flowcharts to illustrate the operational steps performed by the apparatus or system of related embodiments. However, unless otherwise specified, the order in which these steps are described should not be construed as a limitation on the order of execution. Those skilled in the art can adjust the order of these steps based on the knowledge and information conveyed by the embodiments in this specification. Such adjustments include, but are not limited to, reversing the order of steps, merging multiple steps, and splitting a step.

[0027] Interactive display devices refer to devices that use a display screen or virtual display screen as an interactive interface, combining touch, gestures, buttons, and other methods to achieve two-way information transmission between users and the device. Interactive display devices can be applied to consumer electronics (e.g., smartphones / tablets), tactile points for smart wearable devices (e.g., quick control), smart homes and appliances (e.g., smart central control screens, appliance interactive screens), self-service terminals (e.g., self-service cash registers, digital signage), industrial control (e.g., industrial control panels, elevator control panels), medical devices (e.g., medical equipment interactive screens, self-service health terminals), transportation and travel (e.g., in-vehicle infotainment systems, public transportation terminals (e.g., subway ticketing screens)), etc.

[0028] In some embodiments, the screen display interaction device can be implemented using medium-free holographic imaging technology (also referred to herein as a medium-free holographic imaging device or a medium-free holographic imaging interaction device). Medium-free holographic imaging technology generates suspended 3D images (also called medium-free holographic images) in the air using optical techniques, without relying on a physical screen or medium for image display. Combined with interactive technologies (such as gesture recognition and haptic feedback), medium-free holographic imaging technology enables direct interaction between the user and the virtual image.

[0029] In some embodiments, the medium-free holographic imaging interactive device can use sensing technology to identify user operations on the medium-free holographic image (e.g., clicking, swiping, grabbing, etc.), map the user's operations on the medium-free holographic image to specific interactive areas of the medium-free holographic image, and trigger feedback based on the position of the interactive area.

[0030] In some embodiments, sensing technologies may include optical sensor technology, ultrasonic sensor technology, millimeter-wave radar technology, etc. Optical sensor technology can use an infrared sensor to emit infrared light and calculate the reflection time to generate a depth map to determine the position of the user's finger, or use multiple cameras to locate and reconstruct the 3D position of the hand. For example, taking the finger clicking a floating button as an example, the infrared camera calculates the distance between the finger and the infrared camera by the time difference between the infrared pulse emission and reflection, and then constructs a depth map; key point detection is performed on the depth map to build a 3D hand model to obtain the position of each point of the hand in the 3D coordinate system; the position of each point of the hand in the 3D coordinate system is mapped to the virtual coordinate system of the mediumless holographic image by an algorithm; it is determined whether the fingertip is close to the interactive area of ​​the "floating button"; if it is determined that the fingertip is close to or has reached the interactive area of ​​the "floating button", feedback is triggered (e.g., the button is highlighted or deformed and the system responds to a click event, such as opening a menu). Millimeter-wave radar technology can use the emitted millimeter waves with varying frequencies to calculate the frequency difference between the echo and the emitted wave to obtain the distance; it can calculate the angle information by receiving the phase difference of the signals by multiple antennas; and it can determine the coordinate information of the user's finger in three-dimensional space based on the distance and angle information. Ultrasonic sensing technology can collect information synchronously through multiple receivers, calculate distance by calculating the time difference or phase difference between the ultrasonic wave emitted and received, and determine the three-dimensional coordinate information of the user's finger based on the distances corresponding to multiple receivers.

[0031] However, traditional hand localization methods using infrared sensors or multiple cameras require building complex hand recognition models (e.g., skeletal keypoint detection models), which are computationally intensive and inefficient. Ultrasonic sensing or millimeter-wave radar technologies, on the other hand, suffer from weak and noisy finger-reflected signals, requiring specific algorithms to extract effective echoes, which are also computationally intensive and inefficient.

[0032] Therefore, some embodiments of this specification propose a medium-free holographic imaging device and its control method. The medium-free holographic imaging device may include a medium-free holographic imaging component configured to generate a suspended medium-free holographic image in a preset imaging area. The medium-free holographic image includes multiple user-operable interactive areas. The medium-free holographic imaging device may also include a sensor component comprising multiple first sensors configured to generate first perceptual data about the executing subject when the user operates the interactive areas of the medium-free holographic image through an executing subject. The emission beams of the multiple first sensors are parallel to the medium-free holographic image, and the distance between the emission beams and the medium-free holographic image is less than a threshold value.

[0033] Through the above-described configuration of the first sensor, the determination of the three-dimensional coordinates of the user's interactive area in three-dimensional space can be transformed into the determination of the two-dimensional coordinates of the corresponding pixel in the medium-free holographic image interactive area. For example, the two-dimensional coordinates of the pixel corresponding to the interactive area in the medium-free holographic image can be represented by a first direction and a second direction that are perpendicular to each other in the medium-free holographic image where the pixel is located. The second direction is defined as an emitted beam parallel to the first sensor. When the user's interactive area is operated, the emitted beam of the target first sensor is reflected by the user's interactive area and received by the target first sensor, and its emitted beam is parallel to the second direction. Since the position of the emitted beam of each first sensor in the first direction of the medium-free holographic image is known, determining the target first sensor determines the position of the emitted beam of the target first sensor in the first direction of the medium-free holographic image, that is, determining the position of the target interactive area in the first direction. The position of the target interactive area in the second direction is related to the distance between the user's interactive area and the target first sensor, and determining the distance between the user's interactive area and the target first sensor determines the position of the target interactive area in the second direction. According to the embodiments of this application, the position of the target interactive area can be determined without a complex gesture recognition model, and furthermore, the position of the target interactive area can be used to control the medium-free holographic imaging device to provide a response. It can reduce computational complexity, improve interaction efficiency, and reduce costs.

[0034] Figure 1 This is a schematic diagram of a medium-free holographic imaging device according to some embodiments of this description. Figure 2 yes Figure 1 The diagram shows the viewing angle of the medium-free holographic imaging device in the x1 axis direction; Figure 3 yes Figure 1 A schematic diagram showing the viewing angle of a medium-free holographic imaging device along the y1 axis. (See diagram.) Figure 1 As shown, the medium-free holographic imaging device 100 includes a medium-free holographic imaging component 110, a sensor component 120, and a controller 130.

[0035] The medium-free holographic imaging component 110 is used to generate suspended images (also known as medium-free holographic images) in the air using optical technology, without relying on a physical screen or medium for image display. The medium-free holographic imaging component 110 may include an imaging source 112 and an optical transmission component 114.

[0036] Imaging source 112 is used to generate and / or adjust an interactive light field, which can form a light source image 116. Optical transmission device 114 is used to process and transmit the light source image generated by imaging source 112 to project the light source image onto a preset imaging area to form a medium-free holographic image (i.e., a medium-free holographic image 118). Optical transmission device 114 can adjust the divergence angle, focus position, and energy distribution of the light waves in the light source image to ensure the display effect of the medium-free holographic image. The position of the light source image 116 and the position of the medium-free holographic image 118 are symmetrical about the optical transmission device 114.

[0037] A medium-free holographic image may include one or more interactive areas. The interactive area of ​​a medium-free holographic image refers to the spatial range within which a user can interact with the virtual image suspended in mid-air through clicking, touching, or other operations. Users can perform operations on the interactive area of ​​the medium-free holographic image (e.g., clicking, swiping, grabbing) to achieve bidirectional information exchange with the medium-free holographic imaging device 100. The interactive area can be marked on the medium-free holographic image using virtual buttons, focus cursors, etc. For example, virtual buttons can be displayed with a semi-transparent color or an luminous border to mark the operable area (i.e., the interactive area).

[0038] Two-way information transmission between the user and the mediumless holographic imaging device 100 may include the user transmitting instruction information to the mediumless holographic imaging device 100 by operating the interactive area of ​​the mediumless holographic image, while the mediumless holographic imaging device 100 responds (or provides feedback) based on the user's operation of the interactive area.

[0039] The response can include visual response, tactile response, auditory response, etc.

[0040] The visual response may include dynamic or static adjustments made by the medium-free holographic imaging device 100 (e.g., imaging source 112) to the interactive area of ​​the medium-free holographic image operated by the user, such as highlighting or distorting it. Further, for example, the imaging source 112 may adjust the light field to increase the light intensity or color of the interactive area of ​​the medium-free holographic image operated by the user. Yet another example is that the imaging source 112 may adjust the light field to generate halo or ripple animations to distort the interactive area of ​​the medium-free holographic image operated by the user. Still another example is that the imaging source 112 may adjust the light field to magnify the interactive area of ​​the medium-free holographic image operated by the user.

[0041] Haptic response refers to simulating the realistic tactile sensation of a user interacting with an area by providing pressure. For example, a perceptible tactile point can be created at the finger location by focusing sound pressure in the air using an ultrasonic sensor (e.g., a phased array ultrasonic transducer (typically at a frequency of 40-70 kHz)).

[0042] Auditory responses can include providing voice prompts or effects based on interactive areas operated by the user. For example, each interactive area can correspond to a different voice effect. When a user operates a specific interactive area, the medium-free holographic imaging device can generate a corresponding voice effect (e.g., a "click" sound) through an ultrasonic sensor (e.g., an ultrasonic transducer array).

[0043] Imaging source 112 may include a display for generating and / or adjusting light source image 116. The display generates or adjusts the light source image by modulating the phase / amplitude of the light waves.

[0044] In some embodiments, the display may include a display, such as a liquid crystal display (LCD).

[0045] In some embodiments, the display device may include a display and a light source, the light source being a laser diode or an LED light source, which can provide backlighting, enhance display brightness and color gamut, and ensure display light intensity and color accuracy.

[0046] In some embodiments, the display may include a display, a light source, and a modulator. The modulator is used to adjust the image of the light source by modulating the phase / amplitude of the light waves; for example, the modulator may include a phase-type spatial light modulator.

[0047] In some embodiments, the display device can be based on laser scanning imaging. For example, the display device may include a laser diode, a galvanometer, and a modulator. The galvanometer controls the laser diode to emit a laser beam in either raster or vector mode to rapidly scan a virtual plane to form a medium-free holographic image. For example, the laser emitted by the laser diode scans pixels (e.g., the outline of a floating button) point by point on the virtual plane under the control of the galvanometer to form a medium-free holographic image. It should be noted that the medium-free holographic imaging device based on laser scanning imaging may not include the optical transmission element 114, and may not generate a light source image during the imaging process.

[0048] In some embodiments, the optical transmission element 114 may include optical elements or devices as described in any of the Chinese patent applications Nos. 202210060077.2, 202221492951.1, and 201920104395.8. The optical waveguide plate transmits light waves or light fields (i.e., light source images) in a thin-film medium through the principle of total internal reflection, achieving transparent transmission and expansion of the light source image. The optical waveguide plate may also include other geometric optical waveguide plates, diffractive optical waveguide plates, etc. Geometric optical waveguide plates achieve light wave transmission and processing based on prism or mirror arrays. Diffractive optical waveguide plates utilize surface relief gratings or volume holographic gratings to couple light beams, achieving light wave transmission and processing.

[0049] In some embodiments, the optical transmission element 114 may include free optical elements, such as lens groups (e.g., aspherical lenses, Fresnel lenses), mirrors (e.g., high-speed galvanometers or multifaceted prisms), etc.

[0050] In some embodiments, the optical transmission element 114 may include holographic optical elements that enable complex optical device functions (such as lenses or beam splitters) through a holographic recording medium (such as a photosensitive polymer).

[0051] Sensor component 120 is configured to acquire perceptual data related to the user and / or the user's actuator when the user interacts with an interactive area of ​​the medialess hologram via an actuator. The actuator that the user interacts with the interactive area can be a finger, a stylus, etc. Sensor component 120 can determine which interactive area in the medialess hologram the user's actuator is pointing to or located in.

[0052] In some embodiments, sensor assembly 120 (e.g., a first sensor and / or a second sensor) is arranged on the side of optical transmission member 114 facing the medium-free holographic image.

[0053] The sensor assembly 120 may include a plurality of first sensors. Each first sensor can generate and emit a light beam, which is reflected after being transmitted to the execution subject in the operation interaction area to generate a reflected light beam. The first sensor can receive the reflected light beam to generate first sensing data. The first sensors may include infrared sensors, laser sensors, etc.

[0054] In some embodiments, the first sensor may include a light emitter, a light receiver, an optical structure, etc. The light emitter (e.g., an infrared LED) is used to emit a light beam. The light receiver (e.g., a photodiode or phototransistor) is used to receive the reflected light beam and convert it into an electrical signal (i.e., the first sensing data). The optical structure may be used to adjust the angle of the light beam. In some embodiments, the light emitter and the light receiver may be integrated into the same structure.

[0055] The first sensing data can reflect the distance between the executing entity and the first sensor.

[0056] The ranging algorithm for the first sensor includes time-of-flight (TOF) and phase-shift methods. A TOF-based first sensor may include a laser (for emitting laser light), a receiver (for detecting reflected light), and a time-to-data converter (for measuring the time difference between the emitted laser light and the received reflected light). The distance between the actuator and the first sensor can be further determined based on this time difference. A phase-shift-based first sensor may include a laser (for emitting laser light and detecting reflected light), a drive circuit (for generating a high-frequency sinusoidal signal), and a phase-locked loop (for measuring the phase difference). The distance between the actuator and the first sensor can be further determined based on this phase difference.

[0057] To facilitate the explanation of the positions of the various components in the medium-free holographic imaging device 100, such as Figure 1 As shown, a three-dimensional spatial coordinate system and a two-dimensional coordinate system for the medium-free holographic image can be established. The three-dimensional spatial coordinate system includes mutually perpendicular x1, y1, and z1 axes. The z1 axis can be vertical, i.e., the direction from the light source image to the medium-free holographic image or the direction perpendicular to the optical transmission device 114. The plane defined by the x1 and z1 axes can be the front of the medium-free holographic imaging device 100, where the user can typically interact with the device from the front. The plane defined by the y1 and z1 axes can be the side of the medium-free holographic imaging device 100. The two-dimensional coordinate system for the medium-free holographic image includes x2 and y2 axes. The x2 axis is parallel to the x1 axis in the three-dimensional coordinate system; the y2 axis is perpendicular to the x2 axis and parallel to the plane containing the medium-free holographic image. Figure 2 The image shown is a side projection of the medium-free holographic imaging device 100 and the medium-free holographic image 118. Figure 3 The image shown is a frontal projection of the medium-free holographic imaging device 100 and the medium-free holographic image 118. It should be noted that when determining the coordinates of the interactive area, the three-dimensional spatial coordinate system can also be associated with the two-dimensional coordinate system, that is, the two-dimensional coordinates [x,y] and the three-dimensional coordinates [x,y,z] can be associated and mapped. When the two-dimensional coordinates are obtained, the three-dimensional coordinates can be determined through the association relationship. Therefore, when it is necessary to link or cooperate with other devices and the three-dimensional coordinates are needed, they can be used directly.

[0058] The sensor assembly 120 can be positioned on the side of the optical transmission element 114 facing the medium-free holographic image, that is, above the optical transmission element 114 along the z1 axis.

[0059] The plurality of first sensors in the sensor assembly 120 are configured to emit beams parallel to the medium-free holographic image, and the distance between the emitted beams and the medium-free holographic image is less than a threshold.

[0060] The emission beam of the first sensor is parallel to the medium-free holographic image, meaning that the emission beam of each first sensor is parallel to a second direction of the medium-free holographic image, that is, the second direction is parallel to the transmission direction of the emission beam.

[0061] The second direction can be the row direction, column direction, or any direction of the medium-free hologram. For example... Figure 3 As shown, the column direction of the medium-free holographic image is the vertical direction of the medium-free holographic image, i.e., the y2 axis direction; the row direction of the medium-free holographic image is the horizontal direction of the medium-free holographic image, i.e., the x2 axis direction. Figure 3 The second direction shown is the column direction, i.e., the y2 axis direction. The fact that the emitted beam of each first sensor is parallel to the second direction of the medium-free holographic image allows the projection of the emitted beam of each first sensor onto the plane of the medium-free holographic image to cover one or more rows of pixels arranged along the second direction of the medium-free holographic image. For example, if the second direction is the column direction, and the emitted beam of each first sensor is parallel to the column direction of the medium-free holographic image, then the projection of the emitted beam of each first sensor onto the plane of the medium-free holographic image can cover one or more columns of pixels arranged along the second direction of the medium-free holographic image.

[0062] In some embodiments, the distance between the light beam emitted by the first sensor and the medium-free holographic image is less than a threshold, including a distance of 0 between the light beam emitted by the first sensor and the medium-free holographic image, i.e., the plurality of first sensors in the sensor assembly 120 are configured to emit light beams that overlap with the plane of the medium-free holographic image.

[0063] By setting the distance between the emitted beam of the first sensor and the mediumless holographic image to be equal to 0, when the user operates the interactive area of ​​the mediumless holographic image, the user's execution subject needs to reach the location of the interactive area of ​​the mediumless holographic image before the emitted beam of the first sensor can be reflected by the execution subject. Furthermore, the location of the interactive area operated by the user can be determined based on the reflected beam, which can improve the accuracy of the interactive area determination and reduce erroneous operations.

[0064] In some embodiments, the distance between the emitted beam of the first sensor and the medium-free holographic image being less than a threshold includes a distance greater than 0 and less than a threshold, where the threshold can be 0.5 mm, 1 mm, 2 mm, etc. By setting the distance between the emitted beam of the first sensor and the medium-free holographic image to be greater than 0 and less than a threshold, when a user operates the interactive area of ​​the medium-free holographic image, the user's execution subject can reflect the emitted beam of the first sensor even without reaching the interactive area, i.e., at a certain distance from the medium-free holographic image. This allows interaction with the medium-free holographic imaging device to be achieved even when close to the medium-free holographic image, meaning the user's execution subject can interact with the medium-free holographic imaging device without contacting the medium-free holographic image.

[0065] By configuring the emitted beams of the first sensors to be parallel to the medium-free holographic image, when the distance between the emitted beams of multiple first sensors and the medium-free holographic image is 0, multiple interactive regions of the medium-free holographic image can be located on the transmission paths of the emitted beams of multiple first sensors, that is, the emitted beams overlap with the interactive regions of the medium-free holographic image. When the distance between the emitted beams of multiple first sensors and the medium-free holographic image is not 0, the emitted beams of multiple first sensors overlap with the multiple interactive regions of the medium-free holographic image in the projection area of ​​the medium-free holographic image along a direction perpendicular to the medium-free holographic image (also called the projection direction, or the third direction in this paper). For simplicity, this paper can refer to "the emitted beams of the first sensors in the projection area of ​​the medium-free holographic image along a direction perpendicular to the medium-free holographic image" as "the emitted beams in the projection area of ​​the medium-free holographic image". Both "the emitted beams overlapping with the interactive regions of the medium-free holographic image" and "the emitted beams overlapping with the interactive regions of the medium-free holographic image in the projection area of ​​the medium-free holographic image" can be referred to as the emitted beams covering the interactive regions of the medium-free holographic image.

[0066] It should be noted that the light beam itself cannot be projected onto a plane along a direction perpendicular to its transmission direction. The projection area of ​​the emitted light beam of the first sensor in the medium-free holographic image mentioned in this application can refer to the equivalent projection area. The equivalent projection area refers to the projection area formed by the physical object along a third direction on the imaging plane (e.g., the plane where the medium-free holographic image is located) after simulating the emitted light beam of the first sensor as a physical object with the same shape as the emitted light beam. This projection area can be equivalent to the projection area of ​​the emitted light beam of the first sensor in the medium-free holographic image.

[0067] The emitted beams of all the first sensors in the sensor assembly 120 are configured to cover all interactive areas in the medium-free holographic image so as to identify the interactive areas of user operation based on the reflected beams.

[0068] As mentioned earlier, the emitted beams of multiple first sensors cover the interactive areas of the medium-free holographic image. For any first sensor, its emitted beam can cover one or more interactive areas along its transmission path or transmission direction; that is, the position of the emitted beam of the first sensor in the first direction of the medium-free holographic image (the first direction is perpendicular to the second direction) is the same as the position of the interactive area it covers in the first direction of the medium-free holographic image. Figure 3 As shown, the positions of the emitted beams of interactive region A and the first sensor a121 on the x2 axis are the same; the positions of the emitted beams of interactive regions B and C and the first sensor b123 on the x2 axis are the same.

[0069] For two different first sensors, their emitted beams can cover interactive regions located at different positions in a first direction within a medium-free holographic image. For example... Figure 3 As shown, the first sensor includes first sensor a121 and first sensor b123. Interaction region A is located on the transmission path of the emitted beam of first sensor a121; interaction region B is located on the transmission path of the emitted beam of first sensor b123; the emitted beam of first sensor a121 can cover interaction region A, and the emitted beam of first sensor b123 can cover interaction region B. The positions (or coordinates) of interaction region A and interaction region B in the first direction (i.e., the x2 axis) are different.

[0070] The position of each interactive region in the medium-free holographic image can be represented as a first position in a first direction and a second position in a second direction, where the second direction is parallel to the transmission direction of the emitted beam and the first direction is perpendicular to the second direction. The position of the interactive region covered by the emitted beam in the first direction can be determined based on the position of the emitted beam of the first sensor in the first direction. When the distance between the emitted beam of the first sensor and the medium-free holographic image is not zero, the position of the emitted beam of the first sensor in the first direction can refer to the position of the projection area of ​​the emitted beam of the first sensor on the medium-free holographic image in the first direction; when the distance between the emitted beam of the first sensor and the medium-free holographic image is zero, the position of the emitted beam of the first sensor in the first direction can refer to the position of the overlapping area of ​​the emitted beam of the first sensor and the medium-free holographic image in the first direction.

[0071] Furthermore, the positions of the emitted beams from different first sensors in the first direction of the medium-free holographic image are different and known. Therefore, when the user's execution subject operates the target interaction area, the emitted beam of one of the multiple first sensors (i.e., the target first sensor) will be reflected by the user's execution subject to generate a reflected beam. The reflected beam will be received by the target first sensor to generate first perception data. The controller can receive the first perception data and determine the target first sensor. After determining the target first sensor, the position of the emitted beam of the target first sensor in the first direction of the medium-free holographic image can be determined, which is the position of the target interaction area in the first direction.

[0072] When an interactive region exists at each position along a first direction in a medium-free holographic image, the first position of the target interactive region along that first direction is determined, thus identifying which interactive region in the medium-free holographic image the target interactive region belongs to. For example, Figure 3 There is only one interactive region in the column direction where interactive region A is located. If the position of the target interactive region in the first direction is the same as the position of interactive region A in the first direction, then the target interactive region is interactive region A.

[0073] When multiple interactive regions exist at each position in the first direction of a mediumless holographic image, even after determining the first position of the target interactive region in the first direction, it is still impossible to determine which interactive region in the mediumless holographic image the target interactive region belongs to. It is necessary to further determine which interactive region in the mediumless holographic image the target interactive region belongs to based on the second position of the target interactive region in the second direction.

[0074] For example, Figure 3 There are two interactive regions (i.e., interactive regions B and C) in the column direction where interactive regions B and C are located. After determining that the position of the target interactive region in the first direction is the same as the position of interactive regions B and C in the first direction, it can be determined that the target interactive region belongs to one of interactive regions B and C. It is also necessary to determine whether it belongs to interactive region B or interactive region C based on the second position of the target interactive region on the y2 axis. For a detailed description of the determination of the second position, please refer to step 406.

[0075] In some embodiments, in order for the emitted beams of all the first sensors in the sensor assembly 120 to cover all interactive areas in the medium-free holographic image, the emitted beams of the first sensors are positioned differently in the first direction. The first sensors can be arranged in a direction parallel to the first direction of the medium-free holographic image, meaning that the projections of the first sensors onto the plane of the medium-free holographic image along the projection direction are located at different positions in the first direction, thereby making the emitted beams of the first sensors positioned differently in the first direction.

[0076] For example, such as Figure 3 As shown, the second direction is the column direction (i.e., the y2 axis direction), and the first direction is the row direction (i.e., the x2 axis direction). The emitted beam of the first sensor is parallel to the column direction. The projection of the first sensor onto the plane of the medium-free holographic image along the projection direction is different in the row direction (i.e., the x2 axis direction), thus ensuring that the emitted beams of all the first sensors can cover all interactive areas of the medium-free holographic image in the projection area. Figure 1 As shown, the first sensors are arranged parallel to the x1 axis. It should be noted that the first sensors do not need to be completely aligned on the same straight line, that is, multiple first sensors do not need to have the same y1 axis position, but the positions of different first sensors on the x1 axis need to be different, so that the emitted beams of all the first sensors can cover all interactive areas in the medium-free holographic image.

[0077] like Figure 1 As shown, in some embodiments, the medium-free holographic image forms a certain angle with the plane where the optical transmission element 114 is located to facilitate user observation and operation of the medium-free holographic image. For example, the medium-free holographic image includes a first side and a second side, the first side being perpendicular to the second side and parallel to a first direction, the second side being parallel to a second direction, the first side being parallel to the plane where the optical transmission element 114 is located, and the second side forming a certain angle with the plane where the optical transmission element 114 is located. The first sensor can be arranged above the optical transmission element 114 along a direction parallel to the first side, that is, distributed above the optical transmission element 114 along a direction parallel to the x1 axis. When the user observes the medium-free holographic image from the front of the medium-free holographic imaging device 100, the first side of the medium-free holographic image can be parallel to the user's left-right direction, and the second side of the medium-free holographic image is perpendicular to the first side.

[0078] Similarly, if the second direction can be the row direction, then the first direction is the column direction. The emitted beam of the first sensor is parallel to the row direction. The projection of the first sensor onto the plane of the medium-free holographic image is positioned differently along the column direction, thus ensuring that the emitted beams of all the first sensors can cover all interactive areas of the medium-free holographic image within its projection area.

[0079] The emitted beam of each first sensor can cover the interactive area at one location in the first direction of the medium-free holographic image in the projection area of ​​the medium-free holographic image without covering the interactive areas at multiple other locations.

[0080] In some embodiments, the emitted beams of a plurality of adjacent first sensors may cover the same interaction area. First sensors whose emitted beams cover the same interaction area may constitute a group of sensors. For example... Figure 3 As shown, the first sensor a121 may include two sub-sensors, each of which can emit and receive a light beam. These two sub-sensors may also be referred to as the first sensor group. The light beams emitted and received by these two sub-sensors can cover the interaction area A. The first sensor b123 includes a sub-sensor, and the light beam emitted by this sub-sensor can cover the interaction areas B and C.

[0081] In some embodiments, the emitted beam of the same first sensor may not cover different interaction areas.

[0082] In some embodiments, the positions of multiple interactive regions in a medium-free holographic image can be represented as a first position in a first direction and a second position in a second direction, wherein the second direction is parallel to the transmission direction of the emitted beam and the first direction is perpendicular to the second direction, and the multiple first sensors are arranged along a direction parallel to the first direction.

[0083] In this application, the first sensor is configured such that its emitted beam is parallel to the plane of the medium-free holographic image, and the distance between the first sensor and the medium-free holographic image is less than a threshold. Multiple first sensor emitted beams cover the interactive area of ​​the medium-free holographic image's projection area along a direction perpendicular to the medium-free holographic image. This transforms the determination of the three-dimensional coordinates of the user-operated interactive area's execution entity in three-dimensional space into the determination of the two-dimensional coordinates of the pixels corresponding to the interactive area of ​​the medium-free holographic image. The two-dimensional coordinates of the pixels corresponding to the interactive area in the medium-free holographic image (i.e., the position of the interactive area) can be represented by the pixel's position in the first direction (e.g., the row direction, i.e., the y2 axis direction) and the second direction (e.g., the column direction, i.e., the x2 axis position) of the medium-free holographic image. Since the beam emitted by the first sensor is reflected by the user's action and received by the same sensor, and the emitted beam is parallel to the second direction, the position of the emitted beam of each first sensor in the first direction of the medium-free holographic image is known. Therefore, determining the first sensor allows us to determine the position of the interactive area covered by its emitted beam in the first direction. The position of the interactive area in the second direction is related to the distance between the user's action and the first sensor; determining this distance allows us to determine the position of the interactive area in the second direction. According to the embodiments of this application, a complex gesture recognition model is not required to determine the position of the interactive area, reducing computational complexity, improving interaction efficiency, and lowering costs.

[0084] For more details on the location recognition of interactive areas for user-operated media-free holographic images, please refer to [link / reference]. Figure 4 Detailed description is provided.

[0085] To ensure that the emitted beam of the first sensor is parallel to the plane of the medium-free holographic image, the distance between the first sensor and the medium-free holographic image is less than a threshold, and that the emitted beams of multiple first sensors cover the interactive area of ​​the medium-free holographic image in the projection area of ​​the medium-free holographic image along a direction perpendicular to the medium-free holographic image, the number, orientation, arrangement, and internal components of the first sensors can be configured.

[0086] The number and arrangement of the first sensor groups can be set based on the number and position of the interactive regions in the medium-free holographic image. For example, the number of the first sensor groups can be greater than the number of interactive regions in the medium-free holographic image. As another example, the spacing between the first sensor groups can be set based on the spacing between the interactive regions.

[0087] In some embodiments, to make the light beam emitted by the first sensor parallel to the second direction of the medium-free holographic image, the first sensor can be configured based on the medium-free holographic image, so that each column of pixels in the medium-free holographic image can overlap with the projection of the light beam emitted by the first sensor onto the plane of the medium-free holographic image. For example, the orientation, mounting position, internal optical structure, etc. of the first sensor can be configured based on the plane of the medium-free holographic image.

[0088] In some implementations, the number and arrangement of the first sensors can be designed based on the accuracy of the interactive area recognition. The higher the accuracy requirement, the more first sensors can be used, and the smaller the spacing between two adjacent first sensors can be set.

[0089] In some embodiments, sensor assembly 120 may further include a second sensor. The second sensor may be used to acquire second sensing data, which may be used to detect whether the entity or user performing the operation on the medialess holographic image is a preset object. In some embodiments, the preset object may include a finger, a stylus, etc. In some embodiments, the preset object may include a user with a specific identity.

[0090] The second sensor may include an ultrasonic sensor, an image sensor, etc.

[0091] In some embodiments, the second sensor may be disposed opposite to the first sensor. For example, the first sensor may be disposed above the first side of the optical transmission member 114, and the second sensor may be disposed above the second side of the optical transmission member 114. The first side and the second side are parallel.

[0092] In some embodiments, the second sensor may be disposed adjacent to the first sensor. For example, the first sensor may be disposed above the first side of the optical transmission member 114, and the second sensor may be disposed above the second side of the optical transmission member 114, with the first side and the second side being perpendicular.

[0093] The controller can be used to control the operation of other components or elements in the medium-free holographic imaging device 100. For example, the controller can control the medium-free holographic imaging component 110 to generate and display medium-free holographic images. As another example, the controller can control the sensor component 120 to emit light beams in a cyclical or simultaneous manner. In some embodiments, the controller can execute the control method of the medium-free holographic imaging device shown in the embodiments of the present invention. For example, the controller controls a first sensor to emit a light beam toward the area where the medium-free holographic image is located. When a user operates on an interactive area (i.e., a target interactive area) of the medium-free holographic image using an execution subject (e.g., a finger or stylus), the light beam emitted by the target first sensor is reflected by the user's execution subject and received by the target first sensor to generate first perception data. The first perception data may reflect the distance between the target first sensor and the execution subject. The controller can determine a first position of the execution subject (or the target interactive area) in the light beam transmission direction (i.e., the second direction) based on the first perception data. The controller can determine a second position of the execution subject (or the target interactive area) in a direction perpendicular to the light beam transmission direction (i.e., the first direction) based on the target first sensor. The controller can determine the position of the target interactive region on the medium-free holographic image based on a first position and a second position. The controller can generate a response based on the position of the target interactive region on the medium-free holographic image. For example, the controller can generate a control command based on the position of the target interactive region on the medium-free holographic image, which can drive the imaging source 112 to adjust the light field to provide feedback.

[0094] In some embodiments, the medium-free holographic imaging device 100 may further include a housing, the housing comprising one or more cavities. In some embodiments, the imaging source 112 and the optical transmission element 114 are disposed in the same cavity of the housing. In some embodiments, the imaging source 112 and the optical transmission element 114 may be disposed in different cavities of the housing.

[0095] In some embodiments, the two cavities include a first cavity and a second cavity. The medium-free holographic imaging component 110 may be located in the first cavity, and the sensor component 120 may be located in the second cavity. In some embodiments, the medium-free holographic image may be located in the second cavity.

[0096] In some embodiments, the first cavity and the second cavity are separated by a partition. The optical transmission element 114 is disposed on one side of the partition, and the sensor assembly 120 is disposed on the other side of the partition.

[0097] It should be noted that the above description is provided for illustrative purposes only and is not intended to limit the scope of this specification. Various changes and modifications can be made by those skilled in the art based on the content of this specification. Features, structures, methods, and other features of the exemplary embodiments described herein can be combined in various ways to obtain other and / or alternative exemplary embodiments. For example, the functionality of the controller may be implemented based on a cloud computing platform (e.g., public cloud, private cloud, community cloud, and hybrid cloud). As another example, the medialess holographic imaging device 100 may further include communication components to enable data transmission between the various devices. However, these changes and modifications do not depart from the scope of this specification.

[0098] This specification also provides a control method for a medium-free holographic imaging device. Figure 4 This is a schematic flowchart illustrating a control method for a medium-free holographic imaging device according to some embodiments of this specification. In some embodiments, process 400 may be... Figure 1 The controller 130, the electronic device 600 of the medium-free holographic imaging device, or the control device 700 for controlling the medium-free holographic imaging device described herein shall be executed. In some embodiments, such as Figure 4 As shown, process 400 may include the following steps.

[0099] Step 402: Acquire first perception data collected by the first sensor of the medium-free holographic imaging device. The first perception data is generated by the first sensor when the user's execution subject operates on the target interactive area of ​​the medium-free holographic image of the medium-free holographic imaging device. The first perception data is related to the user's execution subject. In some embodiments, step 402 can be performed by the acquisition module 702.

[0100] A medium-free holographic imaging device refers to a device that uses medium-free holographic imaging technology to generate medium-free holographic images as an interactive interface, combining touch, gestures, buttons, and other methods to achieve two-way information transmission between the user and the device. A medium-free holographic imaging device includes a medium-free holographic imaging component and a sensor component.

[0101] In some embodiments, the medium-free holographic imaging component may include an imaging source and an optical transmission element. The imaging source is used to generate and / or adjust an interactive light field, which can constitute a light source image. The optical transmission element is used to process and transmit the light source image generated by the imaging source to project the light source image onto a preset imaging area to form a medium-free holographic image (i.e., a medium-free holographic image). The medium-free holographic image may include one or more interactive areas, which users can manipulate (e.g., click, swipe, grab) through an action (e.g., a finger) to achieve bidirectional information transmission with the medium-free holographic imaging device. The interactive areas may be marked on the medium-free holographic image using virtual buttons, focus cursors, etc. For example, virtual buttons may be displayed with a semi-transparent color or an luminous border to mark the operable areas (i.e., interactive areas).

[0102] The sensor assembly includes multiple first sensors configured to acquire first perceptual data about the user's action subject as the user interacts with an interactive area of ​​a medialess holographic image through the action subject.

[0103] In some embodiments, the subject performing the user's interaction with the interactive area can be a finger, a stylus, or the like. The first sensor-collected data can be used to determine which interactive area in the mediumless holographic image the user's subject is pointing to or located in.

[0104] Each first sensor can generate and emit a light beam. When the emitted light beam is transmitted to the actuator in the user interaction area, it is reflected to produce a reflected light beam. The first sensor can receive the reflected light beam and generate first sensing data based on the reflected light beam. The first sensor is also called a ranging sensor and can include infrared sensors, laser sensors, etc.

[0105] Multiple first sensors are configured to emit beams parallel to the medium-free holographic image, and the distance between the emitted beams and the medium-free holographic image is less than a threshold.

[0106] In some embodiments, the distance between the emitted beam of the first sensor and the medium-free holographic image can be equal to 0 and parallel to the medium-free holographic image. In this case, the emitted beam of the first sensor can overlap with the interactive region of the medium-free holographic image. In other words, the interactive region of the medium-free holographic image is on the transmission path of the emitted beam of the first sensor.

[0107] In some embodiments, the distance between the emitted beam of the first sensor and the medium-free holographic image can be greater than 0 and less than a threshold, and parallel to the medium-free holographic image. The emitted beams of multiple first sensors overlap in the projection area and interaction area of ​​the medium-free holographic image along a direction perpendicular to the medium-free holographic image (also called the projection direction, or the third direction herein). For simplicity, the phrase "the emitted beam of the first sensor in the projection area of ​​the medium-free holographic image along a direction perpendicular to the medium-free holographic image" can be abbreviated as "the projection area of ​​the emitted beam of the first sensor in the medium-free holographic image". It should be noted that the beam itself cannot be projected onto a plane perpendicular to its transmission direction. The projection area of ​​the emitted beam of the first sensor mentioned in this application can refer to an equivalent projection area. An equivalent projection area refers to the projection area formed by simulating the emitted beam of the first sensor as a physical object with the same shape as the emitted beam, along the third direction, on the imaging plane (i.e., the plane where the medium-free holographic image is located). This projection area can be equivalent to the projection area of ​​the emitted beam of the first sensor.

[0108] Both the overlap of the interaction region between the emitted beam and the medium-free holographic image and the overlap of the projection region of the emitted beam on the medium-free holographic image with the interaction region of the medium-free holographic image can be referred to as the emitted beam covering the interaction region of the medium-free holographic image.

[0109] Based on the above configuration, determining the three-dimensional coordinates of the user interaction area's executing entity in three-dimensional space can be transformed into determining the two-dimensional coordinates of the corresponding pixel in the mediumless holographic image. The two-dimensional coordinates of the pixel corresponding to the interaction area in the mediumless holographic image (i.e., the position of the interaction area) can be represented by the position of the pixel in the first direction (e.g., the row direction, i.e., the y2 axis direction) and the second direction (e.g., the column direction, i.e., the x2 axis position) of the mediumless holographic image. Since the beam emitted by the first sensor is reflected by the user's executing entity and received by the same sensor, and the emitted beam is parallel to the second direction, the position of the emitted beam of each first sensor in the first direction of the mediumless holographic image is known. Determining the first sensor allows us to determine the position of the interaction area covered by the emitted beam of the first sensor in the first direction. The position of the interaction area in the second direction is related to the distance between the user's executing entity and the first sensor. Determining the distance between the user's executing entity and the first sensor allows us to determine the position of the interaction area in the second direction. According to the embodiments of this application, the position of the interaction area can be determined without a complex gesture recognition model, reducing computational complexity, improving interaction efficiency, and reducing costs.

[0110] For more information on medium-free holographic imaging devices, please refer to [link / reference]. Figure 1 The detailed description is in the text.

[0111] As mentioned above, the first perception data is generated by the target first sensor when the user's execution subject operates on the target interactive area of ​​the mediumless holographic image of the mediumless holographic imaging device. For example, multiple first sensors emit light beams to the area where the mediumless holographic image is located simultaneously or in a cyclical manner at a preset frequency. When the user's execution subject operates on the target interactive area of ​​the mediumless holographic image (e.g., clicking, grabbing, etc.), the light beam emitted by the target first sensor is blocked and reflected by the execution subject. The reflected light beam is received by the target first sensor and generates an electrical signal (i.e., the first perception data).

[0112] The first perception data can reflect the distance between the user's execution subject and the target first sensor.

[0113] In some embodiments, the first sensing data includes the distance between the user's execution body and the target first sensor.

[0114] In some embodiments, the first sensing data includes the time difference (also known as time of flight) between the time when the target first sensor emits a beam and the time when it receives the reflected beam.

[0115] In some embodiments, the first sensing data includes the phase difference between the phase of the beam emitted by the target first sensor and the phase of the reflected beam received.

[0116] In some embodiments, the number of first target sensors may be one or more.

[0117] Step 404: Determine the first position of the target interaction region in a first direction of the medium-free holographic image based on the target first sensor. In some embodiments, step 404 may be performed by the first position determination module 704.

[0118] Determining the position of the target interaction area in the first direction means determining the first position of the executing entity in the first direction.

[0119] The first direction is perpendicular to the transmission direction of the emitted beam from the first target sensor on the plane of the medium-free holographic image. The medium-free holographic image may further include a second direction, which is parallel to the transmission direction of the emitted beam from the first target sensor on the plane of the medium-free holographic image. The first direction is perpendicular to the second direction. In some embodiments, a two-dimensional coordinate system for the medium-free holographic image can be established using the first and second directions, such as... Figure 3 As shown, the first direction is the x2 axis, and the second direction is the y2 axis.

[0120] In some embodiments, the first direction may be parallel to the row direction of the mediumless holographic image, i.e., the column arrangement direction.

[0121] In some embodiments, the first direction may be parallel to the column direction of the mediumless holographic image, i.e., the row arrangement direction.

[0122] In some embodiments, the first direction may be a direction parallel to any direction of the plane in which the medium-free holographic image is located.

[0123] Determining the first location of the target interaction area may include: determining the position of the emitted beam of the target first sensor in a first direction; and designating the position of the emitted beam of the target first sensor in the first direction as the first location of the target interaction area.

[0124] The position of the emitted beam from the first sensor in the first direction is the position of the area covered by the emitted beam in the medium-free holographic image in the first direction. The position of the emitted beam from the first sensor in the first direction is known or is a system default setting. After determining the target first sensor, the position of the emitted beam from the target first sensor in the first direction can be determined, that is, the position of the interactive area (including the target interactive area) covered by the emitted beam from the target first sensor in the medium-free holographic image in the first direction is determined.

[0125] For any first sensor, its emitted beam can cover one or more interaction regions, meaning the position of the emitted beam and the interaction region it covers is the same in the first direction. For example... Figure 3 As shown, the positions of the emitted beams of interactive region A and the first sensor a121 on the x2 axis are the same; the positions of the emitted beams of interactive regions B and C and the first sensor b123 on the x2 axis are the same.

[0126] For two different first sensors, their emitted beams can cover interactive regions located at different positions in a first direction within a medium-free holographic image. For example... Figure 3 As shown, interactive region A is located on the transmission path of the emitted beam of the first sensor a121, and interactive regions B and C are located on the transmission paths of the emitted beam of the first sensor b123, respectively. The emitted beam of the first sensor a121 can cover interactive region A, and the emitted beam of the first sensor b123 can cover interactive regions B and C. The positions (or coordinates) of the first sensor a121 and the first sensor b123 in the first direction (i.e., the x2 axis) are not the same, and the positions of interactive regions A and interactive regions B and C in the first direction are different.

[0127] Therefore, the position of the interactive area covered by the first sensor in the first direction can be determined based on the position of the emitted beam of the first sensor in the first direction. Furthermore, the positions of the emitted beams of different first sensors in the first direction are known. Therefore, after determining the target first sensor, the position of the emitted beam of the target first sensor in the first direction can be determined based on the target first sensor, and this position is the position of the target interactive area in the first direction.

[0128] To further explain, if one of the multiple first sensors receives reflected light, it can be determined that a user execution entity exists on the transmission path of the emitted beam of that first sensor, meaning that the first sensor can be identified as the target first sensor. Since the position of the emitted beam of the first sensor in the first direction (i.e., the position in the first direction) of the medium-free holographic image is known or pre-set, the position of the emitted beam of the target first sensor in the first direction of the medium-free holographic image can be determined based on the target first sensor. This position is the first position of the target interactive area in the first direction.

[0129] For ease of explanation, the following example illustrates how the first direction is parallel to the row direction of the medium-free holographic image and the second direction is parallel to the column direction of the medium-free holographic image.

[0130] Since the emitted beam of the first sensor is parallel to the column direction of the medium-free holographic image, and the emitted beams of different first sensors are positioned differently in the row direction of the medium-free holographic image, when the user's action object operates on the target interactive area, due to the presence of the user's action object, the emitted beam of a certain first sensor will be blocked and then received by that first sensor. The first sensor that receives the reflected beam is designated as the target first sensor. The position of the emitted beam of this target first sensor in the row direction of the medium-free holographic image is the position of the target interactive area in the row direction. Furthermore, since the position of the emitted beam of the first sensor in the row direction is known, the position of the emitted beam of the target sensor in the row direction can be determined through the target first sensor, which is the first position of the target interactive area in the row direction.

[0131] Further with Figure 3Taking an example, the first direction of the medium-free holographic image 118 is the row direction, i.e., the x2 axis direction, and the second direction is the column direction, i.e., the y2 axis direction. The first sensors are arranged along the first direction. The positions of the emitted beams of different first sensors on the x2 axis of the medium-free holographic image are different. For example, the emitted beam of the first sensor a121 covers the interactive region A of the medium-free holographic image, that is, the position of the emitted beam of the first sensor a121 on the x2 axis of the medium-free holographic image is the same as the position of the interactive region A on the x2 axis; the emitted beam of the first sensor b123 covers the interactive regions B and C, that is, the position of the emitted beam of the first sensor b123 on the x2 axis of the medium-free holographic image is the same as the position of the interactive regions B and C on the x2 axis.

[0132] When the user performs a main operation interaction on area A, the light beam emitted by the first sensor a121 is reflected by the subject. Therefore, the first sensor in the first sensor a121 can be identified as the target first sensor. The correspondence between the position of the first sensor a121 and the position of the emitted light beam of the first sensor a121 in the first direction of the medium-free holographic image (i.e., the position on the x2 axis) is known. By determining the position of the first sensor a121 in the first direction of the medium-free holographic image (i.e., the position on the x2 axis), the position of the interaction area A on the x2 axis can be determined, that is, the position of the column where the interaction area A is located (i.e., the first position).

[0133] When an interactive region exists at each position along a first direction in a medium-free holographic image, the first position of the target interactive region along that first direction is determined, thus identifying which interactive region in the medium-free holographic image the target interactive region belongs to. For example, Figure 3 If there is only one interactive region in the column direction where interactive region A is located, and the position of the target interactive region in the first direction is the same as the position of interactive region A in the first direction, then the target interactive region is determined to be interactive region A.

[0134] When multiple interactive regions exist at each position in the first direction of a mediumless holographic image, even if the first position of the target interactive region in the first direction is determined, it is still impossible to determine which interactive region in the mediumless holographic image the target interactive region belongs to. It is necessary to further determine which interactive region in the mediumless holographic image the target interactive region belongs to based on the second position in the second direction.

[0135] For example, Figure 3There are two interactive regions (i.e., interactive regions B and C) in the column direction of the emitted beam of the first sensor b123. When it is determined that the first position of the target interactive region in the first direction is the same as the position of interactive regions B and C in the first direction, it can be determined that the target interactive region belongs to one of interactive regions B and C. It is also necessary to determine whether it belongs to interactive region B or interactive region C based on the second position of the target interactive region on the y2 axis. For a detailed description of the determination of the second position, please refer to step 406.

[0136] Step 406: Determine the second position of the target interactive region in the second direction of the medium-free holographic image based on the first perception data. In some embodiments, step 406 may be performed by the second position determination module 706.

[0137] The second direction is the direction perpendicular to the first direction. For example, if the first direction is the row direction, the second direction is the column direction. If the first direction is the column direction, the second direction is the row direction.

[0138] In some embodiments, the distance between the execution subject and the target first sensor can be determined based on the first sensing data; and the second position of the target interaction area in the second direction of the mediumless holographic image can be determined based on the distance between the execution subject and the target first sensor.

[0139] In some embodiments, the first sensing data may include the time difference (also known as time of flight) between the time the target first sensor emits a light beam and the time it receives the reflected light beam. The distance between the actuator and the target first sensor can be determined based on the time of flight. For example, the distance between the actuator and the target first sensor may be equal to half the product of the speed of light and the time of flight.

[0140] In some embodiments, the first sensing data includes the phase difference (also known as phase offset) between the emitted and reflected beams of the target first sensor. The distance between the actuator and the target first sensor can be determined based on the phase difference. For example, the distance between the actuator and the target first sensor can be determined based on the following formula:

[0141] D=

[0142] Where c is the speed of light (3 × 10⁻⁶) 8 (m / s), f is the modulation frequency of the light wave (unit: Hz). Phase difference (unit: radians).

[0143] Since the beam emitted by the first sensor is parallel to the second direction, the distance between the executing entity and the target first sensor is the distance in the second direction. Having determined the distance between the executing entity and the target first sensor, the second position of the target interaction area in the second direction can be further determined based on the distance between the target first sensor and the medium-free holographic image in the second direction.

[0144] For ease of explanation, the following example will still be used, where the first direction is parallel to the row direction of the medium-free holographic image, and the second direction is parallel to the column direction of the medium-free holographic image. Figure 3 As shown, the first direction of the medium-free holographic image 118 is the row direction, i.e., the x2 axis direction, and the second direction is the column direction, i.e., the y2 axis direction. The first sensors are arranged along the first direction. When the user performs a main body operation interaction area A, the emitted beam of the first sensor a121 is reflected by the main body, thus it can be determined that the first sensor in the first sensor a121 is the target first sensor. The distance between the first sensor a121 and the main body can be determined based on the first perception data, which is the distance d1 between the interaction area A and the first sensor a121 in the y2 axis direction. The position d2 of the first sensor a121 and the medium-free holographic image on the y2 axis is known or is a system default setting. Furthermore, the position of the interaction area A1 in the second direction in the medium-free holographic image, i.e., the row position (i.e., which row in the column direction), can be determined based on the distances d1 and d2.

[0145] Step 408: Based on the first and second positions of the target interaction area, control the medium-free holographic imaging device to provide a response. In some embodiments, step 408 may be performed by the response module 708.

[0146] By determining the first position of the target interactive area in the first direction and the second position in the second direction, the position of the target interactive area in the mediumless holographic image can be determined, thus identifying which interactive area the user is operating.

[0147] In some embodiments, the response may include a general response, meaning that different interactive areas can correspond to the same response. This general response is a processing of the interactive area, such as highlighting or deformation. For example, adjusting the light field by the imaging source 112 can increase the light intensity of the target interactive area operated by the user in the medium-free holographic image, or adjust the color of the interactive area operated by the user in the medium-free holographic image for highlighting. As another example, adjusting the light field by the imaging source 112 can generate ripple animation or magnify the target interactive area to deform the target interactive area operated by the user in the medium-free holographic image.

[0148] In some embodiments, the response may include a personalized response, i.e., different responses may be provided for different target interaction areas.

[0149] Personalized responses can include visual responses, tactile responses, auditory responses, etc.

[0150] The visual response can be processing of the target interactive area displayed. The visual response can include dynamic or static adjustments, such as highlighting or distorting, made by a medium-free holographic imaging device (e.g., imaging source 112) to the target interactive area in the medium-free holographic image that the user is interacting with.

[0151] Haptic responses can include simulating the realistic tactile sensation of a user interacting with a target area by providing pressure. For example, a perceptible tactile point can be generated at the finger location by focusing sound pressure in the air using a phased array ultrasonic transducer (typically with a frequency of 40-70 kHz).

[0152] Auditory responses may include providing voice prompts or effects based on the target interactive area operated by the user. For example, each interactive area may correspond to a different voice effect. When the user operates the target interactive area, the medium-free holographic imaging device 100 may generate the corresponding voice effect (e.g., a "click" sound) through an ultrasonic array.

[0153] In some embodiments, the response may include executing system functions corresponding to the target interactive area. The system functions can be configured based on the specific application scenario of the media-free holographic imaging device. For example, when the media-free holographic imaging device is applied to a smart home system, the interactive areas in the media-free holographic image can each correspond to different smart appliances. When a user interacts with the target interactive area, the next-level menu of the target smart appliance corresponding to that area can be opened. Furthermore, the media-free holographic imaging device can update the media-free holographic image to display the control panel of the target smart appliance.

[0154] According to some embodiments of the present invention, by configuring multiple first sensors such that the emitted beams of the multiple first sensors are parallel to the medium-free holographic image and the distance between them and the medium-free holographic image is less than a threshold, the emitted beams of the first sensors approximately overlap with or are adjacent to the medium-free holographic image. When the emitted beams of the first sensors approximately overlap with or are adjacent to the medium-free holographic image, the determination of the three-dimensional coordinates of the user operation interaction area execution subject in three-dimensional space can be transformed into the determination of the two-dimensional coordinates of the corresponding pixels of the interaction area of ​​the medium-free holographic image (i.e., the positions in the first direction and the second direction, where the second direction is parallel to the transmission direction or transmission path of the emitted beam). Further, since the emitted beams of the first sensors are reflected by the user execution subject and received by the same sensor, determining the first sensor allows us to determine the position of the emitted beams of the first sensor in the first direction, that is, to determine the position of the interaction area covered by the emitted beams of the first sensor in the first direction. Further, the position of the interaction area in the second direction, i.e., the position in the transmission direction of the emitted beam, can be determined based on the distance between the user execution subject and the first sensor measured by the first sensor. In this application, the determination of three-dimensional coordinates in three-dimensional space is transformed into the determination of two-dimensional coordinates of a mediumless holographic image by configuring the first sensor, which simplifies the calculation process, improves the interaction efficiency, and reduces costs.

[0155] This specification also provides a control method for a medium-free holographic imaging device. Figure 5 This is a schematic flowchart illustrating a control method for a medium-free holographic imaging apparatus according to some embodiments of this specification. In some embodiments, process 500 may be... Figure 1 The controller 130, the electronic device 600 for controlling a medium-free holographic imaging device, or the control device 700 for a medium-free holographic imaging device described herein are executed. In some embodiments, such as Figure 5 As shown, process 500 may include the following steps.

[0156] Step 502: Acquire second sensing data collected by the second sensor of the medium-free holographic imaging device. In some embodiments, step 502 may be performed by the acquisition module 702.

[0157] In some embodiments, the second sensor is used to collect personalized data of the user and / or the user's executive body, such as appearance, biometric features, etc.

[0158] The appearance of a user-executing entity can include its thickness, length, shape, texture, and outline. For example, a user-executing entity can be a finger, and its appearance can include the thickness of the finger.

[0159] In some embodiments, the shape-related data of the user's execution subject can be acquired by a second sensor when the user's execution subject operates on the target interactive area of ​​the mediumless holographic image of the mediumless holographic imaging device.

[0160] A user's biometric features may include their fingerprints, irises, voiceprints, etc.

[0161] In some embodiments, the second sensor may include an ultrasonic sensor, millimeter-wave radar, infrared camera, etc. The shape of the user-executing entity can be determined based on the second sensing data collected by the second sensor.

[0162] In some embodiments, the second sensor may include a fingerprint sensor, an iris scanner, a voiceprint scanner, etc., which can be used to collect the user's fingerprint, iris, voiceprint, etc.

[0163] Step 504: Based on the second perception data, determine whether the user and / or the user's execution subject conforms to a preset object to obtain a determination result. In some embodiments, step 502 may be performed by the response module 708.

[0164] The second perception data can be compared with the preset standards of the preset object to determine whether the user and / or the user execution subject conforms to the preset object.

[0165] For example, the second perceptual data may include the shape of the user's executing subject. By comparing the shape of the user's executing subject (e.g., thickness) with a preset standard range (e.g., finger thickness range), it can be determined whether the user's executing subject conforms to a preset object (finger). If the shape of the user's executing subject deviates from the preset standard range, it can be determined that the user's executing subject does not conform to the preset object; if the shape of the user's executing subject is within the preset standard range, it can be determined that the user's executing subject conforms to the preset object.

[0166] For example, the second sensing data may include the user's biometric features (e.g., fingerprints, iris scans, voiceprints). By comparing the user's biometric features with the biometric features (e.g., fingerprints) of a preset object, it can be determined whether the user matches the preset object. If the similarity between the user's biometric features and the preset object's biometric features is greater than a threshold, the user is determined to match the preset object; if the similarity is less than the threshold, the user is determined not to match the preset object.

[0167] Step 506: Based on the judgment result, determine whether to provide a response to control the medium-free holographic imaging device. In some embodiments, step 506 may be performed by the response module 708.

[0168] If the determination result indicates that the user and / or the user execution subject matches a preset object, the medium-free holographic imaging device is allowed to respond to operations on the medium-free holographic image. For example, in response to the user and / or the user execution subject matching a preset object, the steps in process 400 can be executed to respond to the user's operation.

[0169] If the determination result indicates that the user and / or the user execution subject does not conform to the preset object, the medium-free holographic imaging device is not allowed to respond to operations on the medium-free holographic image. For example, if the user and / or the user execution subject does not conform to the preset object, step 408 is not executed.

[0170] The second sensing data can determine whether the user and / or the user's execution subject matches a preset object, reducing the probability of erroneous operations. For example, when the controller detects that a non-executing subject is performing an operation in the interactive area, it can choose not to respond to the operation. Similarly, when the controller detects that the execution subject is not a preset object bound to the medialess holographic imaging device, it can choose not to respond to the operation, improving the security of user interaction.

[0171] This specification also provides an electronic device for controlling a medium-free holographic imaging device. Figure 6 This is a schematic diagram of a module for controlling a medium-free holographic imaging device, according to some embodiments of this specification. Figure 6 The electronic device shown may be Figure 1 Another specific implementation of the controller shown. For example... Figure 6 As shown, in some embodiments, the electronic device 600 includes a processor 602 and a memory 604. The memory stores a program for a control method for a medium-free holographic imaging device. After the control device is powered on and runs the program for the control method for the medium-free holographic imaging device through the processor, it performs the following steps: acquiring first sensing data collected by a target first sensor, the first sensing data being generated by the target first sensor when the user's execution subject operates on a target interactive area of ​​a medium-free holographic image provided by the medium-free holographic imaging device; determining a first position of the target interactive area in a first direction of the medium-free holographic image based on the target first sensor; determining a second position of the target interactive area in a second direction of the medium-free holographic image based on the first sensing data; and controlling the medium-free holographic imaging device to provide a response based on the first and second positions of the target interactive area.

[0172] This specification provides a computer-readable storage medium storing a program for a control method of a medium-free holographic imaging device. The program is executed by a processor and performs the following steps: acquiring first sensing data collected by a first target sensor, the first sensing data being generated by the first target sensor when a user's execution entity operates on a target interactive area of ​​a medium-free holographic image provided by the medium-free holographic imaging device; determining a first position of the target interactive area in a first direction of the medium-free holographic image based on the first target sensor; determining a second position of the target interactive area in a second direction of the medium-free holographic image based on the first sensing data; and controlling the medium-free holographic imaging device to provide a response based on the first and second positions of the target interactive area.

[0173] Some embodiments of this specification also provide a computer program product, including a computer program that, when at least a portion of the computer instructions are executed by a processor, can implement this specification. Figure 4 and Figure 5 The method illustrated is for controlling a medium-free holographic imaging device. In some embodiments, the computer program product may relate only to computer instructions, which may be carried on a storage medium or processing device. In other embodiments, the computer program product may also be a storage medium or processing device containing the aforementioned computer instructions. The processing device may include one or more processors, and the storage medium.

[0174] In some embodiments, the processor may be a combination of one or more of the following processors: central processing unit (CPU), application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), graphics processing unit (GPU), physical processing unit (PPU), digital signal processor (DSP), field-programmable gate array (FPGA), programmable logic device (PLD), programmable logic controller (PLC), reduced instruction set computer (RISC), and microprocessor.

[0175] In some embodiments, the storage medium may include one or more combinations of the following: mass storage, removable storage, volatile read-write memory, and read-only memory (ROM). Exemplary mass storage may include disks, optical disks, solid-state drives, etc. Exemplary removable storage may include flash drives, floppy disks, optical disks, memory cards, compressed hard disks, magnetic tapes, etc. Exemplary volatile read-write memory may include random access memory (RAM). Exemplary RAM may include dynamic random access memory (DRAM), dual data rate synchronous dynamic random access memory (DDRSDRAM), static random access memory (SRAM), silicon controlled retrieval memory (T-RAM), and zero-capacitance memory (Z-RAM), etc. Exemplary read-only memory may include masked read-only memory (MROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), compressed hard disk read-only memory (CD-ROM), and digital multifunction hard disk read-only memory, etc.

[0176] This specification also provides a control device for a medium-free holographic imaging device. Figure 7 This is a schematic diagram of a control device for a medium-free holographic imaging apparatus according to some embodiments of this specification. In some embodiments, Figure 7 The control device shown can be Figure 1 One specific implementation of the controller. In some embodiments, Figure 7 The control device shown can be Figure 6 One specific implementation of the processor in the control device 700 may include an acquisition module 702, a first position determination module 704, a second position determination module 706, and a response module 708.

[0177] The acquisition module 702 is used to acquire the first perception data collected by the first sensor of the target. The first perception data is generated by the first sensor of the target when the user's execution subject operates on the target interactive area of ​​the mediumless holographic image provided by the mediumless holographic imaging device.

[0178] The first position determination module 704 is used to determine the first position of the target interactive region in the first direction of the medium-free holographic image based on the target first sensor.

[0179] The second position determination module 706 is used to determine the second position of the target interactive region in the second direction of the medium-free holographic image based on the first perception data.

[0180] The response module 708 is used to control the medium-free holographic imaging device to provide a response based on the first position and the second position of the target interaction area.

[0181] In some embodiments, the acquisition module 702 is used to acquire second sensing data collected by the second sensor. The response module 708 is further used to determine, based on the second sensing data, whether the user and / or the user's execution subject conforms to a preset object to obtain a determination result; and to determine, based on the determination result, whether to control the medium-free holographic imaging device to provide a response.

[0182] For more information on each module, please refer to [link / reference]. Figure 4 and Figure 5 The relevant explanations will not be repeated here. It should be understood that... Figure 7 The systems and modules shown can be implemented in various ways. For example, in some embodiments, the systems and modules can be implemented by hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the methods and systems described above can be implemented using computer-executable instructions and / or included in the control code of a processor, such as on a media such as a disk, CD, or DVD-ROM, or in the memory of a programmable device. The systems and modules of this specification can be implemented not only by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips and transistors, or programmable hardware devices such as field-programmable gate arrays and programmable logic devices, but also by software, for example, executed by various types of processors, or by a combination of the aforementioned hardware circuits and software (e.g., firmware).

[0183] It should be noted that the above description of the system and its modules is for convenience only and should not be construed as limiting this specification to the embodiments described. It is understood that those skilled in the art, after understanding the principles of this system, may arbitrarily combine the various modules without departing from these principles to form subsystems connected to other modules. Alternatively, some modules may be split to obtain more modules or multiple units under a single module. Such modifications are all within the scope of this specification.

[0184] It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects may be any one or a combination of the above, or any other possible beneficial effects.

[0185] The basic concepts have been described above. It is obvious that the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, various modifications, improvements, and corrections may be made to this specification by those skilled in the art. Such modifications, improvements, and corrections are taught in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

Claims

1. A medium-free holographic imaging device, characterized in that, include: A medium-free holographic imaging component is configured to generate a suspended medium-free holographic image in a preset imaging area. The medium-free holographic image includes multiple user-operable interactive areas. The positions of the multiple interactive areas in the medium-free holographic image are represented as a first position in a first direction and a second position in a second direction of the medium-free holographic image, wherein the first direction is perpendicular to the second direction. A sensor assembly includes a plurality of first sensors configured to generate first perceptual data about the executing subject when the user interacts with an interactive area of ​​the medium-free holographic image through an executing subject, wherein the emitted light beams of the plurality of first sensors are parallel to the medium-free holographic image, the distance between the emitted light beams and the medium-free holographic image is less than a threshold, a second direction is parallel to the transmission direction of the emitted light beams, and the plurality of first sensors are arranged along a direction parallel to the first direction; and The controller is configured to: acquire first perception data collected by a first sensor of the target, the first perception data being generated by the first sensor of the target when the user's execution subject operates on the target interaction area; determine a first position of the target interaction area in a first direction of the medium-free holographic image based on the position of the first sensor of the target; determine a second position of the target interaction area in a second direction of the medium-free holographic image based on the first perception data collected by the first sensor of the target; and control the medium-free holographic imaging device to provide a response based on the first position and the second position of the target interaction area.

2. The medium-free holographic imaging device according to claim 1, characterized in that, The distance between the emitted beam and the medium-free holographic image is greater than 0 and less than the threshold. The emitted beams of the plurality of first sensors cover the plurality of interactive areas in the projection area of ​​the medium-free holographic image along a direction perpendicular to the medium-free holographic image.

3. The medium-free holographic imaging device according to claim 1, characterized in that, The sensor assembly further includes a second sensor configured to acquire second sensing data, the second sensing data including the appearance of the user execution entity and / or the user's biometric features.

4. The medium-free holographic imaging device according to claim 3, characterized in that, It also includes a controller, which is configured as follows: Acquire the second sensing data; Based on the second perception data, determine whether the user and / or the user execution subject conforms to the preset object to obtain the judgment result; as well as Based on the judgment result, determine whether to control the medium-free holographic imaging device to provide a response.

5. A control method for a medium-free holographic imaging device, characterized in that, include: Acquire first perception data collected by the first sensor of the target, wherein the first perception data is generated by the first sensor of the target when the user's execution subject operates on the target interactive area of ​​the mediumless holographic image provided by the mediumless holographic imaging device; Based on the target first sensor, the first position of the target interactive area in the first direction of the medium-free holographic image is determined; Based on the first perception data, the second position of the target interactive region in the second direction of the medium-free holographic image is determined; as well as Based on the first and second positions of the target interaction area, the medium-free holographic imaging device is controlled to provide a response; The medium-free holographic imaging device includes a sensor assembly, which includes a plurality of first sensors. The emitted beams of the plurality of first sensors are parallel to the medium-free holographic image, and the distance between the emitted beams and the medium-free holographic image is less than a threshold.

6. The control method according to claim 5, characterized in that, Determining the first position of the target interaction region in the first direction of the medium-free holographic image based on the target first sensor includes: Based on the target first sensor, determine the position of the emitted beam of the target first sensor in the first direction; and The position of the emitted beam of the first sensor of the target in the first direction is designated as the first position of the target interactive region in the first direction of the medium-free holographic image, and the emitted beam of the first sensor of the target covers the target interactive region.

7. The control method according to claim 6, characterized in that, Determining the second position of the target interactive region in the second direction of the medium-free holographic image based on the first perceptual data includes: Based on the first sensing data, the distance between the user's execution entity and the target first sensor is determined; and The second position of the target interaction area in the second direction is determined based on the distance between the execution subject and the target first sensor and the distance between the target first sensor and the medium-free holographic image in the second direction.

Images