Data processing method and electronic device

Abstract

The present application provides a data processing method, the method comprising: obtaining depth image information of a target object and touch information on a transparent display screen; establishing a mapping relationship between the touch information and the target object based on the depth image information and the touch information; and controlling the touch information to be displayed on the transparent display screen based on the mapping relationship, so that the touch information moves with the target object. At the same time, the present application also provides an electronic device.

Description

Technical Field

[0001] The present application relates to a data processing technology, and more particularly to a data processing method and electronic device. Background Art

[0002] In traditional scenarios, when a user performs a touch operation on a touch screen, the corresponding touch information can only be displayed through the touch screen, but cannot be combined with people or objects in the real scene, making it impossible to achieve effective interaction between the person in front of the screen and the person behind the screen. Summary of the invention

[0003] In view of this, an embodiment of the present application is intended to provide a data processing method, the method comprising:

[0004] Acquire depth image information of the target object and touch information on the transparent display screen;

[0005] Establishing a mapping relationship between the touch information and the target object based on the depth image information and the touch information;

[0006] The touch information is controlled to be displayed on the transparent display screen based on the mapping relationship, so that the touch information moves along with the target object.

[0007] In the above solution, the step of obtaining the depth image information of the target object and the touch information on the transparent display screen includes:

[0008] Acquire depth image information of the target object collected by an image collection component in a first surface direction of the transparent display screen, wherein the target object is located between the image collection component and the transparent display screen;

[0009] Acquire first touch information collected by a touch detection component in a direction of a second surface of the transparent display screen; wherein the second surface and the first surface are arranged opposite to each other.

[0010] In the above solution, the step of obtaining the depth image information of the target object and the touch information on the transparent display screen includes:

[0011] Acquire depth image information collected by the image acquisition component for the target object in the direction of the second surface of the transparent display screen, wherein the target object is located in the direction of the first surface of the transparent display screen; wherein the second surface and the first surface are arranged opposite to each other;

[0012] Acquire first touch information collected by a touch detection component in a direction of the second surface of the transparent display screen.

[0013] In the above scheme, the method further comprises:

[0014] Acquire second touch information collected by the touch detection component in the direction of the first surface of the transparent display screen; wherein display modes of the first touch information and the second touch information are the same or different.

[0015] In the above solution, the step of obtaining the depth image information of the target object and the touch information on the transparent display screen at least further comprises:

[0016] At a first moment, touch information on the transparent display screen is acquired; at a second moment, depth image information of the target object is acquired; wherein the first moment is less than the second moment; or, the first moment is greater than the second moment.

[0017] In the above solution, the step of establishing a mapping relationship between the touch information and the target object based on the depth image information and the touch information includes:

[0018] A mapping relationship between the touch information and the target object is established based on the depth image information and the coordinate position of the touch information.

[0019] In the above solution, the step of establishing a mapping relationship between the touch information and the target object based on the depth image information and the coordinate position of the touch information includes:

[0020] Determine a first coordinate position of the touch information on the transparent display screen based on regional measurement data of the transparent display screen by a touch detection component;

[0021] and determining a second coordinate position of the depth image information on the transparent display screen based on a spatial position relationship between the image acquisition component and the transparent display screen;

[0022] A mapping relationship between the touch information and the target object is established based on the first coordinate position and the second coordinate position.

[0023] In the above scheme, the method further comprises:

[0024] Detecting the behavior of the target object;

[0025] If the behavior action satisfies the action condition, the current display mode of the touch information is switched based on the behavior action.

[0026] In the above scheme, the method further comprises:

[0027] receiving a mode switching instruction for the touch information;

[0028] The current display mode of the touch information is switched based on the mode switching instruction.

[0029] According to another aspect of the present application, there is provided an electronic device, including:

[0030] An acquisition unit, used to acquire depth image information of a target object and touch information on a transparent display screen;

[0031] an establishing unit, configured to establish a mapping relationship between the touch information and the target object based on the depth image information and the touch information;

[0032] The display unit is used for controlling the touch information to be displayed on the transparent display screen based on the mapping relationship, so that the touch information moves along with the target object.

[0033] The present application provides a data processing method and electronic device, which obtains the depth image information of the target object and the touch information on the transparent display screen; establishes a mapping relationship between the touch information and the target object based on the depth image information and the touch information; and controls the touch information to be displayed on the transparent display screen based on the mapping relationship, so that the touch information moves with the target object. In this way, the touch information can be combined with people or objects in the real scene to achieve the effect of augmented reality. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a schematic diagram of the process implementation of the data processing method in this application;

[0035] Figure 2 This is an illustration of the scene in this application Figure 1 ;

[0036] Figure 3 This is an illustration of the scene in this application Figure 2 ;

[0037] Figure 4 This is an illustration of the scene in this application Figure 3 ;

[0038] Figure 5 This is an illustration of the scene in this application Figure 4 ;

[0039] Figure 6 This is an illustration of the scenario in this application Figure 5 ;

[0040] Figure 7 The structure of the electronic device in this application is shown in FIG. Figure 1 ;

[0041] Figure 8 The structure of the electronic device in this application is shown in FIG. Figure 2 ;

[0042] Fig. 9This is a schematic diagram of the structural composition of the data processing system in this application. DETAILED DESCRIPTION

[0043] The technical solution of the present application is further elaborated in detail below in conjunction with the accompanying drawings and specific embodiments of the specification.

[0044] Figure 1 This is a schematic diagram of the process implementation of the data processing method in this application, such as Figure 1 As shown, the method includes:

[0045] Step 1, obtaining depth image information of the target object and touch information on the transparent display screen;

[0046] Step 2: establishing a mapping relationship between the touch information and the target object based on the depth image information and the touch information;

[0047] Step 3: Control the touch information to be displayed on the transparent display screen based on the mapping relationship, so that the touch information moves along with the target object.

[0048] In the present application, the method can be applied to an electronic device, which can be a personal computer (PC), a server, a signal processor, etc. The electronic device can include an image acquisition component and a transparent display screen, and the transparent display screen can have a touch capability. It can also be connected to an external image acquisition component and an external transparent display screen respectively by wired or wireless means, and the transparent display screen can have a touch capability.

[0049] In Example 1 of the present application, the transparent display screen may have a first surface and a second surface, wherein the image acquisition component may be arranged in the direction of the first surface of the transparent display screen, and the second surface of the transparent display screen may be a touch surface with touch capability.

[0050] When there is a target object between the image acquisition component and the transparent display screen, the electronic device can obtain depth image information of the target object acquired by the image acquisition component in the first surface direction of the transparent display screen.

[0051] Here, the depth image information may be actively sent to the electronic device by the image acquisition component. The electronic device may send an image acquisition request to the image acquisition component, and the image acquisition component sends the depth image information collected for the target object to the electronic device based on the image acquisition request.

[0052] Here, the target object includes but is not limited to people, organisms and objects, wherein objects include solids, liquids and gases.

[0053] In this example, taking the target object as a person, if the person is facing the transparent display screen, the image acquisition component can be located behind the person.

[0054] In the present application, when the electronic device acquires touch information on the transparent display screen, a touch detection component may be provided in the direction of the second surface of the transparent display screen, and the touch detection component may be connected to the electronic device in a wired or wireless manner. When the touch detection component detects a touch operation performed by the user on the touch surface in the direction of the second surface of the transparent display screen, the touch detection component may send the first touch information corresponding to the touch operation to the electronic device. In this way, the electronic device can acquire touch information on the transparent display screen.

[0055] Here, the touch detection component includes but is not limited to a laser radar and an infrared detection component.

[0056] like Figure 2 As shown, it includes a host 10, a transparent display screen 20, a camera 30 and a touch detection component 40, wherein the transparent display screen 20 has a first surface 21 and a second surface 22, the camera 30 is arranged in the direction of the first surface 21 of the transparent display screen 20, the second surface 22 is a touch surface, and the touch detection component 40 is arranged in the direction of the second surface 22 of the transparent display screen 20. When user A is located between the camera 30 and the transparent display screen 20, the camera 30 can capture an image of user A, obtain depth image information of user A (such as skeletal image information of user A), and send the depth image information to the host 10, and the host 10 projects the depth image information of user A onto the transparent display screen 20 according to the spatial position relationship between the camera 30 and the transparent display screen 20 (the depth image information of user A is indicated by a dotted line in the figure). User B is located in the direction of the second surface 22 of the transparent display screen 20. User B writes on the second surface 22 of the transparent display screen 20 by touch, for example, draws a "heart" on the heart part represented by the depth image information. The touch detection component 40 sends the "heart" touch information to the host 10. The host 10 projects the "heart" to the display area of ​​the transparent display screen 20 through the area measurement data of the second surface 22 by the touch detection component 40, and the "heart" is located in the heart part represented by the depth image information of user A. In this way, from the perspective of user B, the "heart" seems to be directly attached to the heart part of user A, and can also move and beat with user A.

[0057] In Example 2 of the present application, the image acquisition component may also be arranged in the direction of the second surface of the transparent display screen, in the same direction as the touch surface of the transparent display screen.

[0058] When the image acquisition component and the touch surface are located in the same direction of the transparent display screen, the target object may be located in the direction of the first surface of the transparent display screen. The image acquisition component located in the direction of the second surface of the transparent display screen may perform image acquisition on the target object to obtain depth image information of the target object, and send the depth image information to the electronic device. Alternatively, the electronic device may send an image acquisition request to the image acquisition component located in the direction of the second surface of the transparent display screen, and the image acquisition component may send the depth image information collected for the target object to the electronic device based on the image acquisition request. In this way, the electronic device can obtain the depth image information of the target object.

[0059] For example, taking the target object as a person, when the person faces the transparent display screen, the transparent display screen can be located between the image acquisition component and the person.

[0060] In this second example, the process of the electronic device acquiring the touch information on the transparent display screen is the same as that in the first example, and will not be repeated here.

[0061] like Figure 3 As shown, Figure 2 The difference is that the camera 30 is set in the direction of the second surface 22 of the transparent display screen 20, and the user A is located in the direction of the first surface 21 of the transparent display screen 20. The camera 30 is used to capture the image of user A, obtain the depth image information of user A (such as the bone image information of user A), and send the depth image information to the host 10.

[0062] Here, the camera 30 can also be set in other directions of the transparent display screen 20. Here, the specific setting position of the camera 30 is not limited, as long as it can capture the image of the target object, obtain the depth image information of the target object, and send the depth image information to the host 10.

[0063] In the third example of the present application, the first surface and the second surface of the transparent display screen can both be touch surfaces, and a first touch detection component can be provided in the direction of the first surface of the transparent display screen, and a second touch detection component can be provided in the direction of the second surface of the transparent display screen, and both the first touch detection component and the second touch detection component can be connected to the electronic device by wire or wireless means. When the first touch detection component detects a first touch operation performed by a user on the touch surface in the direction of the first surface of the transparent display screen, the first touch detection component can send the first touch information corresponding to the first touch operation to the electronic device. When the second touch detection component detects a second touch operation performed by a user on the touch surface in the direction of the second surface of the transparent display screen, the second touch detection component can send the second touch information corresponding to the second touch operation to the electronic device. In this way, the electronic device can obtain touch information from different surfaces of the transparent display screen.

[0064] In this example, a first image acquisition component may be provided in the first surface direction of the transparent display screen, and a second image acquisition component may be provided in the second surface direction of the transparent display screen, and both the first image acquisition component and the second image acquisition component may be connected to the electronic device in a wired or wireless manner. When the target object is located in the first surface direction of the transparent display screen, the image of the target object may be acquired by the first image acquisition component, or by the second image acquisition component, or the image of the target object may be acquired by both the first image acquisition component and the second image acquisition component at the same time, and the acquired depth image may be sent to the electronic device.

[0065] In this example, when the first image acquisition component and the second image acquisition component simultaneously acquire images of the target object, the electronic device can also combine the first deep image information and the second deep image information to form the third deep image information of the target object when receiving the first deep image information acquired by the first image acquisition component and the second deep image information acquired by the second image acquisition component. In this way, the completeness of the depth image information of the target object acquired by the electronic device can be improved.

[0066] like Figure 4 As shown, Figure 3 and Figure 2The difference is that the first surface 21 and the second surface 22 of the transparent display screen 20 are both touch surfaces, and a camera 30-1 is arranged in the direction of the first surface 21, and a camera 30-2 is arranged in the direction of the second surface 22, and the user A is located between the camera 30-1 and the transparent display screen 20, wherein the image of the user A can be simultaneously captured by the cameras 30-1 and 30-2, and the first depth image information and the second depth image information of the user A are obtained, and the first depth image information and the second depth image information are sent to the host 10. After the host 10 receives the first depth image information and the second depth image information sent by the cameras 30-1 and 30-2, the first depth image information and the second depth image information can be merged to obtain a more complete third depth image information of the user A, and then based on the spatial position relationship between the cameras 30-1, 30-2 and the transparent display screen 20, the third depth image information is projected to the display area of ​​the transparent display screen 20. Then, user A can also write on the first surface 21 of the transparent display screen 20 by touch, such as drawing a hat a on the head part represented by the depth image information of user A, and user B can write on the second surface 22 of the transparent display screen 20 by touch, such as drawing a "heart" on the heart part represented by the depth image information of user A, and send the touch information of the "hat" and the touch information of the "heart" to the host 10 through the touch detection component 40, and the host 10 projects the touch information of the "hat" and the touch information of the "heart" to the display area of ​​the transparent display screen 20. In this way, from the perspective of user B, the "hat" seems to be directly worn on the head of user A and can move with user A; and the "heart" seems to be directly attached to the heart part of user A, and can also beat while moving with user A.

[0067] In the above examples 1 to 3 of the present application, after acquiring the depth image information of the target object and the touch information on the transparent display screen, the electronic device can also establish a mapping relationship between the touch information and the target object based on the depth image information and the touch information.

[0068] In one implementation, the electronic device may establish a mapping relationship between the touch information and the target object based on the depth image information and the coordinate position of the touch information.

[0069] Here, the touch detection component can emit a light beam to the touch display area of ​​the transparent display screen. When the user performs a touch operation on the touch display area, the user's hand, arm or stylus and other objects will block part of the light beam, so that the touch detection component can determine the first coordinate position of the touch information corresponding to the touch operation on the transparent display screen based on the regional measurement data of the transparent display screen. A spatial position relationship can be established between the image acquisition component and the transparent display screen, and the second coordinate position of the depth image information of the target object on the transparent display screen can be determined based on the spatial position relationship, so that the depth image information can be displayed on the transparent display screen based on the second coordinate position. The electronic device can establish a mapping relationship between the touch information and the target object based on the first coordinate position of the touch information on the transparent display screen and the second coordinate position of the depth image information on the transparent display screen.

[0070] In the present application, the electronic device can also control the touch information to be displayed on the transparent display screen based on the mapping relationship, so that the touch information moves with the target object.

[0071] Here, when the electronic device controls the touch information to be displayed on the transparent display screen based on the mapping relationship, the current display mode of the touch information may be a follow mode, indicating that the touch information may be combined with the target object and may move in the touch display area as the target object moves.

[0072] For example, a depth camera A is provided in the direction of the first surface of the transparent display screen, a touch detection component B is provided in the direction of the second surface of the transparent display screen, and a user A is located between the depth camera A and the transparent display screen as a target object. The depth camera A can be used to capture an image of the user A to obtain depth image information of the user A, and the captured depth image information is sent to the electronic device, which can project the depth image information onto the display area of ​​the transparent display screen based on the spatial position relationship between the depth camera A and the transparent display screen.

[0073] Here, the depth image information can represent the skeletal feature information of user A. If at this time, user B draws a heart on the part representing the heart in the depth image information through the transparent display screen, the touch detection component B can detect the touch operation of user B, and can send the touch information "heart" corresponding to the touch operation to the electronic device, and the electronic device can determine the coordinate position of the touch information "heart" on the transparent display screen based on the regional measurement data of the transparent display screen by the touch detection component B, so that the touch information "heart" can be projected to the display area of ​​the transparent display screen based on the coordinate position. Then, the electronic device can establish a mapping relationship between the touch information "heart" and user A based on the coordinate position of the depth image information on the transparent display screen and the coordinate position of the touch information "heart" on the transparent display screen, and can control the touch information "heart" to be displayed on the transparent display screen based on the mapping relationship.

[0074] Here, since user A is moving, the position parameters between the various bone features represented by the depth image information of user A remain unchanged. Therefore, when user A moves, the effect viewed from the perspective of user B is that the touch information "love heart" fits the heart of user A and can move with user A, thereby achieving an augmented reality effect.

[0075] In the present application, when the electronic device acquires the depth image information of the target object and the touch information on the transparent display screen, it can acquire the touch information on the transparent display screen at a first moment and acquire the depth image information of the target object at a second moment; wherein, the first moment may be less than the second moment; or the first moment may be greater than the second moment. In other words, the electronic device can first acquire the touch information on the transparent display screen, and then acquire the depth image information of the target object after acquiring the touch information. It is also possible to first acquire the depth image information of the target object, and then acquire the touch information on the transparent display screen after acquiring the depth image information. Of course, the first moment can also be equal to the second moment, that is, the electronic device can also acquire the touch information and the depth image information at the same time. It can be set specifically according to the usage scenario.

[0076] Below, an exemplary description is given of an implementation scenario where the first moment is less than the second moment:

[0077] for example Figure 5As shown, user B first draws a table b and a backpack a on the second surface 22 of the transparent display screen 20 by touch, wherein the backpack a is placed on the table b, and then the touch detection component 40 sends the touch information of the backpack a and the table b to the electronic device 10. After receiving the touch information of the backpack a and the table b, the electronic device 10 projects the touch information of the backpack a and the table b to the display area of ​​the transparent display screen 20. At this time, user A walks from the first surface 21 of the transparent display screen 20, and the image acquisition component 30 can acquire the depth image information of user A, and the image acquisition component 30 sends the depth image information to the electronic device 10. Based on the spatial position relationship between the image acquisition component 30 and the transparent display screen 20, the electronic device 10 can project the depth image information of user A to the display area of ​​the transparent display screen 20. Then when user A walks to the side of table b, the electronic device 10 can establish a mapping relationship between the backpack a and user A based on the first coordinate position of the backpack a on the transparent display screen 20 and the second coordinate position of the depth image information of the user A on the transparent display screen 20, and can control the display mode of the backpack a on the transparent display screen 20 to be a follow-up mode based on the mapping relationship. Since the electronic device 10 does not establish a mapping relationship between table b and user A, the display mode of table b on the transparent display screen 20 is a static mode. In this way, the effect seen from the perspective of user B can be: when user A walks to the side of table b, the backpack a on table b can be directly lifted up by raising his hand, and the backpack a can also be moved, while table b is stationary, thereby achieving an augmented reality effect. In this way, for some teaching scenarios, some relatively abstract teaching contents can be combined with the virtual and the real through the scheme of this application, so that students can understand and accept them more easily, thereby improving the teaching quality.

[0078] In the present application, when the first surface and the second surface of the transparent display screen are both touch surfaces, the display modes of the first touch information on the first surface and the second touch information on the second surface may be the same or different.

[0079] For example, when the display modes of the first touch information and the second touch information are different, the display mode of the first touch information on the transparent display screen can be a following mode, indicating that the first touch information can move with the movement of the target object; the display mode of the second touch information on the transparent display screen can be a static mode, indicating that the second touch information is disconnected from the target object. When the target object moves, the second touch information is static relative to the target object and will not move with the target object.

[0080] When the display modes of the first touch information and the second touch information are the same, the display modes of the first touch information and the second touch information on the transparent display screen may both be follow-up modes; or, the display modes of the first touch information and the second touch information on the transparent display screen may both be static modes.

[0081] In this way, user A and user B located on different sides of the transparent display screen can perform touch operations on the transparent display screen respectively.

[0082] In the present application, the electronic device can also detect the behavior of the target object; if the detected behavior meets the action condition, the current display mode of the touch information is switched based on the behavior.

[0083] Here, multiple behavioral actions can be preset in the electronic device. When the detected behavioral action is a preset action, it can be determined that the detected behavioral action meets the action condition; or, the electronic device can receive multiple touch information. When the detected behavioral action corresponds to target touch information, it can be determined that the detected behavioral action meets the action condition.

[0084] for example Figure 5 As shown, the target object is user A. During the movement of user A, the image acquisition component 30 continuously acquires images of user A and sends the acquired depth image information to the electronic device 10. After receiving the depth image information of user A, the electronic device 10 can compare the depth image information of user A. According to the comparison result, the behavior of user A can be determined. If the user's behavior is detected as "carrying a bag", the electronic device 10 can search for the coordinate position of the depth image information corresponding to the behavior of "carrying a bag" on the transparent display screen 20 in the received touch information. If the touch information corresponding to the coordinate position is detected at the coordinate position of the depth image information, it is determined that the current behavior action meets the action condition, and a mapping relationship between the touch information "package a" and the user A is established, and the touch information "package a" is controlled to be displayed on the transparent display screen 20 based on the mapping relationship, so as to switch the current display mode of the touch information "package a" from the static mode to the following mode, and in the following mode, the touch information "package a" can be lifted by the user A when the user A has the behavior action of lifting the bag, and can move on the transparent display screen as the user A moves.

[0085] For example Figure 6As shown, the electronic device 10 detects that the user A walks to the side of the table b and detects that a behavior action is a "bag placing" action. Based on the coordinate position of the depth image information corresponding to the "bag placing" behavior action on the transparent display screen 20, the touch information "bag a" can be switched from the current following mode to the static mode, so that the touch information "bag a" is placed on the table b. Here, bag a will not move with the movement of user A in the static mode.

[0086] In the present application, the electronic device may also receive a mode switching instruction for the touch information; and switch the current display mode of the touch information based on the mode switching instruction.

[0087] For example, the current display mode of the touch information is the still mode. At this time, the user can send a mode switching instruction to the electronic device through gestures, voice or buttons. The electronic device can control the touch information to switch from the current still mode to the follow mode based on the mode switching instruction.

[0088] The data processing method provided by the present application obtains the depth image information of the target object and the touch information on the transparent display screen; establishes a mapping relationship between the touch information and the target object based on the depth image information and the touch information; and controls the touch information to be displayed on the transparent display screen based on the mapping relationship, so that the touch information moves with the target object. In this way, the touch information can be combined with people or objects in the real scene to achieve the effect of augmented reality.

[0089] Figure 7 The structure of the electronic device in this application is shown in FIG. Figure 1 ,like Figure 7 As shown, the electronic device includes:

[0090] An acquisition unit 701 is used to acquire depth image information of a target object and touch information on a transparent display screen;

[0091] An establishing unit 702, configured to establish a mapping relationship between the touch information and the target object based on the depth image information and the touch information;

[0092] The display unit 703 is used to control the touch information to be displayed on the transparent display screen based on the mapping relationship, so that the touch information moves along with the target object.

[0093] In a preferred embodiment, the acquisition unit 701 is specifically used to acquire depth image information acquired by the image acquisition component in the direction of the first surface of the transparent display screen for the target object, wherein the target object is located between the image acquisition component and the transparent display screen; and to acquire first touch information acquired by the touch detection component in the direction of the second surface of the transparent display screen; wherein the second surface and the first surface are arranged relative to each other.

[0094] In a preferred embodiment, the acquisition unit 701 is further specifically used to acquire depth image information acquired by the image acquisition component for the target object in the second surface direction of the transparent display screen, wherein the target object is located in the first surface direction of the transparent display screen; wherein the second surface and the first surface are arranged relative to each other; and to acquire first touch information acquired by the touch detection component in the second surface direction of the transparent display screen.

[0095] In a preferred embodiment, the acquisition unit 701 is further configured to acquire second touch information collected by the touch detection component in the direction of the first surface of the transparent display screen; wherein display modes of the first touch information and the second touch information are the same or different.

[0096] In a preferred embodiment, the acquisition unit 701 is further used to acquire touch information on the transparent display screen at a first moment; acquire depth image information of the target object at a second moment; wherein the first moment is less than the second moment; or, the first moment is greater than the second moment.

[0097] In a preferred solution, the establishing unit 702 is specifically configured to establish a mapping relationship between the touch information and the target object based on the depth image information and the coordinate position of the touch information.

[0098] In a preferred embodiment, the electronic device further comprises:

[0099] A determination unit 704 is configured to determine a first coordinate position of the touch information on the transparent display screen based on the regional measurement data of the transparent display screen by the touch detection component; and determine a second coordinate position of the depth image information on the transparent display screen based on the spatial position relationship between the image acquisition component and the transparent display screen;

[0100] The establishing unit 702 is further specifically configured to establish a mapping relationship between the touch information and the target object based on the first coordinate position and the second coordinate position.

[0101] In a preferred embodiment, the electronic device further includes: a detection unit 705 and a switching unit 706;

[0102] The detection unit 705 is used to detect the behavior of the target object;

[0103] If the behavior action satisfies the action condition, the detection unit 705 triggers the switching unit 706, so that the switching unit 706 switches the current display mode of the touch information based on the behavior action.

[0104] In a preferred embodiment, the electronic device further comprises:

[0105] A receiving unit 707, configured to receive a mode switching instruction for the touch information;

[0106] The switching unit 706 is further configured to switch the current display mode of the touch information based on the mode switching instruction.

[0107] It should be noted that: when the electronic device provided in the above embodiment performs data processing, only the division of the above program modules is used as an example. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the electronic device provided in the above embodiment and the data processing method embodiment provided above belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0108] Figure 8 This is a schematic diagram of the structure of the electronic device in this application. Figure 2 The electronic device 800 may be a personal computer, a digital terminal, an information transceiver device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc. Figure 8 The electronic device 800 shown includes: at least one processor 801, a memory 802, at least one network interface 804 and a user interface 803. The various components in the electronic device 800 are coupled together via a bus system 805. It is understood that the bus system 805 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 805 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 805 is not shown in FIG. Figure 8 Various buses are labeled as bus system 805 .

[0109] The user interface 803 may include a display, a keyboard, a mouse, a trackball, a click wheel, keys, buttons, a touch pad or a touch screen.

[0110] It can be understood that the memory 802 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 802 described in the embodiments of the present application is intended to include but is not limited to these and any other suitable types of memories.

[0111] The memory 802 in the embodiment of the present application is used to store various types of data to support the operation of the electronic device 800. Examples of these data include: any computer program for operating on the electronic device 800, such as an operating system 8021 and an application 8022; contact data; phone book data; messages; pictures; videos, etc. Among them, the operating system 8021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application 8022 can include various applications, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The program that implements the method of the embodiment of the present application can be included in the application 8022.

[0112] The method disclosed in the above embodiment of the present application can be applied to the processor 801, or implemented by the processor 801. The processor 801 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 801. The above processor 801 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 801 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 802, and the processor 801 reads the information in the memory 802 and completes the steps of the above method in combination with its hardware.

[0113] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to execute the aforementioned method.

[0114] In an exemplary embodiment, the present application also provides a computer-readable storage medium, such as a memory 802 including a computer program, and the computer program can be executed by a processor 801 of an electronic device 800 to complete the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disk, or CD-ROM; it can also be various devices including one or any combination of the above memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc.

[0115] A computer-readable storage medium stores a computer program, which, when executed by a processor, executes any method step in the above-mentioned data processing method.

[0116] Fig. 9 Schematic diagram of the structure of the data processing system in this application, such as Fig. 9 As shown, the data processing system includes:

[0117] A transparent display screen 91 has a first surface and a second surface, wherein the first surface and / or the second surface is a touch surface;

[0118] Here, a touch detection component 911 is further provided in the direction of the touch surface of the transparent display screen 91, which is used to collect touch information corresponding to the touch operation when the user performs a touch operation on the touch surface.

[0119] An image acquisition component 92, located on at least one side of the transparent display screen 91, for acquiring an image of the target object A to obtain depth image information of the target object A;

[0120] Here, the target object A may be located between the image acquisition component 92 and the transparent display screen 91 , and the depth image information of the target object A is represented by a dotted line in the figure.

[0121] The host 93 is connected to the image acquisition component 91 and the touch detection component 911 on the transparent display screen 91, and is used to receive the depth image information acquired by the image acquisition component 92 for the target object A, and receive the touch information sent by the touch detection component 911 for the transparent display screen 91; and is used to establish a mapping relationship between the touch information and the target object A based on the depth image information and the touch information, and control the touch information to be displayed on the transparent display screen 91 based on the mapping relationship, so that the touch information can move with the movement of the target object A.

[0122] The host 93 is equivalent to the electronic device described above. The data processing system and the data processing method embodiment provided above belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0123] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0124] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0125] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0126] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0127] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0128] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data processing method, the method comprising: Obtaining depth image information of the target object and information generated by the touch-sensitive transparent display screen; Establishing a mapping relationship between the information generated by the touch-controlled transparent display screen and the target object based on the depth image information and the coordinate position of the information generated by the touch-controlled transparent display screen; Controlling information generated by the touch-controlled transparent display screen to be displayed on the transparent display screen based on the mapping relationship; The information generated by the touch-sensitive transparent display screen is controlled to be combined with the target object and to move in the touch-sensitive display area following the target object.

2. The method according to claim 1, wherein: The step of obtaining the depth image information of the target object and the information generated by the touch-controlled transparent display screen on the transparent display screen includes: Acquire depth image information of the target object collected by an image collection component in a first surface direction of the transparent display screen, wherein the target object is located between the image collection component and the transparent display screen; The information generated by the first touch-sensitive transparent display screen collected by the touch detection component in the direction of the second surface of the transparent display screen is acquired; wherein the second surface and the first surface are arranged opposite to each other.

3. The method according to claim 1, wherein: The step of obtaining the depth image information of the target object and the information generated by the touch-controlled transparent display screen on the transparent display screen includes: Acquire depth image information collected by the image acquisition component for the target object in the direction of the second surface of the transparent display screen, wherein the target object is located in the direction of the first surface of the transparent display screen; wherein the second surface and the first surface are arranged opposite to each other; The information generated by the first touch-sensitive transparent display screen collected by the touch detection component in the direction of the second surface of the transparent display screen is acquired.

4. The method according to claim 2 or 3, wherein: The method further comprises: Acquire information generated by a second touch-sensitive transparent display screen collected by the touch detection component in the direction of the first surface of the transparent display screen; wherein display modes of the information generated by the first touch-sensitive transparent display screen and the information generated by the second touch-sensitive transparent display screen are the same or different.

5. The method according to claim 1, wherein: The step of obtaining the depth image information of the target object and the information generated by the touch-controlled transparent display screen at least includes: At a first moment, information generated by a touch-controlled transparent display screen on the transparent display screen is obtained; at a second moment, depth image information of a target object is obtained; wherein the first moment is less than the second moment; or, the first moment is greater than the second moment.

6. The method according to claim 1, wherein: The establishing a mapping relationship between the information generated by the touch-controlled transparent display screen and the target object based on the depth image information and the coordinate position of the information generated by the touch-controlled transparent display screen includes: Determine the first coordinate position of the information generated by the touch-controlled transparent display screen on the transparent display screen based on the regional measurement data of the transparent display screen by the touch detection component; and determining a second coordinate position of the depth image information on the transparent display screen based on a spatial position relationship between the image acquisition component and the transparent display screen; A mapping relationship between the information generated by the touch-sensitive transparent display screen and the target object is established based on the first coordinate position and the second coordinate position.

7. The method according to claim 1, wherein: The method further comprises: Detecting the behavior of the target object; If the behavior action satisfies the action condition, the current display mode of the information generated by the touch-sensitive transparent display screen is switched based on the behavior action.

8. The method according to claim 1, wherein: The method further comprises: Receiving a mode switching instruction for information generated by the touch-controlled transparent display screen; The current display mode of the information generated by the touch-sensitive transparent display screen is switched based on the mode switching instruction.

9. An electronic device, comprising: An acquisition unit, used to acquire depth image information of a target object and information generated by a touch-controlled transparent display screen on the transparent display screen; An establishing unit, configured to establish a mapping relationship between the information generated by the touch-controlled transparent display screen and the target object based on the depth image information and the coordinate position of the information generated by the touch-controlled transparent display screen; A display unit, configured to control information generated by the touch-controlled transparent display screen to be displayed on the transparent display screen based on the mapping relationship; The information generated by the touch-sensitive transparent display screen is controlled to be combined with the target object and to move in the touch-sensitive display area following the target object.

Images