Method, device, equipment and readable storage medium for interacting with digital content

Abstract

The present application discloses a method, device, equipment and readable storage medium for interacting with digital content, the method comprising: collecting a depth image of a user through a depth camera, the depth camera being set in a first direction of the user; calculating the position of a human shadow formed by the user on the projection screen according to the depth image and the positional relationship between the depth camera and the projection screen, and the positional relationship between the projection screen and the projector; the projector is set in the first direction of the user, the projection screen is set in the second direction of the user, and the first direction is opposite to the second direction; forming a projection screen image according to the position of the human shadow; the projection screen image includes a human shadow image; identifying the first content and triggering action in the projection screen image; generating the second content according to the first content and the triggering action, and projecting the second content to the projection screen. By implementing the present invention, when the user forms a human shadow on the projection screen, the user can interact with the digital content based on the human shadow image to improve the user experience.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, equipment and readable storage medium for interacting with digital content. Background Art

[0002] If a user stands in front of a projection screen, a shadow will be formed on the projection screen due to the user blocking the projection light, which will block the content at the shadow position, reducing the viewing experience of other users of the content at the shadow position. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a method, device, equipment and readable storage medium for interacting with digital content, so as to solve the problem in the prior art that shadows may block the content on the projection screen, thereby reducing the viewing experience of other users for the content in the shadow position.

[0004] In order to solve the above problems, on the first aspect, an embodiment of the present application provides a method for interacting with digital content, including: collecting a depth image of a user through a depth camera, the depth camera being set in a first direction of the user; calculating the position of a human shadow formed by the user on the projection screen according to the depth image and the positional relationship between the depth camera and the projection screen, and the positional relationship between the projection screen and the projector; the projector is set in the first direction of the user, and the projection screen is set in the second direction of the user, the first direction being opposite to the second direction; forming a projection screen image according to the position of the human shadow; the projection screen image includes a human shadow image; identifying first content and a trigger action in the projection screen image; generating second content according to the first content and the trigger action, and projecting the second content onto the projection screen.

[0005] Optionally, the position of the shadow of the user formed on the projection screen is calculated based on the depth image and the positional relationship between the depth camera and the projection screen, and the positional relationship between the projection screen and the projector, including: calculating a first position of the user's outline based on the depth image; converting the first position into a second position with the projector as the origin of the coordinate system based on the positional relationship between the depth camera and the projection screen, and the positional relationship between the projection screen and the projector; and determining the position of the shadow of the user formed on the projection screen based on the intersection of the line connecting the projector and the second position with the projection screen.

[0006] Optionally, identifying a trigger action in the projection screen image includes: identifying a trigger action in the projection screen image when a collision is detected between the first digital content in the first content and the human shadow image; or identifying a trigger action in the projection screen image when a target action is detected between the human shadow image and the projection screen image.

[0007] Optionally, detecting that the first digital content in the first content collides with the human shadow image includes: identifying the position of the first digital content in the first content; and confirming that the first digital content in the first content collides with the human shadow image when it is detected that the position of the first digital content and the position of the human shadow image have an intersection.

[0008] Optionally, generating the second content according to the first content and the trigger action includes: performing a first operation on the first digital content having an intersection with the human shadow image in the first content according to the trigger action; and generating the second content according to the first digital content after performing the first operation and the remaining first digital content.

[0009] Optionally, detecting that the human shadow image has performed a target action includes: identifying a posture formed by a target part of the human shadow image; and confirming that the human shadow image has performed the target action when the posture is detected to be the target posture.

[0010] Optionally, generating the second content according to the first content and the trigger action includes: confirming the position of the target part according to the first content; and generating the second content corresponding to the trigger action at the position of the target part.

[0011] In a second aspect, an embodiment of the present application provides a device for interacting with digital content, including: a collection unit, used to collect a depth image of a user through a depth camera, the depth camera being arranged in a first direction of the user; a calculation unit, used to calculate the position of a human shadow formed by the user on the projection screen according to the depth image and the positional relationship between the depth camera and the projection screen, and the positional relationship between the projection screen and the projector; the projector is arranged in the first direction of the user, and the projection screen is arranged in the second direction of the user, the first direction being opposite to the second direction; a formation unit, used to form a projection screen image according to the position of the human shadow; the projection screen image includes a human shadow image; an identification unit, used to identify first content and a trigger action in the projection screen image; a generation unit, used to generate second content according to the first content and the trigger action, and project the second content onto the projection screen.

[0012] In a third aspect, an embodiment of the present application provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes a method for interacting with digital content as in the first aspect or any embodiment of the first aspect.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute a method for interacting with digital content as in the first aspect or any embodiment of the first aspect.

[0014] The method, apparatus, device and readable storage medium for interacting with digital content provided by the embodiments of the present application collect a depth image of a user through a depth camera, and the depth camera is set in a first direction of the user; the position of the human shadow formed by the user on the projection screen is calculated according to the depth image and the positional relationship between the depth camera and the projection screen, and the positional relationship between the projection screen and the projector; the projector is set in the first direction of the user, and the projection screen is set in the second direction of the user, and the first direction is opposite to the second direction; a projection screen image is formed according to the position of the human shadow; the projection screen image includes a human shadow image; first content and a trigger action in the projection screen image are identified; second content is generated according to the first content and the trigger action, and the second content is projected onto the projection screen; so that when the user forms a human shadow on the projection screen, a human shadow image can be formed based on the human shadow, and then the human shadow image can be used to interact with the digital content on the projection screen, so that when the human shadow blocks the content on the projection screen, the blocked content can be directly pushed out from the human shadow position, so that other users will not be able to view the content at the human shadow position, and from the user's perspective, it is as if the human shadow can be directly used to interact with the digital content, thereby improving the user experience.

[0015] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of a method for interacting with digital content in an embodiment of the present application;

[0017] Figure 2 A schematic diagram of a scene in which a user's shadow interacts with digital content on a screen in an embodiment of the present application;

[0018] Figure 3 A schematic diagram of the structure of a device for interacting with digital content in an embodiment of the present application;

[0019] Figure 4 This is a schematic diagram of the hardware structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0021] The embodiment of the present application provides a method for interacting with digital content, which can be applied to a scenario in which a user's shadow interacts with digital content on a screen, such as Figure 1 As shown, methods for interacting with digital content include:

[0022] S101. Collect a depth image of the user through a depth camera, where the depth camera is set in a first direction of the user.

[0023] In this embodiment, the first direction is the direction in which the user is facing away, that is, the rear of the user. The depth camera is a 3D camera, which is set behind the user. The depth camera can collect the depth image of the user within the preset spatial range. When the user needs to interact with the digital content on the projection screen, the depth camera needs to be turned on to collect the depth image of the user. When there are one or more users within the preset spatial range, the depth image of the one or more users can be collected. For example, when multiple users stand in front of the projection screen, the depth camera can collect the depth images of multiple users.

[0024] Depth cameras can capture depth data in a scene in real time, greatly promoting the development of visual recognition, such as human posture estimation and human action recognition. Depth data is three-dimensional data, including depth data stream and skeleton data stream. Therefore, the depth camera can be used to collect the depth image of the user, and then by analyzing the first depth image, the contour, skeleton, gesture and other data of each user can be obtained.

[0025] S102. Calculate the position of the user's shadow on the projection screen based on the depth image and the positional relationship between the depth camera and the projection screen, and the positional relationship between the projection screen and the projector; the projector is set in a first direction of the user, and the projection screen is set in a second direction of the user, and the first direction is opposite to the second direction.

[0026] In this embodiment, the user's outline can be calculated based on the depth image. The second direction is the direction the user is facing, that is, the front of the user. Since part of the light emitted by the projector is projected onto the user, a shadow of the user is formed on the projection screen. Figure 2As shown, the user stands in front of the projection screen, facing the projection screen and facing away from the projector, so that the light emitted by the projector is partially projected onto the user, and a shadow of the user is formed on the projection screen. The projection screen is a large-size display or a projection curtain wall.

[0027] When calculating the position of the user's shadow on the projection screen, it is considered that part of the light emitted by the projector is projected onto the user, forming the user's shadow on the projection screen. Therefore, the projector (equivalent to the light source), the user's outline, and the projection screen need to be in the same coordinate system to calculate the position of the user's shadow on the projection screen. The depth image, projector, and projection screen are in different coordinate systems. Therefore, it is necessary to transform the position of the outline of the human figure in the depth image, and calculate the distance between the projection screen and the projector. Therefore, it is necessary to calibrate the position relationship between the depth camera and the projection screen, and the position relationship between the projection screen and the projector in advance. The position relationship includes the position transformation relationship, distance, etc.

[0028] S103. Form a projection screen image according to the position of the human shadow; the projection screen image includes the human shadow image.

[0029] In this embodiment, the size of the projection screen can be preset. If the content to be projected onto the projection screen is empty, a projection screen image can be formed according to the position of the user's shadow on the projection screen and the size of the projection screen, so that the projection screen image only has a shadow image. If the content to be projected onto the projection screen is the first digital content, the position of the first digital content to be displayed on the projection screen can be determined according to the configuration parameters of the projection screen and the built-in display screen of the computer, and then the projection screen image is formed according to the position of the first digital content to be displayed on the projection screen, the position of the user's shadow on the projection screen, and the size of the projection screen. Thus, the projection screen image includes a shadow image and the first digital content. The shadow image is digital information, but it completely overlaps with the shape of a real shadow, so it can be analyzed with digital interface elements based on the shadow image.

[0030] S104. Identify the first content and trigger action in the projection screen image.

[0031] In this embodiment, the trigger action can be a trigger action of the interaction between the human shadow image and the first digital content, or a target action performed by the human shadow image alone, or a target action performed by the first digital content alone. Recognizing the first content in the projection screen image is to recognize the first digital content in the projection screen image and the specific content in the human shadow image. For example, recognize the specific components included in the human shadow image and the positions of each component; recognize the objects, types of objects and positions of objects in the first digital content, etc. Recognizing the trigger action in the projection screen image is to recognize the posture of the human shadow image or the state of the human shadow image and the first digital content. After recognizing the trigger action in the projection screen image, the first content can be operated accordingly.

[0032] S105. Generate second content according to the first content and the trigger action, and project the second content onto a projection screen.

[0033] In this embodiment, the content corresponding to the trigger action can be preset. For example, if the user makes a shooting action, the content corresponding to the shooting action can be to fire a virtual bullet at the position of the hand in the user's figure; if the user's figure touches a bullet or a three-dimensional object or other content, the content corresponding to the touch action can be to bounce off the bullet or three-dimensional object or other content that touches the user or the user's figure. Figure 2 As shown, when the user's shadow image hits the small ball, the small ball is bounced away, and the bounced small ball returns along the original path, while the remaining small balls remain in the falling state. Then, the second content can be generated based on the bounced small ball and the remaining small balls, and then the second content is projected onto the projection screen. From the user's perspective, it is as if the actual human shadow interacts with the small ball.

[0034] When the second content is generated according to the first content and the trigger action, if the trigger action is a trigger action for the human shadow image to interact with the first digital content, the first digital content in the first content can be operated to obtain the second content; if the trigger action is a target action performed by the human shadow image alone, the second digital content can be generated according to the target action to obtain the second content, and then the second content is projected onto the projection screen. Therefore, from the user's perspective, it is as if his own shadow can directly interact with the digital content.

[0035] The method for interacting with digital content provided in the embodiment of the present application collects a depth image of a user through a depth camera, and the depth camera is set in a first direction of the user; the position of the human shadow formed by the user on the projection screen is calculated according to the depth image and the positional relationship between the depth camera and the projection screen, and the positional relationship between the projection screen and the projector; the projector is set in the first direction of the user, and the projection screen is set in the second direction of the user, and the first direction is opposite to the second direction; a projection screen image is formed according to the position of the human shadow; the projection screen image includes a human shadow image; first content and a trigger action in the projection screen image are identified; second content is generated according to the first content and the trigger action, and the second content is projected onto the projection screen; so that when the user forms a human shadow on the projection screen, a human shadow image can be formed based on the human shadow, and then the human shadow image can be used to interact with the digital content on the projection screen, so that when the human shadow blocks the content on the projection screen, the blocked content can be directly pushed out from the human shadow position, so that other users will not be able to view the content at the human shadow position, and from the user's perspective, it is as if the human shadow can be directly used to interact with the digital content, thereby improving the user experience.

[0036] In an optional embodiment, step S102 calculates the position of the shadow of the user formed on the projection screen according to the depth image and the positional relationship between the depth camera and the projection screen, and the positional relationship between the projection screen and the projector, specifically including: calculating a first position of the user's outline according to the depth image; converting the first position into a second position with the projector as the origin of the coordinate system according to the positional relationship between the depth camera and the projection screen, and the positional relationship between the projection screen and the projector; determining the position of the shadow of the user formed on the projection screen based on the intersection of the line connecting the projector and the second position with the projection screen.

[0037] Specifically, the first position of the user's outline can be calculated based on the depth image, and the first position is the position with the depth camera as the origin of the coordinate system. Since the position conversion relationship between the depth camera and the projection screen and the position conversion relationship between the projection screen and the projector are calibrated, the first position can be first converted to a position with the center point of the projection screen as the origin of the coordinate system according to the position conversion relationship between the depth camera and the projection screen, and then the position with the center point of the projection screen as the origin of the coordinate system is converted to a second position with the projector as the origin of the coordinate system according to the position conversion relationship between the projection screen and the projector. Then, with the projector as the origin, a connecting line is drawn between the origin and the first position, and the intersection of the extended line and the projection screen is the position of the user's shadow on the projection screen. This position is converted to a position with the center point of the projection screen as the origin of the coordinate system, and the position of the user's shadow on the projection screen is obtained.

[0038] In this embodiment, since the depth image can recognize the outline of the user, the first position is converted into a second position with the projector as the origin of the coordinate system according to the positional relationship between the depth camera and the projection screen, and the positional relationship between the projection screen and the projector. The position of the user's shadow formed on the projection screen is determined based on the intersection of the line connecting the projector and the second position with the projection screen. The position of the shadow is determined based on the principle of shadow formation, and the position of the user's shadow on the projection screen can be accurately calculated, so that the formed shadow image can be completely overlapped with the real shadow, so that the interaction based on the shadow and the digital machine content is more vivid and specific.

[0039] In an optional embodiment, in step S104, identifying a trigger action in the projection screen image includes: identifying a trigger action in the projection screen image when a collision is detected between the first digital content in the first content and the human shadow image; or identifying a trigger action in the projection screen image when a target action is detected between the human shadow image and the projection screen image.

[0040] Specifically, the trigger action can be actively performed by the human shadow image or by the digital content. If the human shadow image performs the target action and does not collide with the first digital content, the trigger action in the projection screen image is recognized; if the human shadow image performs the target action and collides with the first digital content, the trigger action in the projection screen image is recognized; if the human shadow image does not move and the first digital content collides with the human shadow image, the trigger action in the projection screen image is recognized.

[0041] Since the interaction between the human shadow image and the digital content is required, the triggering action in the projection screen image can be identified by performing collision detection between the first digital content in the first content and the human shadow image or detecting the target action of the human shadow image.

[0042] In an optional embodiment, detecting that the first digital content in the first content collides with the human shadow image includes: identifying the position of the first digital content in the first content; and confirming that the first digital content in the first content collides with the human shadow image when it is detected that the position of the first digital content and the position of the human shadow image have an intersection.

[0043] Specifically, the first digital content in the projection screen image and the specific content in the human shadow image are identified. For example, the specific components included in the human shadow image and the positions of the components are identified; the objects, types of objects and positions of objects in the first digital content are identified. If the first digital content collides with the human shadow image, there is an intersection between the positions of the first digital content and the human shadow image, such as Therefore, in the process of detecting that the first digital content in the first content collides with the human shadow image, the positions of the first digital content and the human shadow image can be compared. When it is determined that the positions of the first digital content and the human shadow image have an intersection, it is confirmed that the first digital content in the first content collides with the human shadow image.

[0044] In this embodiment, since the positions of the first digital content and the human shadow image are known, when detecting whether the first digital content and the human shadow image collide, the positions of the first digital content and the human shadow image are compared, so as to quickly and accurately determine whether the first digital content and the human shadow image collide.

[0045] In an optional embodiment, in step S105, generating the second content according to the first content and the trigger action includes: performing a first operation on the first digital content having an intersection with the human shadow image in the first content according to the trigger action; and generating the second content according to the first digital content after performing the first operation and the remaining first digital content.

[0046] Specifically, the first operation may be to bounce, push, etc. If the first digital content in the first content collides with the human shadow image, the first digital content that collides with the human shadow image of the user or the human shadow image of the user may be bounced. After the first digital content that collides with the human shadow image of the user or the human shadow image of the user is bounced, the bounced first digital content forms a second state, for example, moving in a direction opposite to the original direction of movement. The first digital content after the first operation and the remaining first digital content generate the second content. Figure 2 As shown, if the falling ball hits the user's shadow image, the shadow image will bounce the ball away, and the bounced ball will return along the original path, and the remaining balls will continue to fall.

[0047] In this embodiment, a first operation is performed on the first digital content having an intersection with the human shadow image in the first content according to a trigger action, so that the human shadow image interacts with the first digital content, and second content is generated according to the first digital content after the first operation and the remaining first digital content. From the user's perspective, it is as if his or her actual human shadow can interact with the first digital content, thereby improving the user experience.

[0048] In an optional embodiment, detecting that the human shadow image has performed a target action includes: identifying a posture formed by a target part of the human shadow image; and confirming that the human shadow image has performed the target action when the posture is detected to be the target posture.

[0049] Specifically, if the human shadow image does not collide with the first digital content, or the projection screen image does not include the first digital content, the posture formed by the target part of the human shadow image can be identified. The target part can be a hand, an arm, a foot, etc. Since a certain posture is formed when the target action is performed, the target action in the human shadow image can be accurately identified by identifying the posture formed by the target part.

[0050] In an optional embodiment, in step S105, generating the second content according to the first content and the trigger action includes: confirming the position of the target part according to the first content; and generating the second content corresponding to the trigger action at the position of the target part.

[0051] Specifically, since the first digital content in the projection screen image and the specific content in the human figure image are identified, for example, the specific components included in the human figure image and the positions of the components are identified. Therefore, the position of the target part can be determined according to the first content. Since the target part performs the target action, the second operation should be performed on the target part, and the second content corresponding to the triggering action is generated at the position of the target part. For example, if the user's hand makes a shooting gesture, a virtual bullet can be generated at the position of the hand in the user's human figure image.

[0052] In this embodiment, by confirming the position of the target part according to the first content, the second content corresponding to the trigger action is generated at the position of the target part, so that the second content corresponding to the target action of the target part of the human shadow image can be generated, thereby realizing interaction with digital content based on the human shadow image. From the user's perspective, it is as if the second content can be generated at his or her actual human shadow, thereby improving the user experience.

[0053] The present application also provides a device for interacting with digital content, which can be applied to a scene based on the interaction between the user's shadow and the digital content on the screen, such as Figure 3 As shown, the devices for interacting with digital content include:

[0054] The acquisition unit 21 is used to acquire a depth image of the user through a depth camera, and the depth camera is set in a first direction of the user; the specific implementation method is detailed in the description of step S101 of the above method embodiment, which will not be repeated here.

[0055] The calculation unit 22 is used to calculate the position of the user's shadow formed on the projection screen according to the depth image and the positional relationship between the depth camera and the projection screen, and the positional relationship between the projection screen and the projector; the projector is set in a first direction of the user, and the projection screen is set in a second direction of the user, and the first direction is opposite to the second direction; the specific implementation method is detailed in the description of step S102 of the above method embodiment, which will not be repeated here.

[0056] The forming unit 23 is used to form a projection screen image according to the position of the human shadow; the projection screen image includes the human shadow image; the specific implementation method is detailed in the description of step S103 of the above method embodiment, which will not be repeated here.

[0057] The identification unit 24 is used to identify the first content in the projection screen image and trigger the action; the specific implementation method is detailed in the description of step S104 of the above method embodiment, which will not be repeated here.

[0058] The generating unit 25 is used to generate the second content according to the first content and the triggering action, and project the second content onto the projection screen. The specific implementation method is described in the description of step S105 of the above method embodiment, which will not be repeated here.

[0059] The device for interacting with digital content provided by the embodiment of the present application collects a depth image of a user through a depth camera, and the depth camera is set in a first direction of the user; the position of the human shadow formed by the user on the projection screen is calculated according to the depth image and the positional relationship between the depth camera and the projection screen, and the positional relationship between the projection screen and the projector; the projector is set in the first direction of the user, and the projection screen is set in the second direction of the user, and the first direction is opposite to the second direction; a projection screen image is formed according to the position of the human shadow; the projection screen image includes a human shadow image; first content and a trigger action in the projection screen image are identified; second content is generated according to the first content and the trigger action, and the second content is projected onto the projection screen; so that when the user forms a human shadow on the projection screen, a human shadow image can be formed based on the human shadow, and then the human shadow image can be used to interact with the digital content on the projection screen, so that when the human shadow blocks the content on the projection screen, the blocked content can be directly pushed out from the human shadow position, so that other users will not be able to view the content at the human shadow position, and from the user's perspective, it is as if the human shadow can be directly used to interact with the digital content, thereby improving the user experience.

[0060] Based on the same inventive concept as the one for interacting with digital content in the aforementioned embodiment, the embodiment of the present application further provides an electronic device, such as Figure 4 As shown, it includes: a processor 31 and a memory 32, wherein the processor 31 and the memory 32 can be connected via a bus or other means. Figure 4 The bus connection is used as an example for explanation.

[0061] The processor 31 may be a central processing unit (CPU). The processor 31 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0062] The memory 32 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the interaction with digital content in the embodiment of the present application. The processor 31 executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory 32, that is, to realize the interaction with digital content in the above method embodiment.

[0063] The memory 32 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created by the processor 31, etc. In addition, the memory 32 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 32 may optionally include a memory remotely arranged relative to the processor 31, and these remote memories may be connected to the processor 31 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0064] The above one or more modules are stored in the memory 32, and when executed by the processor 31, the following is performed: Figure 1 Interacting with digital content in the illustrated embodiment.

[0065] For details of the above electronic equipment, please refer to Figure 1 The corresponding related descriptions and effects in the illustrated embodiments can be understood and will not be repeated here.

[0066] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0067] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0068] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable information processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable information processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0069] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable information processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0070] These computer program instructions can also be loaded onto a computer or other programmable information processing device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0071] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for interacting with digital content, comprising: Acquire a depth image of the user by using a depth camera, wherein the depth camera is arranged in a first direction of the user; Calculate the position of the shadow formed by the user on the projection screen according to the depth image, the positional relationship between the depth camera and the projection screen, and the positional relationship between the projection screen and the projector; The projector is arranged in a first direction of the user, and the projection screen is arranged in a second direction of the user, wherein the first direction is opposite to the second direction; forming a projection screen image according to the position of the human shadow; the projection screen image includes the human shadow image; Identifying first content in the projection screen image and triggering an action; generating second content according to the first content and the trigger action, and projecting the second content onto the projection screen; Wherein, identifying the trigger action in the projection screen image includes: In the case where it is detected that the first digital content in the first content collides with the human shadow image, a triggering action in the projection screen image is identified; or, When it is detected that the human shadow image performs the target action, a triggering action in the projection screen image is identified.

2. The method for interacting with digital content according to claim 1, wherein the step of calculating the position of the shadow formed by the user on the projection screen according to the depth image and the positional relationship between the depth camera and the projection screen, and the positional relationship between the projection screen and the projector comprises: Calculating a first position of a contour of the user based on the depth image; Converting the first position into a second position with the projector as the origin of the coordinate system according to the positional relationship between the depth camera and the projection screen and the positional relationship between the projection screen and the projector; Based on the intersection of a line connecting the projector and the second position with the projection screen, the position of the shadow formed by the user on the projection screen is determined.

3. The method for interacting with digital content according to claim 1, wherein the step of detecting that the first digital content in the first content collides with the human shadow image comprises: identifying a location of first digital content in the first content; When it is detected that the position of the first digital content and the position of the human shadow image have an intersection, it is confirmed that the first digital content in the first content collides with the human shadow image.

4. The method for interacting with digital content according to claim 3, wherein generating the second content according to the first content and the triggering action comprises: Performing a first operation on first digital content in the first content that has an intersection with the human shadow image according to the trigger action; The second content is generated according to the first digital content after the first operation is performed and the remaining first digital content.

5. The method for interacting with digital content according to claim 1, wherein detecting that the human shadow image performs a target action comprises: Recognizing the posture formed by the target part of the human figure image; When the posture is detected as the target posture, it is confirmed that the human shadow image performs the target action.

6. The method for interacting with digital content according to claim 5, wherein generating the second content according to the first content and the triggering action comprises: confirming the location of the target part according to the first content; A second content corresponding to the trigger action is generated at the location of the target part.

7. A device for interacting with digital content, comprising: A collection unit, configured to collect a depth image of a user through a depth camera, wherein the depth camera is arranged in a first direction of the user; a calculation unit, configured to calculate the position of the shadow formed by the user on the projection screen according to the depth image and the positional relationship between the depth camera and the projection screen, and the positional relationship between the projection screen and the projector; the projector is arranged in a first direction of the user, the projection screen is arranged in a second direction of the user, and the first direction is opposite to the second direction; A forming unit, configured to form a projection screen image according to the position of the human shadow; the projection screen image includes a human shadow image; an identification unit, configured to identify first content and a triggering action in the projection screen image; a generating unit, configured to generate second content according to the first content and the triggering action, and project the second content onto the projection screen; The recognition unit is further configured to recognize a triggering action in the projection screen image when a collision between the first digital content in the first content and the human shadow image is detected; or, When it is detected that the human shadow image performs the target action, a triggering action in the projection screen image is identified.

8. An electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the method for interacting with digital content as described in any one of claims 1-6.

9. A computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method for interacting with digital content as described in any one of claims 1 to 6.

Images