Interaction processing method and electronic equipment

Abstract

The invention provides an interactive processing method and electronic equipment, and relates to the technical field of computers. The interaction processing method comprises the steps that first information and second information of a target object are acquired, the first information is derived from an optical sensor, and the second information is derived from an inertial sensor arranged on the target object; responding to the condition that the first information does not meet the target condition, entering a first mode which represents a mode in which the second information is used as dominant processing information; in the first mode, processing the second information to obtain first position information of the target object; and adjusting position information and / or display parameters of a virtual object interacting with the target object in the three-dimensional image based on the first position information.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to an interactive processing method and an electronic device. Background Art

[0002] In naked-eye three-dimensional (3D) technology, users observe virtual objects with their naked eyes and perform interactive operations. However, due to factors such as device quality and lighting conditions, users often have spatial perception errors when judging information such as the position and depth of virtual objects. In addition, due to the lack of visual guidance and feedback mechanisms, users do not have a strong sense of immersion, which affects the interactive experience. Summary of the invention

[0003] The present disclosure provides an interactive processing method and an electronic device to at least solve the above technical problems existing in the prior art.

[0004] According to a first aspect of the present disclosure, there is provided an interaction processing method, comprising: acquiring first information and second information of a target object, the first information being derived from an optical sensor, and the second information being derived from an inertial sensor disposed on the target object; in response to the first information not satisfying a target condition, entering a first mode, the first mode representing a mode in which the second information is used as dominant processing information; in the first mode, processing the second information to obtain first position information of the target object; and adjusting position information and / or display parameters of a virtual object in a three-dimensional image that interacts with the target object based on the first position information.

[0005] In one possible implementation, the response to the first information not satisfying the target condition includes: responding to the first information not containing the target object; and / or responding to the optical sensor that collects the first information sending sensor abnormality information.

[0006] In one possible implementation manner, entering the first mode includes: correcting the second information; adjusting the weight of the second information to a target threshold, and adjusting the weight of the first information to zero.

[0007] In one possible implementation, an interaction processing method also includes: in response to the first information satisfying a target condition, entering a second mode, the second mode representing a mode of jointly processing the first information and the second information; in the second mode, processing the second information to obtain the first position information of the target object, and processing the first information to obtain the second position information of the target object; fusing the first position information and the second position information to obtain the third position information of the target object; and adjusting the position information and / or display parameters of a virtual object in a three-dimensional image that interacts with the target object based on the third position information.

[0008] In one possible implementation manner, the fusing the first position information and the second position information to obtain the third position information of the target object includes: based on a weighted average algorithm or a Kalman filter algorithm, fusing the first position information and the second position information to obtain the third position information of the target object.

[0009] In one possible implementation, the adjusting of the position information and / or display parameters of a virtual object interacting with the target object in a three-dimensional image based on the first position information includes: adjusting the position information of the virtual object based on the first position information, so that the distance between the adjusted position information of the virtual object and the first position information satisfies a first range; and / or, in response to the distance between the first position information and the position information of the virtual object satisfying a second range, adjusting at least one of the color parameter, size parameter, brightness parameter and texture parameter of the virtual object to a parameter threshold in a visual feedback rule.

[0010] In one possible implementation, after processing the second information to obtain the first position information of the target object, it further includes: based on the first position information, generating a virtual guide image indicating the first position information and a moving path of the virtual guide image in the three-dimensional image.

[0011] In one possible implementation, an interaction processing method further includes: in response to the distance between the first position information and the position information of the virtual object satisfying a third range, adjusting the position information of the virtual guide image until the virtual guide image and the virtual object satisfy a target position relationship.

[0012] In one possible implementation, an interaction processing method further includes: in response to a distance between the first position information and the position information of the virtual object satisfying a second range, controlling a tactile feedback device disposed on the target object to perform tactile feedback.

[0013] According to a second aspect of the present disclosure, there is provided an electronic device, including:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present disclosure.

[0017] According to a third aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the present disclosure.

[0018] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:

[0020] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0021] Figure 1 A schematic diagram showing a flow diagram of an interactive processing method according to an embodiment of the present disclosure Figure 1 ;

[0022] Figure 2 A schematic diagram showing a flow diagram of an interactive processing method according to an embodiment of the present disclosure Figure 2 ;

[0023] Figure 3 A schematic diagram showing a flow diagram of an interactive processing method according to an embodiment of the present disclosure Figure 3 ;

[0024] Figure 4 A schematic diagram showing a flow diagram of an interactive processing method according to an embodiment of the present disclosure Figure 4 ;

[0025] Figure 5 A schematic diagram showing a flow diagram of an interactive processing method according to an embodiment of the present disclosure Figure 5 ;

[0026] Figure 6A schematic diagram showing a scenario of an interactive processing method according to an embodiment of the present disclosure Figure 1 ;

[0027] Figure 7 A schematic diagram showing a scenario of an interactive processing method according to an embodiment of the present disclosure Figure 2 ;

[0028] Figure 8 A schematic diagram showing a scenario of an interactive processing method according to an embodiment of the present disclosure Figure 3 ;

[0029] Fig. 9 A schematic diagram showing a scenario of an interactive processing method according to an embodiment of the present disclosure Figure 4 ;

[0030] Fig.10 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0031] In order to make the purpose, features, and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0032] Figure 1 A schematic diagram showing a flow diagram of an interactive processing method according to an embodiment of the present disclosure Figure 1 ,like Figure 1 As shown, an interaction processing method includes:

[0033] Step S101, obtaining first information and second information of a target object.

[0034] In this embodiment, the target object is an object that interacts with the virtual object. The target object may be a part of a human body or a movable object. For example, if a user interacts with the virtual object through a finger, the target object is the user's finger. The first information is information about the target object collected by an optical sensor. The optical sensor may be an RGB camera and an infrared sensor, etc. The first information includes a picture of the target object or infrared information, etc. The optical sensor is usually arranged at a position where the target object can be photographed. The second information is information about the target object collected by an inertial sensor arranged on the target object. The second information includes information such as acceleration and angular velocity of the target object. After processing, the first information and the second information can both obtain position information of the target object.

[0035] Step S102: In response to the first information not satisfying the target condition, entering the first mode.

[0036] In this embodiment, the target condition indicates that the position information of the target object can be obtained after the first information is processed. If the first information does not meet the target condition, that is, the position information of the target object cannot be obtained after the first information is processed, the first mode is entered. The first mode indicates a mode in which the second information is used as the dominant processing information. In the first mode, the position information of the target object is obtained only based on the second information collected by the inertial sensor.

[0037] Step S103: in the first mode, the second information is processed to obtain first position information of the target object.

[0038] In this embodiment, in the first mode, the angular change of the target object in each axis can be determined based on the angular velocity in the second information, the displacement of the target object in each axis can be determined based on the acceleration in the second information, and the first position information of the target object can be determined based on the angular change and displacement in each axis. It should be emphasized that the first position information of the target object is the position information of the target object in the naked eye 3D space where the naked eye 3D image is located.

[0039] In one embodiment, the first position information of the target object can also be corrected based on the known patterns and constraints of the target object to further reduce the cumulative error, wherein the known patterns are the various postures involved in the target object's interaction with the virtual object, such as the various gestures of the user's hands during the interaction, and the constraints are the range of movement of the target object during the interaction with the virtual object, such as the height limit of the user's hand during the interaction.

[0040] In one embodiment, a machine learning algorithm can also be used to assist in correcting the first position information of the target object. First, information of target samples acquired by a large number of inertial sensors and precise position information of the target samples are collected and analyzed, a model is trained to learn the relationship between the two, and the first position information of the target object is corrected based on the trained model, thereby further improving the accuracy of the first position information of the target object obtained in the first mode.

[0041] Step S104: adjusting the position information and / or display parameters of the virtual object interacting with the target object in the three-dimensional image based on the first position information.

[0042] In this embodiment, the three-dimensional image is a naked-eye 3D image, and the user can perceive the three-dimensional stereoscopic effect of the naked-eye 3D image through the naked eyes. The naked-eye 3D image includes one or more virtual objects. After the first position information of the target object is obtained in the first mode, the position information of the virtual object interacting with the target object in the naked-eye 3D image can be adjusted based on the first position information. For example, in the process of the target object picking up the virtual object, the position information of the virtual object is adjusted based on the first position information of the target object so that the virtual object moves with the movement of the target object; the display parameters of the virtual object interacting with the target object in the naked-eye 3D image can also be adjusted based on the first position information. The display parameters are parameters that determine the display effect of the virtual object. For example, in the process of the target object approaching the virtual object, if the distance between the target object and the virtual object is less than a certain threshold, it is considered that the target object touches the virtual object. At this time, the color of the virtual object can be changed, or the virtual object can be enlarged to prompt the user that the virtual object has been touched.

[0043] In one possible implementation, before obtaining the first information and the second information of the target object, the parallax between the left and right eye views of the user can be obtained based on the pupil distance of the user to which the target object belongs, and the naked-eye 3D image can be rendered based on the parallax, thereby ensuring that the user's perception of the position of the target object is consistent with the position perception of the virtual object in the naked-eye 3D image, avoiding spatial perception errors when the user judges information such as the position and depth of the virtual object.

[0044] In the present disclosure, when the first information of the target object does not meet the target condition, the first mode is entered in which the second information is used as the dominant processing information, and in the first mode, the second information is processed to obtain the first position information of the target object, and the position information and / or display parameters of the virtual object that interacts with the target object in the naked-eye 3D image are adjusted based on the first position information. Therefore, when the first information collected by the optical sensor does not meet the target condition, the first position information of the target object can still be accurately obtained, and the position information and / or display parameters of the virtual object are adjusted based on the first position information, so that the virtual object changes its position or display effect according to the user's operation, thereby improving the user's immersion in the interaction process and ensuring the user experience.

[0045] In another embodiment, “in response to the first information not satisfying the target condition” in step S102 includes:

[0046] In response to the first information not including the target object; and / or in response to the optical sensor that collects the first information sending sensor abnormality information.

[0047] In this embodiment, if the first information does not contain the target object, it is considered that the first information does not meet the target condition. For example, if the target object is beyond the detection range of optical sensors such as RGB cameras and infrared sensors, or if the target object is blocked by other objects, the first information may not contain the target object. If the optical sensor that collects the first information sends sensor abnormality information, proving that the optical sensor is abnormal, the first information is also considered to not meet the target condition.

[0048] In another embodiment, the “entering the first mode” in step S102 includes:

[0049] The second information is corrected; the weight of the second information is adjusted to the target threshold, and the weight of the first information is adjusted to 0.

[0050] In this embodiment, if the first information does not meet the target condition, the second information needs to be corrected to ensure that the accurate first position information of the target object can be obtained after the second information is processed. It is also necessary to increase the weight of the second information to the target threshold and reduce the weight of the first information to 0 to avoid introducing the first information that does not meet the target condition during the data processing process, thereby improving the accuracy of the first position information of the target object obtained in the first mode.

[0051] In one embodiment, the inertial sensor will produce cumulative errors during long-term use, and the second information can be corrected using zero velocity update (ZUPT) technology, that is, the velocity of the target object is reset when the target object is stationary, thereby eliminating the error caused by acceleration drift.

[0052] In one embodiment, "entering the first mode" may also include adjusting parameters such as the sampling rate, sensitivity, and filter settings of the inertial sensor to enable the inertial sensor to achieve optimal acquisition performance and further ensure the accuracy of the second information subsequently acquired by the inertial sensor.

[0053] Figure 2 A schematic diagram showing a flow diagram of an interactive processing method according to an embodiment of the present disclosure Figure 2 ,like Figure 2 As shown, an interaction processing method includes:

[0054] Step S201, obtaining first information and second information of a target object.

[0055] If the first information does not meet the target condition, the following steps S202 to S204 are performed:

[0056] Step S202, entering the first mode.

[0057] Step S203: in the first mode, the second information is processed to obtain first position information of the target object.

[0058] Step S204: adjusting the position information and / or display parameters of the virtual object interacting with the target object in the three-dimensional image based on the first position information.

[0059] The specific implementation process of step S201 to step S204 is similar to that of step S101 to step S104, and will not be repeated here.

[0060] If the first information meets the target condition, the following steps S205 to S208 are executed:

[0061] Step S205, entering the second mode.

[0062] In this embodiment, the target condition represents that the first information can obtain the location information of the target object after processing. If the first information meets the target condition, the second mode is entered. The second mode represents a mode in which the first information and the second information are used as joint processing information. In the second mode, the first information and the second information are combined to obtain the location information of the target object.

[0063] In one embodiment, based on the quality of the first information and the quality of the second information, the weight of the first information can be adjusted to a first threshold, and the weight of the second information can be adjusted to a second threshold, and the first threshold and the second threshold are not 0, thereby entering the second mode.

[0064] Step S206: in the second mode, the second information is processed to obtain the first position information of the target object, and the first information is processed to obtain the second position information of the target object.

[0065] In this embodiment, the process of processing the second information in the second mode is similar to the process of processing the second information in the first mode in step S103, and will not be repeated here; in the second mode, the optical image in the first information can be used to determine the first coordinate information of the target object in the optical image, and the first coordinate information can be converted into the first three-dimensional space coordinates corresponding to the three-dimensional image, the infrared information in the first information is identified, the second three-dimensional space coordinates of the target object in the three-dimensional image are determined, and the first three-dimensional space coordinates and the second three-dimensional space coordinates are fused to obtain the second position information of the target object.

[0066] Step S207: fuse the first position information and the second position information to obtain third position information of the target object.

[0067] In this embodiment, the first position information and the second position information may be fused based on a weighted average algorithm or a Kalman filter algorithm to obtain the third position information of the target object. For the weighted average algorithm, the weight of the first information may be determined as the weight of the second position information, and the weight of the second information may be determined as the weight of the first position information, thereby fusing the first position information and the second position information.

[0068] Step S208: adjusting the position information and / or display parameters of the virtual object interacting with the target object in the three-dimensional image based on the third position information.

[0069] The specific implementation process of step S208 is similar to that of step S104 and will not be repeated here.

[0070] In the present disclosure, when the first information of the target object meets the target condition, the second mode is entered in which the first information and the second information are used as joint processing information, and in the second mode, the second information is processed to obtain the first position information of the target object, the first information is processed to obtain the second position information of the target object, and then the first position information and the second position information are fused to obtain the third position information of the target object, and based on the third position information, the position information and / or display parameters of the virtual object that interacts with the target object in the naked-eye 3D image are adjusted, so that when the first information meets the target condition, the third position information of the target object is determined in combination with the first information and the second information, and the accuracy of the position information of the target object is further improved, and the position information and / or display parameters of the virtual object are adjusted based on the third position information, so that the virtual object changes its position or display effect according to the user's operation, thereby improving the user's immersion in the interaction process and ensuring the user experience.

[0071] Figure 3 A schematic diagram showing a flow diagram of an interactive processing method according to an embodiment of the present disclosure Figure 3 ,like Figure 3 As shown, an interaction processing method includes:

[0072] Step S301, obtaining first information and second information of a target object.

[0073] Step S302: In response to the first information not satisfying the target condition, entering the first mode.

[0074] Step S303: in the first mode, the second information is processed to obtain first position information of the target object.

[0075] The specific implementation process of step S301 to step S303 is similar to that of step S101 to step S103, and will not be repeated here.

[0076] Step S304, based on the first position information, adjusting the position information of the virtual object; and / or, in response to the distance between the first position information and the position information of the virtual object satisfying the second range, adjusting at least one of the color parameter, size parameter, brightness parameter and texture parameter of the virtual object to the parameter threshold in the visual feedback rule.

[0077] In this embodiment, in a scene where the target object is close to a virtual object, if the distance between the first position information of the target object and the position information of the virtual object satisfies the second range, that is, the distance between the target object and the virtual object is small, it is considered that the target object touches the virtual object. At this time, at least one of the color parameters, size parameters, brightness parameters and texture parameters of the virtual object can be adjusted to the parameter threshold in the visual feedback rule to prompt the user that the virtual object has been touched. The visual feedback rule is a pre-set visual feedback that needs to be performed when the target object touches the virtual object. For example, when the target object touches the virtual object, the virtual object is highlighted or enlarged.

[0078] In this embodiment, in the scenario where the target object controls the movement of the virtual object, the position information of the virtual object needs to be adjusted based on the first position information of the target object. The distance between the adjusted position information of the virtual object and the first position information satisfies the first range, that is, the virtual object is controlled to move with the movement of the target object.

[0079] In this embodiment, the position information and / or display parameters of the virtual object are adjusted based on the first position information, so that the position or display effect of the virtual object changes with the user's operation, thereby improving the user's immersion in the interaction process and ensuring the user experience.

[0080] Figure 4 A schematic diagram showing a flow diagram of an interactive processing method according to an embodiment of the present disclosure Figure 4 ,like Figure 4 As shown, an interaction processing method includes:

[0081] Step S401, obtaining first information and second information of a target object.

[0082] Step S402: In response to the first information not satisfying the target condition, entering the first mode.

[0083] Step S403: in the first mode, the second information is processed to obtain first position information of the target object.

[0084] The specific implementation process of step S401 to step S403 is similar to that of step S101 to step S103, and will not be repeated here.

[0085] Step S404: Based on the first position information, a virtual guide image indicating the first position information and a moving path of the virtual guide image are generated in the three-dimensional image.

[0086] In this embodiment, after obtaining the first position information of the target object, a virtual guide image indicating the first position information can be generated in the naked-eye 3D image based on the first position information. The virtual guide image can be a virtual light spot or a virtual image of the target object. For example, if the target object is the user's hand, the virtual guide image can be a virtual light spot indicating the position of the user's fingertips closest to the virtual object, or it can be a virtual image of the user's hand. The moving path of the virtual guide image can also be generated in the naked-eye 3D image based on the first position information to prompt the user of the movement process of the target object in the naked-eye 3D image.

[0087] Step S405 , in response to the distance between the first position information and the position information of the virtual object satisfying a third range, adjusting the position information of the virtual guide image until the virtual guide image and the virtual object satisfy a target position relationship.

[0088] In this embodiment, if the distance between the first position information and the position information of the virtual object satisfies the third range, for example, if the virtual object is a solid object, but the acquired first position information of the target object passes through the virtual object and is actually located inside the virtual object, then the position information of the virtual guide image needs to be adjusted until the virtual guide image and the virtual object satisfy the target position relationship, such as adjusting the virtual guide image to the surface of the virtual object.

[0089] In one example, if a user wants to hold a virtual object with his hand and move the virtual object, but during the movement of the virtual object, the user's hand cannot maintain a fixed gesture, for example, the user's hand grasps smaller and smaller, and its corresponding first position information has passed through the surface of the virtual object, but the virtual hand still has to remain on the surface of the virtual object, indicating that the user is holding the object and moving it at this time. Therefore, the virtual hand needs to be adjusted to the surface of the virtual object, thereby enhancing the accuracy of the user's interaction with the virtual object, improving the user's sense of immersion in the interaction process, and ensuring the user experience.

[0090] Figure 5 A schematic diagram showing a flow diagram of an interactive processing method according to an embodiment of the present disclosure Figure 5 ;like Figure 5 As shown, an interaction processing method includes:

[0091] Step S501: Acquire first information and second information of a target object.

[0092] Step S502: In response to the first information not satisfying the target condition, entering the first mode.

[0093] Step S503: in the first mode, the second information is processed to obtain first position information of the target object.

[0094] Step S504: adjusting the position information and / or display parameters of the virtual object interacting with the target object in the three-dimensional image based on the first position information.

[0095] The specific implementation process of step S501 to step S504 is similar to that of step S101 to step S104, and will not be repeated here.

[0096] Step S505 , in response to the distance between the first position information and the position information of the virtual object satisfying a second range, controlling a tactile feedback device disposed on the target object to perform tactile feedback.

[0097] In this embodiment, in a scene where the target object is close to a virtual object, if the distance between the first position information of the target object and the position information of the virtual object satisfies the second range, that is, the distance between the target object and the virtual object is small, it is considered that the target object touches the virtual object. At this time, in addition to adjusting at least one of the color parameters, size parameters, brightness parameters and texture parameters of the virtual object to the parameter threshold in the visual feedback rule, a tactile feedback device provided on the target object can also be controlled to perform tactile feedback, such as vibration feedback, so as to prompt the user that he has touched the virtual object based on the visual feedback and tactile feedback, thereby further enhancing the user's sense of immersion in the interaction process and ensuring the user experience.

[0098] In one possible implementation, a tactile feedback device for performing tactile feedback and an inertial sensor for collecting the second information can be integrated on the same smart wearable device. For example, if the target object is the user's finger, the tactile feedback device and the inertial sensor can be integrated on a smart ring, and the user can wear the smart ring on his finger when interacting with a virtual object.

[0099] In order to facilitate the understanding of the present disclosure, the following Figure 6 , Figure 7 , Figure 8 and Fig. 9 An interactive processing method in the present disclosure is explained as follows:

[0100] Figure 6 A schematic diagram showing a scenario of an interactive processing method according to an embodiment of the present disclosure Figure 1 , Figure 6 The video shows a scene where a user interacts with virtual objects in different naked-eye 3D images. Figure 7 A schematic diagram showing a scenario of an interactive processing method according to an embodiment of the present disclosure Figure 2 , Figure 7 by Figure 6Take the scene in the lower right corner as an example. Figure 7 As shown, a naked-eye 3D image 20 is displayed on the naked-eye 3D screen, and a dinosaur is displayed in the naked-eye 3D image 20. The user can sit in front of the naked-eye 3D screen, and the user's hand is the target object; optical sensors such as RGB cameras and infrared sensors (not shown in the figure) are set at positions where the user's hand can be photographed, and are used to obtain first information about the user's hand, such as an image of the user's hand or infrared information, etc.; an inertial sensor and a tactile feedback device for tactile feedback are integrated in the smart ring 10. When the user interacts with the dinosaur in the naked-eye 3D image 20, the user wears the smart ring 10 on his hand, and the inertial sensor in the smart ring 10 can collect second information about the user's hand, such as information such as acceleration and angular velocity of the user's hand.

[0101] Users can Figure 7 The user interacts with the dinosaur in the naked-eye 3D image 20, such as moving, zooming in and out the dinosaur. During the interaction, the optical sensor collects first information about the user's hand, and the inertial sensor collects second information about the user's hand. The first information and the second information are used to obtain the position information of the user's hand after processing. If the first information about the user's hand collected by the optical sensor does not meet the target condition, that is, the first information does not contain the target object or the optical sensor is abnormal, the first mode is entered in which the second information is used as the dominant processing information. In the first mode, only the second information collected by the inertial sensor in the smart ring 10 is processed to obtain the first position information of the user's hand; if the first information about the user's hand collected by the optical sensor meets the target condition, the second mode is entered in which the first information and the second information are used as the joint processing information. In the second mode, the second information is processed to obtain the first position information of the target object, the first information is processed to obtain the second position information of the target object, and the first position information and the second position information are fused to obtain the third position information of the target object. Therefore, when the first information collected by the optical sensor does not meet the target condition or meets the target condition, the first position information of the target object can be accurately obtained. If the distance between the first position information or the third position information of the user's hand and the dinosaur meets the second range, that is, the distance between the user's hand and the dinosaur is small, it is considered that the user's hand touches the dinosaur. At this time, the visual parameters of the dinosaur, such as color, brightness and texture, can be adjusted. Figure 7 The dinosaur displays a luminous effect, and there is a halo around the dinosaur. At the same time, the tactile feedback device set in the smart ring 10 is controlled to perform tactile feedback, such as vibration feedback, so as to prompt the user that he has touched the dinosaur based on visual feedback and tactile feedback, thereby further enhancing the user's sense of immersion in the interaction process and ensuring the user experience.

[0102] When the user touches Figure 7After viewing the dinosaur, the user can control the movement of the dinosaur by moving his hand, so as to move the dinosaur to the position desired by the user. In this process, the position information of the dinosaur is adjusted based on the first position information or the third position information of the user's hand, and the distance between the adjusted position information of the dinosaur and the first position information or the third position information satisfies the first range, so that the dinosaur moves with the movement of the user's hand. Figure 8 A schematic diagram showing a scenario of an interactive processing method according to an embodiment of the present disclosure Figure 3 ,like Figure 8 As shown, the user will Figure 7 The dinosaur located at the center of the naked-eye 3D display screen moves to the left to a position close to the left edge of the naked-eye 3D display screen. It should be emphasized that the dinosaur can keep the luminous effect while the user is moving it. Figure 8 As shown, there is still a halo around the dinosaur.

[0103] Fig. 9 A schematic diagram showing a scenario of an interactive processing method according to an embodiment of the present disclosure Figure 4 ,like Fig. 9 As shown, after obtaining the first position information or the third position information of the user's hand, a virtual hand 30 indicating the first position information or the third position information of the user's hand and a moving path 40 of the virtual hand 30 can be generated in the naked-eye 3D image 20 based on the first position information or the third position information of the user's hand. If the user touches the dinosaur with his hand and moves it, the naked-eye 3D image 20 will display the virtual hand 30 moving the dinosaur and the path of the virtual hand 30 moving the dinosaur. In this process, the display position of the virtual hand 30 can also be adjusted based on the positional relationship between the user's hand and the dinosaur, thereby enhancing the accuracy of the user's interaction with the dinosaur, improving the user's sense of immersion in the interaction process, and ensuring the user experience.

[0104] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0105] Fig.10 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0106] like Fig.10As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0107] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0108] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as an interactive processing method. For example, in some embodiments, an interactive processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of an interactive processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform an interactive processing method in any other appropriate manner (e.g., by means of firmware).

[0109] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0110] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0111] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0112] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0113] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0114] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0115] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0116] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0117] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. An interactive processing method, include: Acquire first information and second information of a target object, wherein the first information comes from an optical sensor, and the second information comes from an inertial sensor disposed on the target object; In response to the first information not satisfying the target condition, entering a first mode, wherein the first mode represents a mode in which the second information is used as the dominant processing information; In the first mode, the second information is processed to obtain first position information of the target object; Based on the first position information, position information and / or display parameters of a virtual object that interacts with the target object in the three-dimensional image are adjusted.

2. The method according to claim 1, wherein in response to the first information not satisfying the target condition, include: In response to the first information not including the target object; and / or, In response to the optical sensor collecting the first information, sensor abnormality information is sent.

3. The method according to claim 1, wherein entering the first mode: include: Correcting the second information; The weight of the second information is adjusted to a target threshold, and the weight of the first information is adjusted to 0.

4. The method according to claim 1, further comprising: include: In response to the first information satisfying the target condition, entering a second mode, wherein the second mode represents a mode of treating the first information and the second information as jointly processed information; In the second mode, the second information is processed to obtain the first position information of the target object, and the first information is processed to obtain the second position information of the target object; fusing the first location information and the second location information to obtain third location information of the target object; Based on the third position information, position information and / or display parameters of a virtual object interacting with the target object in the three-dimensional image are adjusted.

5. The method according to claim 4, wherein the first location information and the second location information are fused to obtain the third location information of the target object. include: Based on a weighted average algorithm or a Kalman filter algorithm, the first position information and the second position information are fused to obtain third position information of the target object.

6. The method according to claim 1, wherein the position information and / or display parameters of a virtual object interacting with the target object in the three-dimensional image are adjusted based on the first position information. include: Based on the first position information, the position information of the virtual object is adjusted, and the distance between the adjusted position information of the virtual object and the first position information satisfies a first range; and / or, In response to the distance between the first position information and the position information of the virtual object satisfying a second range, at least one of the color parameter, size parameter, brightness parameter and texture parameter of the virtual object is adjusted to a parameter threshold in a visual feedback rule.

7. The method according to claim 1, after processing the second information to obtain the first location information of the target object, further comprising: include: Based on the first position information, a virtual guide figure indicating the first position information and a movement path of the virtual guide figure are generated in the three-dimensional image.

8. The method according to claim 7, further comprising: include: In response to the distance between the first position information and the position information of the virtual object satisfying a third range, the position information of the virtual guide image is adjusted until the virtual guide image and the virtual object satisfy a target position relationship.

9. The method according to any one of claims 1 to 8, further comprising: include: In response to the distance between the first position information and the position information of the virtual object satisfying a second range, a tactile feedback device disposed on the target object is controlled to perform tactile feedback.

10. An electronic device, include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

Images